A
REVIEW OF PROGRAMS
FOR
MINORITIES AND MINORITY SERVING INSTITUTIONS
By
Committee of
Visitors (COV)
The National
Science Foundation
February 1-2,
2001
A Committee of Visitors
(COV) Review of Programs for Minorities and Minority Serving
Institutions
EXECUTIVE
SUMMARY
Overview
This Committee of Visitors (COV) report summarizes the findings,
recommendations and strategic outcomes for each of four programs serving
minorities and minority serving institutions. The COV conducted an extensive review
and analyses of the Louis Stokes Alliance for Minority Participation (LSAMP),
Historically Black College and University - Undergraduate Program (HBCU-UP),
Alliances for Graduate Education and the Professoriate (AGEP), and the Centers
for Research Excellence in Science and Technology (CREST) programs on their
implementation strategies and results relative to the Government Performance and
Results Act (GPRA) guidelines, NSF-wide strategic plan, and the EHR directorate
and HRD divisional goals and objectives.
The primary strategic
objectives for the four programs under review by this COV are the
following:
·
To
develop educational capacity at minority serving
institutions;
·
To
leverage existing capacity to substantially increase baccalaureate graduates
from minority serving and majority institutions;
·
To
leverage existing capacity and linkages to substantially increase doctoral
graduates and new faculty production; and
·
To
build research capacity, principally among senior researchers at minority
serving institutions.
To maximize the
effectiveness of the programs under review, four primary coordination strategies
for achieving program linkages have been established, (1) linkages among
programs for minorities and minority serving institutions, (2) linkages with
other HRD programs, (3) linkages with other EHR programs, and (4) linkages with
other NSF programs and other Federal programs with comparable goals and
objectives.
The underlying philosophy of these programs is that they will ultimately
meet national needs with respect to the future workforce in the SMET
fields. This philosophy is
strategically aligned with the following GPRA goals, as listed
below:
People
- A diverse, internationally competitive
and globally engaged workforce of
scientists, engineers, and
well-prepared citizens.
Ideas - Discovery across the frontier of science
and engineering, connected to learning innovation and service of
society.
Tools - Broadly accessible,
state-of-the-art and shared research and education tools.
Approach
In the conduct of the COV,
the committee considered a number of informational sources, including a random
sampling of award and declination files for FY 1998, FY 1999, and FY 2000;
presentations and individual interviews from the program staff and senior
administrators; and printed documents pertaining to NSF strategic outcomes and
evaluation data. The committee paid
particular attention to the integrity and efficiency of program processes and
management through an analysis of the following four
factors:
·
Effectiveness
of each program's use of merit review procedures
·
Program
ratings on the use the new NSF merit review criteria
·
Reviewer
selection
·
Resulting
portfolio of awards
The Committee of
Visitors (COV) was convened by the National Science Foundation on February 1-2,
2001 was chaired by Dr. Orlando L. Taylor, a member of the EHR Advisory
Committee and comprised of the members listed below under the following
sub-groups:
Undergraduate
Sub-Group
Dr. Linda Mantel, University of Portland,
(Sub-Group-Chair)
Dr. Alfredo G. de los Santos, Jr., Arizona State
University
Dr. Charles Pickett, Mississippi Institutions of
Higher Learning
Dr. Robert Harvey, Knoxville
College
Graduate
and Research Institutes Sub-Group
Dr. Fitzgerald Bramwell, University of Kentucky,
(Sub-Group-Chair)
Dr. Jeanette Jones, Alabama A&M
University
Dr. Warren Buck, University of Washington,
Bothell
Dr. Joseph Ortiz, Kansas State
University
SUMMARY OF MAJOR FINDINGS
AND
RECOMMENDATIONS
Major
Findings
The following represents a
summary of some of the major findings of this COV report:
v The overall design,
including appropriateness of review mechanism (e.g., panels, ad hoc reviews,
site visits, etc.) for all programs under review, are adequate based on the use
of merit reviews.
v The six months which the
Foundation allows itself between proposal submission and notification of award
or decline is longer than desirable for the LSAMP program. For the HBCU Program,
the due dates and notification timetables appear adequate.
v All funded proposals
fulfilled the priorities and criteria stated in the solicitations and guidelines
for all programs under review. Within AGEP programs, the alteration of program
criteria encouraged the formation of institutional alliances to achieve
objectives, resulting in an increased emphasis on regional
partnerships.
v Generally, among CREST
programs there was consistency with priorities stated in the program
solicitations, announcements and guidelines. We note the recent suggested addition of
a matching funds requirement in CREST.
v Among programs such as LSAMP
and HBCU-UP, it is very difficult to separate the intellectual merit from the
broader impact of the project. The current program announcement of the
importance of linking the HBCU-UP and LSAMP programs to other NSF-funded
graduate programs (such as AGEP and CREST) and to graduate institutions is
commendable.
v Within AGEP and CREST
programs, program officers adequately monitored review panel activity with
respect to intellectual merit. The impact of diversity and pipeline activity on
basic research programs is adequately addressed.
v Reviewers show deep
commitment to-- and annual reports reveal--the establishment of a community of
graduate programs that are intimately networked with one another and with
undergraduate feeder institutions.
v The identification of and
support for emerging opportunities and the creation and support of existing
opportunities are difficult to distinguish. Existing opportunities appear to
provide a leverage for NSF support and are crucial to the continuing success and
expansion of program efforts.
v Potential support for new
investigators is enhanced through the LSAMP program by the ability of
researchers from small or developing institutions to partner with investigators
from major research institutions.
v Both undergraduate programs
show specific support of minority students, programs, and institutions. Also,
there is a good balance of multidisciplinary and innovative projects in funded
proposals.
v AGEP projects appear to
uphold high professional standards; however, there is a large number of unfunded
proposals that also meet these criteria. Out of seventy proposals submitted
during FY 98-00, twenty-two were awarded. This award ratio of 31% approximates
the Foundation-wide rate.
v Among CREST projects, the
overall quality of the science/engineering programs is very good, with a good portfolio of awards.
v CREST institutions generally
take advantage of emerging opportunities to expand grant award portfolios such
as REU’s, instrumentation awards, and awards from other agencies such as the
Department of Energy, Department of Defense, Federal laboratories and industrial
alliances.
v AGEP projects have broad
support for new and established investigators, benefiting the creation of
faculty mentoring relationships. This is true with respect to post-docs, senior
researchers and new faculty.
v AGEP proposals that are
funded generally emphasize the integration of research and education in an
academic context. CREST institutions often emphasize outreach, mentoring and
research even more than research-intensive institutions.
v AGEP projects have produced
a steady increase in the number of minority scientists in the
pipeline.
v AGEP projects have a
tendency to emphasize incremental advances in research and to adopt
well-recognized pedagogical strategies. The majority of the CREST institutions
have multidisciplinary projects and few high risk projects.
vHBCU and LSAMP programs do
not specifically have the goal of improving mathematics, science and technology
understanding and skills for U.S. students at the K-12 level; however, several
of the LSAMP alliances, e.g., Chicago State and Howard, have teacher preparation
programs as part of their project.
v The vast majority of the
faculty teaching in institutions funded by HBCU-UP is African-American, and a
significant majority of the students in these institutions is also
African-American, thus having the potential to strengthen the diversity of the
scientific workforce.
v Increased numbers of
minority undergraduate students at LSAMP institutions enroll in and complete
college level courses in all areas of SMET, thereby assisting greatly in meeting
the goal of a better-informed citizenry. Across the 27 alliances, over 187,000
students were enrolled in SMET courses in 1999.
LSAMP programs also assist
greatly in increasing the diversity of the scientific workforce. For example, in
1999, there were 20,567 BS degrees awarded in SMET fields through the LSAMP
alliances. [LSAMP MARS
database]
v The increased use and
further development of the new LSAMP-Mars database will make an enormous
contribution, not only to the success of LSAMP, but also to its linkages with
NSF-funded graduate and research programs, by demonstrating the quality and
productivity of LSAMP projects.
v LSAMP funding does not
provide international activity; however, virtually all grantees find creative
ways to prepare their students for work and study in contexts beyond their local
campuses.
v CREST/AGEP programs have
performed excellently in growing the numbers of students exposed to NSF funded
programs. Coordinated efforts exist within AGEP to access and/or improve the
quality of graduate support systems, expand graduate education curriculum
options, gain access to a variety of academic partners and improve the
preparation of graduates for broad career options.
v The transition of the AGEP
to emphasis on institutional alliances has provided a workable mechanism through
which graduate students from diverse backgrounds can gain access to globally
engaged science and engineering professionals with outstanding reputations.
v As noted in a recent
analysis [SED 9255369] of the LSAMP program, the alliance model of LSAMP links
advances in discovery (led by research universities) with the advances in
learning and societal benefits contributed by the other components of an
alliance. [Westat Study of LSAMP Program]
v The institutions that are
part of an LSAMP alliance are proactive in linking research and education.
Curriculum reform is a major component of undergraduate programs in LSAMP
schools.
For example, North Carolina
A&T State University in Greensboro, NC [HRD 9909058], as part of a target
strategy of improving institutional capacity, will revise three “killer” courses
simultaneously: calculus, chemistry, and physics. Teams of faculty from the
three disciplines will work together to develop complementary course objectives,
using active learning pedagogies, and the same web-based technology. Thus,
students in calculus will be taught the mathematics concept that they need in
order to solve the problems that are being assigned in chemistry or physics, and
the homework they turn in will be graded by both the calculus faculty and the
science faculty. Four classrooms will be remodeled to facilitate this new manner
of teaching and will be equipped with the most current technology available.
Faculty in all three departments will be provided with workshops, seminars, and
technical assistance to implement this new way of teaching and learning.
v CREST researchers and
programs require shared use instrumentation, cross campus collaboration, and
inter-university collaboration to enhance the probability of the discovery or
creation of new knowledge. The collaboration in physics of the CREST Center at
Hampton University [HRD 9154080; HRD 9633750] with the University of Michigan
and with Duke University is an excellent example of such collaborations.
v Networking and connectivity
that take full advantage of the Internet and make SMET information available to
all citizens are exploited fully within the AGEP community. Among CREST
programs, this activity is critical to success.
v AGEP projects have created
effective lines of institutional communication and student transition between
participating institutions. This
collective result derived from meeting the needs of individual institutions has
contributed to an effective use of science and engineering resources.
The following represents a
summary of the major recommendations found within this COV
report:
Evaluation
Recommendations
v A longer preparatory period
after the release of the program solicitation and before the review process is
advised.
v In order to provide a fair
and comprehensive evaluation of the CREST/AGEP programs, the COV panel should attend
meetings of the program directors.
v The composition of the external review panels for AGEP programs could be enhanced by increased representation from disciplinary representatives within NSF.
v A review mechanism that embeds professional contributions (e.g., review panel membership, site visit participation, etc.) from other NSF directorates should be implemented for CREST programs.
Program
Recommendations
v The capacity of CREST/AGEP
centers should be maximized.
v Activities to complete the
transition of MGE programs to AGEP programs should be enhanced until all MGE
programs are fully transitioned.
v Linkages between LSAMP
programs in Research I universities, HBCUs, Hispanic-serving institutions,
Tribal colleges, and two-year, four-year, liberal arts and comprehensive
institutions should be improved so that students can be better assured
productive careers in SMET, both in academia and
elsewhere.
v A common set of professional
standards should be established to maximize the potential of engagement of
targeted communities and the scientific community as a whole. CREST proposal
submission and evaluation should place greater emphasis on the research to be
conducted and should be judged in a manner consistent with that used for other
NSF-funded centers.
v A database should be
developed for LSAMP-Mars programs to access, update and track information on
individual students regarding enrollment, degrees obtained, stipends and
awards.
v CREST centers should serve
as nationally recognized exemplars in their areas of
research.
v CREST and AGEP/SMET programs
should work toward greater access to the larger graduate scientific communities
and professional organizations through Internet connectivity.
A. INTEGRITY AND EFFICIENCY OF
THE PROGRAM’S PROCESSES
AND
MANAGEMENT
Based on the COV’s study of 269 proposal actions completed within the past three fiscal
years, comments were provided on each of the following aspects of the
program’s review processes and management.
Constructive comments indicating areas for improvement were encouraged.
1. Effectiveness of the Program’s Use of the Merit Review Process.
The COV considered the following in determining the effectiveness of the program’s use of merit review procedures in assessing integrity and efficiency of processes and management:
a.
Overall design, including
appropriateness of review mechanism (panels,
ad hoc reviews, site
visits);
b.
Effectiveness of program’s
review process;
c.
Efficiency; time to
decision;
d.
Completeness of documentation making
recommendations;
e.
Consistency with priorities and criteria stated in the program’s
solicitations,
announcements, and guidelines.
The COV made the following observations of graduate and undergraduate programs based on the above indicators:
a.
Overall design,
including appropriateness of review mechanism (panels, ad hoc reviews, site
visits)
UNDERGRADUATE
PROGRAMS
LSAMP Program
Merit reviews for the LSAMP program are conducted by a combination of panel reviews and site visits (or reverse site visits) before and/or after funding. Panelists review the applications in advance, and, after extensive discussion at the panel meeting, prepare a panel summary. Annual reports provide information on each project and requests for continuation. They are reviewed by panel and reverse site visit. NSF program officers prepare reports and make recommendations based on the panel reviews and on staff analyses.
In addition, the LSAMP has had Program Effectiveness Reviews, carried out by NSF staff, which involve the preparation of substantial summaries by the funded institutions and participation in a combined reverse site visit. These reviews provide data on the production of minority graduates in SMET fields compared with baseline efforts and compared to state and national trends. Thus, they are measures of the effectiveness of the LSAMP project on individual campuses. In addition, program staff participate in the activities of the funded projects and carry out additional site visits as necessary. These multiple forms of review provide both detailed information on each participating institution, as well as a sense of the institutional situation and commitment to the goals of the program.
HBCU Program
This program has completed only two annual competitions so far. Merit review is carried out as discussed above. In addition, an annual report is submitted that addresses the program goals and objectives included in the proposal. NSF staff makes recommendations for continuation based upon these reports.
GRADUATE PROGRAMS AND RESEARCH
CENTERS
AGEP
A diverse set of reviewers has been systematically assembled to assess the prospects and realization of the recruiting and retention goals of AGEP. External panels have executed a balanced and rigorous review process. The composition of the review panels could be enhanced by increased representation from disciplinary representatives within NSF.
CREST
The overall CREST design incorporates a number of self-checking accountability measures and possesses critical strengths which have been pivotal to the reviews of the proposed activities. The COV recommends that a review mechanism that embeds professional contributions (e.g., review panel membership, site visit participation) from other NSF directorates be implemented.
b.
Effectiveness of program’s review process
UNDERGRADUATE
PROGRAMS
The LSAMP and the HBCU-UP
Programs
Effectiveness of the review process for the two undergraduate programs is indicated by the ability of the panels and the NSF staff to differentiate clearly between competitive and non-competitive proposals, and to give clear indications when non-competitive re-submissions have reached competitive strength. Consistency in the ratings from reviewers are the norm (3 VG, or 2E, 1VG are common for highly competitive proposals) and individual reviews show strong commonality of noted strengths and weaknesses of each project. In the HBCU program, most proposals submitted in the first annual competition were not funded. Some of the institutions whose proposals were not funded in the first annual competition submitted proposals for the second annual competition. Those of high quality were funded. Others were not funded in either annual competition.
GRADUATE PROGRAMS AND RESEARCH
PROGRAMS
AGEP
Individuals with an understanding of and commitment to the goals of the AGEP program have conducted candid and informed reviews. Ample opportunity was provided for proposing institutions to obtain feedback from reviews. It is likely that the thoroughness of the review process was reinforced by site visits.
CREST
As designed, the review process enhances the research competitiveness of the submitting institution. This outcome is reflected in carefully stated reviews by a diverse expert panel review team.
c. Efficiency and time to decision
UNDERGRADUATE
PROGRAMS
LSAMP
Proposals are due each year on October 15. Annual reports from continuing institutions are due October 31. For successful proposals, the date of the Cooperative Agreement ranges from August to October of the following year. Thus, the Foundation’s target of providing notification of award within six months is longer than desirable. However, because of the complexity of the proposals, the multiple institutions that are involved, and the multiple steps involved in the proposal process at both the NSF and among the cooperating institutions, it is often difficult to complete the process more quickly. As of FY 2001, the due date for LSAMP is January 31.
HBCU Program
The due date for proposals is October 15, and the notifications are made and cooperative agreements signed within six months.
GRADUATE PROGRAMS AND RESEARCH
CENTERS
AGEP
A thorough review of selected proposals indicates that a timely review process has been implemented with deliberate, but optimal, responses to proposing institutions.
CREST
Based on a sampling of reviews of proposed activities, the NSF’s response time following proposal submission appears to be good.
d.
Completeness of
documentation making recommendations
UNDERGRADUATE
PROGRAMS
LSAMP and HBCU-UP Programs
Documentation of recommendations is provided by detailed summaries and analyses by the NSF staff of panel findings and responses of institutions to questions arising from reviews.
GRADUATE PROGRAMS AND RESEARCH
CENTERS
AGEP
The completeness of documentation for making recommendations is excellent.
CREST
The completeness of documentation making recommendations is also excellent.
e.
Consistency with priorities
and criteria stated in the program’s solicitations, announcements, and
guidelines.
UNDERGRADUATE
PROGRAMS
All funded proposals fulfill the priorities and criteria stated in the solicitations and guidelines for both LSAMP and HBCU-UP.
GRADUATE PROGRAMS AND RESEARCH
CENTERS
AGEP
There is a close correspondence between promulgated information and the execution of the AGEP review procedures. The alteration of program criteria to encourage the formation of institutional alliances to achieve objectives has resulted in an increased emphasis on regional partnerships.
CREST
Generally,
there is consistency with priorities stated in the program solicitations,
announcements and guidelines. However, there does appear to be an inconsistency
in communicating the need for matching or institutional contributions between
program solicitations and the review process. Explicitly, this can be found in
the apparent contradiction of “cost-sharing on page 14 of the draft CREST Program Announcement (April 15,
2001), a requirement that was not present in the previous Program Announcement and Guidelines NSF
98-19.
2. The
Programs’ Use of the Merit Review Process
The COV
considered the following in determining the effectiveness of each program’s use
of the NSF Merit Review Criteria (intellectual merit and broader
impacts):
a.
Performance
Goal: Implementation of Merit Review Criteria by
Reviewers: NSF performance in implementation of the merit review criteria is
successful when reviewers address the elements of both generic review criteria.
b.
Performance Goal:
Implementation of Merit Review Criteria by Program Officers: NSF performance in
implementation of the merit review criteria is successful when program officers
address the elements of both generic review criteria.
The COV
considered the reviewers and program officers of the programs under review to be
successful in implementing the merit review criteria.
The following observations
were made relative to whether reviewers adequately addressed the intellectual
merit criteria in their reviews:
UNDERGRADUATE
PROGRAMS
LSAMP:
Example
1. “..This is a well thought out
project with the potential of affecting education in the whole state of () by
the creation of bridges between high schools and the universities in the
state.”
Example
2. “The potential for benefits is
much greater than most due to the high number and percentage of Blacks and
Hispanics.”
Example 3. “I rated this proposal as excellent because its
approach is based on important research, because it is innovative in its use of
technology, because it
builds the
ability to change into the curriculum, and because all of these qualities give
it a high potential to accomplish its purposes.”
Example
4. “This is an excellent
opportunity to model successful approaches to increasing minority students.”
HBCU-UP
Example 1:
“The proposal has clearly stated strategies that sufficiently address program
goals. It reflects a complete understanding of the significance of faculty
development, curriculum reform and enhancement, and student research experience
to strengthen the SMET undergraduate education and research infrastructure. The
intrinsic merit of the proposal is strengthened by careful recruiting plans,
proposed integrated enrichment programs, favorable demographics for potential
impact, promise of helpful cooperation from industry, recognition of the
importance of financial assistance and an overall collaborative approach.”
In programs
such as LSAMP and HBCU-UP, it is very difficult to separate the intellectual
merit from the broader impact of the project. The assessments quoted above
demonstrate how reviewers address both of these criteria in an integrated
fashion.
GRADUATE AND RESEARCH
CENTERS
AGEP
The program officers
adequately monitored review panel activity with respect to intellectual
merit.
CREST
The program officers
adequately monitored review panel activity with respect to intellectual
merit.
The impact of diversity and pipeline activity on basic research programs was adequately addressed. The impact of research driven economic activity (e.g., technology licensing, start-up businesses) should be enhanced.
The following observations were made as to
whether or not reviewers adequately addressed the broader impacts
criteria in their decisions:
UNDERGRADUATE
PROGRAMS
Inspection of the reviews reveals a deep and reflective commitment to the consideration of broader impacts in the proposals.
GRADUATE AND RESEARCH
CENTERS
Inspection of the reviews reveals a deep and reflective commitment to the consideration of broader impacts in the proposals. Remarks in the reviews on this subject were plentiful and incisive. The annual reports cite the establishment of a community of graduate programs that are intimately networked with each other and with undergraduate feeder institutions.
B. THE EXTENT TO WHICH PROGRAM
OFFICERS ADDRESSED THE BROADER
IMPACT CRITERIA
The COV made the following observations and recommendations with respect to whether or not the program officers adequately addressed the broader impact criteria in their decisions:
UNDERGRADUATE
PROGRAMS
For the most part, staff
analyses reiterated the comments of reviewers on the merit and impact criteria
for both programs. The COV recommends that the template for staff analyses
include specific reference to the two criteria, which it does not include
currently. The COV applauds the
addition in the current program announcement of the importance of linking the
HBCU-UP and LSAMP programs to other NSF-funded graduate programs (such as AGEP
and CREST) and to graduate institutions.
This linkage is already reflected at places such as Miles College in
Alabama where student participants benefit from technology-enhanced curricula,
faculty mentoring, and financial support directly through an HBCU-UP grant [HRD 9909038]. These same students received faculty
supervised research experiences through the LSAMP-sponsored summer internships
at the University of Alabama at Birmingham [HRD 9940643]. The students also receive graduate
school awareness, preparation and recruitment through the Alabama Alliance for
Graduate Education and the Professoriate [HRD 9817296], which provides
mentoring, and faculty advisement to both existing and potential graduate
students.
Program officers demonstrated an awareness of the many broader impacts that could accompany the funding of various proposals (e.g., regional distribution, diversification of institution type, representation of the disabled and underrepresented populations).
The program officers
adequately monitored review panel activity with respect to broader impacts.
The impact of diversity and pipeline activity on basic research programs was adequately addressed. The impact of research driven economic activity (e.g., technology licensing, start-up businesses) should be enhanced.
C. CONCERNS RELATIVE TO
THE NSF MERIT REVIEW SYSTEM
The COV had the following
concerns with respect to the NSF merit review system.
The COV
examined over two dozen jackets from funded and non-funded projects for the
three review years. The reviewers’
comments ranged from brief to extensive, but they all addressed both the
intellectual merits of the proposal and the broader impacts of the activity.
Both the LSAMP and the HBCU-UP programs responded directly to the NSF strategic
outcome related to people and less so to the outcomes focused on ideas and
tools. The goal of the LSAMP is baccalaureate production, while the goal of the
HBCU-UP is the improvement and enhancement of the institutional capacity to
produce more graduates.
GRADUATE PROGRAMS AND
RESEARCH CENTERS
Identification of promising proposals with respect to eventual integration into competitive review processes depends upon critical evaluation from discipline focused reviewers. NSF in-house expertise could be brought to bear on this opportunity
There is concern that the
review panels and site visit teams do not contain a broad enough representation
of reviewers. The representation can be further broadened by the inclusion of
not only more discipline focused external experts, but also by inclusion of NSF
discipline focused divisional staff. This enhances the probability that the
proposed activities will flourish beyond CREST funding and enhances the
likelihood of increased visibility within the NSF.
3. REVIEWER
SELECTION
The following represents reviewer selection
criteria.
a.
Use of adequate
number for balanced review;
b. Use of reviewers
having appropriate expertise/qualifications;
c.
Use of reviewers
reflecting balance among characteristics such as geography, type of institution,
and underrepresented groups;
d.
As
appropriate, recognition and resolution of conflicts of interest by NSF staff
and adequacy of documentation justifying actions
taken.
Based on the
above criteria, reviewers made the following comments:
HBCU-UP and
LSAMP
a.
Use of adequate
number for balanced review
For both HBCU-UP and LSAMP, panels consist of six to eight reviewers, and each proposal is reviewed by at least three panelists.
b. Use of reviewers having
appropriate expertise/qualifications
Without seeing the CVs of the
panelists, it is difficult to comment precisely on their expertise and
qualifications. However, many of the panelists have national reputations for
scholarship and service in their fields. In addition, many members of the COV
have personal and professional knowledge of the panelists. The division director
approves suggestions for reviewers, and a balance is sought among previous
grantees, seasoned reviewers, and those new to the process.
c. Use
of reviewers reflecting balance among characteristics such as geography,
type of institution, and underrepresented groups
The review teams are well balanced by field of expertise, type of institution and region of the country represented, gender, race, and ethnicity. Large and small, private and public, two-year, four-year, and graduate institutions are represented.
d.
As appropriate, recognition and resolution of conflicts of interest by
NSF staff and
adequacy of documentation justifying actions taken
In every case, when a member of a review panel had a conflict of interest, this was not only well documented, but it was also clear that the panel member did not participate in the discussions related to the proposal in conflict. They did not prepare an independent review of the proposal nor did they sign the Summary Rating Sheet.
CREST/AGEP
a.
Use of adequate number for balanced review
The review panels reflected an excellent cadre of multi-disciplinary scientists
from a variety of backgrounds.
b. Use of reviewers having appropriate
expertise/qualifications
The choice of reviewers
demonstrated a good match of expertise and qualifications to the programs under
review.
c. Use of reviewers reflecting
balance among characteristics such
as geography, type of institution, and
underrepresented groups
The diversity
of each panel was balanced and was reflected by the selection of individuals
from various types of institutions, geographical locations and underrepresented
minority groups.
d. As appropriate, recognition and
resolution of conflicts of interest by
NSF staff and adequacy of
documentation justifying actions taken
The NSF staff was diligent in recognizing and resolving conflicts of interest and providing the necessary documentation to support implemented actions.
B. RESULTS: OUTPUTS AND OUTCOMES OF NSF
INVESTMENTS
4. Resulting
Portfolio of Awards
The reviewers used the following criteria for determining the resulting portfolio of awards:
a)
Overall quality of
science/engineering;
b)
Appropriateness of award,
scope, size, and duration;
c)
Effective identification of
and support for emerging opportunities;
d)
Appropriate attention to
maintaining openness in the system, for example, through the support of new
investigators;
e)
Evidence that proposers had
addressed the integration of research and education in
proposals;
f)
Evidence of
increased numbers of applications from underrepresented
groups;
g)
Balance of projects
characterized as
·
High-risk
·
Multidisciplinary
·
Innovative
UNDERGRADUATE
PROGRAMS
The following comments were made on undergraduate programs based on the above indicators:
HBCU-UP and
LSAMP
a. Overall quality of
science/engineering
The overall quality of science/engineering is assured at the basic level by the fact that NSF only supports institutions accredited by their regional accrediting bodies. In addition, many particular programs (e.g., engineering, chemistry) are accredited by national associations. The goal of the HBCU-UP program is to improve and enhance the capacity of the institutions. Thus, while the overall quality of the science and engineering programs may not be cutting edge to start, their capability will improve. In the LSAMP program, the alliance model supports a range of institutions from Research I to small liberal arts and two-year colleges. Thus, participants in the alliance have the opportunity to observe and take part in cutting edge research. For instance, the Alabama LSAMP [HRD 9940643] consists of Research I universities and HBCUs with varying levels of research support and traditions. The All-nations AMP [HRD 9940632] consists of all 32 tribal colleges, a number of major mid-western research universities, and 20 two-year colleges. In all cases, institutions are expected to remain true to their own goals and missions while improving their capacity to carry out science and engineering programs.
b. Appropriateness of award scope, size,
and duration
The award size, scope, and duration are appropriate. The characteristics of funded grants depend upon program guidelines and proposed activities. For HBCU-UP, in the first year the awards were for three years for a total of about $1.8 million. In the second year, the awards were for about $3 million for 5 years. For LSAMP, the size of the award now depends upon the number of BS degrees anticipated. Most awards are for five years at a time, or one phase. Currently, there are 17 alliances in Phase II (second five years) and 11 in Phase I (first five years). Among the institutions in an alliance, policies on admission of students vary, but systems designed to support students and encourage their persistence to the degree are important components of all programs.
c. Effective identification of and
support for emerging opportunities
It is sometimes difficult to distinguish between the effective identification and support for emerging opportunities and the creation and support of existing opportunities. To increase the number of graduates in SMET within individual LSAMP projects from 3,910 to 20,567 in 10 years required that these students be available and identifiable; but the LSAMPs clearly created a major opportunity for these students to succeed in pursuing and achieving their degree. The success of these projects has provided a basis for increased non-NSF funding. For example, the South Carolina legislature provides $600,000 in direct support of the South Carolina LSAMP [HRD 0043700]; the Illinois Legislature provides $500,000 for the Illinois LSAMP [HRD 0000341], the New Mexico legislature provides $700,000 to New Mexico LSAMP [HRD 9802223]. The CUNY system itself supports the New York AMP [HRD 0043973] with both direct and matching funds of more than $1 million, while Hewlett-Packard supports the All Nations LSAMP [HRD 9940632] with $500,000. These opportunities for leveraging NSF support are crucial to the continuing success and expansion of LSAMP efforts.
d. Appropriate attention to
maintaining openness in the system, for example, through the support of new
investigators
Potential support for new investigators is enhanced through the LSAMP program by the ability of researchers from small or developing institutions to partner with investigators from Research I institutions.
e. Evidence that proposers have
addressed the integration of research
and education in
proposals
Although research itself is not a goal of LSAMP or HBCU-UP, there is clear evidence that the proposals funded integrate research and education. As noted above (2a), one reviewer wrote, “I rated this proposal excellent because its approach is based on important research.”
f. Evidence of increased numbers of
applications from underrepresented groups
Both of these programs are directed specifically to support of minority students, programs, and institutions. In this context the demographic characteristics of the formal principal investigator is irrelevant. In broader terms, these programs are contributing to the growth of a future pool of scientists and engineering researchers that may make application to NSF for awards.
g.
Balance of projects characterized as
1) High Risk
2)
Multidisciplinary
3) Innovative
LSAMP
High Risk projects with the
potential for substantial payoff represent a appropriately modest fraction of
the LSAMP portfolio. For example,
the All Nations LSAMP project is unusual in that all of the participating
institutions are tribal colleges and universities, the vast majority of which
are associate degree granting institutions. This represents a high risk approach
within a program devoted to baccalaureate production. However, this approach has the potential
for resulting in a significant increase in Native American participation in
science and engineering since it engages every tribal college in the
Nation. Additional benefits could
be derived from applying lessons learned to other community colleges with large
enrollments of underrepresented minorities and women. The LSAMP projects exhibit many innovative
and multidisciplinary characteristics.
LSAMP projects emphasize changes in curriculum and teaching methods. This multidisciplinary approach fosters
innovation across all projects activities, i.e., summer bridge programs,
research experience, mentoring, drop-in centers, caring staff, and the Alliance
structure. LSAMP P.I.s are implementing innovative approaches to preparing
students for enrollment into graduate school.
HBCU-UP
Attention to high risk projects
with the potential for high returns is seen within the most recent cohorts of
projects that bring into the NSF grantee fold institutions that have not
previously received more than a very few NSF awards. Working with novice grantees will
require particular attention to project management. Technical assistance is available to all
HBCU-UP projects. The HBCU-UP
projects are multidisciplinary and directly impact large numbers of
underrepresented minority undergraduate students in SMET disciplines. The primary focus of existing HBCU-UP
project activities is to improve the quality of undergraduate SMET education
using innovative approaches to implement effective educational techniques and
practices.
GRADUATE PROGRAMS AND RESEARCH CENTERS
a. Overall quality of science/engineering
AGEP
All research and teaching projects appear to uphold high professional standards. There remain a large number of unfunded proposals that would also meet these criteria. Out of seventy proposals submitted during FY 98-00, twenty-two were awarded. This ratio of awards to declinations suggests significantly high criteria for funding as well as a significantly high unmet demand.
CREST
The overall quality of the science/engineering programs is very good and in general, the older centers have a good level of portfolio awards.
b.
Appropriateness of award scope, size, and duration
AGEP
The scope, size and duration of the awards are proportional to the proposed activities.
CREST
The size and scope of the awards are very good and allow the proposing institution the opportunity to expand and deepen its research capabilities. The duration of the awards gives investigators time to initiate the development of complex infrastructures. The CREST award should have a similar duration and eligibility for renewal of funding as other major infrastructure awards such as STC.
c.
Emerging Opportunities
AGEP
New opportunities for leveraging resources and supporting innovative science are sought in an effective manner.
CREST
Institutions take advantage of emerging opportunities to expand grant award portfolios such as receiving REU’s, instrumentation awards, and awards from other agencies such as the Department of Energy and the Department of Defense, federal labs and industrial alliances.
d. Appropriate attention to maintaining openness in the system
AGEP
Support is broadly spread over new and established investigators. A benefit of this balance is the creation of faculty mentoring relationships.
CREST
The institutions and the NSF provide enough freedom for new investigators to be supported as they are added to the program. This is true with respect to post-docs, senior researchers and new faculty.
e.
Evidence that proposals have addressed the integration of research
and education
AGEP
Proposals that are funded generally emphasize the integration of research and teaching activities in an academic context.
CREST
Outreach, mentoring and research are more integrated in CREST Centers than at many research intensive institutions.
f.
Evidence of increased numbers of applications from underrepresented
groups
Both of these programs are
directed specifically to enhancing the number and quality of minority faculty
researchers. In this context the
demographic characteristics of the formal principal investigator is
irrelevant. In broader terms, these
programs are contributing the growth of a future pool of scientists and
engineering researchers that may make application to NSF for awards.
AGEP
Steady progress in increasing the number of minority scientists appears to be occurring.
CREST
We declined to draw any conclusions from the data that we examined.
g.
Balance of projects characterized as
1) High Risk
2)
Multidisciplinary
3) Innovative
AGEP
There is a tendency to emphasize incremental advances in research and to adopt well-recognized pedagogical strategies. This trend is balanced by the presence of some multidisciplinary activities.
CREST
The majority of the CREST institutions have multidisciplinary projects and few high risks. There is considerable innovation in the manner the projects are organized and in some of the projects themselves. There is some concern that some CREST Centers have not been fully merged into the mainstream and therefore the scientific community may lose the value of their innovations.
B.
RESULTS: OUTPUTS AND
OUTCOMES OF NSF INVESTMENTS
5. Strategic Outcome: PEOPLE - A diverse,
internationally competitive and globally engaged workforce of scientists,
engineers, and well-prepared citizens.
The reviewers
considered a program to be successful when, in the aggregate, results reported in
the review period demonstrated significant progress in achieving one or more of
the following indicators:
a.
Improved
mathematics, science and technology understanding and skills for U.S. students
at the K-12 level;
b.
Improved
mathematics, science and technology understanding and skills for citizens of all ages, so that they can
be competitive in a technological society;
c.
A science and
technology and instructional workforce that draws on the strengths of America's
diversity;
d.
A science and
technology and instructional workforce that has global career perspectives and
opportunities;
e.
Globally
engaged science and engineering professionals who are the best in the world;
and
f.
A public that
is provided access to the processes and benefits of science and
engineering.
The COV considered the programs under review to be successful in meeting
this strategic outcome.
UNDERGRADUATE
PROGRAMS
a.
Improved
mathematics, science and technology understanding and skills for U.S. students
at the K-12 level
Improved mathematics and
science knowledge for students at the K-12 level is not directly among the goals
of the HBCU-UP or LSAMP programs. However, several of the LSAMP alliances,
including Chicago State [HRD 0000341] and Howard [HRD 0000373] have teacher
preparation programs as part of their project.
b.
Improved
mathematics, science and technology understanding and skills for citizens of all
ages, so that they can be competitive in a technological
society
Providing increased numbers
of students enrolled in and completing college level courses in all areas of
SMET assists greatly in meeting the goal of a better-informed citizenry. In
particular, the LSAMP program has been successful in enrolling increasing
numbers of undergraduates each year in SMET courses. Across the 27 alliances,
over 187,000 students were enrolled in SMET courses in 1999. This can only have
a positive effect on the level of general knowledge among all
citizens.
c.
A science and technology and
instructional workforce that draws on the strengths of America's
diversity
The vast majority of the
faculty who are teaching in institutions funded by HBCU-UP are African-American,
and a significant majority of the students in these institutions are also
African-American. Thus, this program directly leads to strengthening the
diversity of the scientific workforce.
LSAMP also assists greatly
in increasing the diversity of the scientific workforce. In 1999, there were
20,567 BS degrees awarded in SMET fields through the LSAMP alliances. For
instance, the Puerto Rico LSAMP [HRD 9623943] awarded 2,726 BS degrees in SMET
fields over the years 1997-99, a 60% increase over their baseline year. The
Florida-Georgia LSAMP [HRD 9703197] has more than tripled the number of BS
graduates at participating institutions. Those institutions in Phase I have
submitted a plan to significantly increase their production of BS degrees in 5
years. Those in Phase II have committed not only to increasing aggregate
numbers, but also to increasing individual retention and progress towards the
BS. Comparison of data from the
LSAMP database and data from the National Center for Educational Statistics
indicates that in 1999, underrepresented minorities who were baccalaureate
recipients in science and engineering fields at LSAMP institutions constituted
31.64 percent of the national production in these fields. The percentage is even higher when
consideration is restricted to engineering, the physical sciences, and the
biological sciences.
Following the progress of
both cohorts has been greatly improved by use of a new data collection and
reporting system, the LSAMP-Mars database, which can be accessed and updated by
project directors. Enrollment and degrees obtained can be tracked for each
individual student, as well as additional information on stipends, other awards,
and graduate enrollment. Development of this database will be an enormous
contribution not only to the success of LSAMP, but also to its linkage with
other NSF-funded graduate programs.
d.
A science and technology and
instructional workforce that has global career perspectives and
opportunities
While LSAMP funding does not
provide international activity, virtually all grantees find creative ways to
prepare their students for work and study in contexts beyond their local
campuses. Student participants typically obtain rich experiences interacting
with students and faculty members broader than their own environments and local
institutions.
For example, participants have opportunities on most campuses to interact with students and faculty from other countries and cultures. By nature of the alliance model, they also have opportunities to interact with individuals on different kinds of campuses, which may very well have significantly different missions and demographics.
Still other students have
opportunities to examine research topics that have global implications, e.g.,
global warming, HIV-AIDS, etc. In
some LSAMPS, for instance Illinois LSAMP (HRD), students get involved in
recruiting and introducing minority transfer students from community colleges
and other undergraduate institutions to their campuses. This AMP works directly
with the Latin American Recruitment and Education Services organization to
implement this activity.
e.
Globally engaged science and engineering
professionals who are the best in the world
The LSAMPS routinely have
regional and national meetings, conferences, and seminars for both faculty and
students from different types of institutions (large and small, public and
private, community college and -year).
These activities encourage students to learn about other cultures, races,
and ethnic groups, preparing them for global opportunities. For instance, students in the Oklahoma
AMP [HRD 9900796] participated and made presentations in the following: The
American Water Resources Association, Brookhaven Semester Program, Oklahoma
Academy of Sciences, Honor Society, National Institute of Science, and many
others.
In addition, both the LSAMP
and the HBCU-UP institutions have established relationships with national
laboratories, corporations and businesses, allowing both faculty and students
opportunities to learn about global career perspectives and opportunities in
these environments. Finally, the institutions in both programs enroll foreign
students, further enhancing the opportunities for both faculty and students to
broaden their horizons and learn about other cultures. [Albany State, HRD
9909028]
f.
A public that
is provided access to the processes and benefits of science and
engineering
The LSAMP and HBCU-UP
programs do not have as part of their primary missions providing the public with
access to the processes and benefits of science and
engineering. However, with the increasing numbers of undergraduates enrolled in
SMET courses, and with the increasing capacity of HBCU institutions, this public
benefit should occur. Public information on the processes and results of science
and engineering is provided through
the dissemination activities (publications, web pages, conference participation)
conducted by the individual HBCU-UP and LSAMP
grantees.
GRADUATE
PROGRAMS AND RESEARCH CENTERS
GENERAL
COMMENTS
There is documentation that
indicates that the Division and Directorate have enabled investigators to
collaborate with K-12 institutions. This is performed through reviews, program
director meetings, and continual discussions with
investigators.
The following comments were
made by the graduate programs based on the above
criteria:
CREST/AGEP
a.
Improved mathematics, science and
technology understanding and skills for citizens of all ages, so that they can
be competitive in a technological society
As stated above, there is
documentation that indicates that the Division and Directorate have enabled
investigators to collaborate with K-12 institutions. This is performed through
reviews, program director meetings, and continual discussions with
investigators.
The Division and the Directorate have performed excellently in growing the numbers of students exposed to NSF funded programs. Coordinated efforts exist within AGEP to access and/or improve the quality of graduate support systems, expand graduate education curriculum options, gain access to a variety of academic partners and improve the preparation of graduates for broad career options.
b.
A science and technology and
instructional workforce that draws on the strengths of America’s
diversity
The programs under review target increased production, participation, and quality of science and technology professionals from underrepresented minority groups. Their successful operation will directly contribute to the increased diversity of the science and technology and instructional workforce.. For example, the University of California AGEP [HRD 9978897, HRD 9978892, HRD 9978896] has committed to tripling the number of minority doctoral recipients within five years.
c.
A science and technology and
instructional workforce that has global career perspectives and
opportunities
Each program provides an opportunity for participating institutions to leverage their resources with a community of like-minded institutions sharing a commitment to enhance recruitment, retention, advancement and career success of students.
The University of
Alabama at Birmingham AGEP project has formed a statewide consortium with Auburn
University, the University of Alabama,
the University of Alabama at Huntsville, and Alabama A&M University
to collaboratively acquaint minority students within the Alabama / Mississippi
geographic region with science, mathematics, and engineering doctoral programs
and fellowship opportunities at consortium institutions. [HRD
0043739]
d. Globally engaged science and engineering professionals who are the best in the world
The transition of the AGEP to eligibility for graduate alliances has provided a workable mechanism through which graduate students from diverse backgrounds can gain access to globally engaged science and engineering professionals with outstanding reputations.
e.
A public that is provided access to the processes and benefits of science
and engineering
Public
information on the processes and results of science and engineering is provided through the dissemination
activities (publications, web pages, conference participation) conducted by the
individual AGEP and CREST grantees.
6. Strategic Outcome: IDEAS: Discovery across the frontier
of science and
engineering, connected
to learning, innovation and service to society.
The COV considered a program to be
successful when, in the
aggregate, results reported in the review period demonstrated significant
progress in achieving one or more of the following indicators:
a.
A robust and
growing fundamental knowledge base that enhances progress in all science and
engineering areas including the science of learning;
b.
Discoveries
that advance the frontiers of science, engineering, and
technology;
c.
Partnerships
connecting discovery to innovation, learning, and societal advancement;
and
d.
Research and
education processes that are synergistic.
The COV considered the
programs under review to be successful in meeting this strategic outcome.
UNDERGRADUATE
PROGRAMS
a.
A robust and
growing fundamental knowledge base that enhances progress in all science and
engineering areas including the science of learning;
The primary missions of LSAMP and HBCU-UP do not encompass growth of the fundamental knowledge base per se. However, the alliance model that pervades LSAMP does include the benefits of increased knowledge that are expected from Research I institutions. LSAMP and HBCU-UP students engage in discovery experiences on campus, at National Laboratories, and during industrial internships.
b. Discoveries that advance the frontiers of
science, engineering, and technology
See a.
above
Again, the alliance model of LSAMP provides partnerships
among Research I (discovery institutions) with the advances in learning and
societal benefits that are found among all the other components of an alliance.
c. Partnerships connecting discovery to innovation, learning,
and
societal
advancement
Many of the smaller schools
that are part of an LSAMP alliance are proactive in linking research and
education. For example,
Morehouse College has established a Behavioral Neuroscience Undergraduate
Fellows Program and a neuroscience minor that will be expanded into a
baccalaureate/master’s dual degree program in neuroscience. The BS/MS program is an
inter-institutional enterprise involving Emory University, Georgia Institute of
Technology, Georgia State University, Morehouse School of Medicine, and the
colleges of the Atlanta University Center.
The minor in neuroscience and the pipeline it offers to the programs of
the Center for Neuroscience are unique among Historically Black Colleges and
Universities. [HRD 9815529].
Curriculum reform is a major
component of undergraduate programs in LSAMP schools. This process requires
understanding of the current research on teaching and learning and its
application to all SMET classes.
Some of these applications include the following: bridge programs,
supplemental instruction in SMET courses, cooperative learning, undergraduate
research experiences, and enhanced use of educational technology.
GRADUATE PROGRAMS AND
RESEARCH CENTERS
a.
A robust and growing
fundamental knowledge base that enhances progress in all science and engineering
areas including the science of learning;
AGEP
Substantial progress is
being made by funded investigators and students across many functions of
scientific inquiry. Young
investigators are learning the customs and habits of dedicated
professionals.
The AGEP project at
the University of California, Berkeley (Berkeley Edge Program) has developed a
career management series that provides graduate students with role models and
multiple visions of how they might use their education and degrees beyond those
they encounter on a daily basis. Particular attention is devoted to the
introduction of career management skills that will enable students to compete
more effectively for the best postdoctoral positions and junior faculty
positions when they graduate. [HRD 9978896]
CREST
There are indicators of
robust activity in some, but not all, CREST Centers. For example, Dr. Oliver K. Baker, a
faculty member at Hampton University, has been one of the research group leaders
with CREST support [HRD-9154080, and HRD-9633750] since 1991. He is also the principle investigator on
a recent award (PHY 0072686) for a $1.8 million award from the Elementary
Particle Physics program. This
award represents a very substantial validation of the effectiveness of CREST
support in helping an institution like Hampton and minority faculty like Baker
achieve national research competitiveness.
Other indicators include but
are not limited to a variety of workshops sponsored by the Centers, Visitors
Programs, and Publication rates.
The science of learning as found in the Centers should be enhanced;
however, caution should be taken against forcing any one Center to cover such a
wide area of disciplines that it dilutes productivity.
b.
Discoveries that advance the
frontiers of science, engineering, and technology
AGEP
Each of the projects
addresses issues at the limits of current knowledge and engages young scientists
in this pursuit.
CREST
Peer recognition of
discovery by Center researchers should be enhanced. On the other hand, there are Centers in
which discoveries are well documented.
For example, CREST researchers have been responsible for such advances as
extra-solar planet discovery using new methods [HRD 9706268] and probes which
may aid in breast cancer and other tumor discovery and treatment [HRD
963750].
c.
Partnerships connecting
discovery to innovation, learning, and societal advancement
AGEP
Engagement in scientific
inquiry inculcates the values of enterprise, scholarship and service to society
through the examples of accomplished academicians.
CREST
Centers have good partnerships connecting discovery to innovation, learning and societal advancement. For example, working in collaboration with the NSF-funded engineering research center at Mississippi State University, the CREST project at Tennessee State is engaged in the development, application, and evaluation of one-dimensional icing models with important implications for the safe operation of commercial aircraft [EEC 9730365]. The Tennessee State CREST center is also working with the engineering research center at the California Institute of Technology to develop improved visual telepresence systems which may lead to widespread use of robots in environments or situations harmful to humans [EEC 9730980].
More national recognition awards from peers will further accent this reality. For example, Dr. Alfred Z. Msezane, PI of the Clark Atlanta CREST project [HRD 9154077] received the 1999 Edward A. Bouchet Award of the American Physical Society “[f]or continued outstanding contributions to theoretical atomic physics and leadership in the creation and administration of a highly regarded research centers of excellence.” Dr. Msezane was also named a fellow of the American Physical Society the same year.
d.
Research and education
processes that are synergistic
AGEP
Young investigators are quickly integrated into the academic culture that combines teaching and research. The exercise of one function stimulates the other and the next generation of researchers participates fully.
The AGEP project at
the City University of New York (CUNY) [HRD 9978851] has established a community
of minority scholars in conjunction with the NYC AMP program [HRD0043973] to
provide a cadre of minority role models, peer colleagues, and supportive faculty
at every level for every minority participant. All participants are immediately
placed in research labs and assigned doctoral faculty
mentors.
CREST
The big strength of CREST is
the synergistic play between research and education. For example, the California State
University at Los Angeles CREST center has established a partnership with an NSF
sponsored ecological analysis and synthesis center at the University of
California at Santa Barbara (UCSB) which has resulted in the formation of a
workgroup on specially structured dynamics involving Stanford, UC Berkeley,
UCSB, and Cal State LA. The
collaboration will expose CREST students and faculty to the nation’s top experts
on modeling population dynamics in marine landscapes.
7.
Strategic Outcome: TOOLS -
Broadly accessible, state-of-the-art information-bases and shared research and
education tools
Performance
Outcome: The COV
considered programs to be successful when, in the aggregate, as a result of its
investments, results reported in the review period demonstrated significant
progress in achieving one or more of the following
indicators:
a.
Shared use
platforms, facilities, instruments, and databases that enable
discovery;
b.
Shared use
platforms, facilities, instruments, and databases that enhance the productivity
and effectiveness of the science and engineering
workforce;
c.
Networking and
connectivity that takes full advantage of the Internet and makes SMET
information available to all citizens;
d.
Information
and policy analyses that contribute to the effective use of science and
engineering resources.
The COV considered the programs under review to be successful in meeting
this strategic outcome. Success is
within the context of development of databases of virtual learning/research
communities.
UNDERGRADUATE
PROGRAMS
HBCU-UP
a. Shared use platforms, facilities,
instruments, and databases that
enable
discovery
Not relevant to HBCU-UP or LSAMP.
b. Shared use platforms, facilities,
instruments, and databases that enhance
the
productivity and effectiveness of the science and engineering
workforce
Not directly relevant, but
development of the LSAMP-Mars database will make linkages between students with
the bachelor’s degree and graduate programs that they can then enter more
seamlessly.
c. Networking and connectivity that
takes full advantage of the Internet
and makes SMET
information available to all citizens
Not directly relevant to
HBCU. It will take place as part of the alliance model of LSAMP, for example,
the LSAMP Virtual Institutes used networking to connect faculty and students
across LSAMP alliances in various focus areas [HRD 9743298, HRD 9743694, HRD
9743693, HRD 9743339, HRD 9743338, HRD 9701775].
d. Information
and policy analyses that contribute to the effective use of
science and
engineering resources
Not directly
relevant to HBCU-UP or LSAMP as primary objectives. However, a special project in the LSAMP
program [HRD 9729401] did examine the correlation of student demographic factors
with institutional characteristics in fostering LSAMP student success. This and similar education research
projects will better inform the design and implementation of projects to
increase participation in the science and engineering enterprise by
underrepresented minorities.
a.
Shared use platforms,
facilities, instruments, and databases that enable
discovery
AGEP
Within a relatively short
period of time, through the use of regional Alliances, the AGEP program has
achieved tremendous success by leveraging shared use platforms, facilities,
equipment and data bases that help to create a more nurturing graduate
environment to enhance recruitment, productivity, effectiveness and retention of
SMET graduates.
CREST
CREST researchers and
programs require shared use instrumentation, cross campus collaboration, and
inter-university collaboration to enhance the probability of the discovery or
creation of new knowledge. There was significant evidence to support this type
of activity among many CREST Centers and research intensive universities. The
collaboration in physics of the CREST Center at Hampton University with the
University of Michigan and with Duke University is an excellent example. [HRD
9154080, HRD 9633750]
b.
Shared use platforms,
facilities, instruments, and databases that enhance the productivity and
effectiveness of the science and engineering workforce
CREST/AGEP
Where possible, each program
attempts to train graduate students and research assistants using cutting-edge
instrumentation and training facilities. Some of this training and research
activity result in research monographs, volumes, papers and presentations. The
contribution of graduate students and research assistants to these activities
enhances the productivity of the workforce, particularly in the area of trained
SMET scientists. The CREST Center in combinatorial chemistry at Jackson State
University [HRD 9805465] is a good example.
c.
Networking and connectivity
that takes full advantage of the Internet and makes SMET information available
to all citizens
AGEP
Networking and Internet
connectivity are exploited fully within the AGEP community. However, efforts to engage the larger
graduate scientific community in AGEP/SMET information access are encouraged
The AGEP project at
the Georgia Institute of Technology (GIT) was awarded a supplement to develop a
cyber network (CyberNet). CyberNet activities will include linking all AGEP
projects with existing LSAMP institutions to provide a mechanism for PIs and
coordinators to exchange program information and a mechanism for tracking AGEP
enrollment and graduation across all institutions. CyberNet is envisioned to
evolve into a comprehensive cyber infrastructure for SEM minority education at
each stage of the educational continuum for the nation. Components will be
designed to enhance the recruitment, retention, and graduation activities of
pre-college through doctoral educators as well as promote greater community
awareness of SEM opportunities. [HRD9817632]
CREST
Networking and Internet
connectivity are critical to the success of the CREST centers. These are often
successful within CREST Centers and in connection with research intensive
universities. However, increased efforts to engage the larger research community
(industry, discipline focused professional organizations, foreign centers) with
CREST Centers are encouraged.
d.
Information and policy
analyses that contribute to the effective use of science and engineering
resources
AGEP
The creation of effective
lines of communication and participation between AGEP institutions, such as the
bridge program, based on individual institutional needs, contributed to an
effective use of science and engineering resources.
The American Association for
the Advancement of Science (AAAS) and the Georgia Institute of Technology AGEP
jointly sponsored Workshop 2000: A
National Dialogue to Move Minority Graduate Education Forward. The workshop
audience included NSF AGEP grantees as well as leaders from higher education,
corporations and foundations, and government. A highly interactive format
featuring keynotes, panels, and breakout groups allowed participants to interact
and engage in dialogues leading to action agendas throughout the educational
continuum. Workshop 2000 represented
a major step toward building a community of students, scholars, and leaders
linked by a solid commitment to increasing opportunities in SMET for
underrepresented minorities. [HRD 9817536]
CREST
CREST researchers appear to
be well informed and sharply discipline focused. Thus, research resources are
leveraged to achieve the greatest productivity. This is reflected in
conferences, presentations, papers and other contributions to the research
literature.
4.
Areas of Emphasis:
Investments
and available results that demonstrate the likelihood of strong performance in
the future based on the following areas of emphasis and their relevance to
program activities. NSF-supported examples that relate to or demonstrate the
relevant strategic outcomes are provided.
a.
Strategic Outcome: People
·
K-12 systemic
activities
·
Enhancing instructional
workforce/professional development
- Centers for Learning and
Teaching (CLT)
- Graduate Teaching Fellows in
K-12 Education
·
Broadening participation
- Tribal
Colleges
- Partnerships for
Innovation (PFI)
·
Addressing
near-term workforce needs
- Advanced Technological
Workforce program (ATE)
Comments: The
HBCU-UP and LSAMP programs address the broadening participation goal and
contribute toward addressing near-term workforce needs. Additionally, the HBCU-UP program,
through attention to faculty development, enhances the instructional
workforce.
b. Strategic Outcome:
Ideas
·
Appropriate
Balance of Portfolio (high risk, multidisciplinary, or innovative research) for
each NSF
program
·
Investment in
three initiatives:
- Information Technology
Research (ITR)
- Nanoscale Science and
Engineering
- Biocomplexity in the
Environment
·
Investments in
non-initiative fundamental research:
- Mathematical Sciences
Research
- Functional
Genomics
- Cognitive
neuroscience
Comments: The
HBCU-UP and LSAMP programs do not address these particular program
activities.
c. Strategic Outcome:
Tools
·
Investments
in Major Research
Equipment:
- Terascale Computing
System
·
Continuing
investments:
- Major Research
Instrumentation Program (MRI)
- Science and Engineering
Information/reports/databases
- New types of scientific
databases and tools for using them
Comments: The
LSAMP program addresses this outcome through development of the LSAMP Virtual
Institute , a companion platform to
the National SMETE Digital Library. [HRD 9743298, HRD 9743694,
HRD 9743693, HRD 9743339, HRD 9743338, HRD 9701775].
The HBCU-UP program does not address these particular program
activities.
GRADUATE
7.
Areas of Emphasis: Strategic
Outcomes relevant to program activities,
investments and available results that demonstrate or do not demonstrate
the likelihood of strong performance in the future. NSF-supported examples that
relate to or demonstrate the relevant strategic outcomes are
provided.
a.
Strategic Outcome: People
K-12 systemic activities
Enhancing instructional workforce/professional
development
· Centers for Learning and Teaching (CLT)
· Graduate Teaching Fellows in K-12 Education
Broadening participation
· Tribal Colleges
· Partnerships for Innovation (PFI)
Addressing near-term workforce needs
· Advanced Technological Workforce program (ATE)
GRADUATE PROGRAMS AND
RESEARCH CENTERS
CREST/AGEP
The resources of these
programs are employed in raising a new generation of academic scientists. This
resource allocation will have a substantial impact on the demographics of the
technological workforce and future faculty. In this way, these programs are dressing
three of the PEOPLE strategic outcomes: (1) enhancing the instructional
workforce, (2) broadening participation, and (3) addressing near-term workforce
needs. While it is too early to tell how career trajectories will evolve in the
future, it is likely that these programs will have a significant impact.
b.
Strategic Outcome:
Ideas
Appropriate Balance of Portfolio (high risk, multidisciplinary, or innovative research) for each NSF program
Investment in three initiatives:
· Information Technology Research (ITR)
· Nanoscale Science and Engineering
· Biocomplexity in the Environment
Investments in non-initiative fundamental research:
· Mathematical Sciences Research
· Functional Genomics
· Cognitive neuroscience
CREST/AGEP
The representation of
scientific specialties embodied in these programs is comprehensive. Since the researchers are engaged in
frontier research, it is likely that significant technological developments will
emerge from these activities.
a.
Strategic Outcome:
Tools
Investments in Major Research Equipment:
· Terascale Computing System
Continuing investments:
·
Major Research
Instrumentation Program (MRI)
·
Science and Engineering
Information/reports/databases
·
New types of scientific
databases and tools for using them
CREST/AGEP
Investment in a variety of
infrastructure resources has been made under the auspices of these
programs. In addition, data
pertaining to the reasons that students seek or avoid academic careers have been
gathered.
SUMMARY OF
RECOMMENDATIONS
9.
Program areas
that the COV believes need improvement.
UNDERGRADUATE
PROGRAMS
Comments: The COV
(undergraduate) was greatly impressed with the quality and quantity of materials
available for review. The staff performed in an outstanding manner in preparing
for the process and in providing information as needed throughout. Two comments
about the process: It would have been helpful to see the GPRA questions in
advance, as the COV members reviewed the materials at home. There was such an
abundance of information that it was not clear in advance what was to be most important. However, we compliment
the staff, both in the program and at NSF, generally, for providing most helpful
guidance and guidelines for the actual process. The templates helped to focus
our review of materials, our questions, and our discussions. We greatly
appreciate this assistance.
GRADUATE AND RESEARCH
CENTERS
There is a need to maximize
capacity in CREST Centers. Failure
to do so or failure to implement a structure to do so reflects negatively on
NSF. CREST researchers have
demonstrated nationally competitive research capabilities. It is strongly urged that NSF technical
staff in division-specific areas with input to CREST proposed activities be
members of all review teams and maintain that membership throughout award
durations. Additionally, it is
strongly urged that relevant divisions will overlap in CREST proposed activities
and find ways to partially fund such activities.
AGEP is currently
undertaking activities to enhance the transition of MGE to AGEP through the
modification of existing MGE cooperative agreements. These efforts should be
continued until the MGE program is fully transitioned to the AGEP
program.
10. Programs’ performance in meeting program-specific goals and objectives (non-GPRA outcomes).
UNDERGRADUATE
PROGRAMS
Comments: The HBCU-UP
program is in its second year; therefore, it is difficult to evaluate. However,
the LSAMP program has completed 10 years and is now entering Phase III. The 350 colleges and universities
participating in this program represent about 10% of all institutions of higher
education in the country. Schools represented include Research I universities,
HBCUs, Hispanic-serving institutions, Tribal colleges, and two-year, four-year,
liberal arts and comprehensive institutions. Thus the potential for a major
improvement in the situation of underrepresented students is enormous.
The COV is
greatly impressed with the extensive scope of the evaluation, assessment, and
review processes that LSAMP has undergone. It is probably one of the
best-documented programs at NSF (annual reports, brochures and publications,
conferences, PER, COV). It has been meeting its overall goals, although
individual institutions have had variable measures of success at various times.
In terms of the overall goal of increasing the number of underrepresented
students who receive bachelor’s degrees in SMET, it is unquestionably
successful.
The next step,
ensuring that these students go on to productive careers in SMET, both in
academe and elsewhere, is also critical to ensuring a workforce in science that
represents the population.
Enhancing the linkages between these programs and others funded by NSF
and elsewhere will be a most important step.
GRADUATE
PROGRAMS AND RESEARCH CENTERS
The program’s performance in meeting program-specific goals and objectives (non-GPRA outcomes).
Conclusive judgements are
premature on the number of research careers that have been stimulated by this
program. It is possible to discern
some preliminary trends, however.
Highly competent and dedicated researchers have turned their attention to
the problem of minority recruitment and retention for academic careers. Personal relationships based on shared
research interests have been established between investigators who otherwise
would not have collaborated. The
human capital of minority communities has been focused on an area of national
need, the training of the next generation of academics. Thus, program
performance has been excellent as measured by traditional and non-traditional
outcomes.
11. The COV
reviews process, format and core questions.
UNDERGRADUATE PROGRAMS
Comments: See above
(10)
GRADUATE
PROGRAMS AND RESEARCH CENTERS
CREST/AGEP
a.
The committee feels that
given the magnitude of the data collected for each of the programs, additional
time is required to provide a fair and comprehensive evaluation of the
program. It is recommended that the
COV panel attend the program directors’ meetings to gain additional insight into
the strategies utilized by the various programs in attaining their
goals.
The preparation and review
of proposals should provide greater weight to the specific research to be
performed so as to balance attention to administrative concerns. CREST proposals must be seen as research
proposals and evaluated by the same criteria applied to other NSF Centers. Maximization of the potential of the
target community as well as the scientific community as a whole requires the
evaluation of proposed work with a common professional
standard.
ADDITIONAL
RECOMMENDATIONS
Emerging
Opportunities
CREST
Institutions should take advantage of emerging opportunities to expand grant award portfolios such as receiving REU’s, instrumentation awards, and awards from other agencies such as the Department of Energy and the Department of Defense, federal labs and industrial alliances.
Networking and Internet
connectivity are critical to the success of the CREST centers. These are often
successful within CREST Centers and in connection with research intensive
universities. However, increased efforts to engage the larger research community
(industry, discipline focused professional organizations, foreign centers) with
CREST Centers are encouraged.
CREST
The overall CREST design incorporates a number of self-checking accountability measures and possesses critical strengths which have been pivotal to the reviews of the proposed activities. The COV recommends that a review mechanism that embeds professional contributions (e.g., review panel membership, site visit participation) from other NSF directorates be implemented.
LSAMP and HBCU-UP
In discussing whether reviewers adequately addressed the intellectual merit criteria in their reviews, for the most part, staff analyses reiterated the comments of reviewers on the merit and impact criteria for both LSAMP and HBCU-UP programs. The COV recommends that the template for staff analyses include specific reference to the two criteria, which it does not include currently.
Following the progress of
both cohorts has been greatly improved by use of a new data collection and
reporting system, the LSAMP-Mars database, which can be accessed and updated by
project directors. Enrollment and degrees obtained can be tracked for each
individual student, as well as additional information on stipends, other awards,
and graduate enrollment. Development of this database will be an enormous
contribution not only to the success of LSAMP, but also to its linkage with
other NSF-funded graduate programs.
CREST
There are indicators
of robust activity in some, but not all, CREST Centers. These indicators include but are not
limited to a variety of workshops sponsored by the Centers, Visitors Programs,
and Publication rates. The science
of learning as found in the Centers should be enhanced; however, caution should
be taken against forcing any one Center to cover such a wide area of disciplines
that it dilutes productivity.
CREST
Centers have good partnerships connecting discovery to innovation, learning and societal advancement. More national recognition awards from peers will further accent this reality.
LSAMP
Development of an LSAMP-Mars
database will make linkages between students with the bachelor’s degree and
graduate programs that they can then enter more seamlessly.
AGEP
Networking and Internet
connectivity are exploited fully within the AGEP community. However, efforts to engage the larger
graduate scientific community in AGEP/SMET information access are encouraged.
Signatures of
COV Members:
Dr. Fitzgerald Bramwell (Graduate and
Research Institutes Sub-Group-Chair)
Dr. Warren Buck
Dr. Alfredo G. de los Santos,
Jr.
Dr. Robert
Harvey
Dr. Jeanette Jones
Dr. Linda Mantel (Undergraduate Programs
Sub-Group-Chair)
Dr. Joseph Ortiz
Dr. Charles Pickett
Dr. Orlando Taylor (COV
Chair)
RESPONSE
TO COMMITTEE OF VISITOR'S REPORT
Louis Stokes Alliance for
Minority Participation (LSAMP)
Historically Black College
and University - Undergraduate Program (HBCU-UP)
Alliances for Graduate
Education and the Professoriate (AGEP)
Centers for Research
Excellence in Science and Technology (CREST)
Division of Human Resource
Development (HRD)
Directorate for Education
and Human Resources (EHR)
The
program staff of the Louis Stokes Alliance for Minority Participation (LSAMP)
program, the Historically Black College and University - Undergraduate program
(HBCU-UP), the Alliances for Graduate Education and the Professoriate (AGEP)
program, and the Centers for Research Excellence in Science and Technology
(CREST) program sincerely thank the members of the Committee of Visitors (COV)
for their hard work in the conduct of a thorough evaluation, and the production
of a positive report. The Division of Human Resource Development acknowledges
and appreciates the outstanding job done by the COV during the February 1-2,
2001 meeting. The following
paragraphs provide the specific responses and comments in addressing the
recommendations of the report.
The
program staffs from each of the reviewed programs agree with the Committee's
recommendations, and will commence immediately pursuing solutions. In seeking possible solutions to the
findings and implementation of the strategies, the programs will seek the
commitment of support and resources from NSF/EHR management for those elements
beyond the purview of the program and division level. This commitment from senior management
at the NSF will encourage participation and institutionalization of the policies
necessary to influence the diversity objectives of both the programs and
recommendations from the COV.
THE PROGRAM STAFFS HAVE
ALREADY BEGUN TO ADDRESS THE FOLLOWING RECOMMENDATIONS OF THE
COV:
EVALUATION RECOMMENDATIONS
·
In order to provide a fair
and comprehensive evaluation of the CREST/AGEP programs, the COV panel should
attend meetings of the program directors.
RESPONSE: Committee of Visitors (COV) Panels are
not standing committees and COV members do not have official NSF appointments,
therefore the participating members are only authorized to conduct their
comprehensive program evaluations on scheduled COV meeting dates.
Dr.
Orlando Taylor, the COV chair, attended the March 26 – 27, 2001 joint PI/PD
meeting for HRD’s minority programs. That meeting afforded Dr. Taylor ample
opportunity to interact with Principal Investigators from all of HRD’s diversity
continuum programs. This practice will continue in cases were the opportunity
becomes available.
·
A review mechanism that
embeds professional contributions (e.g., review panel membership, site visit
participation, etc.) from other NSF directorates should be implemented for CREST
programs.
RESPONSE: Historically, the CREST program staff
have used the expertise of various disciplinary program directors throughout the
NSF as participants on Proposal Review Panels, Project Site Visits, Reverse Site
Visits, providing ad hoc reviewers recommendations for Center proposals and
component proposals.
Program staffs have also
used NSF disciplinary program directors for outreach, and participation in
Project Directors meetings and Performance Effectiveness Reviews. The CREST program staff have continued
to nurture these collaborative efforts and have made them a systemic part of the
CREST review process.
PROGRAM
RECOMMENDATIONS
·
The capacity of CREST/AGEP
Centers should be maximized.
RESPONSE: The revised CREST solicitation, although
still under review, will provide for both co-funding and award supplements both
of which will be designed to attract more funding and more researchers to CREST
projects. The CREST program will
continue to encourage participation of NSF program staff from relevant
disciplines in the CREST proposal review process by broadening their
participation on panels and site visit teams and also continuing to consult with
them during the annual review of projects.
Following the March, 2001 -
Joint HRD PI Meeting, AGEP PIs agreed to:
a) expand the number of departments involved on their campuses, b)
recruit additional faculty to participate in AGEP activities (especially
mentoring), and c) explore linkages with IGERT projects on their campuses.
Additionally, AGEP project directors are aggressively pursuing ways to increase
joint activities between alliance projects.
·
Activities to complete the
transition of the MGE projects to AGEP projects should be enhanced until all MGE
projects are fully transitioned.
RESPONSE: All MGE projects are: a) involved in
formal and informal alliances, or b) conducting primary project activities that
include active participation of a variety of other colleges and
universities.
·
Linkages between LSAMP
projects in Research I Universities, HBCUs, Hispanic-Serving Institutions,
Tribal Colleges, and two-year, four-year, liberal arts and comprehensive
institutions should be improved so that students can be better assured
productive careers in SMET, both in academia and elsewhere.
RESPONSE: The
LSAMP Project Community spans greater than 350 diverse academic institutions,
including Research I and II, HBCUs, HSI's, Tribal Colleges, State, Private
Colleges and Universities, and Women Colleges, are connected via both formal and
informal partnerships through the LSAMP alliance structure. Cross linkages already exist within
LSAMP and also extend to the AGEP, CREST, and HBCU-UP institutions. In addition, the subject of cross
linkages of the aforementioned programs was discussed at length at the March
2001 Joint PI/PD Meeting.
A
strong example of the HRD MSI program "Linkage Model" with substantive statewide
collaborations is in Alabama, where cross linkages occur among the HBCU-UP,
LSAMP, CREST and AGEP projects.
This collaborative effort among the MSI program staff is a continued
effort that is being nurtured to the level of institutionalization of this "Linkage Model" among existing
co-located HRD MSI projects. The
example in the state of Alabama, shows how student participants of the Miles
College HBCU-UP (HRD 9909038) benefit from technology-enhanced SMET curricula,
faculty mentoring and financial support.
However, faculty supervised research experiences are acquired through
LSAMP-sponsored summer internships at the University of Alabama, Birmingham (HRD
9940643) and the CREST project at the Alabama A&M University (HRD
9353548). The Alabama Alliance for
Graduate Education and the Professoriate program (HRD 9817296) provides graduate
school awareness, preparation and recruitment activities targeting Miles College
undergraduates. Once admitted to
University of Alabama graduate programs, students benefit from AGEP-sponsored
faculty mentoring, and advising, and are eligible for financial support to
insure graduate degree completion within a reasonable
timeframe.
·
A database should be
developed for LSAMP-MARS programs to access, update and track information on
individual students regarding enrollment, degrees obtained, stipends and
awards.
RESPONSE: The
Georgia Institute of Technology AGEP is taking the lead in developing a database
on undergraduate student database, initially populated with LSAMP student
information, to be used as a recruitment resource for graduate programs. This
effort has drawn significant interest from NSF’s Office of Integrative Affairs,
especially as it relates to enhancing the recruitment of underrepresented
minorities at Science and Technology Centers (STCs).
·
CREST and AGEP/SMET programs
should work toward greater access to the larger graduate scientific communities
and professional organizations through Internet
connectivity.
RESPONSE: CREST Centers have extensive Web Sites
in addition to producing paper copy publicity. These sites already contain
attractive descriptions of Center activities, personnel and research outcomes.
Centers will be encouraged to have links from popular Web Sites at their partner
research organizations (e.g., universities, research laboratories and industry)
in addition to professional societies. NSF will maintain links to CREST Center
Web Sites from the CREST program homepage. CREST Centers will also be encouraged
to develop links from popular sites that advertise opportunities for graduate
study in relevant disciplines and from sites that are accessed by pre-college
and undergraduate students and their instructors for information about
science.
The
AGEPs are doing the following activities to address this
recommendation:
·
Provided
supplemental support to the Georgia Institute of Technology AGEP to develop a
Cyber Network to link all AGEP projects via the Internet. This site, once fully
developed will serve as a link between AGEP and the larger graduate
research/education community.
·
Implementation
of an active dialogue with the Woodrow Wilson Fellowship Foundation (WWFF) to
explore ways in which the WWFF can enhance the effectiveness of AGEP in
accomplishing its objectives. Particular interest has been expressed in
expanding the AGEP approach to include a larger number of graduate institutions,
perhaps through the WWFF “Towards a More Responsive Ph.D.”
program.
·
The
initiation of discussion between the Council of Graduate Schools (CGS), the
Division of Undergraduate Education (DUE), and AGEP program staff to explore the
expansion of Preparing the Future Professoriate (PFF) to involve AGEP projects.
This effort is particularly noteworthy because of the extensive involvement of
professional associations in the administration of the
PFF.
·
The
development of a Special Project concept with the Southern Regional Education
Board (SREB), a participant in the Compact for Faculty Diversity, to increase
the number of AGEP students participating in its annual Institute for Teaching
and Mentoring.
EVERY EFFORT WILL BE MADE TO
INCLUDE THE FOLLOWING COV RECOMMENDATIONS INTO THE FY 2002 PROGRAM AGENDAS AS
APPROPRIATE:
EVALUATION
RECOMMENDATIONS
·
A longer preparatory period
after the release of the program solicitation and before the review process is
advised.
RESPONSE: HRD Programs would like to resume fixed
application deadlines that spread through the year to balance proposal
processing workloads that would be announced several years in advance. In the
case with the CREST program compression of the preparation period has resulted
because of the extensive time required after receipt of the application for
technical review of the many subprojects proposals typical and unique to a
proposed CREST Center.
·
The composition of the
external review panels for AGEP program could be enhanced by increased
representation from disciplinary representatives within NSF.
RESPONSE: AGEP review panels are
currently small (5 – 8 members). All panelists being used have extensive NSF
disciplinary panel experience, and some are former NSF program officers. We will
continue to exercise the current high standards in our reviewer selection, and
will also continue to solicit recommendations for reviewers from NSF research
directorate colleagues.
PROGRAM
RECOMMENDATIONS
·
A common set of professional
standards should be established to maximize the potential of engagement of
targeted communities and the scientific community as a whole. CREST proposal submission and evaluation
should place greater emphasis on the research to be conducted and should be
judged in a manner consistent with that used for other NSF-funded
centers.
RESPONSE: The revised CREST solicitation currently
under review has been modeled after the NSF STC program solicitation guidelines.
CREST program staff have made tremendous effort in adopting these guidelines for
CREST Center projects. The proposed
new guidelines will allow each subproject proposal to be formatted in the
standard NSF research proposal format.
·
CREST Centers should serve
as nationally recognized exemplars in their areas of
research.
RESPONSE: The revised CREST solicitation has
discontinued the requirement for CREST Centers to be regional resources with the
following new requirement which elevates the Centers status to that of being
nationally recognized resources for research in the Center’s area of
specialization.