December 5, 1996
(Last updated December 23, 1996)
Earlier this month, the National Science Foundation (NSF) held a press conference to announce the findings of its Task Force on Merit Review. The task force is seeking comments from the scientific community on its proposed changes to the merit review process. Since NSF is a key funding agency for the membership of many AGI member societies, this update is intended to facilitate feedback from individual societies by providing background information and NSF contact points for comments. If you have any questions, please contact AGI's Government Affairs Program at firstname.lastname@example.org or contact NSF directly at email@example.com.
Each year, NSF uses merit review to award funding for approximately 10,000 of the 30,000 new proposals received, in the process generating 170,000 reviews. These reviews are based on criteria established by the National Science Board (NSB) in 1981. The criteria are (1) research performer competence, (2) intrinsic merit of the research, (3) utility or relevance of the research, and (4) effect on the infrastructure of science and engineering. Since the NSF has changed dramatically in the fifteen years since the criteria were created, a task force was established to determine if the criteria were still relevant to NSF's goals and purposes.
Task Force Findings
The task force found several problems with the current criteria. In their view, the criteria lack clarity and uniformity in their application. In particular, the criteria on research utility and effect on infrastructure are not well understood and often ignored. Additionally, the current criteria do not reflect the NSF strategic plan, which places great value on the integration of research and education, as well as non-research activities, such as human resources, facilities, and centers. Therefore, the task force proposed changes to the merit review criteria.
Proposed Criteria Changes
Two questions were proposed to replace the current criteria: What is the intellectual merit and quality of the proposed activity? and What are the broader impacts of the proposed activity? The attached sheet from NSF outlines a number of additional questions to be considered when assessing how well a given proposal meets the criteria. The task force recommends that reviewers be instructed only to address the criteria they feel qualified to evaluate and are relevant to the proposal at hand. Furthermore, other considerations may be used for evaluating each project as deemed necessary, and there is no predetermined weighting of criteria #1 and #2.
The task force is seeking comments on the proposed changes before they make their recommendation to the NSB in the Spring of 1997. Comments are due by January 15, 1997. While all comments and suggestions are welcome, they are particularly interested in your views on questions such as: Are the proposed criteria clear? Would they be easier to use than the current criteria? Would the proposed criteria elicit useful input and comments from reviewers? Would they improve NSF's ability to foster linkages across disciplines? Would they contribute to the integration of research and education? Are there further improvements to the criteria that you would recommend? The recommendations and a feedback form can be found on NSF's World Wide Web home page. Please direct e-mail correspondence to: firstname.lastname@example.org.
Current Criteria (adopted in 1981)
1.Research performer competence -- relates to the capability of the investigators, the technical soundness of the proposed approach, and the adequacy of the institutional resources available.
2.Intrinsic merit of the research -- the likelihood that the research will lead to new discoveries or fundamental advances within its field of science or engineering, or have substantial impact or have substantial impact on progress in that field or in other science and engineering fields.
3.Utility or relevance of the research -- the likelihood that the research can contribute to the achievement of a goal that is extrinsic or in addition to that of the research itself, and thereby serves as the basis for new or improved technology or assist in the solution of societal problems.
4.Effect on the infrastructure of science and engineering -- the potential of the proposed research to contribute to better understanding or improvement of the quality, distribution, or effectiveness of the nation's scientific and engineering research, education, and manpower base.
# 1 What is the intellectual merit and quality of the proposed activity?
The following are suggested questions to consider in assessing how well the proposal meets the criterion: What is the likelihood that the project will significantly advance the knowledge base within and/or across different fields? Does the proposed activity suggest and explore new lines of inquiry? To what degree does the proposer's documented expertise and record of achievement increase the probability of success? Is the project conceptually well designed? Is the plan for organizing and managing the project credible and well conceived? And, is there sufficient access to resources?
#2 What are the broader impacts of the proposed activity?
The following are suggested questions to consider in assessing how well the proposal meets the criterion: How well does the activity advance discovery and understanding while concurrently promoting teaching, training, and learning? Will it create/enhance facilities, instrumentation, information bases, networks, partnerships, and/or other infrastructure? How well does the activity broaden the diversity of participants? Does the activity enhance scientific and technological literacy? And, what is the potential impact on meeting societal needs?
Contributed by Kasey Shewey, AGI Government Affairs
The text of this update was sent to the presidents and executive directors of AGI's member societies as a memorandum on December 5th. Copies were sent to AGI's Executive Committee, Member Society Council representatives, and Government Affairs Program Advisory Committee members and liaisons.
Last updated December 23, 1996