Enhancing Peer Review at NIH - Scoring and Review Changes

Enhancing Peer Review Home > Scoring and Review Changes

Scoring and Review Changes

New Scoring System

Beginning with the summer 2009 review cycle, assigned reviewers have been instructed to score each of five review criteria, and the overall impact/priority of each application, on a 9-point rating scale according to the following descriptions and additional guidance:

Impact	Score	Descriptor	Additional Guidance on Strengths/Weaknesses
High	1	Exceptional	Exceptionally strong with essentially no weaknesses
	2	Outstanding	Extremely strong with negligible weaknesses
	3	Excellent	Very strong with only some minor weaknesses
Medium	4	Very Good	Strong but with numerous minor weaknesses
	5	Good	Strong but with at least one moderate weakness
	6	Satisfactory	Some strengths but also some moderate weaknesses
Low	7	Fair	Some strengths but with at least one major weakness
	8	Marginal	A few strengths and a few major weaknesses
	9	Poor	Very few strengths and numerous major weaknesses
Non-numeric score options: NR = Not Recommended for Further Consideration, DF = Deferred, AB = Abstention, CF = Conflict, NP = Not Present, ND = Not Discussed
Minor Weakness: An easily addressable weakness that does not substantially lessen impact Moderate Weakness: A weakness that lessens impact Major Weakness: A weakness that severely limits impact

Applications that are not discussed at the meeting will not be given an overall impact/priority score, but the applicant, as well as NIH staff, will see the preliminary scores for each of the review criteria as additional feedback on their summary statement. The Scoring System and Procedure document was provided as guidance to all reviewers.

Rationale for the New NIH Grant Application Scoring System

The prior scoring system of 1.0 to 5.0 in 0.1 increments served NIH well for many years, but its weaknesses became increasingly evident as the quality and quantity of applications increased and NIH budgets to fund grant applications tightened. The new scoring system is being implemented to address the following issues:

For even the most experienced reviewers, it is difficult to make 41 reliable discriminations of application merit. Based on measurement science, prior experience, and feedback from various constituencies, a 9-point rating scale with descriptors associated with each rating option was adopted.
Reviewer ratings became increasingly positive, compressing the score range, and effectively reducing the usefulness of scores for NIH funding decisions. In the new scoring system, the descriptors associated with each rating were designed to encourage use of the full scoring range.
To provide additional feedback to applicants, program staff, and other consumers of the summary statement, assigned reviewers also provide rating of the specific review criteria using the same 9-point scale.

This page was last reviewed on October 22, 2009