Peer Review College

Evaluation of the Peer Review College

The Evaluation addressed four objectives:

Consider the extent to which the College is fulfilling its original objectives.
Assess whether the College and associated review process are delivering the quality of peer review necessary to identify the best science.
Determine whether the College has the confidence of the environmental sciences community.
Recommend any changes necessary to improve the performance of the College and extend its use.

After wide consultation and data analysis, the Project Board agreed on five overarching recommendations:

Recommendation 1: The quality and balance of College membership should be improved.
Recommendation 2: NERC should implement changes to optimise the effectiveness of moderating panel meetings.
Recommendation 3: NERC needs to take steps to increase the level of confidence in its peer review process.
Recommendation 4: Dialogue between NERC and College members, and between NERC and the research community, should be improved.
Recommendation 5: Some changes should be made to the contractual agreement with College members.

Details of these recommendations and the reasoning behind them are included in the Evaluation Final Report:

The recommendations were endorsed by NERC's Science & Innovation Strategy Board (SISB) in October and NERC Council in November 2008.

The consultation with current and former College members and the wider community (via an open web-based questionnaire) were key elements of this Evaluation. NERC would like to thank everyone who responded to these consultations for their contribution to developing the recommendations.

Implementing the recommendations

Implementation is in progress. So far the following actions have been taken:

Recommendation 1

We have changed the Peer Review College recruitment process. Instead of relying on self-nomination, we are also inviting nominations of others. Those submitting a self-nomination will need the support of their Head of Department or equivalent. More detail is provided in the current call for nominations.
We have introduced guidelines for membership.
Members in the third year of their term are now given the opportunity to state whether or not they would like the opportunity to continue for a further year if they are needed. Those leaving the College are encouraged to reapply in future if they wish.

Recommendation 2

A Pool of College Chairs has been introduced.
We are piloting a change to the moderating panel meeting process, so each proposal will be read by at least two introducers and two extra 'readers'. College members are still encouraged to read as many of the proposals being considered by the panel as possible.
Panel members are now asked to act as moderators, rather than referees. Although they should be able to give their own views, they should not usually raise substantive new issues to which applicants have not had the opportunity to respond and should not overrule judgements of experts in a field.
Panel policy papers have been updated and we are reminding Panel members of the importance of reading them before the meeting.

Recommendation 3

NERC staff are spending more time checking whether or not reviews are of sufficient quality. Those which are too brief or contain inappropriate comments are now being returned to reviewers via the Je-S system. We are monitoring those who provide inadequate reviews, or return reviews late, more closely and taking action when necessary.

Recommendation 4

The College website is being updated.
An exit questionnaire is now sent to those leaving the College.
More individual feedback if being provided to members.

Recommendation 5

The maximum number of meetings that full College members are asked to attend has been changed to a usual maximum of three a year, with members only exceptionally being asked to attend four meetings. For 'half' members, the maximum is now two. However, a two-day meeting will count as one meeting.
The College will now run from 1 July to 30 June, instead of 1 June to 31 May, so members do not 'roll off' the College just before moderating panel meetings that are held in June.

Information about the Peer Review College.

Information about responsive mode funding.

For further information about the Evaluation, please contact Anne McFarlane.