Doctoral training partnerships mid-term evaluation 2016-17

Evaluation report published July 2017

Following completion of doctoral training partnership (DTP) evaluation data gathering activities in early 2017, NERC commissioned the Careers Research & Advisory Service (CRAC) to produce an independent summary report highlighting the main findings of the evaluation and lessons learned from the first round of NERC DTPs (DTP1). The full independent report is available below.

NERC DTP mid-term evaluation independent report (PDF, 1.1MB)

NERC is pleased to receive the recommendations made in the independent DTP evaluation report. These recommendations will be used to inform decision-making regarding commissioning of DTP2 based on lessons learned from DTP1. NERC reserves the right to implement recommendations as appropriate and as advised by NERC Council.

NERC's management response to the recommendations within the independent report is available below.

NERC DTP mid-term evaluation management response (PDF, 277KB)

Background to NERC DTP commissioning

In 2013, the DTP1 competition awarded funding to 15 DTPs for a total of 240 notional studentships per annum for five cohort intakes starting in October 2014. The final intake for the current DTP1 awards will therefore be in October 2018. Full details of the DTP1 allocations can be found on the main DTP page.

Mid-term evaluation and future commissioning of DTPs

At the time of award, all DTP1 grantholders were informed of NERC's intention to conduct a mid-term evaluation of its DTPs in 2016-17. This evaluation will supplement DTP annual reports and annual visits as part of NERC's ongoing performance monitoring of DTPs.

It is also NERC's intention that commissioning of future DTPs should be informed by lessons learned from DTP1, and the mid-term evaluation will therefore also help inform (but not determine) the commissioning process for DTP2 in 2018.

The framework for this evaluation (developed in consultation with NERC's Training Advisory Board (TAB) and NERC DTPs), and the proposed timetable, are outlined below.

Evaluation framework and timetable

The evaluation had two main aims - firstly to monitor individual DTPs' ongoing performance against their original proposals and NERC's priority success criteria; and secondly to inform (but not determine) the commissioning process for future DTP awards. The evaluation also provides evidence of the wider impact of the DTP scheme.

Evidence gathering

The evaluation collected evidence through two routes:

  1. Targeted surveys to all members of the NERC community (not just the DTPs).
  2. Follow-up interviews during on-site visits to the DTPs by members of NERC staff and TAB.

All NERC DTPs were consulted on the types of evidence to be gathered in the evaluation.

Evaluation outputs

  1. Individual progress reports for each DTP, which provide a focus for dialogue between NERC and each individual DTP as part of ongoing monitoring and act as a useful indicator of changes in individual DTP performance over time.
  2. A summary report highlighting common themes and issues, lessons learned from DTP1, and an overall perception of the success of the DTP scheme.
  3. Learning points to inform the commissioning process for DTP2.
  4. Examples of good practice within the DTP scheme.

Timetable

Surveys were circulated in April 2016, with responses submitted in June 2016. Survey responses were used to inform the follow-up interviews with each DTP and also fed into the summary report. Interviews were conducted as part of NERC's annual on-site visits to its DTPs, with additional involvement of TAB members. A timetable of evaluation activities and next steps is provided below:

Indicative timetable of activities
Timing Activity Status

April - June 2016

Surveys open - responses by 10 June 2016 Complete
June - September 2016 NERC and TAB analyse survey responses and other data to develop interview questions Complete
September 2016 - February 2017 On-site visits to DTPs conducted - interviews with DTP partners and students Complete
February - May 2017 NERC analyses all data and produces evaluation reports Complete
March 2017 Individual progress reports provided to DTPs following moderation Complete
June 2017 TAB discussion of evaluation outcomes and recommendations for DTP2 commissioning mechanism Complete
July 2017 Publication of independent evaluation report and NERC's management response Complete
September - December 2017 DTP2 commissioning mechanism discussed and approved by NERC Council Pending
January 2018 DTP2 commissioning process implemented (mechanism TBC) Pending
September 2018 DTP2 funding confirmed Pending

Evidence for the DTP evaluation was gathered from all members of the NERC community, including those not currently involved in a NERC DTP. Dedicated surveys were developed for several key target audiences: DTP lead organisations; DTP academic partner organisations; DTP non-academic partner organisations; DTP advisory boards (where applicable); students (both DTP and others); research organisations / higher education institutions (HEIs) not currently part of a DTP; NERC-relevant end-user organisations not currently working with a DTP.

Surveys

The surveys are now closed. NERC would like to thank all those who took the time to submit survey responses for their valuable contribution to this evaluation exercise.

Further information and contacts

For more information on the DTP evaluation, please contact NERC Research Careers Team at .