Scientific excellence as one of the FP7 key concepts was, of course, under closer inspection by the HLEG. The new grants by the European Research Council (ERC) in FP7-Ideas were, once again, praised as a major success story and as a prime example for promoting scientific excellence. With many of the ERC grants not yet finished at the time of the HLEG report, each million euro of EU funds in FP7-Ideas projects was already worth more than eight publications, almost more than twice as in other Specific Programmes. But this is not too unexpected, if bottom-up research projects by outstanding scientists with a proven track record are compared to top-down projects of large consortia with a focus on innovation and commercial exploitation. Another aspect mentioned in the HLEG report was the impact of an ERC grant on career advancement as analysed in the EURECIA survey. While an ERC grant clearly improved the status of a grantee at an institution, there were only marginal advantages on his career advancement, due to the novelty of ERC grants in FP7 and rigid structures at hosting organisations.
To give proof of scientific excellence with respect to output as well as impact, different bibliometric indicators were put forward by the HLEG. The overall increase of publications during FP7, accompanied by a further rise of their quality and citation rates, was emphasised. Again not surprising, if taken into account that FP7 in comparison to FP6 was extended by two years and that the available budget was even tripled. The HLEG report also mentioned a study on the overall EU share of worldwide publications. This analysis demonstrated that the EU share of the top 1% of most-cited articles increased from roughly 28 to 30% between 2002 and 2012. Which share of this rise might be attributable specifically to Framework Programmes, including FP7, was however not addressed.
The HLEG also reflected on cases of outstanding scientific achievement in FP7. For example, it was stressed that 11 Nobel Prize winners and five Field medallists were among the ERC grantees of FP7. But is this really relevant? Does it not just mean that excellent scientists who have made important discoveries in the past and in most cases outside FP7-funded projects are able to write successful grant applications? In conclusion, the HLEG stated that although bibliometric indicators gained high significance and acceptance in evaluations, they are far from being perfect indicators for scientific excellence. Moreover, the HLEG gave a clear rebuff to those requesting that frontier research, as performed in ERC projects, should have to demonstrate immediate impact at a very early stage after their completion. It was further suggested that ERC projects might be the right test object to follow up the gap between scientific discovery, invention and possible commercial exploitation in a more systematic manner.
Another chapter of the HLEG report dealt with the effects of FP7 on value creation and economic growth. It focussed on the question as to how new findings in EU-funded projects may translate into productivity, economic growth and employment. According to the HLEG report, the earliest step along the value chain of a funded project is easier to follow, for example, by counting associated publications, patents or prototypes. Project-specific economic effects might not, however, be traceable until ten or more years after the execution of a project and may persist for 25 years or even longer. Effects on employment were also difficult to capture. But as one target of the Europe 2020 strategy is to employ 75% of the 20-64 year-olds, FP7 has also had to prove its suitability as a job creation programme. The HLEG struggled to address these economic impacts and argued that (so far, only a few reports had touched on this issue) the financial crisis between 2008 and 2012 had made many previous impact assessments worthless and the current available models, predicting economic effects of whole Framework Programmes, need further improvement.
Based on key indicators and model calculations from previously published studies by the French Paul Zagame and colleagues, the HLEG came up with some estimates. The basis of calculation was a €50 billion FP7 budget. The leverage of each FP7 euro, which was applied, was 74% or €37 billion in total. This means that each euro invested into the Framework Programme by the EU, was topped up by other organisations, such as research organisations, universities and industries by 74 cents. The value of proposal writing was estimated to be in the range of €3 billion, so that in total the direct investment into R&D caused by FP7 was around €90 billion. Using a multiplying factor of 6.5, which was already used by Zagame et al., the indirect economic FP7 impact by new technologies, products and markets over a 25-year period was calculated to be in the range of €500 billion. Combing all data, the HLEG suggested that each euro contributed by the EC incurred at least 11 additional euros in direct and indirect effects. Translating this monetary impact into job numbers, the HLEG proposed that 130,000 jobs for researchers were directly created by FP7 over a ten-year period (1.3 million persons-year at a rate of €70,000). Indirect effects were suggested to create 160,000 additional jobs over a 25-year period (4 million persons-year). These are barely noticeable effects, if one considers that the whole workforce in the EU is more than a factor of 1,000 higher.
One superordinated goal of FP7 was to contribute to Europe becoming the world’s leading research area. In financially difficult times, when national budgets for research and innovation were cut, the FP7 budget was kept constant. In the Austrian Research and Technology Report 2014, it was estimated that FP7 funds contributed, on average, to about a quarter of competitive national R&D funds in the EU member states, which means a significant impact. The overall conclusion of the HLEG was that FP7 clearly strengthened the overall European Research and Innovation System and gave proof of evidence of the added value of FP7. For example, it was stressed that the culture of cooperation and networking clearly improved throughout FP7, which brought together not only individuals and institutions from different countries but also actors from different sectors. At the same time, the share of partners from outside Europe remained low in FP7 and the HLEG criticised the missing strategy within FP7 to counteract this weakness.
New instruments, such as the five Joint Technology Initiatives (JTIs) set up as private-public partnerships in strategic areas of European interest, such as innovative medicine or embedded computing systems, generated critical mass and scope and led to novel technologies with high innovation potential. At the same time, the HLEG noted that synergies between established and new programmes as well as sub-programmes have to be improved, and duplicate efforts avoided. FP7-People with more than 50,000 mobile researchers also clearly contributed to the human research potential of Europe. As expected, the UK was the most attractive country for MC fellows and displayed a net gain of 3,000 researchers. Countries, such as Germany or France did not show any net gain; while Mediterranean countries, such as Italy or Spain, had a net outflow of 1,500 and 700 researchers, respectively. The HLEG noted that broader conclusions on outflow trends over time have not been possible due to the lack of long-term data. Along those lines, the HLEG also criticised the low survey response rate of MCA fellows, which allowed only an incomplete picture of this funding measure.