When science meets politics: the EU’s impact assessment review

Since the turn of the century, the European Union has striven to improve its decision-making by requesting a detailed impact analysis before taking new major policy initiatives.

Impact assessments (IAs) are now routinely conducted to predict the likely consequence of EU legislation on citizens and the wider economy, in a detailed cost-benefit analysis.

The European Commission is currently reviewing the guidelines on how it carries out IAs and has committed to publish new ones by the end of 2014.

The debate may sound technical but could have far-reaching consequences: an IA may lead the Commission to make substantial amendments to a legislative proposal, scale it down or even drop it altogether. Products and technologies may be approved or rejected as a result, leading to new growth opportunities or job losses depending on the outcome.

And some questions can easily turn explosive when science is thrown into the mix – on topics such as climate change, renewable energies, GMO authorisation, shale gas or the regulation of endocrine-disrupting chemicals.

On each of these, the Commission seeks advice from the scientific community to determine what form of regulation is needed, if any, applying the precautionary principle.

It is therefore no surprise that IAs have attracted attention and sometimes, criticism.

Some see them as just another layer of bureaucracy and a further impediment to faster and leaner law-making. Others have been suspicious of the IA process, saying the “evidence” presented there is prone to political manipulations and influence from a wide variety of interest groups, including powerful business lobbies.

Even the scientists feel they are being manipulated by politicians seeking to push specific policy initiatives.

A big challenge for the incoming European Commission will be to disconnect its evidence gathering processes from the “political imperative” that’s driving policy proposals, said Anne Glover, the EU’s chief scientific advisor.

The EU: bigger on big things, smaller on small things?

José Manuel Barroso, the outgoing President of the European Commission, made a pledge before he ended his second term in office: the EU Executive would stop spending its time on petty things like regulating olive oil or the amount of water used in toilet flushes to focus on the bigger issues. 

“Europe should only act where it can add real value – big on the bigger things, small on the smaller things,” Barroso said in a speech to European leaders in October 2013. “Useless laws weaken necessary laws,” he added, quoting Montesquieu.

One of the tools to achieve this law-making process are impact assessment studies.

IAs are hardly a new feature in EU policymaking. Originally introduced at the turn of the century  under the Commission’s ‘Better Regulation’ programme (since rebranded ‘Smart Regulation’), they are now being followed up as part of the Regulatory Fitness and Performance programme (REFIT), which aims to make EU law simpler, and to reduce regulatory costs on businesses.

IAs are meant to act as a 'reality check' on policymakers by requiring  “an integrated analysis of costs and benefits” of proposed legislation and an assessment of the economic, social and environmental effects of every major new policy initiative.

The European Commission is currently reviewing the guidelines on how it carries out IAs and has committed to issue new ones by the end of 2014.

The matter may sound technical but is in fact highly political. Essentially, the guidelines determine who the Commission seeks advice from and how. After going through the IA procedure, draft legislation may be amended or even dropped altogether, in an effort to make the EU leaner and more efficient.

Precautionary principle

Weighing scientific evidence is a key part of this process, based on the precautionary principle.

Enshrined in Article 191 of the Treaty on the Functioning of the European Union (EU), the precautionary principle holds that anticipated harmful risks to areas such as health or the environment should take precedence in policy-making, where a scientific consensus exists.

In preparing its environmental laws, the treaty adds that the Union “shall take account of:

• available scientific and technical data,
• environmental conditions in the various regions of the Union,
• the potential benefits and costs of action or lack of action,
• the economic and social development of the Union as a whole and the balanced development of its regions.”

In practice, implementing the precautionary principle has proved controversial however, as it touches upon politically sensitive topics like the safety evaluation of foodstuff, pesticides, or genetically modified organisms (GMOs).

In all cases, policymakers are supposed to look at the evidence and evaluate the degree of uncertainty before making any decision.

This is where things get tricky: when is there enough scientific evidence to take a decision? Where to draw the line?

Risk analysis: the work of EU agencies

The job of weighing the science – and therefore the risk – is often delegated to external agencies, like the European Food Safety Authority (EFSA), the European Chemicals Agency (ECHA) or the European Medicines Agency (EMA). There are also numerous expert groups that the European Commission consults on a regular basis to inform its decisions, many of which are composed of scientists.

EFSA for instance recently published its assessment of aspartame, the low-calorie sweetener used in drinks and weight-control products and regularly issues scientific opinions about the safety of food products.

The vast majority of those opinions are uncontroversial. But the agency has suffered criticism for its ‘revolving door’ practice: experts sitting on panels and in decision-making bodies often have a background in the food industry, or return there after leaving EFSA, leading to conflicts of interest.

In 2012, the chair of EFSA's management board, Diána Bánáti, was forced to resign after it emerged that she had failed to mention she was also a member of ILSI-Europe, a body funded by the food and agro-chemical industry.

To address the issue, EFSA unveiled new rules for its in-house staff, as well as its outside experts, including specified lists of activities that would preclude scientific experts from serving on advisory panels.

Scientists previously employed by industry must now take a two-year "cooling-off" period before they can sit on EFSA’s scientific panels and scientists who receive more than 25% of their research funding from industry face other restrictions on the roles they can undertake at the authority, for example.

Following the EFSA scandal, European institutions came to the realisation that common rules were needed to police the 31 agencies currently informing EU policymakers. A common approach was agreed in 2012 followed up a year after by a new set of set of guidelines on the prevention and management of conflict of interests in decentralised agencies.

Evidence-based policymaking?

The EFSA case illustrates just how contentious science has become as a policy-making tool.

Following an impact assessment, controversial technologies such as GMOs or animal cloning might be approved in Europe as being safe or, on the contrary, banned because of the environmental and health risks they pose.

The stakes are sometimes so high that politicians are tempted to manipulate the evidence collected in IA studies in order to favour one outcome over another.

Anne Glover, the EU’s chief scientific advisor, is no stranger to the issue. Describing her role at the European Commission, Glover recently admitted that she found it difficult to disentangle the evidence gathering process from what she calls the “political imperative” that drives policy.

In order to back-up their policy proposals, Glover said EU commissioners may sometimes request that specific “evidence” is presented in IA studies.

“Let’s imagine a Commissioner over the weekend thinks, ‘Let’s ban the use of credit cards in the EU because credit cards lead to personal debt’. So that commissioner will come in on Monday morning and say to his or her Director General, ‘Find me the evidence that demonstrates that this is the case.’”

The Commissioner’s staff might resist the idea but in the end, she says, “they will do exactly what they’re asked” and “find the evidence” to show that credit card use leads to personal debt, even though this may not be the case in reality.

“So you can see where this is going,” Glover said: “You’re building up an evidence base which is not really the best.”

Often, the European Commission will also contract external consulting firms to do the research groundwork that feeds into an impact assessment study.

But consulting firms have little incentive to contradict the Commission’s political agenda if they want repeat business, Glover said. This leads them to look for evidence that will back up the Commission’s agenda, leading to a distortion of the ‘scientific’ data presented to policymakers.

To be fair, the Commission is not alone in trying to manipulate evidence, Glover said. The same goes for the other two EU institutions – the European Parliament and the EU Council of Ministers, which represents the 28 EU member states.

“What happens at the moment – whether it’s in Commission, Parliament or Council – is that time and time again, if people don’t like what’s being proposed, what they say is that there is something wrong with the evidence. So everybody blames the evidence and nobody is honest about the fact that in many cases, understanding the evidence is the best possible platform to make the logical extension into policy. But they don’t like it so they say ‘We need more evidence’. And of course scientists can always produce more evidence.”

Contested impact assessments: the REACH case

There are countless examples of topics where EU policymakers have bickered over the evidence, including on the safety of nanoparticles, the impact of biofuel crops over food prices or chemical substances with hormone-disrupting effects.

In fact, the battle extends far beyond the EU institutions and spills over to the private sector and non-governmental groups trying to influence policy.

Perhaps the most politicised case to date was the REACH regulation on chemicals, which gave rise to one of the most epic lobbying battles in the EU's history, generating dozens of impact assessment studies before it was eventually adopted in 2006.

The bitter campaign saw chemical companies warn they could be forced to close factories and leave Europe because of the extra costs generated by the EU law. These included widely publicised industry-funded impact assessment studies which claimed that REACH would cost billions of euros to implement, causing millions of job losses in Germany.

At one point, EU officials arranged a meeting to try and make sense of 36 different impact assessment studies on REACH, most of them focusing on the legislation’s projected disastrous cost to businesses.

The Commission’s own initial impact study, meanwhile, had sought to balance those costs with the expected benefits of REACH for public health and the environment.

Ironically, the final impact study, which brought together representatives from all sides of the debate – from environmental NGOs to the chemical sector – ended up broadly confirming the Commission’s initial assessment that the costs of REACH would be largely bearable for the chemicals industry.

Quality standards

It was in the aftermath of the REACH battle that the European Commission adopted its impact assessment guidelines, which are now being revised.

The guidelines include minimum quality standards for conducting IAs such as data collection and making sure all the relevant stakeholders are consulted in the process.

“Good quality data – facts as well as figures – are an essential part of any IA. You need them to define the problem and the baseline scenario, and to identify the impacts of alternative options for dealing with the problem. Particular attention needs to be paid to quality and credibility of data,” the Commission states in its IA review document.

As shown by the REACH saga, stakeholders – whether businesses or citizen’s groups – can have a huge influence on the credibility of EU policies and are therefore seen as an essential part of the IA process.

“A poor public consultation process inevitably weakens the Commission's case,” the EU executive notes in the IA review consultation document, reminding officials that consulting those affected by new policies is “a Treaty obligation and a mandatory component of all IAs.”

Based on the data, impact assessment should also list clear policy options for politicians to choose from, highlighting several alternatives in terms of the expected benefits and costs.

Overall, the Commission says a good quality IA should always have the following attributes:

• Comprehensive: “IA analysis should be comprehensive, considering relevant economic, social, and environmental impacts of alternative policy solutions.”
• Proportionate: “The scope and depth of the IA should be proportionate to the type of initiative, the importance of the problem, and the magnitude of the expected impacts.”
• Evidence-based: “All Commission proposals should be based on the best available evidence and scientific advice, or a transparent explanation of why some evidence is not available and why it is still considered appropriate to act.”
• Open to stakeholders' views: “Stakeholders' views must be collected on all key issues and reported on in the IA Report. Every effort should be made to ensure that the Commission has sought and considered a wide and balanced range of views. The reasons for disagreeing with dissenting views must be explained.”
• Unbiased: “IA analysis must be objective and balanced. An IA should inform political choices with evidence - not the other way around.”
• Transparent: “The credibility of IA hinges on the transparency with which results are presented, estimations explained, choices justified and limits acknowledged.”
• IAs should also be embedded in the policy cycle, drawing on lessons from the past, and conducted in cooperation with other Commission services to bring different perspectives on a given issue.

To ensure those standards are respected, an Impact Assessment Board (IAB) was appointed in 2006 as “an independent quality checker” for IAs to ensure they are subject to “rigorous scrutiny”. It brings together the bosses of the all the Commission’s main departments.

But Anne Glover, says the board’s composition should be revised to make it less political. In particular, she points out that there is no scientist sitting on the board, saying “There is an opportunity there".

Politicians retain the last word

Meanwhile, the Commission’s effort to improve its impact assessment process has attracted praise across the board.

In a 2010 report, the European Court of Auditors said that IAs had been "broadly effective" in supporting EU decision-making. Their findings were “helpful”, especially because they presented decision-makers with policy options to choose from, in the auditors' view.

However, they also remarked that the Commission's IAs were not updated to take into account amendments made to legislation during the decision-making process. The European Parliament and Council of Ministers, the two law-making bodies of the EU, rarely carry out such assessments themselves, they noted.

The Parliament took the criticism on board and created a specific department in 2012 to repeat IAs whenever “substantive amendments” are made to legislative proposals. The new Parliament department may even take the initiative in some areas, such as making “in-house appraisals” of IA studies conducted by the Commission. The Council, for its part, agreed to discuss Commission IAs more frequently than in the past, without committing any further.

But no matter how irrefutable the evidence may be, at the end of the day politicians will – and should – always retain the last word, the Commission warned.

“The IA supports and does not replace decision-making – the adoption of a policy proposal is always a political decision that is made only by the College of Commissioners,” the Commission said.

For the EU’s chief scientist, Anne Glover, this is acceptable as long as all parties understand that policy decisions are taken by politicians, not scientists, and that the evidence-gathering process is transparent.

What she suggests is that the Commission creates a special department whose role would be to assess policy proposals against the evidence – “a central service which would be the evidence portal.” The service would take "questions" submitted to it by the Commission directorates and bring together the evidence to substantiate the issue at hand. Once formulated, the evidence base would be sent back to the policymakers who could then look at policy options based on the analysis.

According to Glover, such an open process would compel politicians to justify their decisions – and explain why they may have chosen to disregard scientific evidence along the way.

“If [politicians] choose to develop a policy that’s not based on the evidence, the biggest difference is that they would have to say, ‘I accept the evidence but for other reasons – social, economic, ethical, philosophical – we’re doing this’. And I think that’s quite justifiable because as I scientist I’m not saying that all that matters is evidence.”

“I don’t do policy for science, I do science for policy.”

BusinessEurope, the employers’ association representing national business federations in 35 European countries, strongly defends the role of science in EU policy-making.

In a letter to the outgoing President of the European Commission, José Manuel Barroso, it praised the “positive steps” that were taken under his leadership to encourage the use of “high quality science whenever officials develop policy, laws and rules,” saying it “welcome(s) these initiatives.”

“Evidence-based decisions, derived from the best available science, build trust, enhance legitimacy, and reduce the risk of regulatory failure. They also provide a predictable policy context within which trade and investment flow freely and innovation flourishes,” BusinessEurope wrote in the letter.

The EU employers’ group particularly welcomed the appointment of Anne Glover as Chief Scientific Advisor (CSA) to the Commission President. “We see the CSA as a key proponent in science-based decision-making,” it said, encouraging Barroso’s successor Jean-Claude Juncker to “further institutionalise the role of scientific evidence in the political process.”

“The CSA should have a formal role in the Impact Assessment Board,” BusinessEurope said, as well as during the so-called “interservice consultation” stage where policy proposals are discussed between Commission departments. “The CSA should also have the mandate to voice scientific evidence in the Commissions decision-making process. The College of Commissioners should then lay out its argumentation, in case it chooses to overrun the official scientific opinion.”

The role of the EU’s Chief Scientific Advisor has fuelled criticism from transparency campaigners, who have called on the incoming Commission President Jean-Claude Juncker to scrap the position.

“The post of Chief Scientific Adviser is fundamentally problematic as it concentrates too much influence in one person, and undermines in-depth scientific research and assessments carried out by or for the Commission directorates in the course of policy elaboration,” wrote the Corporate Europe Observatory (CEO) in a letter to Jean-Claude Juncker.

According to the CEO, the role of Chief Scientific Adviser has been “unaccountable, intransparent and controversial,” adding that the nature of Anne Glover’s advice to the Commission President remains unknown and confidential. The letter also cited “one-sided, partial opinions” Glover gave to the media in the debate on the use of genetically modified organisms (GMOs) in agriculture, saying her views did not represent the scientific consensus.

“We hope that you as the incoming Commission President will decide not to nominate a chief scientific adviser and that instead the Commission will take its advice from a variety of independent, multi-disciplinary sources, with a focus on the public interest,” CEO said.

Anne Glover, the EU’s Chief Scientific Advisor (CSA), believes science should remain independent from politics and sees her role as a non-partisan expert whose job is “to challenge” what the Commission says or proposes.

“Of course I am aware of the political implications of some of the things I say. But I try to focus everything I do on the evidence,” she recently told an event organised by Eurochambres, the Association of European Chambers of Commerce and Industry.

However, Glover is keen to underline that the nature of her role at the Commission implies a necessary degree of confidentiality. Asked by EURACTIV to disclose the kind of scientific advice Barroso had sought from her during her two-year mandate, she refused to answer in detail, arguing the process was intentionally not transparent.

“Why is it not transparent? Because if it is transparent, then everybody will try to defend their position. And I’m not interested in that, I’m interested in getting the best evidence possible.”

Still, Glover does believe that the scientific evidence put forward in the Commission’s impact assessment studies should be made more transparent. She even recommends to “develop a new system of evidence-gathering within the Commission” which is “entirely disconnected” from political “imperatives”. Her reasoning is that every stakeholder should be able to look at the evidence on which politicians base their decisions and hold them to account.

“You would be balancing the evidence base and making an analysis to determine why [you are] giving more weight to that evidence over that evidence. And that would be transparent. So every stakeholder whether a citizen, a business, a politician, a scientist – can look at the reasoning that’s behind all of that. And that is all doable, it is not a fantasy. It would be quite easy to achieve.” (Read more).

For the European Ombudsman Emily O’Reily, the Commission’s IA review should be seen in the broader context of how the EU executive deals with external expert advice.

In May this year, O’Reily opened an investigation into the composition and transparency of Commission expert groups, arguing they should adhere to “the highest standards of ethics, transparency and good administration.”

"EU legislation and policies are getting more and more complex, especially in key areas such as public health, science and research. The Commission relies on the advice and input of EU agencies such as EMA, EFSA or ECHA as well as of hundreds of expert groups, covering a broad range of scientific areas. It is of utmost importance for the Commission to be as transparent as possible about the advice it seeks and takes from those groups and agencies.”

O’Reily said she wanted to “investigate which expert groups may lack a balanced representation of interests” and whether the appointment of members "in a personal capacity" was problematic. She focused on the composition of groups advising the Commission on agricultural policy, saying more than a third of the EU's budget goes into this crucial policy area.

Ragnar Löfstedt, the director of the King’s College Centre for Risk Management in London, spoke to EURACTIV in an interview last year about the role of science in decision-making.

In his view, the precautionary principle is often “misused” by politicians who may be campaigning against technologies like GMOs or hazardous chemicals. He likes to cite an EU Parliament report by Swedish MEP Åsa Westlund’s on endocrine disruptors as a worst-case example, saying the document presents “woolly” evidence to argue that the feared effects of endocrine disruptors should “override any evidence-based reasoning” and justify a ban.

In reality, Löfstedt says Sweden can afford to be anti-chemicals because it does not have a chemical industry. But when its own economic interests are on the line, the country suddenly becomes much less of a green warrior. For example, he notes that Stockholm has fought hard to maintain Baltic Sea fishing despite scientific evidence that the fatty species caught there – herring and salmon – contain high levels of dioxins.

“Why is this? This is about saving jobs along the northern coast of Sweden. So is this hypocrisy? Yes, it’s hypocrisy. But basically with regard to those fatty fish, the Swedes are letting their own population be poisoned by eating the fatty fish because they want to protect jobs. But it’s very easy for them to be anti-chemicals because there is no chemical industry …”

At the same time, Löfstedt also laments that scientific arguments have been used by industry lobbyists to delay policy decisions by instilling doubt in the minds of policymakers.

“The classic example comes from the tobacco sector. There is very clear evidence that Philip Morris and others tried to halt legislation on tobacco by throwing in scientific uncertainty. And to be very frank, I think that is disgraceful and it is a horrible thing they have done. It gives us academics who have been devoting our entire lives to the field of risk a bad name. What happens now is that campaign groups will say because of what Philip Morris and those folks did, the purpose of all risk assessments is basically about trying to help business slow down the regulatory process. And that simply is not the case … but people will believe it.”

Colin Macilwain, a columnist writing in the US scientific journal ‘Nature’, says the term “sound science” has been hijacked by lobbyists in the “global war” on over how science should be used in regulating everything from smoking to pollution.

“The term 'sound science' may sound innocuous — comforting, even. Don't be fooled. In policy circles, its use is now pretty-much confined to the determined band of brothers who make their livings trying to roll back government regulation, by fair means or foul,” Macilwain wrote.

Here too the tobacco industry is under scrutiny. Macilwain cites British American Tobacco (BAT), the world's second-largest tobacco company, for putting up The Advancement of Sound Science Coalition in 1993 to question early evidence linking passive smoking to cancer. Macilwain says the coalition “pioneered an effective approach that abuses science to hold back regulation for long enough for BAT and others to haul in extraordinary profits. Today, regulatory efforts such as plain packaging and bans on menthol cigarettes are held up by legal challenge, often using language inserted into global trade agreements by industry lobbyists.”

“'Sound science' is thus science that big business knows it can trust. In its name, businesses that sell contentious products are working night-and-day to deflect rules and regulations by exploiting a schoolboy image of science to make their case,” Macilain wrote.

Jacqueline McGlade, the Executive Director of the European Environment Agency (EEA), has argued in favour of following the precautionary principle in decision-making rather than what she calls “scientific inertia”.

Speaking to EURACTIV in an interview last year, McGlade said “scientific uncertainty is not a justification for inaction” while responding to early warnings “can provide immense savings” for companies and society at large.

Taking endocrine disrupting chemicals as an example, McGlade said it is hard for scientists to identify precise thresholds of exposure below which there are no effects to humans. But there is often sufficient scientific evidence to suggest precautionary measures should be taken, she argued.

“Where the 'knowledge-to-ignorance ratio' is high (implying much knowledge and little practically necessary ignorance), as with, for example, lead, asbestos and mercury, there is little need for either more research or for precautionary measures – in such cases we need to take preventative action.”

• 1 July 2014: European Commission launches public consultation on the review of its impact assessment guidelines
• 30 Sept. 2014: Public consultation closes
• By end 2014: Commission expected to publish new impact assessment guidelines

European Commission

• Impact Assessment
• Commission Impact Assessment Guidelines
• Public consultation on the revision of the Commission's Impact Assessment guidelines | Public consultation questionnaire
• Impact assessments in other EU institutions
• Register of Commission expert groups | Expert groups explained
• The precautionary principle
• Communication from the Commission on the precautionary principle (2000) | FR | DE

Other EU institutions and bodies

• European Court of Auditors (ECA): Special Report: Impact Assessments in the EU institutions: Do they support decision-making? | FR | DE | (Sept. 2010)
• European Ombudsman: Investigation into Commission's expert groups (14 May 2014)
• European Food Safety Authority (EFSA): Corporate documents
• European Food Safety Authority (EFSA): Openness and transparency
• European Medicines Agency (EMA): Transparency | Access to documents
• European Chemicals Agency (ECHA): Independence and transparency
• European Environment Agency (EEA): Governance
• European Environment Agency (EEA): Late lessons from early warnings: science, precaution, innovation (2013)

Business & Industry

• BusinessEurope: Letter to Commission President Barroso (6 May 2014)

Think tanks and academia

• Nature: Beware of backroom deals in the name of 'science'

EURACTIV coverage

• How the Commission ‘blocked’ key environmental plans (5 Aug. 2014)
• EU twisting facts to fit political agenda, chief scientist says (27 May 2014)
• Special Report: Risk vs. hazard in policymaking (June 2013)
• Risk expert: Policymakers often misuse precautionary principle (June 2013)
• EEA chief: 'Scientific uncertainty is not a justification for inaction' (June 2013)
• Auditors assess role of impact studies in EU decision-making (29 Sept. 2010)