This article is part of our special report European gas markets.
The integration of European gas markets has made strides over the last fifteen years, bolstering energy security in Eastern EU countries as a result, says Jonathan Stern. But the collapse of Dutch production means Russia will probably remain at the centre of the game for years to come, he cautions.
Jonathan Stern is the head of the Natural Gas Research Programme at the Oxford Institute for Energy Studies. He spoke to EURACTIV’s energy and environment editor Frédéric Simon.
- “Huge progress” made since start of gas liberalisation process
- “Near-total” price convergence in North-West Europe, less in east and south
- Despite new interconnections, Europe is more dependent on Russian gas than ever
- “Collapse” of Dutch production a major source of concern
- The role of gas in Europe “will not diminish significantly before 2030”
- Carbon constraint means it will be “very difficult to build any new natural gas assets after 2025”
The European gas market looked very much like a patchwork of national markets when the first gas directive was adopted, in 1998. How much progress has been achieved since then? What were the main milestones?
A huge amount has been achieved, particularly in North West Europe, and to a lesser extent in Central Europe and Italy. Essentially, the further south and east you go, the less the integration of markets, but in general I would say there has been a huge amount of progress.
The first big milestone was the European Commission’s sector investigation of 2007, when the competition directorate concluded that too little progress had been achieved on the liberalisation of gas markets in Europe and told energy utilities what they needed to do.
They basically said what we knew already: the gas market was segmented, prices were not competitive, etc. And they told utilities they should do much better.
The next major milestone happened the following year, in 2008. And that was what I call a perfect storm for the gas industry – with recession, collapsing demand, massive increases in oil prices, and hence gas prices that were linked to oil prices.
That essentially led to the main utilities losing loads of money, and having to renegotiate their contracts. And this is when the whole world changed, particularly for the big gas markets in Europe.
The third milestone was the development of a liquid gas hub in the Netherlands, the TFF. There was already the NBP in the UK. But to have a continental European gas hub was a critical development. It had been in existence already in the mid-2000s, but starting from 2011 it became a serious gas hub.
And from 2014 onwards, its liquidity meant that there was now a real rival to the NBP. And it has since become in many metrics a bigger hub than the NBP.
And what did that change?
What it changed is that we now have a real gas market in North West Europe and other parts of the continent, where prices are set by supply and demand. So we have a liquid market which means that we have real competition and a very good progress towards a single gas market in North West Europe – meaning approximately 60 to 70% of the gas sold in Europe, and if you add Italy approaching 80%.
And that has an effect on the whole of continental Europe.
Price convergence and market liquidity are the two key indicators the European Commission looks at when assessing its liberalisation policy. How much of that has actually taken place over time?
In North West Europe, I would say we are more than 90% towards price convergence. We have done studies on this, showing that price convergence in that region is near-total.
You can find little periods where the prices diverge a little bit, like when some pipelines and interconnectors undergo maintenance. But essentially you have almost complete price convergence in North West Europe.
In Central Europe, you have a good degree of price convergence but not to the same extent as North West Europe. And in Southern Europe, with the exception of Italy, you have much less convergence.
So that means gas suppliers there can set their own terms?
That’s right. It means that you do not yet have enough interconnection and competitive conditions in a lot of the smaller gas markets, especially in South East Europe. And you don’t have the connectivity between Spain and the rest of Europe to ensure that you have meaningful price convergence with North West Europe.
For these regions, does that mean they are essentially dependent on Russia as a single gas supplier?
You can’t generalise, you have to look at individual countries.
- Hungary and Poland have made significant progress, same for the Czech Republic and Slovakia.
- In some former Yugoslav Republics, Bulgaria and Romania, there is much less progress.
- In Spain, there is a different type of progress because they have their own market for LNG and pipeline gas – they’re still an LNG island in some ways.
- Italy is very close to being fully integrated with North West Europe, via Austria, although it’s not completely the case yet. But in Southern Europe, Italy is the beacon of competitive gas markets.
So the regulators are right to push for further liberalisation because this will mean a more diverse range of suppliers…
It will. But there are still issues to be resolved: Interconnections are gradually getting built. Which means these markets will eventually have a direct connection to a liquid gas hub, whether it’s in the Netherlands or Austria – they will be within range of a market price. And that will have a big impact on their markets.
The problem in many of these markets is that they are still dominated by a single, often state-owned, domestic player. And that means there are limits to the amount of competition that can be expected. But they will have access to a more diverse gas supply than simply Russia. And they will also be able to negotiate potentially more advantageous terms.
You say “will”, can you say when?
It depends on when interconnections will be finished. Virtually every country in Europe, except some of the very small former Yugoslav republics, these pipeline links are either underway or in advanced planning stage.
One of the main objectives of the EU’s liberalisation policy was to improve security of supply, especially after the first gas crisis between Ukraine and Russia in 2006. How successful has the EU been in that respect?
Moderately, I would say 7 out of 10.
….which is not so bad.
Not so bad at all. The problem here is that interconnectors allow these countries to get hold of other gas. The question is, will other gas be available?
But if there were to be another Russia-Ukraine gas crisis today or tomorrow, would Central and Eastern European countries do better than 2009? The answer is definitely yes.
But they would still be in deep trouble.
Yes and no. What has happened, totally counter-intuitively, is that Europe has become steadily more dependent on Russian gas. And the big irony is that this happened against all the policy statements and ideological commentary that were made since the last two Russia-Ukraine crises, the 2009 gas crisis and the 2014 political crisis.
Today, we are in a much better position to deal with a gas crisis – thanks to new infrastructure, liberalising infrastructure, allowing for two-way flows of gas, facilitating storages and new LNG import capacity.
But that is not the same as saying that Europe has diversified its gas supplies.
And that is probably getting worse with North Sea gas production having passed its peak…
The key development that nobody outside the gas industry recognises is the collapse of Dutch production at the Groningen field. This is what is absolutely unbelievable for me: when politicians and media commentators speak about gas security problems, they only talk about Russia and nothing else.
Whereas when gas industry people speak about security of supply, what they speak about is the decline of production in the Netherlands, the UK, and further in the future Norway. Although in general things are much more positive in Norway, we have questions about some day-to-day reliability issues.
On the UK and Netherlands side, the state of North Sea reserves is probably beyond repair…
The only question is to the speed of the decline. And the problem is that we did not realise until relatively recently that in the Dutch gas sector it would decline very quickly.
The gas market is closely linked to infrastructure, and you underlined some of the efforts currently underway. Broadly speaking, what major improvements have been made over the past ten years on the infrastructure side?
The big improvement is that more European countries now have LNG import terminals. Let’s leave Spain aside because it is still much more dependent on LNG than any other European country. Many European countries which ten years ago or even five years ago did not have an LNG import terminal, now have one.
However, the actual utilisation of EU LNG terminals over the past few years has been only around 25% of capacity. So despite the fact that we have the infrastructure to import, we haven’t been importing LNG over the last few years, for commercial reasons.
Now, in terms of pipeline infrastructure, the big initiative over the last ten years has been the Southern Gas Corridor.
…which hasn’t happened.
Exactly. The Southern Gas Corridor was such a big initiative, so much emphasis was placed on it.
So why has it not happened? If you’re optimistic and charitable, you’ll say it needs more time, it could well happen in the 2020s or the 2030s. If you’re less charitable, you’ll say that, although there is an awful lot of gas in the Caspian, Central Asia and the Middle East, it’s been impossible to actually get it out of those countries and into Europe, for political and commercial reasons.
And also maybe because of falling demand and improved energy efficiency?
No. It’s due to the fact that although we’ve got rising imports of gas, Caspian gas cannot compete with Russian gas in particular, or gas from other sources. Caspian gas has to come from a very long way, and from countries that many importers do not consider as secure sources of supply, or transit.
Are you saying the Southern Gas Corridor is a kind of white elephant?
I think it would be too harsh to say that. What I would say is that it’s proved to be much more difficult than was initially thought.
And it’s been unlucky as well. The Southern Gas Corridor started being developed as liberalisation started to happen in Europe. If it had been developed in the 1990s, it would have been much easier: The big monopolists were still in place, they still had their financial power, they could sign long-term contracts, and they could sell gas at oil-linked prices – which would have made the creation of a southern corridor so much easier.
In today’s environment, it’s very difficult. Basically, what we’re telling these suppliers is that they can only sell gas to Europe at hub prices. And that they’ll have to operate in a competitive environment, so it will be up to them to take the price risk.
And from a transit point of view, Erdoğan’s Turkey is becoming a more challenging partner for Europe. So offering Ankara further leverage on Europe at the moment doesn’t look very attractive politically.
Returning to European gas markets as such, the so-called “use-it-or-lose-it” rule has been in place for some years now. It requires transport capacity booked but not used by energy companies to be made available to other network users. How much change has this brought to the market?
I think it’s brought a certain amount of change. But the change has been more in relation to the fact that people know it exists, rather than actual gas flows.
The point about use-it-or-lose-it is that it provides protection against hoarding of gas capacity by incumbents, who cannot claim anymore that they have a long-term ship or pay a contract which takes up all of the capacity. The fact that the market knows that is important in itself.
But the rule has only been operational on many markets for a relatively short time. We haven’t seen it in action everywhere. But I haven’t heard any complaints about use-it-or-lose-it, except in some Central European countries, where the incumbents still dominate. There, you do hear complaints, but not in the liberalised liquid markets.
What does that tell us about these Central European countries?
It tells us that many of them complain loudly about their positions but are not willing to liberalise their markets. Basically, they want the best of both worlds: They don’t want to change their market structure by privatising their companies and encouraging new entrants. But they want the benefits of liquid and competitive markets at lower prices.
That sounds contradictory. Liberalisation has improved security of supply in Western Europe, which is the big worry of Central and Eastern European countries. So they should be interested in doing the same, shouldn’t they?
Yes, but many Central and Eastern European countries want to maintain security of supply through control by government.
How much of the infrastructure is now capable of running with “reverse flow” capacity? Has this been a significant element in improving the fluidity of EU gas markets?
All of it should be capable of running reverse flow because that was part of the regulation on gas security, which should now be fully operational. And I don’t know anywhere where it’s not operational.
I believe this has been very significant in terms of market fluidity. But it’s the same story as with general competition: The further east and south you go, the more problematic it becomes.
That said, this is one of the issues which should be resolved by the Gazprom case with DG Competition, which should ensure that Gazprom is required to allow reverse flow wherever it’s requested.
And how is that process going?
The investigation has been running for seven years now but it’s been held up by the threat of a legal challenge from Poland. The last we’ve heard is that it’s very close to concluding.
Turning to LNG and shale gas – they’ve both gained importance in recent years, and are seen by policymakers as a new way of diversifying supplies. Have they had a significant impact on the EU market yet?
This is actually a very complicated question.
Let’s dispose of shale gas immediately – shale gas in Europe has failed, I don’t expect it to succeed now, even in the UK where the government still seems very keen on it.
LNG has been a success in potential diversification – a lot of countries now have a receiving terminal. In some countries where the receiving terminal was specifically to diversify away from Russian gas – like Poland and Lithuania – it’s had a big impact in diversification because they went from a single supplier to being able to import from a range of LNG suppliers.
In many countries, receiving terminals have been running at very low capacity since they were created in the early to mid-2010s. But what’s important is that they could receive a lot more LNG if there was a problem with pipeline gas, or if there was a price signal suggesting that LNG was cheaper than pipeline gas. At the moment, LNG is more expensive and is being shipped to markets other than Europe that are ready to pay a higher price.
On shale gas, you didn’t mention Poland as a potential developer but I remember they had big plans at some point…
You remember correctly. They drilled 50 wells but the big international companies concluded there was no serious commercial opportunity there and almost all of them pulled out. What remains is a few smaller companies doing a little bit of production but nothing much.
Shale gas can also be imported, via LNG…
Yes, but although many people want to look at US LNG as the saviour of Europe from Russian gas, actually very little US LNG has arrived in Europe so far. More may arrive later but again, this is a commercial issue: is it more profitable for Americans to export their LNG to Europe or export to South America and Asia? They will, of course, look for the highest bidder.
Looking forward, how do you see the evolving share of LNG, shale and piped gas in Europe?
I think it’s almost impossible to answer that question, to be honest. Because it depends on so many different global dynamics.
Let me give you an example: The dynamics of global LNG depends on Asian countries import somewhere around two thirds to three-quarters of global LNG. Much of it goes to countries which do not have any pipelines alternatives. So when they need gas, they will take the LNG because they will always be prepared to pay higher prices than European countries which have a pipeline alternative.
So the general consensus is that for the next few years – until 2019, possibly 2022, 2023 or even 2025 – there will be a global surplus of LNG and Europe will be the beneficiary. And we will see more LNG in Europe. Exactly how much more is not possible to say.
Regional gas trading hubs were pushed by policymakers in Brussels as a way of strengthening the resilience of markets in Central and Eastern Europe. How successful has this been so far?
It’s been a great success, I would say. But it is not clear how much of that success is due to policymakers and how much is due to market forces.
In other words, I don’t think that TTF in the Netherlands became a big success because policymakers said ‘We need a liquid hub’. It became a big success because the market needed a price reference once it became absolutely clear that there was no basis for continuing to price gas in relation to oil.
Having said that, the fact that the Third Gas Directive and the subsequent Gas Target Model mandated that there should be a hub in each country was clearly important. So I think policymakers can take some credit. But if we hadn’t had the 2008 recession, I think the whole process would have been much slower, despite the policies.
Do you see a regional trading hub emerging in Central Europe, maybe in Austria, of the same magnitude as the British or the Dutch one?
No. Because basically, hubs are about first mover advantage. There is a good example with the US and Canada. Once you have a dominant hub, creating an additional one of equal dominance simply doesn’t happen.
Can they be strong regional hubs nevertheless?
They can be strong national or small regional hubs. But they will always a strong relationship to TTF because this is where the liquidity is.
Have minimum storage requirements imposed at EU level made a difference to supply security?
Probably but they haven’t been tested out yet. If we had another 2009 situation with Russian gas – and it really is only Russian gas we’re talking about – would Europe do better? I would say absolutely yes. But part of that is storage, part is reverse flow and part is because the markets are more liquid.
Natural gas consumption in Europe last year reached its highest level since 2010, according to EU figures published in April. What conclusions do you draw from this? Does it show that European markets are still not functioning as they should?
It shows they are functioning properly. This is the result of basic economics. One of the reasons why demand went down so dramatically was that prices in 2010-2014 were disastrously high due to some extraordinary conditions that we will not see again. Since 2014, prices have fallen dramatically and demand has gone back up again.
Another reason is the extraordinary strength of coal prices that nobody expected, and the phasing out of coal and some nuclear stations in North West Europe. So it’s not a single reason but it does show that basic economics apply to the gas industry.
How do you see the evolving dynamics between domestic gas production and imported gas in Europe in the coming years? With falling production in the North Sea and complications with the Southern Gas Corridor, that doesn’t leave Europe with many other options than Russia, right?
Again, these are complicated dynamics. On domestic gas, what we have is rapidly falling availability from the Netherlands, slowly falling availability in the UK, flat or slightly increasing production in Norway (at least for a few years), and then a whole range of other suppliers, apart from Russia.
North African gas supplies are a serious problem because we just don’t know how much gas is going to be available from those countries. And that’s a big issue in Southern Europe, which is not often talked about. In Libya, there is a serious political situation. And in Algeria, there are plenty of reserves but they are not developing them quickly enough. So we should not be expecting more gas coming out of North Africa and in a worst case situation, there could be less gas coming from there.
Does import dependency makes the case for liberalisation even more urgent? Does it help in a way?
Yes, liberalising access to markets does help and is the right thing to do, as long as there is gas and LNG available. But what people have not got their heads around is the gas availability situation as opposed to liberalisation conditions.
That speaks in favour of the Southern Gas Corridor, then?
If you look back to the mid-2000s, you will find people from the Commission saying that by 2020, there will be 100 billion cubic meters (bcm) of gas flowing through the Southern Corridor via three pipelines.
You could say that, even then, these expectations were a bit of an exaggeration. But nobody expected that there would be a corridor carrying so little gas by 2020. I don’t want to be too negative about this because I still think the Southern Gas Corridor could be a bigger success. But it’s already taken a long time and it’s going to take quite a lot longer.
What do you tell those – usually environmentalists – who argue more pipelines aren’t needed because the demand can be met with things like efficiency improvements or electricity.
I’d say fantastic news, but you’d better be right. All those people in North West Europe who say we don’t need any more gas, we’ve got efficiency, we’ve renewables, battery storage, etc. – fine. But please be sure that you’re right. Because if you’re not right, the pipelines will not be there and there will be no time to build them.
The consequences of that may not be disastrous – they may just be that we will be paying more for our energy – but it’s important that customers and governments understand this.
A fourth gas directive was tabled by the European Commission in November last year, which proposed extending so-called “ownership unbundling” requirements to third country pipelines – which is mainly seen as an attack on Nord Stream 2. Do you see this move as being justified from a market liberalisation perspective or a security of supply perspective?
No, I don’t. And I think it brings regulation into disrepute. In other words, we’re doing what we often accuse other governments of doing – basically making rules on the hoof: you see something you don’t like and you make a regulation to stop it.
We’ve had 30 years of gas regulation in Europe. We’ve created a framework and told countries that they need to stick to it. And when the Russians respect that framework, the Commission then makes new regulations to stop those pipelines. I’m against that because it is impacted by political views, which is exactly what we said we didn’t want.
Has Gazprom really complied with the EU regulatory framework? They’ve been trying to resist it too, haven’t they?
Of course they have. I spent three years sitting on an EU committee with the Russians telling us not to introduce this regulatory framework. They were telling us that we were going to stop Russian gas coming into Europe and that it was a stupid thing to do because we should be encouraging gas coming into Europe.
We kept saying to them was that we were going to make our rules and that they would have to live by them. So we did make our rules, which was the third gas directive. And, with some struggle, they agreed to live by them.
Now, we’re saying that this hasn’t done what we wanted, which is to stop Nord Stream 2. So we’re going to introduce new rules.
The goal here is to decrease the amount of gas imported from Russia…
That’s right. But the problem is that there isn’t any other gas. At least not in the short term. And that’s what people can’t accept. I do have sympathy for those who question Europe’s need for more gas and say there are alternatives to balance supply and demand. But you then have to accept that energy may end up costing more as a result.
One of the arguments for Nord Stream was to accompany Germany’s nuclear phase-out and avoid a capacity shortage being filled with coal, which emits more CO2. Do you see gas as a necessary “transition fuel” away from coal – a kind of lesser evil in a way?
I’ve written a lot on this. And you need to be very clear on the definitions. People bandy around this phrase ‘transition fuel’ as if they know what it means but they don’t give us a precise definition.
I have defined it as follows: The role for gas in Europe, particularly North West Europe, will not diminish significantly before 2030, and in some countries even beyond that date. I would suggest that qualifies gas to be defined as a transition fuel.
Looking at the 2040s and the 2050s is a different story. But if countries are going to meet carbon reduction targets, they cannot retain coal in their energy balances. Now, in some countries, like the UK for example, we have a lot of very old coal-fired power stations that we should have retired many years ago because they’re so inefficient. And we’ve made a big fuss about the fact that we’re retiring them now when they should have been phased out long ago anyway.
That’s not the same in countries like Germany where you have relatively new and efficient coal-fired power stations. But you can’t resolve the CO2 problem without retiring at least some of these stations. Maybe you can fill some of that gap with renewables and efficiency but you can’t fill all of it. And therefore you will need more gas.
But you can’t make broad sweeping generalisations either across countries or even across time frames within the same country. You have to be specific about the countries and the periods under consideration.
In the case of Germany, the role of gas in the transition phase is to fill the gap left by the nuclear phase-out because there are no guarantees that other sources of energy will be available to fill that gap.
Again, you need to be careful about what kind of gap you’re talking about. Because on some days or weeks, you can run a huge amount of wind and solar power. But in the cold, dark winter months in Northern European countries, you will need something to fill those gaps. And that will probably be gas.
In the long run – say 2030 and 2050 – do you see gas as a clean fuel in its own right?
I believe that on CO2 grounds (we could also talk about methane leakage), it will not be possible to represent gas as a clean fuel beyond 2030 unless carbon capture and storage can be developed on a large scale. Elsewhere in the world, it will, but not in Europe, and particularly not in North West Europe. And potentially, even in the 2020s, there will be pressure to phase-out gas if possible on the basis of CO2 and possibly methane emissions.
So you seem to agree in a way with environmentalists who warn against building new gas infrastructure because it could remain unused.
That’s a very difficult judgement to make. Because there are two types of risks. One is to build assets that will be stranded before they’re paid off. And the other risk is not building assets which will provide us with the energy we need. And the difficulty is to know what those assets should be and how quickly we need to pay them off.
But assuming that North West Europe is going to meet carbon reduction targets, it’s going to be very difficult to build any new natural gas assets after 2025.
Do you think Nord Stream 2 is a project that is driven by market demand?
Yes, now it is. This is an important point: When Nord Stream 1 was first presented, it was put forward as four lines. They only built 2, which came online in 2012. Now, think about what would have happened if they hadn’t been built given the problems we’ve experienced with supply since 2014.
When the next two lines were proposed after 2012, people questioned whether they were really needed. And then in 2014, with Ukraine and Crimea, everyone said ‘No, we don’t want Russian gas’. And then, people suddenly began to see what was happening with North West European supply. And the moment you see the drop in exports from the Netherlands, you realise the market case for Nord Stream 2.
But even if Nord Stream 2 is needed in the early 2020s, you might question whether it will still be needed in the 2040s. It’s an interesting question and one which is much more difficult to answer.
To conclude, policymakers in Brussels are looking at ways to encourage different varieties of “green gas” coming from renewable energy sources – whether biogas produced from agriculture, synthetic gas or hydrogen produced from renewable electricity. Gas industry representatives say these could represent up to 76% of gas in Europe by 2050. What do you make of that?
What I didn’t get from that report is what they thought European demand would be in 2050. Because if it’s 76% of a relatively small market, I agree. What I question is the Ecofys report which said there could be 100bcm of renewable gas in Europe by 2030. Yes, there could, but I wonder who is going to pay for it because it’s very expensive. We can do it but that would be mainly biogas or biomethane.
Hydrogen is more complicated. Because you either have to produce it through steam reforming methane but then you have to store the CO2. Or you have to produce it through electrolysis.
Now, we don’t have any large scale electrolysis going on in Europe, we have a megawatt here and a couple of megawatts there. But no large-scale electrolysis projects. There is quite a bit of hydrogen produced around the world but not with carbon sequestration and storage.
So we need to make more progress on technologies before we can say that renewable gas actually can be a commercial success. Because at current costs, there are more economic sources of energy.
Hydrogen is often put forward nowadays as a way of decarbonising the so-called hard-to-abate sectors of industry, like steelmaking. Is that something you’ve been looking into?
Hydrogen is great, but if you’re going to store the CO2 (which you need to do for it to be really renewable gas), then you need structures in which to store the CO2. For political reasons those structures generally have to be offshore. And all of that adds to costs.
In the UK, we have the best structure of industry and opportunities for storage to test that out. But we still don’t have ongoing projects. We have proposals but not ongoing projects.