The court said setting carbon limits is a matter for member states rather than the EU. The ruling could force the European Commission to review its quotas and undermine the fledgling carbon market.
Estonia and Poland have been fighting for more generous national caps on industrial carbon emissions, arguing that their industry would be hamstrung under the EU scheme.
A Commission spokesperson said the EU executive would consider appealing the decision, which was described as "extremely disappointing". An appeal process could take more than a year.
Under the scheme countries get a certain allowance of carbon emissions rights which they apply to industry, such as power plants and steel mills.
"The Commission exceeded its powers" by imposing a ceiling on carbon emissions, said the EU Court of First Instance, Europe's second highest court, in its statement.
Poland, Estonia and other East European countries argued that the Commission had unfairly trimmed their quotas, or national allocation plans (NAPs), under the second trading phase of the scheme from 2008-12.
Concern for future of carbon market
The news sparked concern among EU carbon market participants that the ruling, if upheld, could cause an unravelling of the market, which depends on a tight cap on emissions.
If their cap is raised, as Poland and Estonia want, the price of EU allowances (EUAs) could tumble. In addition, several more countries have objected to their quotas, including Czech, Hungary, Bulgaria, Latvia, Lithuania and Romania.
"It is certainly a big issue as far as other outstanding national allocation plan decisions are concerned," said Graham Stuart, partner at the law firm Baker & McKenzie.
Prices for EUAs were down 3.9% at €13.20 euros ($19.54) a tonne in the wake of the decisison. "It's bearish news. It sets a precedent for other countries," Reuters quotes one trader as saying.
The Commission had cut by 27% Poland's original request for 284.6 million tonnes of EUAs annually from 2008-12, and had cut Estonia's requested quota by 48%.
EU member states alone had the power to take final decisions fixing the quota, the court said on Wednesday. The Commission only had powers to review the quotas, and was wrong to dismiss these solely on the grounds of unreliable data, it added.
(EurActiv with Reuters.)