The 28 EU member states currently have a patchwork of varying tariffs for cybercrime.
The decision mandates national maximum sentences of at least two years in prison for attempting to illegally access information systems.
The maximum penalty for attacks against infrastructure such as power plants, transport, or government networks will be set at five years or more, higher than the current tariff in most member states.
The decision also increases the penalties for illegally intercepting communications, or producing and selling tools to do this.
Cybercriminals often infect computers to form armies of zombie PCs known as "botnets" by sending spam emails containing malicious links and attachments, and by infecting legitimate websites with computer viruses.
Some botnet creators rent or sell infected machines on underground markets to other cybercriminals looking to engage in a wide variety of activities including credit card theft and attacks on government websites.
In June, Microsoft helped to break up one of the world's largest cybercrime botnets, believed to have stolen more than $500 million (€387m) from bank accounts.
Under the new EU rules, companies that benefit from botnets or hire hackers to steal secrets will be liable for any offences committed on their behalf.
The European Parliament in Strasbourg voted 541 to 91 with nine abstentions on the proposal by the European Commission, the EU executive.
However, Denmark has chosen to opt out of the rules, wanting to keep its own system in place. The UK and Ireland, for their part, signalled they would apply the directive into their national law, despite Britain's decision to opt-out from EU Police and crime laws.
EU governments are expected to formally adopt the directive shortly at a forthcoming meeting of the EU Council of Ministers. Once done, they will have two years to translate the decision into national law.