Partido Popular won an absolute majority of 186 seats in the 350-seat lower chamber of the Spanish parliament.
This is three seats more than obtained by José María Aznar in 2000 when his opponent was Joaquín Almunia, now a European Commissioner.
The centre-left Spanish Socialist Party PSOE, led by Alfredo Pérez Rubalcaba, won 110 seats, 59 short of the result obtained in 2008 under the leadership of outgoing Prime Minister José Luis Rodríguez Zapatero.
This is only the second time since Spain returned to democracy in the 1970s that a party won an absolute majority in Parliament.
"Difficult times are coming," Rajoy, 56, a former interior minister, told supporters.
“We are going to govern in the most delicate situation Spain has faced in 30 years,” he said.
Rubalcaba, an interior minister until July, acknowledged his defeat and thanked his supporters for the 7 million votes the party received in "extremely difficult circumstances".
Among the other parties represented in the elections, Izquierda Unida (United left), a leftist organisation, captured 11 seats in what is seen as a good result.
The Basque separatist Amaiur enters parliament for the first time with seven seats. Convergència i Unió, or CiU, the Catalonia nationalist party, obtained 16 seats, five more than in the former parliament. The party Unión Progreso y Democracia (UPyD) of Rosa Díez, a former Socialist MEP of Basque origin, took 5 seats and will be able to form its own group.
The victory of the centre-right was anticipated, with voters venting their rage on the Socialists, who after seven years in power leave the country with 5 million people out of work and the EU's highest unemployment rate.
The Spanish crisis, magnified by the flop of the real estate bubble, generated the movement of "Indignados", who inspired protestors across Europe, as well as the Occupy Wall Street movement in New York.
Rajoy is not expected to attend the next European summit on 9 December as he will be sworn around 20 December.
Spain is the fifth EU country, after Ireland, Portugal, Greece and Italy, to have a new government as a result of the unfolding eurozone crisis. In Slovakia, the government was defeated as a result of a vote in Parliament on the EU's bailout fund. The country will hold early elections on 10 March.
Spain will have a government resulting from elections, unlike crisis-hit Greece and Italy, where the governments have been appointed as a result of political bargaining.
