Zeman, 68, who favours more integration within the European Union, won by 54.8% to 45.2% over his conservative opponent, Foreign Minister Karel Schwarzenberg, according to results from 99.9% of voting districts.
Zeman had accused Schwarzenberg of favouring foreign interests in a bitter electoral campaign.
Zeman, an economic forecaster who was a Communist Party member before the Soviet invasion of Czechoslovakia in 1968, is expected to steer Czechs closer to Europe's mainstream.
The anti-EU rhetoric of outgoing President Klaus, who succeeded late playwright Václav Havel, has pushed the country towards the margins of the EU.
Czech presidents do not wield much day-to-day power but represent the country abroad and appoint prime ministers, central bankers and judges.
'I want to be president of the bottom 10 million'
Zeman said he wants to overcome divisions provoked by the election in the central European country of 10.5 million people. The final stage of the campaign was marked by doubts cast on the national loyalties of Schwarzenberg, a prince from a centuries-old aristocratic family who lived much of his life in Austria.
Zeman promised to tackle graft, an issue which has dominated political debate for years.
"I want to be president of the bottom 10 million. These include voters of Miloš Zeman as well as Karel Schwarzenberg. I do not want to be president of mafias that act as parasites on this society," Zeman said.
Zeman served as Social Democrat prime minister in 1998-2002 under a power-sharing deal with Klaus' right-wing party that critics saw as a breeding ground for corruption.
Schwarzenberg conceded defeat and congratulated Zeman, but relations between the centre-right cabinet and the new president may be strained.
Zeman, who has a folksy manner and a well-advertised appetite for sausages and alcohol, appeals to poorer and rural voters, unlike the government, which has raised taxes, cut social benefits and suffered several corruption scandals.
During his premiership, Zeman was credited with privatising the main banks and attracting foreign investment. Opponents criticise his friendship with former communist officials and businessmen with links to Russia.
Previously, Czech presidents were elected by parliamentary votes that involved a lot of back-room dealing, which led to popular demand for a constitutional change approved last year.
Ghosts of the past
The finale of the campaign was marked by appeals to nationalism, unusual for the Czech Republic, whose biggest trading partner is Germany.
Zeman accused Schwarzenberg of backing the cause of some three million ethnic Germans, known as Sudeten Germans, who were expelled from then-Czechoslovakia after World War Two.
Schwarzenberg has said that in today's world, the expulsion could be seen as a war crime, but denied allegations he would open the door for demands to return confiscated property.
Klaus backed Zeman in the vote, saying he wanted a president who had lived in the country all his life, unlike Schwarzenberg, whose family has large land holdings in Austria where he lived in exile during the 1948-1989 communist rule.
Schwarzenberg said the election was won by lies.
"The difference of 10 percentage points was the result of this kind of campaign," he said. "It is impossible to defend against certain type of bad-mouthing."