Negotiators from the two sides are in talks to form a coalition government. They said the agreement would involve listed companies and those with a works council.
Three weeks of coalition talks have made modest progress. Both sides are making concessions but the conservatives want their victory in September's election reflected in the deal.
Germany introduced voluntary targets for women in top management positions in 2001, but little changed. In 2011 blue-chip companies agreed to try to boost women on boards, again through voluntary targets.
The centre-left SPD had pushed for 40 percent women on boards in stages from 2021. The conservatives had resisted fixed targets.
"This is an important signal to improve the career chances for women and for greater equality in the labour market," said Manuela Schwesig, who led the talks for the SPD.
Annette Widmann-Mauz, leading the talks for Merkel's conservative Christian Democrats (CDU) and Christian Social Union (CSU), called the agreement a breakthrough for women.
Under the new law, companies unable to appoint women to at least 30 percent of open board seats from 2016 would be required to leave those seats vacant.
"It's a toad that we're going to have to swallow," said CDU parliamentarian Michael Fuchs. "There are companies where that's going to be difficult."
Women currently hold about 12% of corporate board seats, according to German media reports.
Among the 30 largest DAX companies, women have 101 of the 488 board seats, or 22%, according to the DSW, Germany's largest association of private investors.
The European Commission last year proposed requiring companies with more than 250 workers and listed in European Union countries to have 40% of women on their boards by 2020. But Germany and other EU countries resisted. Germany said those rules should be set at the national level.
Norway, which is not an EU member, imposed a 40% quota in 2003, a target reached in 2009. Norwegian companies can be liquidated if they fail to reach the target.