The deal, which is expected to be signed in coming weeks, could snare up to just short of 1,000 tax cheats over two years.
Swiss Finance Minister Eveline Widmer-Schlumpf said the deal "creates legal certainty and will strengthen the competitiveness and the reputation of Switzerland as a financial centre in the long run."
In Berlin, a German government source said the accord "should mark a real new start in the relationship with Switzerland."
Under the deal, German taxpayers would be given a one-off chance to make an anonymous lump sum tax payment, with the tax rate to vary between 19 and 34 percent of the assets.
Any taxes collected from these voluntary disclosures would be offset against the two billion franc advance payment and refunded to the Swiss banks.
In future, all investment income and capital gains arising from assets held by German taxpayers would also be covered by a withholding tax of 26.375 percent.
To prevent new undeclared funds from entering Swiss banks, German authorities would be able to request information on suspected tax cheats.
The number of requests made is expected to reach between 750 to 999 for two years, said the Swiss ministry, which added that the accord should be in force by 2013.
Germany and Switzerland have locked horns over the small alpine state's banking secrecy rules, which Berlin says help to shield tax cheats.
Tensions between the two boiled over in July 2010, when German authorities raided branches of Credit Suisse in Germany in a tax evasion probe based on information they purchased for a reported 2.5 million euros.
Switzerland angrily condemned the move, saying that the data was stolen in violation of its banking secrecy law.
With Wednesday's deal, the two neighbours agreed to set aside their dispute over the stolen bank client data.
"Germany no longer sees any reason for the purchase of stolen bank client data. Switzerland undertakes to waive criminal prosecution of persons on account of involvement in illegally acquiring bank data," said the Swiss ministry.
In addition, German authorities would drop any plans to prosecute any bank employees who may have helped their clients to hide assets from the taxmen.
The latest dual taxation agreement counts among a series of bilateral deals the Swiss have been signing in order to meet tougher international standards after a post-crisis clampdown on offshore and banking havens.