The deal negotiated by Berlin and Bern would see Swiss banks deduct taxes from German clients and transfer the tax revenues to Berlin, allowing the clients to remain anonymous.
It was due to take effect in January 2013 but needs to be ratified by both parliaments.
Germany's lower house has already approved it but members of the opposition in the upper house, the Bundesrat, had already threatened to block the deal which they see as being too soft on tax-dodgers.
Finance Minister of western North Rhine-Westphalia state Norbert Walter-Borjans, of the main opposition Social Democrats, told the Bundesrat it was a deal which made "honest taxpayers feel like fools".
But German Finance Minister Wolfgang Schaeuble called for backing for the deal, saying: "The agreement tries to find a better solution for a situation which is unsatisfactory."
Neighbours Germany and Switzerland have been embroiled in a spat over tax since 2010 when German authorities raided branches of Credit Suisse bank in 13 German cities after buying data on suspected tax frauds.
Switzerland reacted angrily, saying the data were stolen in violation of its banking secrecy laws.
As much as 180 billion euros ($222 billion) in German assets are hidden in Switzerland, according to unconfirmed media reports.