"We are investigating several hundred domestic clients of Swiss bank UBS on suspicion of tax evasion," said Bernd Bieniossek, a prosecutor in the western city of Bochum.
Some 50 tax investigators took part in the nationwide raids earlier on Monday, Bieniossek said.
The raids were in connection with data recovered from six computer discs purchased by the German state of North Rhine-Westphalia (NRW) containing information on some 7,000 UBS clients based in Germany suspected of hiding cash from the taxman.
Another public prosecutor in the city of Mannheim said last Thursday they were investigating the German branch of UBS over suspicion of tax evasion.
UBS chief Axel Weber, also a former head of the German central bank, said in a German press interview in August that his bank had "zero tolerance" for tax fraud.
But NRW state premier, Hannelore Kraft, told the Bild mass circulation daily in September the data purchased recently indicated Swiss banks were offering clients tips on how to transfer cash away from Switzerland to Asia before a planned tax deal between the two countries comes into force.
A spokesman for UBS told AFP on Monday that the bank "clearly supports the need for its German clients to conform to the tax laws."
"In 2009, UBS conducted a thorough examination of its cross-border activities and, where it was considered necessary, adapted its rules," the spokesman said.
"Any UBS employee who does not adhere to these rules is punished by measures up to and including dismissal," added the spokesman.
A tax deal between the two countries, aimed at ending such disputes, is to take effect in January 2013 but still needs to be ratified by both parliaments.
The double taxation agreement, signed by ministers earlier this year, would see German citizens with assets parked in Switzerland's notoriously secretive banks paying a tax rate of 26.4 percent on these holdings.
But opposition lawmakers in the German upper house, the Bundesrat, have threatened to block the deal.