"We are investigating officials of the bank whose names are still not known, and customers," Peter Lintz, the public prosecutor in the city of Mannheim, said on Thursday.
"It concerns the suspicion of aiding tax evasion," Lintz said.
"Staff of the bank are suspected of having helped customers take money from Germany to Switzerland," he said, confirming a report in the Stuttgarter Nachrichten newspaper.
During a search of the Frankfurt premises of the bank in May, evidence, primarily electronic data, was seized and is currently being examined, he said.
UBS chief Axel Weber said in a German press interview in August that his bank had "zero tolerance" for tax fraud.
Germany and Switzerland have drawn up a tax deal aimed at ending spats over alleged tax evasion but the treaty is currently facing opposition by some lawmakers in the German upper house of parliament.