Jérôme Cahuzac stepped down on Tuesday after prosecutors announced there would be a full criminal inquiry into allegations that he had an undeclared UBS bank account in Switzerland.
The 60-year-old former cosmetic surgeon is also suspected of moving assets to Singapore in order to hide them from the tax authorities.
Cahuzac said he had resigned to ensure that his case did not interfere with or overshadow the functioning of the government and reiterated he was innocent.
The magistrates in charge of the investigation, Roger Le Loire and Renaud Van Ruymbeke, will be seeking the cooperation of the Swiss and Singaporean authorities as they try to establish whether Cahuzac had accounts in either country.
Although politicians across the political spectrum insisted the former minister should be presumed innocent, there was widespread acknowledgement that his position in the government had been made untenable by the announcement of the probe.
Cahuzac had been charged with overseeing the complex array of spending cuts and tax rises required if France is to meet its commitments to cutting its deficit in line with EU targets.
"You don't replace Jérôme Cahuzac, you can only succeed him," the new budget minister, Bernard Cazeneuve, said on Wednesday in a tribute that underlined his outgoing colleague's status as a heavyweight member of President François Hollande's cabinet.
But aides to Hollande indicated that the president had been left with no option but to get rid of the budget minister, particularly as his brief included responsibility for clamping down on exactly the kind of tax dodging that the former minister is accused of.
"It was best to act quickly, before the judicial soap opera became a political one," a source close to Hollande told AFP.
As well as dealing a serious blow to Hollande's administration, Cahuzac's resignation represented a landmark moment in the French media landscape.
The story was broken, in January, by Mediapart, a news website with a hard-hitting style and investigative culture that contrasts with the generally more cautious approach of traditional print and broadcasting media in France.
The announcement of a criminal investigation based on Mediapart's reports is a significant boost to the credibility of the site, acknowledged editor-in-chief Francois Bonnet.
"Our only satisfaction is in seeing the quality of our journalism recognised," Bonnet told AFP.
"For months we have been in the crossfire from a section of our own profession, which has been extremely disagreeable, so it is excellent news that our work has resulted in an independent inquiry, which is what we have asked for since the beginning."
According to Mediapart, Cahuzac held funds in an account with UBS until 2010 and, in breach of French law, failed to declare the account to the French tax authorities.
The existence of the account was reportedly confirmed by Cahuzac himself in a message he left, apparently inadvertently, on the answer machine of a political rival in 2000.
Prosecutors announced on Tuesday that police experts had concluded it was highly likely that the voice on the recording was Cahuzac's.
The former minister has denied ever holding a UBS or any other overseas bank account and dismissed the allegations against him as "slanderous."
Story continues below…