The 79-year-old Swiss official, Fifa president for 17 years and only reelected at Fifa's 65th congress in Zurich on Friday, said a special congress would be called as soon as possible to elect a successor.
His resignation is a stunning turnaround coming after he vowed not to step down in the face of widening corruption allegations.
But Blatter told a press conference that the scandal-tainted Fifa needs "profound reconstruction" and that he had "thoroughly reconsidered" his presidency since his reelection.
He added that the vote "does not seem to be supported by everybody in the world of football."
His resignation came less than a week after Swiss police arrested seven Fifa officials, including two vice presidents, at a Zurich hotel.
The arrests were carried out on behalf of US prosecutors who accuse the seven, plus eight other suspects, of involvement in $150 million of bribes.
Blatter's change of heart also came after a New York Times report that his right-hand man, Jérôme Valcke, had approved the payment of $10 million from South Africa to an account controlled by Jack Warner, former head of the North and Central American and Caribbean confederation (Concacaf).
According to US investigators the money was a bribe promised to Warner, from Trinidad and Tobago, and his deputy Chuck Blazer, from the US, to secure the 2010 World Cup for South Africa.
Warner already faces charges from the US Department of Justice.
South Africa has strongly denied any wrongdoing.
And earlier today Fifa issued a statement denying that Valcke, the body's secretary general or any other senior official was involved in the "initiation, approval and implementation" of the transfer of money.
The organization said it has acted as an intermediary between South Africa and a World Cup legacy project to "support the African diaspora in Caribbean countries".
Fifa said that a former finance committee chief who died last year, Argentine Julio Grondona, authorized the payment which went to Jack Warner.
Blatter also faced opposition from Uefa, the Swiss-based European football body, whose chief Michel Platini called on him to resign.
Responding to the Fifa president's announcement in a statement issued on Uefa's website, Platini said: "It was a difficult decision, a brave decision and the right decision."
Asked whether the Frenchman would be a candidate to succeed Blatter, the Uefa press service said there would "no more statements today."
While Platini had called several times for Blatter to resign, he announced last year that he would not be a challenger to Blatter at last week's congress.
However, Platini, who turns 60 later this month, has not ruled out standing in the future.
Blatter has resigned. Can't quite believe it. FIFA always appeared to be such a fine upstanding organisation.— Gary Lineker (@GaryLineker) June 2, 2015