The Swiss national team had hoped to lure Koller to replace Ottmar Hitzfeld, who recently announced he will step down after the World Cup competition, for which Switzerland has qualified.
"The OeFB sealed a further collaboration with Marcel Koller today for at least another two years," the Austrian federation said in a statement on its website.
The contract runs until December 2015 but will be automatically extended for one year if the national team qualifies for the European championships in France in 2016.
"I began this project with the OeFB two years ago and I want to continue with it," Koller said in the statement.
OeFB President Leo Windtner also welcomed the "continuity" afforded by Koller staying on.
Austria, ranked 53rd in the world, briefly held hopes of making it to their first World Cup since 1998, but eventually finished third in Group C behind eternal rivals Germany and Sweden.
Koller's decision leaves Switzerland searching for other candidates to take over the national team.
The 52-year-old former Grasshoppers player began coaching Austria in 2011 after the team failed to qualify for Euro 2012.