Bach, 59, replaces Belgian Jacques Rogge, 71, who is stepping aside after 12 years at the head of the IOC.
The German, a gold medal winner in fencing at the 1976 Montreal Olympics, was elected after two rounds at the 125th session of the IOC in Buenos Aires, Argentina.
Bach was chosen over five other candidates, including Switzerland's Denis Oswald, a lawyer and professor at the University of Neuchâtel who was critical of the German's candidacy.
“I want to lead the IOC according to my motto ‘unity in diversity,’” Bach told the IOC delegates following his election.
“This means I will do my very best to balance all the different interests of stakeholders of the Olympic movement."
IOC vice-president since 2000, he likened the job of president to being a conductor of an orchestra.
Before being elected Bach said his first priority would be to ensure the Sochi 2014 Winter Games in Russia get under way without problems.
The Games there have faced delays, budget overruns and concerns about the climate being too warm for winter sports.
Among other ideas, Bach proposed creating an Olympic TV Channel to boost the presence of Olympic sports between the Games, held every four years.
Swiss rival Oswald, director of the International Centre for Sports Studies, who won a bronze medal in rowing at the 1968 Olympics, said Bach has taken on board some of the criticism aimed at him during the campaign.
Oswald, who finished fourth in the election with just five votes, had launched an astonishing attack on Bach on Monday, accusing him of using his position to advance the interests of companies he was linked to, especially with Kuwait.
The German has a close friendship with Kuwaiti IOC member Sheikh Ahmad al-Sabah, to the point that an Argentinian newspaper posted on its front page last week a cartoon depicting a grinning Sheikh wearing a t-shirt with Bach's face on it and with his thumb raised in the air.
However, Oswald — the respected long-time president of the Rowing Federation — said he had the feeling Bach had taken on board his remarks.
"I place my confidence in him that he accepts there were problems, I am not the only one to have made these observations," he told AFP.
"However, we can work well together."
Oswald, who said on Monday his values were not those shared by Bach, said that the world of the IOC had not changed radically with Bach's election.
"Each president is different, the trajectory has changed a little bit,” he said.
“It is an evolution rather than a revolution."