His resignation, which has been accepted by the group's board, will be effective on July 15th, LafargeHolcim said in a statement.
Olsen's departure follows an inquiry into the indirect financing by Lafarge of armed groups in Syria, a country torn by civil war, to keep one of its cement plants operational.
"My decision is driven by my conviction that it will contribute to addressing strong tensions that have recently arisen around the Syria case," said Olsen.
"While I was absolutely not involved in, nor even aware of, any wrongdoing I believe my departure will contribute to bringing back serenity to a company that has been exposed for months on this case," he said.
In March, LafargeHolcim admitted that it had resorted to "unacceptable practices" to continue operations at one of its now-closed factories in Syria.
The admission comes after sources close to the case told AFP in January that the French government had filed a legal complaint against Lafarge for buying oil in Syria to power the Jalabiya factory, in violation of sanctions.
The factory closed down before French cement maker Lafarge merged with Switzerland's Holcim in 2015.
The plant, located in northern Syria some 150 kilometres northeast of Aleppo, was finally evacuated in 2014.
Lafarge is suspected of sourcing oil locally to operate the factory in defiance of a 2012 EU ban on purchases of Syrian oil as part of a sanctions package targeting the regime of Syrian President Bashar al-Assad.
LafargeHolcim said on Monday that the search for Olsen's successor would begin "immediately."
Supervisory board chief Beat Hess would take over in the interim.
It's the second time the company has been in the news this year.
In March Paris dumped the cement maker as a supplier for its 'Paris Plage' beach over the company's readiness to supply cement for US President Donald Trump's border wall with Mexico.
"We will do without their services," Paris Mayor Anne Hidalgo's deputy Bruno Julliard told the city council, citing Lafarge's readiness to "work on the nefarious project" of a wall along the US-Mexico border.
The council later backed the decision in a vote.