The pipe, which had weakened with age, burst early on Sunday near the organization's headquarters and large amounts of water gushed into the basement, IOC chief Christophe De Kepper told reporters in Lausanne.
In some places, the water stood as high as two metres (6.6 feet), De Kepper said.
Around 20 to 30 percent of the IOC's recent paper archives were damaged, he said.
But De Kepper said none of the organization's historic archives were destroyed since they were housed separately at the IOC museum.
The server rooms in the headquarter basement were meanwhile all flooded, knocking out the building's internet and telephone connections.
Some 150 of the organization's staff would be relocated to another IOC building in the city on Monday, De Kepper said.
Some parts of the headquarters building were not expected to be operational until Wednesday, he said.
De Kepper said it was too early to estimate how much it would cost to repair the damage.
A camping site and several restaurants were also flooded when the pipe burst, and the main highway was closed for several hours early on Sunday, civil protection services said in a statement.
Around 100 people, drawn from the fire brigade, city water and road services, civil protection units and police, rapidly arrived on site and
managed to halt the leak and begin pumping out the flooded buildings.
The flooding at the IOC was not linked to the heavy rain that has hit the canton of Vaud and neighbouring regions in recent days and that has caused some flooding elsewhere.