The IOC announced the decision after examining evidence of state-sponsored doping over several years that reached its zenith at the 2014 Winter Olympics in Sochi, Russia.
The IOC also banned Russia's Deputy Prime Minister Vitaly Mutko for life over his involvement in the scandal -- he was the Russian Sports Minister during the Sochi Games.
That will raise questions as to whether Mutko can continue in his role as head of the organizing committee for the 2018 football World Cup, which Russia will host.
Mutko, who was also banned from the Rio 2016 Summer Games, had been implicated in the WADA (World Anti-Doping Agency)-commissioned McLaren report.
Nations have in the past been barred from taking part in the Olympics -- notably South Africa during the apartheid years -- but none has ever been handed a blanket suspension over doping.
Russian athletes, however, would be able to take part in the Games, the IOC said, as independent competitors "under the Olympic flag".
"This was an unprecedented attack on the integrity of the Olympic Games and sport," said IOC president Thomas Bach in a statement.
"The IOC... has issued proportional sanctions for this systemic manipulation while protecting the clean athletes.
"This should draw a line under this damaging episode and serve as a catalyst for a more effective anti-doping system led by WADA."
The IOC's decision to choose a more moderate path, instead of a blanket ban, does offer some Russian athletes a path to competing in the Games in Pyeongchang, South Korea -- although that will be by invitation only and dependent on a stringent testing programme.
"The IOC, at its absolute discretion, will ultimately determine the athletes to be invited from the list," said the IOC's statement, ruling out all those to have previously committed a doping violation.
The US Olympic Committee praised the IOC's "strong and principled decision.
"There were no perfect options, but this decision will clearly make it less likely that this ever happens again," it said.
Those athletes who do go to the Games, which start on February 9th, will participate under the name "Olympic Athlete from Russia".
They will compete with a uniform carrying that name, while the Olympic anthem -- and not the Russian one -- will be played at any medal ceremony.
For Grigory Rodchenkov, the former Russian laboratory chief and whistleblower who lifted the lid on the cheating scheme, the IOC's action was a needed step to clean up the Olympic movement.
"It was the most elaborate and sophisticated doping system in the history of sports. If it did not carry the most significant sanction it would simply have emboldened Russia and other countries who don't respect the rules", Rodchenkov's lawyer, Jim Walden, told reporters on a conference call.
Russian officials have previously met doping accusations with defiance.
Mutko has said the allegations were an attempt "to create an image of an axis of evil" against his country while Putin has warned that a Russia ban would cause "serious harm to the Olympic movement".
He said forcing Russian athletes to compete under a neutral flag would amount to a national "humiliation."
That has fuelled speculation that Moscow would instruct its athletes to boycott the compromise solution decided by the IOC.
"An Olympic boycott has never achieved anything," Bach said, insisting that given the window left open for clean athletes to compete, a boycott was unwarranted.
But the IOC expulsion sparked immediate outrage in Russia.
Pyotr Tolstoy, deputy speaker of the Russian parliament's lower house, the State Duma, has already called for a boytcott.
"They are humiliating the whole of Russia through the absence of its flag and anthem," he said in televised remarks.
The president of Russia's Bobsleigh Federation, Alexander Zubkov told Russian TV that the IOC decision was a "humiliation."
"This is a punch in the stomach", he said.
The IOC's decision comes just days after the draw for next year's World Cup, which Moscow hopes will elevate the nation's status as a sporting superpower.
Russia have been stripped of 11 of their 33 Sochi medals for cheating, meaning they have lost their position at the top of the medals table to Norway.
Last month, athletics' ruling body the International Association of Athletics Federations also maintained its two-year suspension of Russia from the sport, imposed over doping claims..