The decision, taken by the 15 members of the IOC executive board in the Vaud capital leaves the sport grappling against seven other disciplines for inclusion at the Games, the location of which will be decided later this year.
"It's a real shock. Wrestling was not on the radar," an IOC source told AFP.
"It was a very close vote between wrestling and modern pentathlon, maybe one or two votes separating them. The trouble was while modern pentathlon and taekwondo did effective lobbying, wrestling thought they were safe and did none at all."
The loss of wrestling, which will remain on the programme for the 2016 Olympics in Rio de Janeiro, sees the likely disappearance of one of the few sports that survived from the ancient Games to those founded in the modern era by Baron Pierre de Coubertin.
It first appeared in 708 BC and has only ever not appeared at an Olympic Games in 1900.
The sport, which includes freestyle and Greco-Roman wrestling, still has a slim chance of regaining its berth as all IOC members at the body's congress vote on the decision in Buenos Aires in September.
Going against the executive committee vote would be unlikely but not impossible if some IOC members not on the board vote against the decision in anger at being presented with a virtual fait accompli.
Also lobbying for inclusion in 2020 are squash, roller sports -- which both failed in previous bids -- softball/baseball, karate, wushu, which is popular at the Asian Games, the water sport wakeboard and sport climbing.
Squash and karate are generally seen to be leading the race to win the sole spot for the 2020 Games which will be held in either Tokyo, Madrid or Istanbul.
Softball and baseball, which were ditched as Olympic sports for the 2012 Games in favour of golf and rugby sevens, have united this time round to regain a spot and will learn their fate at a meeting in St Petersburg, Russia, in May.
The survival of modern pentathlon is a major coup for Juan Antonio Samaranch Junior, vice-president of the sport's governing body and a member of the IOC executive board, who is also a prominent member of Madrid's bid for the 2020 Games.
The discipline had generally been seen to be at risk as it is expensive to compete in and does not attract a broad range of nations entering.
Invented by de Coubertin, it first appeared at the 1912 Games and consists of five disciplines: fencing, horse riding, swimming, running and shooting.
The idea behind it stemmed from the skills required to be a 19th century cavalry officer.
One of its most famous exponents was legendary US Second World War General George Patton, who finished fifth in the 1912 Games.