The 31-year-old is the third Swiss player to withdraw after Roger Federer and Belinda Bencic who was also scratched from the competition through injury.
"I am very sad," said Wawrinka, the former French and Australian Open champion.
"Going to Rio was close to my heart. After Beijing (2008) and London (2012), I wanted to live these Games in Brazil.
"Sadly, it's not possible. I wish good luck to all the Swiss athletes who will go to Rio and I will support them from distance."
Wawrinka aggravated his back injury in Toronto last week where he reached the semifinals.
He won the Olympic doubles gold medal with Federer at the 2008 Games but lost in the first round of the singles in London to eventual champion Andy Murray.
Big-serving Australian Sam Groth was lined up to replace Wawrinka in the draw.
"Late call-up to join the @AUSOlympicTeam in Rio. Couldn't be more pumped," Groth tweeted.
World number three Federer opted out of the Games last week and shut down his 2016 season to rest a knee injury.
One day later, teenager Bencic, the world number 19, withdrew with a wrist injury.
The tennis, which starts on Saturday, has been hit by a number of pull-outs over injury, fears of the Zika virus or scheduling complaints.
Milos Raonic, Tomas Berdych, Dominic Thiem, Richard Gasquet, John Isner, Feliciano Lopez and Nick Kyrgios are not playing.
The women's singles are stripped of Simona Halep, Victoria Azarenka, who is pregnant, Bencic and Karolina Pliskova.
Maria Sharapova, silver medallist behind Serena Williams in 2012, is serving a two-year doping ban.
The men's doubles was also tainted when defending champions Bob and Mike Bryan opted out, insisting "our family's health is now our top priority".