The Swiss fourth seed dropped the opening set but he recovered and was cruising to victory before Troicki forced him into a fourth-set tiebreaker.
Wawrinka needed three match points before completing a 3-6, 6-2, 6-2, 7-6 (9/7) victory over the 29th-seeded Serb in two hours, 32 minutes on Rod Laver Arena.
"It was close. Tough tiebreak for sure. Served twice for the match. Doesn't happen often," Wawrinka said.
"But it's not a big deal. At the end I won in four sets. I'm still here. I know I'm playing well, playing better and better."
Wawrinka maintained his 100 percent record against Troicki, winning all eight of their matches with the loss of only three sets to the Serb.
"It's never easy to play against him. I had some tough battles, like today, four sets," he said.
"But in general, I enjoy playing his game. I can control quite well the game. If I serve well, I know I have a big chance."
The Swiss broke Troicki's service seven times and gave up his own serve five times.
Wawrinka, who beat Rafael Nadal in the 2014 Australian Open final, will face Andreas Seppi in the last 16 after the experienced Italian got past Belgium's Steve Darcis in four sets.
Wawarinka said he felt confident in his game heading into the business end of the tournament.
"I know I'm playing really well on the practice court. I playing a good part of the match today okay," he said.
"But again, what's important is to keep winning.
"It doesn't matter the way you're playing, especially at the beginning of a Grand Slam. But I just know that my level is there, so I'm not worried."
It was the fifth straight year Wawrinka has reached the fourth round in Melbourne, and comes after earlier wins over Slovak Martin Klizan and American Steve Johnson.
Last year as the fourth seed, Wawrinka fell to Milos Raonic over five sets in the round of 16.
Wawrinka is chasing his fourth Grand Slam title after his other wins at the 2015 French Open and last year's US Open.