Hot off his victory at the Swiss Indoors in Basel at the weekend Federer downed Chardy 7-6, 6-7, 6-4 at the Bercy indoor stadium.
But it took two hours and 31 minutes to seal the win after Federer lost the second set.
Federer, the 2011 Paris champion, showed nerves of steel as he saved four sets points in the opening frame before wearing down the plucky Chardy.
The Swiss world number two was pushed all the way and had two match points in the second set, before eventually grinding down his opponent.
The 17-time Grand Slam winner leaned on his wealth of experience as Chardy, the world number 30, produced some inspired tennis in front of a raucous Bercy crowd.
"It was a very, very difficult match," said Federer.
"I had two match points in the second set and was a bit worried but I tried to keep coming forward and I'm very, very satisfied," he said.
"He's got a great forehand and a great serve but I played well and I'm very happy, albeit a bit tired."
Earlier, fellow countryman Stan Wawrinka advanced to the third round of the tournament, along with other winners Andy Murray, David Ferrer, Grigor Dimitrov and Milos Raonic.
Wawrinka, world number four, snapped a three-game losing streak as he qualified for the third round with a 6-4, 7-6 (8/6) victory over Dominic Thiem.
"I'm trying to find my confidence, I have some victories but I know that my level of game is there," The Lausanne native said.
"What I still miss is winning matches."