The upset loss, in just over two hours, came less than two weeks after the 32-year-old Swiss player became father to his second set of twins.
The fourth-seeded Basel native, who has never won the Rome title, decided to come and have a go on the clay of the Foro Italico anyway after consulting with wife Mirka and his team.
With him in Rome was part-time coaching consultant Stefan Edberg.
Conditions were brutal for a match played amid great bursts of red clay blowing regularly through the air.
Federer swept the opening set but began to fade in the second as a charged-up Chardy, ranked 47th in the world, made his move.
The Swiss 17-time Grand Slam player, who skipped last week's Madrid Masters to take paternity leave, said on Tuesday that while he hoped to win the match against Chardy “at the moment I really have other priorities”.
At a post-game press conference Federer admitted to feeling angry and disappointed.
"But then I was like: 'OK, let's just do the press conference and go home; let's not waste too much more time,' he said.
"I'm happy," Federer said.
"I'm playing well, so I always see things quite positive – but surely it wasn't fun leaving the court . . ."
Story continues below…
In other play, Andy Murray took revenge for a loss a year ago as he defeated Marcel Granollers to open with a 6-2, 7-5 victory into the third round.
The Spaniard had benefitted in 2013 when the Scot had to quit their second-round contest with the back pains which eventually drove him to surgery last autumn.
The match was also played in swirling dustbowl conditions which bothered both players throughout.