In one of the highest quality singles matches ever seen in the 29-year history of the event, Federer showed plenty of the class which has won him a record 17 Grand Slams, but couldn't quite hold off the bold 19-year-old German.
Despite the result, Switzerland kept their hopes of reaching Saturday's final alive with a 2-1 win over Germany -- Belinda Bencic beating Andrea Petkovic in the women's singles and then teaming with Federer to win the deciding mixed doubles.
The men's singles was a classic battle between the old and the new, with Federer left to rue his failure to serve out the first set when leading 5-3.
In the end, two booming first serves from Zverev, the first teenager to make the top 20 since Novak Djokovic in 2006, closed out the match in front of a record Perth tennis crowd of 13,785.
He took the match ultimately decided by three tie-breakers, 7-6 (7/1), 6-7 (4/7), 7-6 (7/4) in two hours and 30 minutes.
It was Zverev's second straight win over the Swiss champ, having beaten him in Germany last June, and the result also mirrored that of an unofficial match on the practice court on New Year's Eve.
Federer was upbeat despite his defeat to a player he believes will be a star for the next decade.
"Who really cares ... it was good to play for two and a half hours," he said.
It was only the 35-year-old's second match since a six-month layoff due to knee and back injuries following his win over England's Dan Evans on Monday.
But Federer -- who is out of the top 10 for the first time since late 2002 -- said he was on the right track for the upcoming Australian Open.
"I am very happy, I have played five good sets so far," he said.
"I am very pleased, practice has gone great, the matches have been a lot of fun and I am playing good tennis."
Wawrinka on winning form
Meanwhile in Brisbane, reigning US Open champion Stan Wawrinka was in impressive form as he saw off Serbia's Viktor Troicki in straight sets to move into the quarterfinals of the Brisbane International on Wednesday.
In a high quality match, the Swiss world number four took just under 90 minutes to defeat Troicki 7-6 (7/5), 6-4.
Wawrinka, 31, has traditionally played in Chennai at this time of year but said he wanted to change his Australian Open preparations.
"I'm quite happy with my level to start the year," he said.
"It's always tough to play against Troicki, and I'm very happy for the first match.
"I'm just enjoying a new tournament that I never played, that I'm playing for the first time.
"New facility, new court, new fans, new city, and that's what make the difference. Especially at 31 years old, when you've been on the tour for so long, to play some new tournaments, it makes you a little bit fresher mentally, also."
Wawrinka will now take on unseeded Briton Kyle Edmund in the quarterfinals of the season-opening event.