The Swiss defending title-holder was stunned 7-6 (7/4), 2-6, 6-3 by the world number 70, who is yet to claim his first career title and has only won three matches in total at the Grand Slam tournaments.
Federer, 34, gave a thin smile and a wave to the crowd, which included his idol Rod Laver, as he left a shocked Qi Zhong Stadium, where he lifted the trophy last year and won the 2006 and 2007 Tennis Masters Cup.
"I just think the first round here in Shanghai has always been historically quite difficult, getting used to the conditions and the surface and the balls," said Federer, who saved five match points against Leonardo Mayer in his opening match last year.
"The balls play very different than in other places. Last year I got lucky. This year I didn't. So it's a pity."
Ramos-Vinolas is the lowest-ranked competitor to beat the world number three in more than two years, since he lost to Argentina's 114th-listed Federico Delbonis in the Hamburg semi-finals in July 2013.
Federer, who had a bye in the first round, had played Ramos-Vinolas only once before, in the first round of Wimbledon in 2012, dropping just three games in a 6-1, 6-1, 6-1 dismissal of the Spaniard.
"Before the match I thought I'm not going to win," said the Spaniard.
"But when I saw how I was playing, it was not the same as Wimbledon," he said.
"It was my first Wimbledon. I didn't feel comfortable during the match.
"But here, when I saw that we were very, very similar during the first set, I thought, 'Okay, maybe I can do a good match.'