According to final official results, the "Yes" camp claimed 60 percent support and a victory in 19 of Switzerland's 26 cantons, meeting the two criteria needed for a win.
The government as well as most lawmakers and political parties supported the proposal.
Under it, the grandchildren of immigrants will be able to skip several steps in the lengthy process of securing a Swiss passport, although approval of their citizenship will still not be automatic.
The right-wing Swiss People's Party (SVP), the largest party in Switzerland's parliament, fought against the measure by putting Islam and national identity at the centre of the debate.
Reacting to the defeat, SVP lawmaker Jean-Luc Addor said his side was "alone against everyone in this campaign".
"The problem of Islam, I'm afraid, it will catch up with us in a few years," he told RTS television.
According to a migration department study, fewer than 25,000 people in the country of about eight million currently qualify as third-generation immigrants, a definition meaning they have at least one grandparent who was born in Switzerland or acquired residency.
Nearly 60 percent of that group are Italians, followed by those with origins in the Balkans and Turkey.
During the campaign, Addor warned that third-generation immigrants in Switzerland will increasingly be people "from sub-Saharan Africa, the Horn of Africa, Syria or Afghanistan."
The "No" camp faced heavy criticism over a widely-distributed poster showing a woman staring out from under a black niqab with a tagline urging voters to reject "uncontrolled citizenship".
The SVP was not officially responsible for the poster.
It was commissioned by the Committee Against Facilitated Citizenship, which has several SVP members and was co-chaired by Addor.
Lawmaker Lisa Mazzone, whose Green Party supported the reform, told RTS her side was "pleased it was able to prevail in a campaign where the opponents were completely hysterical".
Sophie Guignard of the Institute of Political Science at the University of Bern told AFP that mainstream politicians and journalists viewed the niqab poster as "a violent attack against Muslims".
But Addor defended it again on Sunday by saying its intent was to "affirm the identity of this country and the need to preserve it."
There are currently an estimated 350,000 Muslims in Switzerland.
Sunday's referendum was part of Switzerland's direct democracy system that includes four votes each year on subjects affecting federal as well as local laws and institutions.
In a 2004 ballot that also saw brazen anti-Muslim messages play a prominent role during the campaign, Switzerland rejected automatic citizenship for third-generation immigrants who were born in the country.
The SVP in 2009 also successfully persuaded voters to approve a ban on new mosque minaret construction.
Ahead of this year's vote the government asserted that most of the people concerned had spent their entire lives in Switzerland and knew no other home, arguing that clearly entitled them to a facilitated citizenship process.
Polls leading up to the vote indicated a win for the "Yes" camp but some analysts said an upset remained possible, noting Swiss political history and recent trends elsewhere in the West.
Experts pointed to the apparent rising fortunes of right-wing parties across Europe along with US President Donald Trump's election win and his travel ban against seven mainly Muslim countries, which has now been rejected by two US federal courts.