The Swiss Broadcasting Corporation reported that 50.34 percent of the electorate favoured the initiative, with a majority of citizens in German-speaking and Italian-speaking cantons in support, and most from French-speaking cantons opposed.
The decision, obligating Switzerland to set its own quotas for immigrants from the EU, puts Switzerland on a collision path with the union, which has warned the mountain country cannot pick and choose its bilateral relations.
Swiss President Didier Burkhalter said the federal government would examine in the next few weeks ways to "rework" relations with the European single market, Switzerland's largest trading partner.
But Burkhalter underlined that the freedom of movement accord, and other agreements with the EU, would remain in place until a new deal is put in place.
The initiative backed by Swiss voters binds the government to renegotiate the labour market agreement within three years.
Voters in a majority of cantons (17) overall approved the proposal "against massive immigration", tearing up the country's commitment to the freedom of movement of people deal known as the Shengen agreement..
Polling agency gfs.bern institute said its surveys showed that 50.4 percent of Swiss voters backed the SVP initiative, which was opposed by the government, most parties and the business community.
Neutral Switzerland is not in the EU but is ringed by countries in the 28-nation bloc.
Since 2007, residents from 15 countries in the EU have enjoyed an equal footing with locals on the job market of Switzerland, a country of 8.1 million.
The Swiss later expanded the freedom of movement agreement to eight other European countries in 2011.
That is part of a raft of deals signed with the EU in 1999 after five years of talks, approved by Swiss voters in a 2000 referendum and then phased in.
But the referendum coalition helmed by the SVP — the largest party in Switzerland's parliament — argued that opening the door fully to EU citizens was a huge mistake.
Supporters of the initiative said that the recent rise in the number of foreigners — who now account for almost quarter of the population — was feeding problems ranging from housing shortages to transportation gridlock.
"The people have taken back their destiny over immigration," said SVP ideologue Christoph Blocher, while leader Toni Brunner hailed "a turning point in our immigration policy".
The government, most political parties and the Swiss business and industry federations warned that ripping up the labour accord would kill off the related economic deals and dent Switzerland's credibility as a partner for the EU.
They also said that slapping restrictions on hiring EU citizens would be a disaster, arguing that the steady stream of foreign labour is a driving force of the wealthy Swiss economy with virtually full employment and an ageing population.
Until 2007, Swiss firms had to clear bureaucratic hurdles before being allowed to recruit a non-resident, with official quotas for foreign employees set down for each business sector.
Brussels has warned that it is not ready to cherry-pick the binding package of deals negotiated painstakingly in the 1990s, seen as a way for Switzerland to enjoy the benefits of access to the EU market without joining the bloc.
The EU capital is already battling internal dissent over its own borderless labour market — west European countries complain about competition from citizens of eastern member states — and has ruled out reopening the issue with Switzerland.
Such arguments did not sway the campaign coalition, which said national sovereignty was at stake.
It argued that the arrival of 80,000 new residents annually in recent years has been an economic and social disaster, and not only because EU citizens have allegedly undercut Swiss workers.
It says that overpopulation has driven up rents, stretched the health and education systems, overloaded the road and rail networks, and eaten into the landscape due to housing construction.
Referenda are the core of Switzerland's system of direct democracy, and the coalition mustered more than 135,000 signatures to force a vote.
Immigration and national identity are traditional headline issues in a country with a long history of drawing foreign workers and some of Europe's toughest rules for obtaining citizenship.
But over recent years, the proportion of foreigners has risen from around one-fifth of the population to roughly a quarter.
The majority of recent immigrants are from neighbouring Germany, Italy and France, as well as Portugal.