According to official results 60.8 percent of voters in the canton rejected the initiative, with only 39.2 percent saying yes.
The canton's support for two languages at primary school level will be a relief for the Swiss federal government, which has long insisted on the point, fearing that if only one language was taught then schools would choose English instead of another Swiss national language.
Aiming in part to prioritize national languages across the country, a 2004 federal language teaching strategy and a 2007 ‘harmonization agreement' – approved by the cantons and the people – decreed that two languages should be taught at primary school, at least one of which should be a Swiss national language.
But citing lack of resources and time, schools in some cantons want to teach just one language, with some saying that one should be English.
Though education policy is generally set at cantonal level, the row was inflamed last year when the federal government said it would intervene if the cantons failed to prioritize national languages.
In Zurich, the clear rejection of the initiative on Sunday was in part due to the fear that English would be booted off the timetable. According to news agencies the Zurich government had indicated that if the people voted ‘yes' to one only language at primary level then that language would be French.
According to an opinion piece in the Neue Zurcher Zeitung, a ‘yes' vote in Zurich would have had “devastating results” since it would have countered the harmonization agreement that Zurich previously signed up to and therefore sparked conflict with the federal government.
Zurich's education minister Silvia Steiner told the media she was “very relieved” at the result and hoped it would give a signal to other cantons to follow suit.
Indeed, Zurich's decision has already caused similar initiatives in other cantons to be dropped.
The parliaments of Schwyz and Nidwalden have now backtracked on the issue, according to news agencies.
However, the subject is still likely to come up elsewhere. Thurgau parliament recently voted to abolish the teaching of French at primary level, and the move will likely be put to a referendum.
The cantons of Lucerne, Graubünden and Basel-Land are also likely to vote on the issue of two languages later this year.
The language debate is primarily an issue in German-speaking areas of Switzerland. In French-speaking Western Switzerland German has an uncontested place on the primary school curriculum.
For some French-speaking commentators the outcome of Zurich's vote is affirmation of the people's commitment to minority languages and the four-language identity of Switzerland, reported Neue Zurcher Zeitung.