Final results showed that almost 62 percent of the electorate shot down a plan pushed by left-leaning parties.
Proponents of the proposed single public health insurance system claim the current system is busting the budgets of ordinary residents.
But going public would have been a seismic shift for a country whose health system is often hailed abroad as a model of efficiency, but is a growing source of frustration at home because of soaring costs.
"Over the past 20 years in Switzerland, health costs have grown 80 percent and insurance premiums 125 percent," ophthalmologist Michel Matter told AFP.
"This is not possible anymore," said Matter, who heads the Geneva Physicians Association, which backs calls to scrap the current system.
"It has to change."
Campaigners who championed the push for a publicly run insurance scheme have said it is the only way to rein in rising premiums and guarantee they are used efficiently and transparently.
Sunday's referendum came after reformers mustered more than the 100,000 signatures required to hold a popular vote, a regular feature of Switzerland's direct democracy.
The rejection of the plan by nearly two-thirds of voters is a major blow for pro-reform campaigners, given that recent polls had shown the No vote was likely to be 54 percent.
In a 2007 referendum, 71 percent rejected similar reforms.
The current system, which was used as a model for US President Barack Obama's controversial healthcare reform, requires that every resident in the wealthy nation of eight million hold basic health insurance and offers freedom of choice among the 61 companies competing for customers.
In a country where the average monthly net salary is 4,950 Swiss francs (4,100 euros, $5,268), health premiums are around 400 francs per adult per month.
That does not include out-of-pocket spending on treatment such as dental care, not covered by basic insurance.
Premiums vary by insurer, age and region of residence, and clients can cut them by opting for an annual deductible — a sum they pay from their own pockets — of up to 2,500 francs.
Figures issued by the insurance industry last week showed that premiums would rise by four percent in 2015, far ahead of overall consumer price inflation.
Critics say the current system is unfair because basic coverage costs a millionaire no more than it does a low-paid worker.
Studies show that almost one-fifth of those on low incomes have skipped at least one monthly payment in a country where rents and retail prices are among Europe's highest.
The reformers also allege that insurers have too much political clout, with research showing that 14 percent of lawmakers have links to health firms or the sector's lobby groups.
But for Switzerland's cross-party government and its right- and centre-dominated parliament, the current system has proven its mettle and is debt-free, unlike the health services of France, Italy or Britain.
Supporters of the status quo argue that higher premiums are inevitable given an ageing population and costly cutting-edge medical care, and say shifting to a public system would generate few savings.