The couple intended to give their daughter four names, with the fourth being J in homage to two great-grandparents called Johanna and Josef, reported news agencies on Tuesday.
But the name was refused by the cantonal registry office.
On Tuesday the Zurich civil court upheld that decision, saying J could be confused with a reference number or abbreviation.
The court rejected the couple's argument that it would be pronounced the English way, Jay, pointing out that in Swiss national languages French and German the letter is pronounced very differently.
Neither did the fact it would be the child's fourth given name hold any sway. In the official Swiss register all given names are accorded equal weight and can be used by the person as their preferred first name.
If the parents really wanted to unite the two great-grandparents' names in one, they could use the name Jo, said the court in its verdict.
Under Swiss law parents may name their children whatever they want so long as it doesn't harm the interests of the child, for example by leading to confusion over to the sex of the child or opening them up to prejudice.
In 2014 another couple in the canton of Bern were ordered to change the name of their newborn son because their chosen name – Jessico – was considered too feminine.