Cavusoglu had planned to stage a rally in Zurich on Sunday to drum up support for Turkish president Erdogan ahead of Turkey's referendum on April 16th.
However last week Zurich authorities requested that the Swiss federal government cancel the visit, saying it posed a security risk.
Bern rejected that request, saying there was “nothing to justify” cancelling it.
Defending the government's stance to broadcaster RTS on Saturday, Swiss foreign minister Didier Burkhalter said it was important to guarantee the fundamental right to freedom of expression, particularly in order to show other countries -- including Turkey -- that they too should guarantee freedom of speech.
Kurdish leaders were allowed to express themselves in a rally last year, he said, therefore Turkish leaders should also be allowed to speak freely too.
But any such rally could be cancelled or adapted at the last minute should the government judge it to pose security concerns, he added.
Despite the government's backing, the Hilton hotel near the Zurich airport, where Cavusoglu had been scheduled to speak, said it would not allow the event to go ahead there out of concern for the safety of its guests.
Turkey then cancelled the trip altogether but did not specify any reason, according to news agencies.
In a statement obtained by news agencies, Zurich's head of security expressed his “great relief” that the visit was finally cancelled by the Turkish authorities, saying that groups opposing the rally had been likely to show up, posing a security risk for all involved.
On Friday another rally supporting the Turkish referendum was cancelled in the Swiss canton of Aargau over security concerns.