Among those on board was The Washington Post's Tehran correspondent Jason Rezaian, who had been detained in Iran for nearly 18 months.
"Overwhelmed to greet Jason #Rezaian, his wife, Yegi, mother, Mary + Saeed #Abedini + Amir #Hekmati to #Geneva after safely departing #Iran," senior US diplomat Brett McGurk wrote on Twitter.
An AFP reporter at Geneva airport saw a Swiss air force plane touch down at around 6pm, on the same tarmac where a large, official US aircraft was stationed.
The Swiss department of foreign affairs earlier said that after landing in Switzerland, the freed prisoners, all of whom have dual US-Iranian citizenship, later left for an American base in Germany.
Neither US nor Swiss officials had explicitly named those on board the Swiss-bound plane, but McGurk's tweet confirmed reports that the three freed prisoners were Rezaian, Saeed Abedini, a Christian pastor, and former US Marine Amir Hekmati.
Iran had announced their release on Saturday, just hours before Tehran's historic nuclear deal with world powers was implemented, in exchange for Washington pardoning seven Iranians accused of sanctions-busting or violating
"When Americans are freed, that's something we can all celebrate," US President Barak Obama said in a televised statement from the White House.
But he also sounded a note of caution, adding that the US would continue to have problems with the Iranian government's "destabilizing activities" in the region, including its support for militant groups.
'Fourth prisoner in Tehran'
The Swiss foreign affairs department said the prisoner swap followed 14 months of confidential discussions in Switzerland.
The fourth Iranian-American released as part of the swap, Nosratollah Khosravi-Roodsari, has not yet left Tehran, senior US officials said.
Speaking on condition of anonymity, officials said Khosravi-Roodsari - about whom little is known - elected not to board the plane for Switzerland.
"Those who wished to depart Iran have left," said one official.
Another said "it's his free determination where he wants to go."
A fifth American, Matthew Trevithick, was released in a separate process, a US official has said.
The Post had earlier reported that the flight out of Iran was delayed because Rezaian's mother Mary and wife Yeganeh, who also were on the plane, initially did not appear on the flight manifest.
US Secretary of State John Kerry told reporters travelling with him from Vienna to Washington that Iranian Foreign Minister Javad Zarif resolved the problem.
"Zarif had no question about it," Kerry was quoted as saying.
"It was part of the agreement, it was clearly stated," he said.
"The problem was, one of the guys on the ground, at a military base, didn't have it on the manifest."
Story continues below…
Iranian state television said the seven Iranians - Nader Modanlou, Baharam Mechanic, Khosrow Afghahi, Arash Ghahreman, Tooraj Faridi, Nima Golestaneh, and Ali Saboonchi - "will be freed today".
The family of former US Marine Hekmati, who faced a death sentence as an alleged spy, welcomed the news that he had left Iran.
"It is hard to put into words what our family feels right now.
"But we remain in hopeful anticipation until Amir is in our arms."
Washington Post publisher Frederick Ryan said in a statement: "We are relieved that this 545-day nightmare for Jason and his family is finally over."
The 39-year-old Rezaian, a dual US-Iranian citizen born in California, was detained in Iran on July 22nd 2014.
The Post statement thanked those "around the world who have spoken out on Jason's behalf and against the harsh confinement that was so wrongly imposed upon him."
Iran's ambassador to the United Nations, Gholam Ali Khoshroo, said on Saturday that Switzerland played a "positive role" in the prisoner swap.
The UN nuclear watchdog announced late Saturday that Iran had complied with its side of the July 2015 accord, allowing the lifting of sanctions.