Fabrice Anthamatten, 39, was sentenced to 10 years in jail for raping a woman and was close to serving the end of his penalty at Champ-Dollon prison, according to reports.
But he went missing after 11am when he was last seen in Geneva in the presence of Adeline Morel, a 35-year-old sociotherapist, who was in the process of driving him, alone in a car, to an equestrian therapy session, said police.
The pair did not show up for the session and failed to return to the Champ-Dollon prison, located in the canton of Geneva.
The therapist, meanwhile, has not been responding to her mobile phone since Thursday morning.
Geneva and Vaud cantonal police regard the disappearance as worrying and have issued an appeal for help from the public.
The prisoner and therapist were travelling in a white Citröen Berlingo with licence plates GE 719 149.
Police, who issued photos of the pair, described Anthamatten as a caucasian, 181 centimetres tall, with shaved head and glasses.
He was wearing a blue K-way jacket, a wool-skin jacket turned inside out, brown trousers with multiple pockets and a black cap with ear flaps.
Police said Adeline Morel is 170 centimetres tall, of a normal build and with untied hair.
She was wearing trousers and a long beige jacket over a pullover and was carrying a handbag.
Police say anyone with information about the missing pair or the vehicle they were in should phone 0800 800 844 or contact the nearest cantonal police office in Geneva or Vaud.