Lampre rider Modolo fended off the challenge of stage three winner Peter Sagan and John Degenkolb down the stretch while Mark Cavendish, who won Tuesday's fourth stage, saw his hopes dashed after crashing 300 metres from the finish.
"The man to beat was Peter Sagan," Modolo, who burst clear of the Slovak in the final corner of the 184-kilometre ride from Ossingen to Buren an der Aare, told Swiss television.
"It's the biggest victory of my career," added the Italian, who has raced sparingly since a fall during April's Paris-Roubaix.
Martin, who has worn the leader's yellow jersey since winning the opening time-trial, crossed with the peloton to maintain his six second lead over Dutchman Tom Dumoulin.
Prior to Wednesday's stage Wiggins, the 2012 Tour de France champion, withdrew from the race following his crash a day earlier that left him with grazes and swelling to his right leg.
Wiggins had also been suffering from a mild chest infection.
His disappointing display in Switzerland could well end his chances of riding in next month's Tour de France.
Team Sky are set to announce their nine-man line-up for the Tour, which starts July 5th in Leeds, England and runs until July 27th, following the conclusion of the Swiss race.
Thursday's sixth stage will see riders tackle four climbs, including the ascent of the category one Grand Chaumont, on a 184km trip from Buren an der Aare to French-speaking Delemont.