But with the army significantly downsizing its presence, Egyptian billionaire Samih Sawiris has jumped in with an offer to buy up the facility and to transform it into a luxury playground for the rich and famous.
"At first they thought I'm crazy but later they offered to let me realise this project," Sawiris said.
Sawiris, whose father founded the Egypt's Orascom, has his work cut out -- he needs to transform the village with a view that is largely bare and rocky, and which is only accessible by the winding Gotthard pass, into an attractive resort.
But the Egyptian believes that he has obtained a "super deal." According to the project's website, the company spent 36 million francs ($43.7 million) to acquire the 1.3 million square metre site.
By 2013, the first five-star hotel as well as several villas are expected to be up in the village located in central Switzerland.
Within the next decade, the site should boast more than 800 rooms in six hotels, 490 apartments in 42 buildings, and 20 to 30 private villas.
Apartments currently being marketed for sale are going for at least one million francs each.
The turf has already been planted on a future 18-hole golf course and construction is ongoing on roads and bridges to improve access.
In addition, there are plans for conference and concert facilities holding up to 600 people, 2,000 parking lots and a sports and leisure centre.
The ski facilities will be modernised and a funicular railway built to hoist people from the village up to the ski runs.
In all, the development should provide employment to 2,500 people.
The village of 1,500 inhabitants hopes that this project will fill the economic gap caused by the army's departure.
Andermatt's ex-mayor Karl Paletti said it was lucky that Sawiris had chosen the spot for his project.
"We are very dependent on the army and with their departure, it was our only chance," he said, pointing out that the construction had halted the exodus of the village's inhabitants as some had managed to find work on the site.
Asked if he was concerned that the village could lose its character with the arrival of the rich and famous, Paletti said: "We will finally have a good mix of population."
Nevertheless, Sawiris had not obtained the land without a fight.
His plans were initially contested by farmers who were against seeing their pastures turned into a golf course but Sawiris had finally managed to secure their accord.
Since then, his company has poured 185 million francs into the development, and costs are expected to rise further, said the Egyptian magnate, who has built holiday resorts across Egypt, the United Arab Emirates, Oman and Morocco.
He explained that he had picked Andermatt because it is a "pretty and small village, which has not been disfigured by urbanisation and nature is very close by."
For now, his bet seems to have begun to pay off.
About 115 million francs worth of homes have already been sold and there are orders worth 22 million francs for others.
Most of the buyers are Swiss, European and Egyptian, according to Orascom data.