The find was made by archaeologists during building work on the site of the the Arsenaux cultural centre in the city, Sion authorities said in a statement on Thursday.
The archaeologists uncovered evidence of human habitation, including graves and the remains of houses, which date back as far as the Neolithic period (5,500-4,800 BC).
Construction work at the site – intended to be a new centre for the city's archives – has been suspended until in mid-September so archaeologists can carry out their work.
The most recent remains will be carefully excavated and documented so that archaeologists can concentrate on the oldest, Neolithic, remains.
That period of history isn't well known in Switzerland, said the authorities, but it's thought to be when the first farmers arrived in the Valais from Italy.
“This construction site offers the rare opportunity to uncover remains over a big enough area to be able to trace the development and organization of these first villages,” they said.
Sion already has several important archaeological sites showing it has been inhabited for around 10,000 years.
The area has several tombs and cemeteries dating back to the Neolithic period, which last year were the subject of an important exhibition exploring the history of funeral rites in Switzerland.
Construction work at les Arsenaux will resume at the end of the summer unless another important find is made, added the statement.