The four parents of the two families were aged from 27 to 33. An infant from one family and a child of school age from the other family died in the fire.
Three children from the same two families remained in hospital, police said in a statement.
Some of the children were said to go to local kindergartens.
Four of the apartments in the building were rented out by the city of Solothurn to house asylum seekers, the city confirmed on Tuesday. Eleven people lived in these four apartments.
The Ethiopian family had seen their bid for asylum in Switzerland rejected, the head of social services in Solothurn, Claudia Hänzi, told local daily the Solothurner Zeitung.
The residence status of the family from Eritrea is not known.
Hänzi said the city only rented dwellings for asylum seekers that had the necessary safety infrastructure and that the families housed there must be capable of living independently.
She stressed, too, that professional teams were in regular contact with all residents to provide assistance.
Initial investigations suggest Monday's fire was caused by a cigarette. One person was arrested and remains in custody, regional daily the Berner Zeitung reported on Tuesday.
Some 20 people were in the apartment building when the fire broke out. Witness described horrific scenes with people stuck in the building screaming.