The win, thanks to Bambase Conte's early goal and a double from David Mokwa, secured second place in the standings ahead of Paderborn on goal difference, with fans streaming onto the pitch to celebrate.
Paderborn will face VfL Wolfsburg in a two-legged relegation/promotion playoff.
Elversberg suffered heartbreak last season in only their second-ever campaign in the second tier, losing out on promotion in a playoff with Bundesliga side Heidenheim, but now become the smallest town to have a team in the German top flight.
Spiesen-Elversberg has a population of around 13,000 while the club's ground has a capacity of 10,000, and Elversberg have achieved three promotions in the last five seasons in a remarkable rise from the lower leagues.
The club, founded in 1907, were the butt of a German train company's joke last season.
Deutsche Bahn posted an image of a train with only one carriage, saying that was the only special service needed for their playoff with Heidenheim, but Elversberg have had the last laugh.
Heidenheim, meanwhile, will swap places with Elversberg next season following relegation from the Bundesliga.
