Marcel Gaus netted the winner for the hosts deep into second-half stoppage time to complete a sensational comeback and trigger wild celebrations from the home crowd as the Bayern players looked on in complete shock.
Bayern, who last lifted the trophy in 2020, had taken a first-half lead through Thomas Muller before Saarbruecken levelled in first-half added time.
"This is a brutal blow for us," said Muller, with his team next facing title rivals Borussia Dortmund in the Bundesliga on Saturday.
"To be eliminated for the third time in the second round of the Cup is not what we strive for. We have to congratulate Saarbruecken for their fight. A bit of luck is also part of it."
"Then we are hit on the break in the 96th minute but we had wasted too many chances before that."
Bayern coach Thomas Tuchel made several changes to the team that crushed Darmstadt 98 8-0 on Saturday, leaving top scorer Harry Kane on the bench along with Kingsley Coman and Jamal Musiala while giving 20-year-old Frans Kratzig his first start.
The game had been in doubt due to heavy rain but it was the hosts who troubled Bayern more early in the match before the visitors opened their account with their first chance when Kratzig set up Muller in the 15th minute.
Bayern also had to take defender Matthias de Ligt off midway through the first half with the Dutch international suffering a knee injury.
It got worse before the break for Bayern as defender Kim Min-jae's pass was intercepted and Patrick Sontheimer tapped in from close range.
The Bavarians struggled to convert chances and paid the price when the hosts launched a quick break and a cutback from the right sailed across the entire box past three Bayern players before Gaus thundered in the winner with a low left-foot drive in the sixth minute of added time.
