The hosts took the lead three times in the first half after Leverkusen had twice pulled level. One of the goals were scored by Adam Hlozek (20), new signing from Sparta Prague, who recorded his first goal for Bayer.
Elversberg, newly promoted to the third division, refused to give up and bagged a fourth with Kevin Conrad's 74th minute header.
Leverkusen, who had been wasteful up front especially in the first half and with big gaps at the back, only managed to pull a goal back through Patrik Schick in the 89th.
They failed to get past the first round of the competition for the first time in 11 years and they will need to raise their game next week when they face Borussia Dortmund in their Bundesliga opener.
Dortmund booked their German Cup second round ticket on Friday with a dominant 3-0 over third tier 1860 Munich.
