South Africa lost four tests in a row prior to a rain-affected draw in Australia earlier this year, though they were on their way to defeat in that match too.
The batting remains the major concern with the side having not scored more than 255 in any of their last 10 innings, which has led to recalls for Aiden Markram and Ryan Rickelton, and a potential first cap for Tony de Zorzi.
"It's the start of a new journey and we start with a clean slate. We want to go out there and play the way we want to play. It’s going to be exciting to see how that looks on the field," Bavuma told reporters on Monday.
"The batting unit (struggled) in the last two series in England and Australia, but those are tough tours, generally ones that separate the good guys from the really good guys.
"The guys who are still here want to man up as the batting unit, we need to score runs to give the bowlers the room to do what they need to do."
South Africa will likely play an extra batsman at number seven with Heinrich Klaasen to keep wicket, and go in with an all pace attack on the bouncy Highveld wicket in Pretoria.
"That will still be decided by the coach (Shukri Conrad). We have resources whether we decide to go with a pace (bowling) attack, or with a spin attack. We can back up whatever tactic we go with."
Bavuma made his test debut against the West Indies in 2014 and admits they are a special opponent for him.
"West Indies was the team I supported growing up. So there is that sentiment when it comes to them. I made 10 runs on my debut so that was nothing to write home about, hopefully this (first game as captain) goes better.
"They play old fashioned cricket. Batters grind it out, bowlers are looking to hit their areas outside off stump. They have guys who can stand up to the challenge.
"Their bowling attack has guys who can get the ball through (quickly), which is something you need in South Africa."
BRATHWAITE BRACED FOR CHALLENGE
West Indies captain Kraigg Brathwaite believes his players must not get ahead of themselves and take each hour as it comes.
The Caribbean side are coming off a 1-0 series win in Zimbabwe earlier this month, but before that were thumped in Australia late last year and will face similar seamer-friendly conditions on their short tour of South Africa.
"It won’t be easy, South Africa are a tough team at home, but we are looking forward to playing some hard days of test cricket," Brathwaite told reporters on Monday.
"It is important we take control of every hour, and not look at the big picture, just go from session to session."
West Indies have lost 12 of their previous 15 tests in South Africa and won just one, though this is the first time they have toured the country in the longest format since 2015. Their only victory was a surprise success in Gqeberha 16 years ago.
Brathwaite, who opens the batting, concedes it will be a big step up from what they faced in Zimbabwe and it will be up to the batters to lay the platform for success against the home side's vaunted pace attack on helpful wickets.
"Zimbabwe isn’t a top team in the world so we are really looking to challenging ourselves here. We need to improve as batters and it is important as openers to set the foundation or the team," he said.
"It looks a good wicket, stats say batters can get runs on this pitch and the spinners will turn it later in the game.
"We have some very good bowlers. As a group we look forward to bowling at any batting line-up in the world. Discipline is very important, if we stick to our plans, the results will take care of themselves."
