Rossi told the www.formula1.com website at this month's Miami Grand Prix that he would make changes before the end of the season if performance does not improve, and "the buck stops with Otmar".
Alpine finished fourth overall last season but are currently sixth, level on points with fifth-placed McLaren but behind on placings.
The situation has been made worse by Spaniard Fernando Alonso, who left Alpine at the end of last year, standing on the podium four times with Aston Martin who have gone from seventh to second overall.
Australian Oscar Piastri, the former Alpine reserve who left after a contract tug-of-war last season, is now racing for McLaren.
"We're working hard to make sure that we deliver on improving this year's car the best we can," Szafnauer told reporters.
"I think we're not happy because we're not Red Bull. However, within our immediate competition, we made gains on both Ferrari and Mercedes, and the outlier this year is Aston, going from seventh to be the second, third fastest car.
"So, we hit most of our targets, not all of them over the winter, and for us to hit all of them we have to make some changes within the organisation and those changes are coming."
Szafnauer, who joined Alpine last year from Aston Martin, said he had no warning about Rossi's comments and pointed out that the changes were already being made before them.
The Formula One veteran said he had spent the first six months or more assessing the team structure and systems and now had a good understanding of the situation and plans were in place.
He said budget cap restrictions limited the amount of wind tunnel time available.
"It's not a matter of working harder or working more, like it was in the past. You know, I remember the days of Brawn when I was there, we were running three tunnels. You can't run three tunnels any more," he added.
"So it's not a matter of quantity. It's a matter of quality, and getting the right quality takes time, and that's people. So we've got the plans in place, we're talking to the right people. It just takes time."
