Despite their record January transfer window spending of around 300 million pounds, Chelsea have won one of their last 10 matches in all competitions.
Following last week's 1-0 home defeat by bottom-club Southampton, they sit 10th in the Premier League standings, 11 points behind rivals Spurs in fourth, albeit with a game in hand.
"Supporters, rightly so, are upset because we lost at home to Southampton. Supporters care, when they are upset they make they let their feelings be known and we accept that," Potter told reporters ahead of Sunday's league clash with Spurs.
"There's nothing I can say that will make the supporters – if they are against us – with us. The solution is that you have to win football matches.
"If you lose 1-0 at home to Southampton, who are bottom of the league, you expect fans to voice their criticism... Results haven't been good enough, a 1-0 defeat to Southampton isn't good enough, they are entitled to be angry."
Potter added that Chelsea's problems are nothing new and that they began even before the season started, with a poor pre-season.
His comments echo the sentiments of his predecessor Thomas Tuchel, who had expressed concern over the team's lack of commitment during their pre-season tour of the United States.
"Speaking to a few of the experienced guys, they said it was the worst pre-season they've had," Potter said.
"Organisationally, the tour didn't work as well as they'd like. I wasn't there so I can't say.
"We thought we made progress, but the first half against Southampton was below par. The sky can fall. These are some of the facts. Inconvenient facts, but that's the situation."
Chelsea have struggled to score goals in recent weeks, finding the net just once in their last five games in all competitions.
"The quality is there. When you integrate young players, it's not always easy for things to click," Potter said.
"The most difficult part of the game is the attacking part. To get that fluidity, certainty in our attacking play, it's a process.
"We've spoken about it with individuals and as a group as to how to attack against Tottenham. We need to focus on the next game. They are a team that don't give much away, defend deep, we'll have to think about that."
Potter said he will be able to call upon most of his squad for the game against Spurs, except for N'Golo Kante and Christian Pulisic, who "are joining the warm-ups and will be more integrated next week".
