The Swedes followed up their opening 5-1 win over Tunisia with a loss by the same scoreline to the Dutch to sit in third in Group F, one point behind both the Netherlands and Japan.
"We can't defend in the way that we did, we have to improve that situation," Potter told reporters.
"Japan are strong on the wings and they're strong through the middle as well, so we meet a very good team and we have to make sure that we are better than we were in that respect."
The Japanese are unbeaten in both matches having twice battled back from a goal down to draw with the Netherlands before handing the Tunisians a 4-0 defeat that eliminated the African side.
Japan a collective unit
Daichi Kamada has scored in both matches for Hajime Moriyasu's side while Ayase Ueda put in a commanding performance against Tunisia, scoring twice and setting up a third to highlight the quality throughout the Japan team.
"Whoever plays offers a threat, it's not about individuals with the Japanese team," said Potter.
"I think their biggest strength is collectively. You don't know who's going to play, you can't second-guess their position. They have a collective threat in attack, a collective defence and that's something we have to be very, very aware of."
The Swedes carry a significant attacking threat of their own in the shape of Alexander Isak and Viktor Gyokeres.
Gyokeres won the Premier League with Arsenal in his first season with the club while Isak struggled with injuries following his move to Liverpool. However, both have already contributed with goal involvements in the tournament so far.
"I don't think it will be fear, I think it's respect," said Potter. "Respect for what they've achieved, in terms of one has won the Premier League and one has had a great career and a big move to a top team.
"Although the season he has had has been disrupted by injury, Alex is still a top, top player."
