James, sustained the injury late in the second half of Tuesday's 2-0 victory at AC Milan in the Champions League and was withdrawn after receiving medical treatment on the pitch.
Potter was unable to provide a timeframe for James' recovery but any long-term absence would be a blow for England, with Manchester City right back Kyle Walker also a doubt for the November 20th-December 18th World Cup after undergoing groin surgery.
"Yesterday he felt not too bad, but until you get these things checked you never know," Potter told reporters before Chelsea's Premier League trip to Aston Villa on Sunday.
"Until we get that, there's not too much I can add. It would be speculation on my perspective. Let's wait and see. There's no point looking at a doomsday scenario just yet."
Chelsea's defence had already been weakened by Wesley Fofana sustaining a knee injury against Milan at Stamford Bridge last week and Potter said he was unlikely to return before the World Cup.
Midfielder N'Golo Kante, who has not featured since mid-August due to a hamstring problem, also remains sidelined after suffering a setback in his recovery and will see a consultant at the weekend, Potter said.
Chelsea will travel to Villa Park in fine form, having picked up their fourth successive win across all competitions to remain unbeaten since Potter took charge in September.
"I've been impressed with the guys and their attitudes have been fantastic," Potter said. "(We) had a few games in a short space of time, so we've had to use the group as wisely as possible.
"The main focus is going game to game and progressing. I don't think we're at the limit and there is more to come from the team and individual."
Chelsea are fourth in the league standings with 16 points from eight games, while Steven Gerrard's Villa are 16th with nine points from as many matches.
