Jack Grealish's 81st-minute goal appeared to have won the game for champions City but Dejan Kulusevski's header in the 90th minute condemned City to a third successive draw.
Guardiola and his side were also furious when Grealish went through alone deep in stoppage time but play was stopped by referee Simon Hooper for a foul on Erling Haaland in the build-up when he had initially allowed the advantage.
After the dust settled on a frenetic encounter, City were left in third place in the standings, three points behind leaders Arsenal and one behind Liverpool. Guardiola had also regained his composure after losing his cool in the closing stages.
His side have now conceded eight goals in their last three Premier League games and they also let in two against RB Leipzig in the Champions League in midweek.
"The performance was quite similar to what we have done lately," Guardiola told a press conference. "The people say you're concerned about goals you concede of course but the chances we concede, no.
"It's not the first time we are in this situation where we are playing good but results don't come."
Of the late controversy, Guardiola refused to blame the decision for his side's failure to win.
"I make mistakes, the players make mistakes," Guardiola said. "It surprised me because when Erling went down if you whistle then it's fine but when he stands up and continues and the ref made the gesture to play on, then after he makes the pass he stops the game.
"After (Haaland) made the pass he stopped the game. I don't want to criticise them. In the touchline sometimes I lose my mind, my gesture was not proper.
"I would say we didn't draw for that. They are happy, Tottenham for the point. We are less happy but the performance is similar to what we have done lately."
Injury-hit Tottenham ended a three-match losing run in the league and manager Ange Postecoglou said he hoped his team were emerging from a tough period which began with a loss to Chelsea.
Injuries and suspensions have played havoc with his side after a superb start to the season, but Sunday's draw at City should send them into Thursday's home clash with West Ham United in good heart.
"I always feel that if you can get through these tough periods in the manner of staying true to the team you want to be then you come out of it stronger, irrespective of the knocks you get along the way," the Australian said.
