In a game of two contrasting halves, Youssef Chermiti's spectacular first-half double looked to have put Rangers in control before Celtic roared back after the break to level the score and stay two points off Rangers with a game in hand.
"At the moment, the feeling is more disappointment because you dropped two points after a 2-0 lead," Rangers coach Danny Rohl told BBC.
"It's always a little bit lucky for a team to get a late penalty. This is normal, I would say it's the same when we get a penalty late in the game."
Chermiti produced a stunning overhead kick from Andreas Skov Olsen's cross after eight minutes to give Rangers the perfect start.
Celtic's defensive frailties were exposed again in the 26th minute when a simple ball in behind caused panic, and Chermiti pounced, nipping the ball away from defender Julian Araujo before slipping it past the goalkeeper to make it 2-0.
But the visitors were transformed after half-time and pulled one back in the 56th minute when Benjamin Nygren picked out Kieran Tierney near the six-yard box and the Scotland international nodded beyond Jack Butland to spark hope.
"When we came out for the second half, we were a totally different team. We're unlucky not to get all the points in the end," Tierney told Sky Sports.
"We let ourselves down first half, it wasn't how we wanted to play. Second half, we showed the character. We're here for this ongoing fight."
Rangers had Butland to thank as the keeper produced a string of saves, but Celtic's persistence paid off when a VAR check ruled that Dujon Sterling had handled the ball in the box, and a penalty was awarded late in the game.
Hatate stepped up to the spot in the 91st minute, and Butland saved twice amid the mayhem, but the Japanese midfielder forced home at the third attempt to complete a remarkable comeback and share the spoils.
"It was obviously a great game. We were well and truly second best in the first half, for almost all of it. They got off to a great start," Celtic boss Martin O'Neill told the BBC.
"The second half belonged to us, and we got an equaliser which we thoroughly deserved.
"If you're 2-0 down at half-time, you have to question yourself. I spend my life questioning myself, to tell you the truth. But anyway, we turned it around second half."
Catch up on all the match stats here.
