Koulibaly joined Chelsea from Serie A side Napoli last month and gained the respect of fans after he called Terry to ask if he could take the number worn by him in his nearly two-decade-long career at the London club.
"It was a nervous call. I was waiting to sign my contract and it happened the day before. I wanted to know which number was free, but the manager (Thomas Tuchel) didn't tell me 26 was," Koulibaly told reporters.
"John left it in 2017 and nobody took it. I wondered if it was retired or nobody wanted it... I called John but he didn't believe me at first, he put down the phone."
Koulibaly said Terry spoke to him later after confirming with the club that the call was not a prank.
"I asked him respectfully, I know it's an important number for him but it's important for me too as I wore it in Napoli," Koulibaly said.
"I know what he did for the club, the supporters and the city so I preferred to ask him and he gave me his answer and I was happy. I wanted to tell everyone, as I didn't want to be disrespectful to a club legend."
Koulibaly said he hoped to have a successful spell at Chelsea.
"I'm 31 but I know that I can give some years here, give everything for the club," he said. "They have confidence in me so I want to show that they made the right decision with me. I want to help this team of fantastic players and supporters."
The defender said however that he would need time to adapt to English football.
"I cannot be the player everybody is waiting for in my first game, I'll take my time," he said. "In two or three months I'll be that player, but at the moment, I'm taking things easy, game by game."
Chelsea begin the new campaign with a trip to Everton on Saturday.
