The podium was the Spaniard's first since he left Ferrari at the end of last year and the team's first in a grand prix that went the full distance since Lance Stroll was third in Azerbaijan in 2017.
George Russell was second for Williams in Belgium in 2021, but that race started behind the safety car and was stopped after three laps due to heavy rain, with only half-points awarded.
Sunday in Baku, in dry conditions with genuine pace, was on merit.
"I cannot describe how happy I am, how good this feels. This is even better than my first ever podium that I did," said Sainz, who started on the front row alongside Red Bull's race winner Max Verstappen.
"We've been fighting hard all year and finally today, we just proved that when we have the speed - we've had it all year - and everything comes together, we can do some amazing things together.
"Today we nailed the race. Not one mistake, and we managed to beat a lot of cars that yesterday I wasn't expecting to beat."
The 15 points Sainz scored were one less than his entire haul from the previous 16 races and kept former champions Williams firmly fifth in the standings despite teammate Alex Albon failing to score.
Williams have now scored 101 points, more than they have managed in the previous seven seasons combined and Sainz said the podium vindicated his decision to join the team after leaving Ferrari.
He now has his first podium for Williams before his Ferrari replacement, seven-time world champion Lewis Hamilton, has managed to do the same at the Italian team.
"I think we've proven to everyone the massive step that we took compared to last year. We are on the rise; we are in the right direction," said Sainz.
"We've had a lot of bad luck, a lot of incidents - very difficult to convert all that pace into results. But now I understand why it all happened, because the first podium needed to come like this.
"It's just life, you know? Life just sometimes brings you those bad moments to give you a very nice one, and this stays much better than any other thing that I was expecting."
Sainz said getting Williams back to winning races was his "life project".
"This is everything that I care about and I will put the next few years of my life, all my effort, to doing that," he added.
Team boss James Vowles said it had been an "exceptional drive" by the Spaniard and hailed a "reset point".
"He needs this and you'll see now the positive momentum it generates," he added.
