Les Bleus almost blew a 19-0 lead after both sides were reduced to 14 men following red cards for Scotland lock Grant Gilchrist and France prop Mohamed Haouas in the opening 13 minutes.
Yet they managed to score four tries to secure a bonus point and deprive their opponents of one, which left both teams on 10 points, level with England and five behind Ireland.
"We have not let that trophy go yet," said Galthie after his team recovered from their 32-19 defeat in Dublin against Ireland in the previous round.
"Two years ago we lost here against Scotland and this time we went for the bonus point and we took the bonus point away from them. It was 5-0."
Galthie was happy his team were back in contention after suffering a confidence-shaking first defeat in 15 tests.
"Getting back on our feet after falling down, that's what we wanted, that was the idea of this game for us," he said.
"We had tough moments but we managed to have them followed by very strong moments and we can be happy about that."
Captain Antoine Dupont, who set up Gael Fickou for a last-minute try, drew positives from the clash.
"There is a lot of satisfaction," he said.
"We were solid in defence against a very dangerous side, so we can be happy about our performance."
Charles Ollivon added: "We suffered but stuck together and we never doubted ourselves."
Scotland, who beat France on their previous visit to Paris two years ago, were left with a bitter taste in their mouths.
"We felt like we could win that. After a really tough start, we got back into it, had ascendancy in the second half. We couldn't get over the line in the last 10 minutes," said centre Huw Jones, who scored two of his team's three tries.
"You've got to put the first 10 minutes behind you, both teams had to adjust to going a man down. We wanted to play, I felt like we had a lot of attack in their half and just weren't converting chances. That was the main thing. We felt like we were chasing the game."
Scotland head coach Gregor Townsend (49) said he was proud of his team.
"We probably produced our best rugby of the tournament and didn't win," he said.
"It's weird saying that when you've had two victories but a lot of our play was outstanding. There was effort, high skill and then there was resilience to go a man and points down and come back into the game."
