Alcaraz blew away Alexander Zverev on Tuesday in a repeat of last year's final in the Spanish capital, but Khachanov put up stiffer opposition.
"Winning this match will give me a lot of confidence, Karen was at a great level and took me to the limit," said Alcaraz who unleashed 31 winners.
"It was decided by small details and I am very, very happy to be able to get through this round."
The Russian, who knocked out compatriot Andrey Rublev this week and also reached the Australian Open semi-finals, showed no fear in the first set despite his opponent's form.
Eventually, Alcaraz found a decisive break to take a 4-3 lead with a strong forehand which Khachanov could not return successfully.
The Spaniard consolidated and served it out.
In the second set, Khachanov broke for a 3-1 lead when Alcaraz went long after saving a first break point.
The world number 12 forced two more in the sixth game but could not convert either as Alcaraz fought hard to hold.
Alcaraz's perseverance paid off as he broke with a volley to get back on serve at 4-5.
The US Open champion consolidated his break and then earned another when Khachanov hit the net with an attempted forehand winner.
Aiming for his 10th tour-level title the crowd-pleasing home favourite sealed his progress with a powerful forehand drive.
If Alcaraz lifts the trophy on Sunday in the Spanish capital, he will knock Novak Djokovic (35) from the top spot in the rankings simply by playing a match at the Italian Open in Rome next week.
Djokovic missed the Madrid Open with an elbow injury.
Alcaraz will next face Borna Coric of Croatia who eased past German lucky loser Daniel Altmaier 6-3, 6-3.
Swiatek marches on
Women's world number one Iga Swiatek (21) raced into the semi-finals with a 6-0, 6-3 victory over Croatia's Petra Martic (32) in the last eight.
The world number one, playing the event for only the second time, will take on Russian 12th seed Veronika Kudermetova (26) on Thursday for a place in the final.
Swiatek lost just eight points in a completely one-sided first set before one break of serve in the second set proved enough to send her into a fifth successive WTA Tour semi-final.
The Polish star has now won eight straight matches as she bids to hit top form in time for the start of her French Open title defence later this month.
First WTA 1000 semi-final
Kudermetova ousted world number three Jessica Pegula (29) 6-4, 0-6, 6-4 in a topsy-turvy clash on Wednesday to reach the Madrid Open semi-finals.
The 12th seed beat the American five-time Grand Slam quarter-finalist in three sets to reach a WTA 1000 event semi-final for the first time, recovering after a collapse in the second.
"To beat Jessica it feels very nice, I'm so proud of myself because every match I played almost three hours," said Kudermetova.
"And today in the second set I start to play so badly.
"But before the third set, I said 'Veronika, you need to try to play your best, because the fans are here. I want to show you some energy'."
The Russian battled well to take the first set in the Madrid sun but crumbled in the second.
Kudermetova won three three-set matches to reach the quarter-finals and appeared to tire, making several unforced errors as last year's Madrid Open finalist triumphed easily in just 22 minutes, winning 83 per cent of points.
However, after a medical time-out, she rallied in the third, with each player breaking the other twice for 3-3, before Kudermetova managed it again for a 4-3 lead which she was able to consolidate before serving it out.
Pegula saved one match point but Kudermetova clinched progress with her fourth ace of the match.
