India restricted South Africa to 106-8, a total that had looked improbable after a spectacular top-order collapse left the touring side reeling at 9-5 in 2.3 overs.
Rahul made 51 not out and Yadav, the world's second-ranked Twenty20 batsman, remained unbeaten on 50 off 33 balls as India romped home with 20 balls to spare.
India's pace attack, led by Arshdeep Singh, had laid the foundations of the victory on a seaming track at the Greenfield International Stadium.
They blew away South Africa's top order within 15 deliveries to seize early control of the match.
Deepak Chahar dismissed touring captain Temba Bavuma in the first over and sent back Tristan Stubbs in his next.
In between, Singh sent down a three-wicket over to break the back of the South African line-up.
The left-arm seamer got Quinton de Kock to drag a ball onto his stumps and removed Rilee Rossouw and David Miller in successive deliveries to leave the tourists in tatters.
Aiden Markram (25) and Wayne Parnel (24) offered resistance with the bat but it was Keshav Maharaj's counter-attacking 41 that helped South Africa post a 100-plus score.
India endured their own top-order woes when they began their chase.
Captain Rohit Sharma fell for a duck and Virat Kohli made three but Rahul played the anchor and Yadav the aggressor in an unbroken 93-run third-wicket stand to put India 1-0 ahead in the three-match series.
The teams move to Guwahati for the second match on Sunday.
For the home side, batsman Deepak Hooda has been ruled out of the series with a back injury, while pace spearhead Jasprit Bumrah missed the opener with a back niggle. Both are part of India's Twenty20 World Cup squad in Australia next month.
