Chasing 189 for victory on a spicy pitch, India slumped to 39-4 but Rahul's unbroken 108-run partnership with Ravindra Jadeja guided them home with 10.1 overs to spare.
Earlier, India's bowlers, complemented by some excellent catching, bundled out Australia for 188 inside 36 overs at the Wankhede Stadium.
Mitchell Marsh, returning to international cricket after ankle surgery, smashed 81 off 65 balls but his exit triggered a collapse.
With India's regular captain Rohit Sharma absent due to family commitments, Hardik Pandya won the toss and elected to field at the Wankhede Stadium.
You can see all the stats here
By the second over, India had drawn first blood.
Travis Head, who made five, charged out against Mohammed Siraj but ended up deflecting the ball onto his stumps.
Marsh steadied the ship with a 72-run stand with Steve Smith, who is leading Australia in the absence of Pat Cummins who stayed back in Sydney following the death of his mother.
David Warner also missed the match through injury though Australia expect the opener to play in the next two ODIs.
Smith made 22 before Pandya removed him caught behind.
Marsh, in his first knock as an ODI opener, hit 10 fours and five sixes and Australia looked pretty comfortable at 129-2 in the 20th over before the wheels came off their innings.
Jadeja removed Marsh and took a stunning catch to send back Marnus Labuschagne (15) before Mohammed Shami's triple strike derailed Australia.
Josh Inglish (26), replacing an unwell Alex Carey at wicketkeeper, dragged the ball onto his stumps, Cameron Green (12) had his off-stump uprooted and Marcus Stoinis (5) perished in the slip.
Glenn Maxwell made eight on his return from a broken leg and Mohammed Siraj (3-29) dismissed the scoreless Sean Abbott and Adam Zampa in successive overs as Australia folded in 35.4 overs.
India's start was even worse and their top four were back in the hut inside 11 overs after Mitchell Starc (3-49) struck three blows that included the prized wicket of Virat Kohli, who managed four.
Rahul added 44 runs with Pandya to arrest the slide and killed off the contest with his collaboration with Jadeja, who remained not out on 45, which included the winning boundary.
The second match is scheduled in Visakhapatnam on Sunday, while Chennai hosts the final ODI on Wednesday.
India won the preceding four-test series 2-1.
