The pendulum swung this way and that in an enthralling first half in Leeds, with two tries from Jahrome Hughes and Dylan Brown, either side of a stunning effort from Australia's in-form Josh Addo-Carr, giving the Kiwis a 14-10 lead at the break.
The second half was much tenser, with tempers boiling over on several occasions, but it was Australia who kept their heads with Murray's try under the posts, and Nathan Cleary's conversion, in the 56th minute proving to be the difference.
New Zealand threw everything at the holders as they looked to sneak back in front with Peta Hiku thinking he had scored a dramatic late try, only for it to be disallowed, as Australia hung on.
Australia will face the winner of Saturday's second semi-final between hosts England and Samoa at the Emirates Stadium in London as the Kangaroos look to win their third successive World Cup crown.
"I am really happy for the boys, it was a real test match," Australia coach Mal Meninga said. "We haven't had a real test match for a number of years.
"I am pleased with the effort and commitment, both teams really went for it. It was one of the best test matches I have been involved in. I just said to the players they have given everything for their country, and found that bit extra to find victory."
In a carnival atmosphere in Leeds, New Zealand tried their best to rile their Trans-Tasman rivals ahead of kickoff with a fierce Haka in their opponents' faces.
It was the first time the pair had met since 2019 and first clash at the World Cup since 2013, when Australia romped home in the final in Manchester, but the Kiwis seemed to be determined to make amends.
They deservedly took the lead after 11 minutes following a fine offload from Joseph Manu for Hughes to go over for a converted try.
But there is no stopping the lightning Addo-Carr at this tournament as he equalled Valentine Holmes' record of 12 tries in a single World Cup with a sensational score, showing incredible pace before plucking the ball out of the sky.
The Kiwis took the lead again through a Jordan Rapana penalty, but Holmes went over down the other end to swing the scoreline back in Australia's favour.
Coming into the match having left it late to overcome Fiji in the quarter-finals, few fancied New Zealand to land a blow on the holders, but they kept coming, scoring another fine try, finished off by Brown.
The Kiwis had another try disallowed early in the second half for offside and seemed to tire from there on in and the Aussies capitalised.
Murray was the match winner, as he pierced the New Zealand defence to go over, before Hiku's luck ran out, as he was adjudged to have come from off the field before grounding his late attempt.
"I am proud of the players that's for sure," New Zealand coach Michael Maguire said. "There were just moments out there that didn't go our way.
"The players deserved the chance to play in the final. I am devastated for the players."
