South Africa won the toss and sent England into bat, reducing the visitors to 14-3 before Buttler (131 from 127 balls) and Malan (118 from 114) shared a stunning fourth-wicket stand of 232 in a total of 346 for seven.
The home side were always up with the run rate, but lost wickets at regular intervals.
Heinrich Klaasen (80 from 62 balls) and opener Reeza Hendricks (52) top-scored, while Archer took a career best 6-40 as the hosts were bowled out for 287.
South Africa won the series 2-1 to keep alive their hopes of automatic qualification for the World Cup in India later this year, though they may still have to win both ODIs against Netherlands in March and April to make sure.
England were in early strife when a triple early blow from Lungi Ngidi (4-62) had Jason Roy (1), Ben Duckett (0) and Harry Brook (6) back in the pavilion inside the sixth over.
After a period of consolidation, however, Buttler and Malan took to the bowling and smashed the home attack to all parts. England struck more sixes (19) in their innings than fours (16) as batsmen opted for power over finesse.
The visitors scored a remarkable 217 runs in the final 20 overs of their innings for their second total above 340 in as many matches.
A number of the South Africa batsmen got starts, but England kept chipping away at the wickets with Archer easily the most impressive as he begins to find top form following his return from long-term injury.
The victory ended a run of five successive losses for England in ODI cricket as they build towards the defence of their World Cup title.
