The American, ranked 87th, ousted the world number eight, who made five double faults and only forced one break point in the match.
Nakashima edged the first set by breaking the Russian in the ninth game for 5-4 and then surviving a break point before serving out.
Rublev, second seed at the tournament, saved four second-set break points but tumbled to defeat in the tie-break after sending a backhand long.
The Russian smashed his racket repeatedly on the ground before storming off.
Later Tuesday, Rafael Nadal makes his return against Flavio Cobolli.
The 22-time Grand Slam champion has not played on the ATP Tour since January in Brisbane, where he felt a hip injury flare up.
Nadal is the record 12-time Barcelona Open winner and he said on Monday making his return at the tournament was a "gift".
The Spaniard missed the majority of the 2023 season injured and expects this year to be his last as a professional.
Nadal's compatriot Carlos Alcaraz, champion in Barcelona in 2022 and 2023, withdrew on Sunday with a forearm injury.
