Wales ran in four tries to collect a bonus point and although it was a third win in five games, Tandy felt his side needed to be more ruthless.
"It's a bonus-point win. It's good to start," Tandy said.
"I can't run away from the performance. There are elements we need to improve. I thought we'd be better. We weren't ruthless enough."
Tandy suggested rust may have been to blame as Wales played their first test since July.
"Getting a hit out for the first time since the summer, it does get clunky," he said.
"It helps to review after a bonus point win.
"We'll enjoy it but the beauty of this group is the hunger to get better quicker. We'll definitely have to do that."
Next up for Wales are New Zealand next weekend and Australia. Tandy knows his side cannot afford to offer up cheap opportunities in those games.
"We'd go seven points up and there'd be a sloppy moment," he said.
"There was some excitement in what we tried to do.
"Japan were effective close to the line. We managed the game well outside of that. Defensively, we improved."
Japan coach Eddie Jones felt it was a missed opportunity for his side, who had 62% of the possession but struggled with entries into the Wales 22.
"That could have been a historic win for us," Jones said.
"But you've got to be good enough to take advantage of those mistakes. We weren't quite good enough. It was a missed opportunity.
"I'm proud of my team. We had a young team, but they stuck at it. We had a few times where we didn't take chances and Wales did."
Japan face England on November 14.
