Ilkay Gundogan's penalty gave Germany the lead in the first half but Japan bounced back in the second half with goals from substitutes Ritsu Doan and Takuma Asano.
It was Japan's first-ever win over Germany, with the teams having met twice in international friendlies before Wednesday's World Cup clash.
The defeat was the third in a row in the opening match of a major championship for Germany after a loss to Mexico at the 2018 World Cup, when they bowed out in the first round as champions -- their earliest exit in more than 80 years - and to France at Euro 2020.
"With this defeat and zero points, we are under pressure, no question about it," Flick told reporters. "We can only blame ourselves. It's a great disappointment.
"We were on the right path in the first half, we had 78% possession and were ahead 1-0. Then we had good chances in the second half that we didn't take advantage of.
"Japan were simply more efficient today. We made mistakes that we should never commit especially in a World Cup and those are the things that we need to improve on."
NEUER FRUSTRATED
Gundogan said players lost the ball too easily.
"We made it too easy for them. Especially the second goal, I don't know if ever there has been an easier goal scored in the World Cup," he said.
Captain and goalkeeper Manuel Neuer said the team lacked the urgency to score after they lost their lead.
"I am totally frustrated and angry that we let this game slip, it was not necessary," Neuer said.
"After the break we no longer had this game flow and did not play with the same confidence as in the first half.
"We are now under pressure from the start. That was also the most important game of how you start in a tournament but we messed it up."
Japan lead Group E, which also includes Costa Rica and Spain who meet later in the day.
