The match looked destined to end in a 1-1 draw before Declan Rice (24) and Gabriel Jesus (26) scored for Arsenal deep in injury time.
United had thought they were en route to victory after Alejandro Garnacho (19) found the net in the 88th minute, but it was ruled out by VAR for offside, a decision that had Ten Hag seething.
"We have to improve, definitely," Ten Hag told Sky Sports. "But you see, and also what we have seen last year, we can beat everyone.
"It was such small margins today, so it could also have been our side. So when Garna is finishing (for the disallowed goal), it's a goal that we're winning this game. So it's that close.
"We needed a little bit more luck to win the game. (Garnacho's goal) was not offside. It was the wrong angle," he added of the video review.
Ten Hag zeroed in what he thought were a few late officiating blunders, including a tackle from Arsenal defender Gabriel that sent newcomer Rasmus Hojlund (20) sprawling that the Dutch manager argued should have been a penalty.
He also said that Gabriel appeared to block Jonny Evans (35) in the penalty area for Rice's 96th-minute winner.
"It's a penalty on Hojlund," Ten Hag fumed. "And at the end we concede a goal, and it's a foul on Jonny Evans. It's so clear and obvious."
United captain Bruno Fernandes (28) said it was a tough way to lose.
"We conceded the late goal after trying to get the draw," United captain Bruno Fernandes said. "It was a good game for both teams but they got the points in the end.
"It is what it is, if it is an offside we can't get on that. It was a good game, I am disappointed by the result, but the performance of the team, the passion was there. If we keep doing this consistently we will get results."
United head into the international break with just six points from four games.
