The hosts were 2-0 down and in disarray inside five minutes with Taiwo Awoniyi (26) scoring a solo effort on the break and Willy Boly (32) heading in a second goal for the visitors.
But Christian Eriksen (31) pulled one back and goals by Casemiro (31) and a Bruno Fernandes (28) penalty sealed the points, even if the display was unconvincing again at Old Trafford.
"It's a horror start giving them two goals and making mistakes, but I'm very pleased with the comeback," Dutchman Ten Hag told reporters. "The leadership, how we stayed calm and composed, stick to the plan and belief and turnaround.
"We played very good football, out of the plan and principles and rules we scored good goals."
United began the Premier League season with an unconvincing 1-0 defeat of visitors Wolverhampton Wanderers but were outplayed at Tottenham Hotspur last week in a 2-0 reverse.
So Saturday's win was welcome, although Ten Hag said it was natural that sides have teething problems early in the season.
"Every team in the start has areas they have to improve and we have some, we have to progress there if you want to be successful but you see this team as big characters, big personalities," he said.
"There's big spirit in this team, they found a way to win."
Skipper Fernandes was inspirational and Ten Hag was effusive in his praise for the Portugal midfielder.
"In all aspects he played a very good game at the highest level, keeping calm, using his emotions at the right moment, he was brilliant in attacking, making a difference, very pleased with his performance," he said.
There was one casualty of the win with defender Raphael Varane (30) forced off with an injury.
"He had complaints, we didn't want to risk him, we have Harry Maguire (30), Victor Lindelof (29), who could replace him, so we don't take a risk," said Ten Hag.
