It was an incredible final few minutes for a team who looked as if they had blown their chance of three points following Bruno Fernandes’ failure from the penalty spot in the 58th minute.
United needed a positive reaction following that 6-1 thumping by Spurs before the international break, but their hopes suffered a blow when Emil Krafth’s cross came off defender Luke Shaw and beat keeper David de Gea.
Harry Maguire, who like his side has endured an unsettled period, equalised to haul the visitors back into the game.
And it looked as if the match was set to turn in their favour in the second half when referee Craig Pawson, after consulting the pitch-side monitor, judged Jamal Lewis to have fouled Marcus Rashford and awarded a penalty.
However, Fernandes, who previously had a 100% record from the spot for United, saw his fierce strike brilliantly pushed away by Karl Darlow.
The Portugal midfielder was not to be denied though and with four minutes remaining he finally beat the Magpies keeper with a thumping finish, before Aaron Wan-Bissaka, with his first for the club, and Rashford ensured United’s second win of the season.
“The boys came together really well. The are looking forward to every game. Today I rested a few and they have come on and done the business for us,” Solskjaer said.
Solskjaer lauded captain Maguire who has seen some struggles for both club and country in recent weeks.
“Harry is a character, he has great resilience and wants to play. There is no hiding place and if you go away you prolong the criticism, He wanted to play and even though he wasn’t 100% fit, he played well. He is a very well liked lad,” added Solskjaer.