In a poor first half the Algerian succeeded in man-marking the Belgian to such an extent that De Bruyne and Hazard were all bet invisible.
The Red Devils looked shaky and although they had the lion's share of possession were unable to create anything in the way of chances.
Algeria took the lead 25 minutes when Feghouli (photo) scored from the penalty spot after a foul by Vertonghen. The goal was Algeria's first in the World Cup Finals since 1986.
Belgium's reaction was a series of long-range shots. Good work from Hazard on the stroke of half time provided a chance for Chadli. However, his shot lacked power and the score remained 1-0 in the Algerians favour.
Second half subs made the difference
The tide started to turn in Belgium's favour once Dries Mertens was brought in in the second half. However the real turning point when 19-year-old Divock Origi was brought in to replace an all but invisible Lukaku.
After just minutes on the field Origi can close to an equaliser. However, good work by the Algerian keeper M'Bolhi denied the youngest ever Red Devil. With Belgium still a goal down Coach Marc Wilmots brought in Manchester United's Fellaini (photo).
This turned out to be a good move because just minutes after entering the game the midfielder with the iconic haircut equalised with a superb header. Belgium was level with 20 minutes to go.
This gave the Belgian fresh hope Mertens scored what was to be the winning goal of the match after some excellent work by De Bruyne and Hazard 10 minutes from time.
Fellaini came close to make it 3-1 a couple of minutes from time. However it was not to be and the match ended 2-1 to Belgium. With the match between Russia and South Korea having ended in a one-all draw, Belgium now leads Group H. The Red Devils' next match is against Russia on Sunday evening.