Sweden advanced to the next round as Group F winners after a 3-0 win over Mexico, who still progress thanks to Germany's 2-0 defeat to South Korea.
In a must-win match for the reigning champions, Germany could not live up to the expectations as a resilient South Korea team managed to oust the champions with a 2-0 win. South Korea also needs Sweden to lose against Mexico in the other group match to finish with a better goal difference.
The four-time winners had never finished outside of the top seven since their last first-round exit back in 1938, but they have fallen in Group F to break 80 years of nearly constant success.
They lost their opening game 1-0 to Mexico and needed Toni Kroos's dramatic late free-kick to beat Sweden 2-1.
The loss capped off Germany's most disappointing campaign of recent times and continuing the mythical "curse of the World Cup champion". I'm still angry and disappointed and at the same time I feel sorry for all our fans who supported us in the stadium or at home in front of the tv or at public viewings.
Germany won last year's Confederations Cup in Russian Federation even though Loew opted to leave his World Cup-winning stars at home, boosting his reputation further. "We need to draw the right conclusions".
South Korea, however, finished its campaign in Russian Federation on a high, scoring two goals in injury time.
Kim was not so profligate when he fired home from close range - although he was initially given offside before a VAR review sparked renewed celebrations - and Son completed the job. "We looked at their strategies for the past four days".
South Korea coach Shin Tae-yong did not reveal which player would captain his side against Germany, just saying the role would not be for the faint-hearted.
The South Koreans were hit with 16 fouls and four yellow cards, as they fought to find a way to do what many in the sporting world believed was impossible.
From 30 yards out, it should have been fodder for World Cup-winning goalkeeper Manuel Neuer, but after fumbling Jung's dipping drive Neuer scrambled desperately to clear as Son raced to pounce.
The result sparked huge celebrations among the legions of Mexican fans in Russian Federation, and also back home in Mexico City, not least when locals mobbed the South Korean embassy and hoisted the ambassador onto their shoulders, before marching him around in thanks for his team's heroics in helping Mexico into the last 16. "I'm a very lucky guy to play for my country and I'm very happy and lucky to have this country", he added.
With no option other than a victory to advance, Low inserted Mario Gomez and Muller for Khedira and Goretzka.
Loew, who recently signed a contract extension, said he would take time to consider his future.