Bremen was one of four Bundesliga sides to be knocked out by lower-ranked opposition in the first round of competition, with Hamburger SV, Nuremberg and Eintracht Frankfurt all losing on the hottest day of the year across much of Germany.
Taylor's scoring exploits began after former Juventus forward Eljero Elia put Bremen ahead with an exquisite strike just before halftime.
The 30-year Taylor equalized and then cancelled out Niclas Fuellkrug's 67th-minute strike to take the game into extra time, in which he set up Dima Nazarov for the third goal.
Bremen defender Sokratis Papastathopoulos was sent off in the 108th minute, and Taylor completed his hat trick 10 minutes later.
"Taylor was ice-cold in front of goal," said Muenster coach Pavel Dotchev. "We're very happy to progress."
Another third-division club, Karlsruher SC, came from behind to eliminate Hamburg with a 4-2 win, while fourth-tier side TSV Havelse beat Nuremberg 3-2 with Marc Vucinovic scoring in the first half of extra time to seal a historic win for the little-known club.
Frankfurt got off to a bad start at second division Erzgebirge Aue when goalkeeper Kevin Trapp was sent off in the 19th minute for a foul on Jakub Sylvestr.
Thomas Paulus scored the resulting penalty, Sylvestr made it 2-0 in the 61st and he wrapped up a 3-0 win in the final minute.
Fourth-tier side Wormatia Worms provided another shock result with a 2-1 win over Hertha Berlin, which was relegated from the Bundesliga last season.
There were no such problems for Schalke, which enjoyed a comfortable 5-0 win at third-division club Saarbruecken through two goals each from Ciprian Marica and 18-year-old Julian Draxler. Greek defender Kyriakos Papadopoulos had given the 2011 Cup winners the lead.
"Some players were out, others replaced them well," Schalke coach Huub Stevens said. "That's how it's fun."
Hannover beat Noettingen 6-1 and second-division side 1860 Munich thrashed fifth-tier team Hennef 6-0 to progress.
Arminia Bielefeld overcame Paderborn 3-1, while Aalen beat Ingolstadt 3-0 in a tie between second division sides.
In the late games, Mainz strolled to a 4-0 win over fifth-tier club SV Rossbach/Verscheid, Fortuna Duesseldorf beat third-division side Wacker Burghausen 1-0 and Kaiserslautern won 3-1 at Hansa Rostock.
Temperatures earlier reached 40 degrees Celsius at some venues before kickoff, forcing stewards to use water cannons to cool fans.
Games were also held up midway through each half so players could take on liquids.
Defending champion Borussia Dortmund progressed to the second round on Saturday with a 3-0 win at fourth-tier club Oberneuland, while last year's beaten finalist, Bayern Munich, visits Jahn Regensburg on Monday.