Al-Baghdadi was born near Samarra, Iraq, in 1971. He did his graduation, masters ad PhD in Islamic studies from the Islamic University of Baghdad.
After the US invasion of Iraq in 2003, Baghdadi helped to found the militant group Jamaat Jaysh Ahl al-Sunnah wa-l-Jamaah (JJASJ), in which he served as head of the sharia committee. Al-Baghdadi and his group joined the Mujahideen Shura Council (MSC) in 2006, in which he served as a member of the MSC's sharia committee.
Following the renaming of the MSC as the Islamic State of Iraq (ISI) in 2006, Baghdadi became the general supervisor of the ISI's sharia committee and a member of the group's senior consultative council.