The couple married in 2000 but the relationship ended in 2003.
Thornton blamed his insecurities for the break-up of the marriage during a recent appearance on Nightline.
"I blew it because I didn't think I was good enough for her," he said. "She has one way she wanted to live her life and I had another way to live mine, and I was just too insecure."
Thornton continued: "I did feel like the Phantom of the Opera hiding in the catacombs. People have actually said that I didn't deserve to be with her."
Thornton also laughed off the rumours surrounding his relationship with Jolie, saying: "We were supposedly vampires who had a dungeon and we drank blood.
"We didn't even wear blood vials around our necks. She bought these clear lockets - they were little bitty things, they were literally this big. Angie and I literally poked each other in the finger with a pen and rubbed a little bit of it on that locket. When we were apart, it was a symbolic thing to do."
He added that he is still "good friends" with Jolie, who penned the foreword to his memoirs, saying: "We had a great time together. We had a great marriage and I chickened out because I didn't feel good enough. That's all that happened. It was no big deal - we never hated each other."
Watch Billy Bob Thornton speaking to Nightline below: