kody brown, robyn brown

During Sunday’sSister Wives: One on Onespecial, the Brown family patriarch said he doesn’t think he’s “hit rock bottom” despite experiencing ongoing anger issues amidthe “unraveling” of his family unit.

“Something about the struggles in the family just made me kind of go,Is this what real life is like? Is this what real marriage is like? Is this how it is?” he told host Sukanya “Suki” Krishnan. “I’ve been thinking about this. Like, all I do when I’m not working, it’s on my mind,” he continued. “It’s rolling through and I’m replaying and rehashing discussions and rehashing my life and how I started this family out and everything.”

Reflecting onhis experience with polygamyas a whole, Kody said, “I was so anxious to be living the principle of plural marriage that I was not vetting who I was marrying to be compatible with me.”

“The blame lays on one thing and one thing only. If you’re going to marry and you’re going to marry for eternity, you should start out in love,” he explained. “I thought it was acceptable. I thought I could make up the difference. I thought [about] everything in this situation with all three wives, ‘This isn’t about being, like, in love. This is about growing a family together.'”

Kody Brown is pictured discussing ‘Sister Wives’ on ‘Good Morning America’ on September 21, 2011.Ida Mae Astute/Disney General Entertainment Content via Getty

Kody Brown talks about their show Sister Wives on Good Morning America

Ida Mae Astute/Disney General Entertainment Content via Getty

According to Kody, “There was a safety and a vulnerability that I had never experienced in my life. I met her and I started weeping for joy and didn’t stop for months.”

“Let’s just say the two of us had a click whereas I was working for years with these others to get them to get this place,” he claimed. “Never had the click.”

The entire Brown family is pictured.Puddle Monkey Prods/Kobal/Shutterstock

Janelle Brown, Kody Brown, Meri Brown, Robyn Brown, Christine Brown

“I did my best. I don’t know what else I’m supposed to do, but I did my best to not have it be something that was painful for them,” she said. “I constantly was suppressing and hiding my relationship in any kind of connection I had with Kody.”

“At that point, Kody and I were already really, really struggling. So, that didn’t help,” she said. “It broke my heart. That was one of my first moments of heartbreak was that moment. I was just sitting at the edge of my bed and watching him get ready to go on a date with her before they were married. He’s like, ‘I don’t know what’s wrong with me.’ And I said, ‘Oh, you’re in love with her.'”

Christine, who isnow married to David Woolley, said of Kody: “He looked at me, he’s like, ‘I am, I am in love with her.’ And then he got really giddy. … It hurt. It hurts so bad. It’s so bad because you realize at that point, everything now is going to change.”

“It was a call for us to put more effort into the other relationships, in my mind. But that became a grind,” he added. “It was constantly, constantly trying to improve your love and prove yourself and and and like, kind of doing this dance constantly. And some years were good because the dance was working really well. And what I’m talking about is these struggles in relationship when we don’t have the basis of passionate love.”

Robyn Brown, Meri Brown, Kody Brown, Christine Brown and Janelle Brown from “Sister Wives” arrive at the grand opening of Mike Tyson’s one-man show “Mike Tyson: Undisputed Truth - Live on Stage”

Ethan Miller/Getty

“I just think that maybe her relationship with Kody was so very different from the get-go,” Janelle also shared. “Maybe it was hard for him because it really felt like he more and more pulled away from wanting to be a plural husband.”

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Sister Wivesairs Sundays at 10 p.m. ET on TLC.

source: people.com