My boyfriend and I have been dating for a little over a year. We’ve had our moments, but lately it seems that all we do is fight and push each other’s buttons.

The catalyst of our fights is usually his college ex-girlfriend. They were together for five years and lived together for two. He comes from an extremely rough past (he was abandoned as a child, went through foster care and bounced from house to house, some abuse, etc.) His girlfriend’s family basically adopted him, helped him though college, put a roof over his head for several months, and was the first real “family” he ever had.

She was his first and only “love,” but five years into their relationship she ended up cheating on him with a mutual friend, and they broke up. Despite the nature of their break up, they remained friends and her family continued to reach out and offer him emotional support. Three years have passed since they broke up, and she has since moved away. He and I have been dating for a year now, and I’m his first relationship after breaking up with her.

When he and I started dating, his ex-girlfriend reacted with a lot of jealousy. She started to contact him more and more frequently, calling or texting at odd hours of the night just to “check in.” He never hid it from me. As the messages became more frequent and emotional (“you’re still my best friend and I feel like I’m losing you,” or “I miss having you in my life… it’s not the same without you”), he agreed that she was crossing the line. After several months of us fighting over it, he finally told her that she had to stop contacting him.

He was pretty emotional after their conversation, and admitted that he wasn’t ready to let her go. We got into a fight about it, and he got more emotional than I have ever seen him. In tears, he confided that she was the only girl he has ever truly loved… and that our relationship didn’t compare to theirs, and that he wasn’t sure if he would ever feel the same way about me. He admitted that he couldn’t delete her number yet, and that he still felt a lot of platonic love for her and was confused about the whole situation. I reacted with anger and we ended up taking not talking for a week after that.

When we finally reunited, we went back and forth about our relationship, their relationship, and other issues. He basically told me again that he had a lot of love and emotion for me, but just wasn’t in love with me yet and wasn’t sure if he ever could be. He also promised that he wasn’t in love with her anymore, but still felt loyalty towards her and her family because of how they helped him. We decided to get back together and work on our relationship.

We’re back together now, and I almost feel more conflicted than if we had just broken up. The whole experience was a huge blow to my ego. Now, I don’t even like to hear him say “I love you,” because I know it doesn’t mean “I’m IN love with you.” I haven’t been able to feel close to him since everything happened. I don’t feel any trust or excitement for him. I just feel like I’m with a guy whose heart already belongs to another woman. I don’t know what to do. Obviously I love him, but I can’t continue on like we were before, now that I know where his head and heart are. Still, he told me that he’s willing to work with me to get better. I just don’t know what it will take, or if it’s worth working… or if we can even fix this.

Has anyone been in a similar situation? Any outside opinions or advice?