Oh, I can tell immediately from your post you are NOT in the least judgmental and are only offering me help while still trying to understand yours!!
Your post makes me so sad for you because I hear in your voice all the joy I felt too, at the very beginning.
Now, while nothing has changed OVERTLY in his actions TOWARDS me (still calls as much, we still have fun, he is very patient etc), I can somehow understand EVERYTHING you are saying because it all rings a bell.
First off, you used the word, toxic for his relationship with her. And she is BP. Both of these things coincide with what my BF(boyfriend) have experienced. My boyfriend’s wife (that sounds so awful) is Histrionic personality.

I feel so bad for what you are enduring. Having gone from all the joy to this is so hard. Especially when you feel he is projecting behaviors from his marriage onto you.
You suggested that it was related to when you got married and how the responsibilities have caused him to revert to behaviors in his marriage.
What I also heard you say that would bother me is that he is emotionally tied to his former wife.
He is stilling doing things for her to avoid drama?

Avoiding drama at all cost is something that will make you feel ‘in the dark’ with him because you feel you cannot get close to him on an emotional level.

I would suggest that you offer him as much space as you can so that he feels as if you ‘backing away’ from him to a degree. In that way he can experience what it feels like not to have your loving support.
It is not manipulative at all to create some anxiety in him, because his thinking is askew and needs to be reprogrammed. He is seeing you a whipping post instead of the love of his life and before this sours you toward him, action must be taken.
Because he doesn’t know how to take the proper action, you have to be the one.
So, back away from him and be emotionally cool.
Not cold or detached. That would be counter productive.
Give him some emotional space so he can experience what it feels like not to be surrounded by your love, goodness and support.
This will cause him to “FEEL” emotions he needs to deal with.
He will want to come back to you to ‘recreate’ those warm feelings.
When he does, do not give them without first telling him that the warmth comes with the flow of love that you two once had,
Let him see that the responsibility for the relationship lies with both of you. NOT JUST YOU!

Most likely, he didn’t have time to grieve his marriage ending and even though it is toxic and terrible, we still need to grieve.
Maybe gently suggest that to him when things get better and see if he is receptive to that.

He may feel that he cannot give to you right now and that is making him display the passive aggressive behaviors to you.

Don’t allow yourself to get sucked into a never ending cycle of bad treatment by him. It will kill your love and it will kill your self esteem too. It will hurt you.
If you give him time and he doesn’t change and begin to talk to you and open up to you and get back to where you were in the joy stages of your relationship then it might be time to think about your life and what you need.
Non stop supportiveness by one partner is called “CODEPENDENCE.”
Make sure you aren’t codependent by googling it and reading up on it.
I am codependent in my marriage and even though my husband had been separated from me for 5 months I still can fall into that role. I think it is my job to fix everything about him and our relationship.
I was worn out and couldn’t do it any more. I had walked on eggshells for many many years.
That is why when my boyfriend walked into my life I fell so hard for him. He gave me everything that I had not gotten from my husband.

I hope you will find peace and the joy you found with your husband. You sound like such a wonderful person. Thank you for your post to my question.