I really do feel like I’m going to destroy this woman and her kids.

This line of thinking is not helping you in any way, John. This is you trying to take 100% of the responsibility when no one and I mean NO ONE forced her to move in with you. SHE chose to move in with you and stay in this relationship, even though she’s known for most of the relationship that she has been coming in second to your ex. She is the one who is responsible for the position she’s in because she put herself there.  You know what I mean?  If she gets destroyed, it’s due to her own choices… just like your feelings are due to yours.

Aside from that, the only thing that blaming yourself is doing is making you drag this on because you feel bad (and your mind is likely coming up with every excuse that it can to try and avoid or put off the conversation because it’s going to be uncomfortable, right?), but dragging this out is likely to make things worse in the long run, especially if there is a window of opportunity right now where she may be able to move back into her old place with her brother now that he’s single again.  She needs to be able to see if she can do that before that window of opportunity closes.  Then hopefully that will lead to her just getting that place back for herself at some point.