I had a similar situation some years ago with my ex bf… He came to visit me in Australia after we hadn’t seen each other for 5 months, and I instantly had a sort of panic attack. We traveled together and I felt – bored, as if there was something missing, even though I did love him very much.
The thing is: I still love him very much. And I love his beautiful daughter and his amazing wife. But the very best thing is I’m actually IN love with my current boyfriend – and even after 5 years I still feel excitement with him and would not want to travel without him.

Your feelings seem very clear about this, and like Ed said, you need to follow them and you don’t have to feel guilty about it. It will be hard for him, but maybe with a bit compassion and honesty he will heal and find something that gives him what he needs.