Feeling uncomfortable in this circumstance is natural, so don’t beat yourself up about that. It sounds like a conversation is in order for the two of you to get past this. But first, you have some homework to do on your own. Ask yourself what you are willing to accept in this situation where they want to remain friends. Are you okay with all of you hanging out but not them hanging out alone? Are you comfortable with them talking on the phone or doing activities? What exactly is okay for you? Once you have those answers, have a conversation with your boyfriend about what is okay with you and what is not and see if the two of you can come up with a compromise of how to deal with this. If he is unwilling to have the conversation , that is another red flag.
It is possible for exes to remain good friends and that does happen, but both of them have to be willing to be sensitive to the feelings of new partners brought into the situation (in this case you). If this feels uncomfortable to you, trust that feeling and work towards a solution.
~Jodi