I think all (at least most) men think that way, but only a couple (or at least a few) would actually act on those fantasies. From what you wrote, I doubt your DH would actually act on the suggestion! But maybe keep him away from alcohol and definitely from the couple. Who needs that energy around? It’s more trouble than it’s worth, look at the arguments it started! It doesn’t seem fair to me that he’s angry at you for not maintaining a friendship with people that would make anyone (or at least most people!) uncomfortable!