Well By My Thinking The Guy is just possessive . He just wants to feel like He Matters the most to you And That You must make him feel like He is the Most special Male You have met . So it might be just an ego issue .

Or As Inky said , he might be a control freak preying on Controlling You . Some people do get a kick out Of That .

My Advice is to go hang out with him – Go to watch a movie , Go Shopping or whatever . The more he Hangs out with you , The more he gets comfortable with you …. the lesser he has to think about mistrust or him not feeling Supreme . The more You stay away From him or Him From You , The More distrust he feels towards You . Just Somehow push or nag him to come with you …. even if he doesnt want to .
It will all work out .