Bail. Unless you want to waste more time being confused and at times abused by this guy, bail.
Its clearly emotional abuse, the “go away, no come back” thing. Not cool. Bail. Find someone who isn’t into the games if you want to be in a relationship. Don’t start confusing yourself – you said you didn’t want a relationship, what’s changed apart from his begging? If you proceed with one now with him, this is just a taster of the times you’ll have together. This is about him having control. Bail.