He says he does not love me but has feelings for me…sometimes he says they are strong, sometimes not.

Oh, I am sorry you find yourself in this situation.

It sounds like you want way more from him than he is able to give.  It sounds like you are saying you are unhappy with him not reciprocating the feelings you have and you wish to be partners with him, more than “secret lovers.”

If I am hearning this correctly, this sounds emotionally devastating.  While a lot of the behavior between you two seems to indicate a serious long term relationship, there is the issue of him being aware of where he is at and such and him being clear that he is not where you are at.

Maybe look at this:

I keep trying to leave but I have a hard time staying away.

How long have you managed to stay separate from him?  What happens that causes you to return to this situation?