If you have stages thats you really dont care about speaking to him and feel annoyed then it could be a couple of things. a) you are both that much constantly with each other all the time that is really become so intense that you need your space and your 'you time'.
or b) its not love, its actually lust. If you are disliking him when you are not with him, then melt when you see his face, sounds more like an attraction to me as opposed to love. They say absence makes the heart grow fonder, but in your case its a dislike, so somethings not quite right. I think you should take some time apart to see if you do actually miss him or not. if you dont miss the guy and disliking him, then hes not the right guy.