If you're sure, do it as soon as possible as it is unfair on him if you stay with him, but don't want to be with him. However, I have had this time in my relationship where I don't really want to see my boyfriend at all and just want time by myself, but now I'm glad that I am with him because it's just that small amount of time away from him that makes me realise that I love him and I couldn't think of being with any other person.
If you decide to break up with him, do it to his face and I'm sure even if he gets angry you know that at least he knows earlier rather than later. If he really loves you, he will respect whatever decision you make.