There is nothing wrong with going to university when you are in a relationship. You can still have a great time and meet lots of new people, you just can't get involved with anyone be that relationship wise or sexually. Look at it this way, what would you rather have? A relationship with someone you care about or a possible one night stand with someone you wont see again? I know which one I would choose.
Long distance relationships are hard, there is no doubts about it. Some people make them work and others don't, it depends on the individual and the situation. They are scary at first, been there and done that myself, but if you believe it's worth it then why not give it a try?
I'm not someone who believes that you should give up your own possible happiness for university. Why should people give up great oppertunities just because of university? I mean, you have a chance to be happy with someone but you are willing to give that up so that just incase you meet someone there? I don't know, I wouldn't do it. If I was interested in someone, and they were interested in me then I wouldn't let that chance pass me by.
At the end of the day, only you can make this decision. You need to figure out if you want a partner or if you want to have casual fun at university. What's more important to you, more sexual partners/flings or a proper relationship? If you really are having your doubts, I don't think it's okay for you to lead him on, give him the idea that something may happen because that will only hurt him.
There is no harm in having a relationship and ending it if things get tough or if you meet someon else down the line. Not all relationships are meant to be, but if you are really interested in this guy then maybe it's worth a shot? Six months down the line you could be asking yourself "What if?" if you don't do something about your feelings. If you give it a go and it doesn't work out, at least you tried right?