I'm 25 and living in halls for my first year, I expected there to be more people my age but I am the oldest out of nearly 1000 on our hall campus. As long as you don't make a problem about it people generally don't care, if they do it's because they are probably too immature to have been allowed out of their village, but I imagine they will grow up as the year goes on. After a while it's not that different from going out with mates at home, apart from the occassional obligatory drunken mess you will see lieing in the corridor.
Certain Uni's do have a higher percentage of mature students, such as Plymouth which has 27% undergraduates over the age of 21, whereas Birmingham only has 3%.