I have no experience on the subject, but I don't know anyone who cares or is able to tell the difference between 18 and 20 years old, myself included. Everyone I've spoken to has told me that the halls are a great place to meet people and made it seem like it's almost mandatory to be in halls at least for your first year at Uni purely for social reasons.
(I've seen in another post that some universities may assign you to other people who are in a similar situation if you request so just talk to them about it if you're worried.)