Agreed - when I attended an open day at Cambridge, almost everyone (both boys and girls) wore jeans and casual tops. If my memory serves me correctly, one or two did wear a suit-type thing, but to be honest this just looked way over-the-top and quite silly compared to everyone else. The point of open days is to give you a relaxed idea of the uni/dept/college before the more formal interview days, so just wear what's comfortable.
Which open day are you planning to attend, btw?