Take it, no one will care. I took teddy bears to uni, no one batted an eyelid.
If anyone does comment on it, the best thing to do is be completely unembarrassed. Once the person sees that it doesn't bother you, they'll get bored and stop the comments. Simples.
I took about 10 teddies with me and an owl cushion and I survived, so don't worry My housemates all have bears, I admit we are all girls but no one cares, we like each others teddies because they're all so cute!
Take it. All of the girls in my flat took a teddy, including me. Admittedly he got booted off the bed whenever I shared my bed, however it's a piece of home isn't it. Even one of the tough sports lads in my dorm took a teddy. No one cares.