It's not just for the bedroom/ fetish events. As long as you wear it well it will not be trashy. You've chosen trousers and a pencil skirt, so it's not as if you'll have loads of flesh hanging out. The people who have already commented are probably the type of people who will go on a night out in a pair of jeans. Blergh.
It's usually better to wear latex with latex, so you could invest in a latex top to wear with all of them. Though libidex make beautiful stuff, there are other sites where you can get things more cheaply if you just go for something simple. You could also try wearing your latex with silk/ chiffon/ lace fabrics. I'd say just steer clear of cotton.
The bottom line is WEAR them, for goodness sake. I wear my latex to the pub.
Oooh, also, make sure you know how to clean and store it properly! You wouldn't want all that money to go to waste when your latex starts prematurely rotting.