Working out to any degree will help your self esteem too. It keeps you fit and it does good thing for your mind.
There's no excuse. If you can't afford the gym (it's really not that bad if you cut down on all the meaningless crap we spend money on, cut down on drinking, whatever your thing is), then there's still no excuse - the correct nutrition is available to you, and if you know where to buy, you can get it cheap (markets, etc.). You can do many great compound mass building excersizes without access to a gym.
If you want to be a slim guy without much muscle mass, then maybe it's not for you (though you could run, that's healthy for anybody - keeps your mind active too. Do early morning runs or maybe a night run, whatever you like best). But why not try out getting a great impressive body? Wouldn't that help your cause? If it would make you feel good, do it. But keep in mind that these superficial things may not have any effect on anybody else - they are also for ourselves, our own self esteem.
What do you dress like? Look man, accept your natural looks - and learn to work with what you have. Everybody has something about them they don't like, be it certain facial features or body image, whatever - but that doesn't mean other people see it in the same way. What you perceive yourself to look like and what others do can often be very different.
Take small steps. Maybe you're blaming your looks when you're really not putting yourself out there, afterall, nobody knows anything about you - when you meet a stranger, what else do they have to go on but your looks if you're not giving them a hell of a lot more to work on, and if you're quite self conscious then you're probably not a highly charasmatic guy who instantly transforms the room.
Think about that. You can blame your looks to a certain point, then it's all you. You take control. You make people see that you're interesting. Strangers know nothing about you. The smallest of details can make the world of difference.