I wouldn't do it simply to save money.
Its possible, but you'll end up getting into the whole thing, then it becomes a hobby rather then just transport, and costs spiral.
Provided you have the know how to do your own maintenance etc you should be able to keep one cheaply. Reliability isn't an issue if you look after the thing, though naturally this requires work. I have to do oil changes every 3000 miles, it eats spark plugs, the electrics are constantly in need of fixing (and the only things on there that are electric are the lights).
That said, its great fun, grinds petrol stations to a hault as people ask questions about it, and when you turn off the motorway for a pee people follow you just to talk about the car.
For me it all started off with a drunken ebay purchase, I ended up with the thing as a heap of parts really which I assembled over the summer. I would recommend doing it this way as it makes fixing it later so much easier.
Generally speaking old cars will use more fuel, I get 35mpg if I drive like a loon, and about 50 mpg if I scrimp it, but thats only because I drive a 0.6.
I think it has to be a labour of love rather than a cost saving exercise really in short.