I got my first car from a dealer at someone's house. He was very nice, let me take the car on a test drive and once we'd finished I decided that it was the car I wanted (it's in great condition and has low miles + it's a bit unique for younger drivers). I bought it that day and collected it around 5 days later. That was and still is my car and I've had no issues at all with it.
You've just got to be brave and trust the dealers. If you don't then take a good look around the car you're interested in to see if there are any faults or anything. You can even get it checked over by a mechanic or someone who's trained to do that.
But you're just gonna have to do it one day if you want to buy a car. Not all car salesmen are bad, you will be able to tell immediately if they are bad because they'll try to make you buy some other car that you don't want. Good ones will sit down with you and will let you look at the car, drive it and come to a decision on it.