You don't have to have a massive crush on someone to go out with them, no. I used to think like that.
But when I met my current boyfriend, I wasn't sure if I actually felt anything besides a deep fondness for him, as he is very nice and sweet.
I decided to go out with him anyway, and I really don't regret it, as I'm comfortable and we treat each other with respect.
If someone is nice, and you like them, then try it out even if you're not 'head over heels'! It's better than thinking you've found 'the one' and having them turn out to be horrible.