You did the right thing. Next time just make it clear from the start that you do not wish to initiate a conversation. I've been in situations like that before and its just creepy.
p.s. and ignore the people on here who say that you are making a fuss. They were not in your shoes, and they have no idea how it made you feel.