Is it rude when people ask where do you come from?

Is it just me, or is it a tad rude and obtrusive?

It annoys me when people ask me, not because I'm ashamed (not at all), but rather they act like I owe it to them to tell them where I come from.

I know often they don't mean it in a mean way, but I would never go round asking people where they come from, never mind asking strangers.

I got asked today at a fast food restaurant by one of the staff. As usual, the conversation went in the usual way;

him: where are you from?
me: UK
him: ah, but where
me: London
him: i mean what country

I felt a tad offended after this to be honest. I feel like a doormat when people think they can just ask anything they want. But at the same time i don't want to turn around and tell them it's none of their business, because they may not mean it in a sinister way, nor do i have anything to hide.