If you're trying to make out as if terrorism doesn't use Islam as a platform then you're just being wilfully ignorant. It's natural for people to be more cautious of something that has a recent track record of holding violent outburst, even if it is a minority.
This sort of prejudice is not exclusive to race or gender and happens often outside of those circumstances except it's acceptable.
If a dog bit you, would you keep dogs at an arm length? Of course you would, you prejudiced monster. /Sarcasm
Not to say it's right, but it's instinct.