Unless you are uber fit and all guys in the street want you only then you can assume that you are a challenge for him. Otherwise he is being genuine.
I think im alright, I attract more white men than black boys though I am black, but I think the only thing that could give him that impression is when we was out at a party we was talking causally and some boy just came up to em and started whispering in my ear though he clearly knew i was talking i told im i was not looking, it happened like 3 times the whole night but i dont know if that would have an effect on the way he is with me