Not necessarily.
It may simply be a case of us not understanding or being aware of time travelers. It's difficult because we don't actually know how time travel would work physically for the traveler, or what humanity would look like by such a point.
The lack of the stereotypical time traveler would indicate that time travel in the form of hopping into a machine and being physically transported back, is impossible.
Also, to put things in perspective, if you take a square mile somewhere on the planet and check for planes, it is quite conceivable that no planes will fly above that square mile. This does not mean planes do not exist.