Whenever someone creeps me out, I creep them out even more.
Some random person who lived in the flat over form ours kept doing weird things with my laundry (weird shapes and such, public dryer you see) and kept hassling me and staring at me from the window. I "mailed" myself some strange goods, I won't say what they were, had my friend give it to them as the mail man, and that sorted them right out. The mail was left shoddily packaged and it must have accidentally opened or something, as planned, and after that she stopped.