I'd just say pay up, an appeal would be relatively useless. You might win an appeal, but chances are you wont. If it's truly designed to catch people out, the council wont care for your appeal.
You might have some luck if you get in touch with the local MP for that area, they may do something about signage situation. But as far as appealing goes, again, I'd say it'd be a waste of time.