I think you have to have pencil for Maths exams for the graphs and drawing - therefore the scanners should pick it up and you should be fine.
However, consider it a lesson learned and don't forget about your pen in future... If you have to, leave the pencil questions until the end, so you can't just forget to switch back after doing one...