I failed my first one for "not stopping at a zebra crossing" when some man was just loitering 3 metres away showing no intent to cross, and all other traffic besides me was also just driving over it. Totally convinced I'd passed til he told me at the end, and I couldn't even have guessed where I failed! You win some you lose some.
Maybe he didn't see you checking, maybe you didn't check properly, hard to tell from your description. There is a blind spot that you can't see just looking out the driver's window, unless you throw your whole body out and somehow curve around to look. The one you check if you're changing lanes on the motorway. I guess you now know for next time! :P