Lots of people saying that as long as the two of you are happy then no one else matters.. this is sadly (in my opinion anyway) just not the case.
Sure, foremost a relationship is between 2 people, but it affects others round you. Your families and your friends. You'll have to think about how you will socialise together. Is he going to be embarrassed by you being underage? (Getting ID'd in pubs for example) Would that upset you? And how is he going to feel about hanging out with your teenage friends?
I've had some of the best times of my life with men 8, 9, 10 years older than me, but they haven't been without problems. Best of luck with it!
And whoever said he might have a "hidden agenda", I think you're wrong. Of course he -might- but no more so than the next guy who might be the same age. An ex of mine was really confused inside because he had feelings for me but didn't want to use me in any way. Older guys are not -just- in it for the sex.