I don't understand why so many people are saying cricket. It's always seemed pretty simple to me, and I can understand most of the rules even though I don't watch it regularly. I'd agree that rugby union must be quite confusing to newcomers though, I only really starting understanding the rules hen I began playing myself.
I'd have to say American Football as the most complicated though. I'm a pretty big American football fan, and I still don't understand half of the rules! The general idea of it is quite simple to pick up, but beyond that it can be so confusing at times...