Some people only like to be "friends" with people who are relevant to their lives NOW, rather than be connected to hundreds of people they'll probably never ever see again. Harsh as it sounds, he was probably just being professional when he was being nice to you, and you might not be the kind of person he'd actually be friends with outside of a work environment. There's nothing wrong with that, though; colleagues are to work with predominantly, not to be best friends with (although obviously people do make friends at their place of work).
If you don't work for him anymore, he probably wondered what was the point in keeping you added as a friend.
Either that, or you play Farmville. Enough said.