Past his prime Federer has probably been the best opponent to Djokovic since 2011. What makes people think that Djokovic is on par - or better - than prime Federer?
IMO, in their primes, Djokovic > Nadal on grass and hard. Nadal > everyone on clay. Federer > Djokovic on grass and >= on hard and clay. Federer > Nadal on grass and hard.
All in all, I give Federer the edge, and Djokovic and Nadal pretty much tied second (because of Nadal's clay dominance).
BUT, bear in mind that we are currently in Djokovic's reign of dominance - he's won half the slams since the beginning of 2011. His achievements will rise the most out of the three for the rest of their careers (Nadal can only win on clay IMO, maybe one or two somewhere else).