I agree with the comments about Matt Smith not being able to 'do' the depth of emotion that David Tennant could. I'd sort of forgotten, until I started watching a few episodes, and realised quite how much of that has been lost. There's no real emotional impact now, just constant whimsy. There were always quirks and whimsy, but that was just one aspect of the Doctor, not his sole defining characteristic. I like Matt Smith, but I think we are ready for a new Doctor now, preferably someone with a bigger emotional range.
(And yes as others have said this does come down in part to the writers, but not entirely - there have been scenes with plenty of scope for raw emotion and Matt Smith just doesn't quite manage them in my opinion.)