This is actually a great question but is one that I can't actually answer properly; I think it depends on so many factors. For instance, what university you go to does make a difference. Also, it depends on the person - the experience is subjective - because some people (by which I mean me) can't adapt very well to the change of pace or style.
Most of all though, I think what makes it difficult (it isn't a bad thing that it is difficult by the way! It should be!) is the sheer number of occasions you find yourself blankly staring at maybe one proof that is written amongst hundreds in your notes and spending hours trying to understand that minute section of it that makes no sense (argh!!). Or perhaps the issue is one of the problems on your problem sheet that has frustrated you for days. Indeed, this is probably what it is. But as I've said, that is a good thing: resilience cometh.