For this post I will asssume that you are part of the investment team in AM (minority of roles yet the most selective), not in a funds/sales roles. For IB I'll assume that you are talking about IBD-M&A/Coverage.
Differences:
A lot of them, hours, sizes of teams, intensity, stress, pay, are all higher in IB.
Skills:
IB: Multitasking, sucking up bull****, managing confliciting deadlines, valuations, powerpoint, industry research, sleeping 4 hours per night, working in a team (read being ****ed by your MD).
AM: deep diving into a sector and understanding all the line items dynamics to the unit economics, valuation
Pay:
IB wins in most if not all cases unless you become a hot shot managing $Bn for your clients
IB is surely more challenging, pressured and has longer hours, but in AM people can track your performance more easily - ie do you make your loose money? I won't address the prestige bit as it's just useless. I don't think anyone working in AM would survive more than a month in IB, and loads of IB folks would get bored in AM.
Both are very competitive, would say AM investing roles is harder to get into.