You may be delighted to see I integrated it using your substiution and obtained answer (B) as well. The correct answer is still (E) however! I'm not clear what the problem is just yet.
However, looking at the graph, there is a problem at x = 4.
but dont we have to integrate before we think about substituting the u?
edit: if u < 2 cause im squaring it its still +ve?
You didn't do that in your above post, or note that the sqaure root of a function whose square has been completed is the modulus. The integral shown a few lines above in your working is not valid for u < 2.
If you are unsure, see the 1959 IO question I posted up a few days ago. It is exactly the same as this (without the integrating), but you certainly get an idea of where the modulus function comes in.