It basically means cutting down on crap. My understanding is to eat mostly from the fruit and veg aisle (apart from condiments etc and tinned fruit/veg because it's cheaper). Healthy eating doesn't have to be expensive, and to be honest you could cook yourself a couple of meals on the amount of money you spend on a couple of packets of chocolate biscuits which will do you no good. And, although you said you're not a vegetarian, I find it really helps my budget to eat vegetarian a couple of days a week.
Personally, it's all about balance. If I ate "clean" 100% of the time then I'd never get to enjoy dessert or an Indian takeaway, so I aim for about 80% of the time. That way you get the healthiness without feeling like you're constantly on a diet, which would be utterly depressing!