You are correct, but solubility is a combination of several factors.
1. The lattice enthalpy
2. The hydration enthalpy
3. The entropy change
The first factor is endothermic and the second exothermic. Both are also affected by changing the ionic radius.
The endothermic lattice enthalpy gets smaller as the ion increases in size, but the exothermic hydration enthalpy also gets smaller.
It is the difference between these two that give the trend (if we assume that entropy change remains more or less the same)
So, if the lattice enthalpy decreases MORE than the hydration enthalpy, then overall dissolution is more favourable.