Priority is based on atomic number first (only mass number for isotopes of the same element).
You look at the atoms directly bonded to each C of the C=C. In your examples these are often both Carbon. In this case you go to the next atoms bonded to these. When you have CH2CH2Cl and CH2Cl the next atoms in the second one include Cl which is higher atomic number than the C which is next in the first chain. This makes CH2Cl higher priority even though it is a smaller group.