Electronics and some sterics.
For a primary alcohol to undergo Sn1, you'd have to form a primary carbocation. These are generally very unstable, as alkyl groups donate electron density, and so, the more adjacent alkyl groups the greater stabilisation of the positive charge. So conversely, tertiary carbocations are much more stable due to having 3 as opposed to 1 electron donating alkyl group.
Additionally, Sn2 at a tertiary centre would be made more unlikely due to steric hinderance, difficult for the nucleophile to attack in with the (relatively) large alkyl groups surrounding, compared to the primary case.