Most startups have no need for an app, so don't create them.
Startups which have an app as part of their business model often have founders who are able of creating apps, as that's why they've created the startup. (They may have worked as app developers for another company, for example, before deciding to branch out in their own.)
Startups which have an app as part of their business model but don't have the expertise to create one will need to hire an individual or a company to build it. As you say, this is not cheap. If the founders cannot fund this themselves, then they will have to secure outside funding. Given that their business model requires an app, then they have little choice but to do so.
Just to reiterate my first point, this isn't an issue for the vast majority of startups as they have no need for an app. They may want to have one, but fulfilling that desire will need to wait until the business has generated sufficient profit that doing so becomes viable.