Maybe not a first hire, but a lot of startups hire them too late. There's a lot of extra stuff that gets done to run even a small office or team. The right admin can do an immense amount of things. Interview support, basic book keeping, office logistics, food, basic HR & payroll, fielding phone calls, scheduling calls, etc.
I think it depends a lot on the nature of the startup. If you're doing things that don't scale, then you might have enough work for a part-time general admin/support person quite early on.
The idea is to find the things where there's not a lot of value in having the founder do the work and find the cheapest way to get it done.
There's a few options for how to do that. You can go to a temp agency, or you can get a recent university graduate with generalist skills pretty cheaply.
That's not true for all startups, nor even most, but I've seen startups where the founder was spending more than a day a week doing data entry.
