The 10X productivity thing may be a myth. What I've seen in 35+ years programming is developers fall into three broad categories:
1. Can't get anything done on schedule or in budget.
2. May eventually get something done, late and over budget.
3. Delivers what the customer needs on time and within budget.
The first two groups just cost the customer time and money, create uncertainty and stress for the customer, and may or may not deliver something of value. A better developer doesn't have to be 10x better (however that is measured). They only have to deliver what the customer needs within the time and budget parameters.
From the customer's point of view the relevant scale is not 1 - 5 - 10 times productivity from the developer. The relevant scale is 0 - 1, where anything much less than 1 means their business problems weren't addressed in a useful way. It doesn't really matter that one programmer had 10x more commits on GitHub than another programmer if the customer can't ship packages because the programmer didn't correctly add shipping charges to the invoice (real example).
Anyone who is freelancing, leading a project or team, or calling themselves a ninja, rockstar, wizard, etc. should not be surprised or indignant that customers don't understand their requirements and usually can't communicate their needs in a complete and consistent format. That problem has been known since the 1950s. Blaming the customer is not an option. An effective developer will concentrate on business needs, ask the questions and get the answers, and work with the customer to nail down requirements.
When I get involved with a broken/stalled project the first thing I do is ask the customer to make a short list of the problems and missing features in decreasing priority order. What I always get, without exception, are business requirements, not technical glitches. The root cause of a missing business requirement may be technical (e.g. failed PCI compliance audit because of vulnerable PHP code), but all the customer cares about is the business requirement. It doesn't matter if I am 10x faster or smarter than whomever was working on the code before as long as I can fix the problem.
>...developers fall into three broad categories: 1. Can't get anything done on schedule or in budget. 2. May eventually get something done, late and over budget. 3. Delivers what the customer needs on time and within budget.
And since software estimates is a known hard problem[1], the only different between 2 and 3 is often just the fact that 3 has learned to appropriately inflate his/her estimates.
I say that as someone who has oscillated between 2 and 3 over my career, although the trend over time is toward 3.
I was making a distinction between the customer's schedule and budget and the developer's estimates.
If your customer (or employer) has no schedule or budget, great, you've landed a job at the next cool startup.
For every other business that is paying for software development the value of the software usually has budget and time constraints. When a customer/employer asks developers for an estimate they are either asking for a commitment to deliver within time/budget constraints, or starting a negotiation where features are changed or removed, or "resources" (more developers) are added.
You are right that it's very hard to get estimates right -- one of the biggest unsolved problems in software development, and one that has been known and written about for decades.
For the jobs I take on, I ask the customer to break down the work into a list of bugs or missing features or missed requirements. Then I ask the customer to put the list in priority order and assign a dollar value and schedule. Then I determine if I believe I can commit to addressing the list, item by item, within their cost/schedule constraints. If I don't think so I tell the customer, and they either change their requirements and constraints, or they go look for someone else -- I don't charge for determining if I can commit or not.
You can do this in terms of a MVP (minimum viable product). If my customer has nothing -- green fields development -- another way to look at that is none of their requirements have been met. So let's put those requirements in priority order and assign maximum costs and schedule to them. I've never been entirely successful at getting customers to actually do this for new development (one reason I rarely take on green fields projects anymore), but when I have tried it they have generated requirements with costs and times that are at least as realistic than any estimate I might attempt.
I think this is an excellent approach. The only problem is finding clients that are both capable and willing to sit down and hammer out this type of informal spec.
1. Can't get anything done on schedule or in budget. 2. May eventually get something done, late and over budget. 3. Delivers what the customer needs on time and within budget.
The first two groups just cost the customer time and money, create uncertainty and stress for the customer, and may or may not deliver something of value. A better developer doesn't have to be 10x better (however that is measured). They only have to deliver what the customer needs within the time and budget parameters.
From the customer's point of view the relevant scale is not 1 - 5 - 10 times productivity from the developer. The relevant scale is 0 - 1, where anything much less than 1 means their business problems weren't addressed in a useful way. It doesn't really matter that one programmer had 10x more commits on GitHub than another programmer if the customer can't ship packages because the programmer didn't correctly add shipping charges to the invoice (real example).
Anyone who is freelancing, leading a project or team, or calling themselves a ninja, rockstar, wizard, etc. should not be surprised or indignant that customers don't understand their requirements and usually can't communicate their needs in a complete and consistent format. That problem has been known since the 1950s. Blaming the customer is not an option. An effective developer will concentrate on business needs, ask the questions and get the answers, and work with the customer to nail down requirements.
When I get involved with a broken/stalled project the first thing I do is ask the customer to make a short list of the problems and missing features in decreasing priority order. What I always get, without exception, are business requirements, not technical glitches. The root cause of a missing business requirement may be technical (e.g. failed PCI compliance audit because of vulnerable PHP code), but all the customer cares about is the business requirement. It doesn't matter if I am 10x faster or smarter than whomever was working on the code before as long as I can fix the problem.