I'm gonna do this as two posts to split up the how and the why. Here is why and some hints at how:
Perf analysis was how I got through some of my most mundane college homework assignments instead of (or sometimes, in addition to) procrastinating.
Then during my formative years I saw the following script play out several times:
Management and sales have a problem. They say the customers are upset that the code is so slow (maybe the customer has expectations, or maybe a competitor or their old system went a lot faster). It's taking 300 units of time now and they think it should take 100. So the leads and the heavies wander off and come back with 20% easy, the next 10 harder, 6, 4, and then they declare that they've done all they can. So now we're around 180 units of time. That's all the customer gets. See this <flame chart precursor> chart? Everything is nice and flat. Our hands are tied now. Business is... not happy. They're glad they have something to show the customer, but it's really just a bone.
So between tasks I'd start poking around, because why not. A pinch here, an architectural tweak there, and maybe 25% just gruntwork, and I'd find another 40% out of the "we've done all we can" code, which is just a hair away from what the business asked for.
You can imagine that you invite a tremendous amount of well-deserved scrutiny on yourself doing stuff like this. But that's okay: the most important part of optimization is not the code speed but the bang for the buck. You can certainly interpret "Low hanging fruit" to fulfill that concern, in practice most people don't (for some, lack of imagination, for others, ego-protection). But once you consider everything, including, for instance, externalities on QA, or the misapprehensions of customers who expect every release to be faster... there are a lot of more complicated things to worry about than how to rearrange this block of code to improve branch prediction. Not touching it for 10% is just a heuristic.
Perf analysis was how I got through some of my most mundane college homework assignments instead of (or sometimes, in addition to) procrastinating.
Then during my formative years I saw the following script play out several times:
Management and sales have a problem. They say the customers are upset that the code is so slow (maybe the customer has expectations, or maybe a competitor or their old system went a lot faster). It's taking 300 units of time now and they think it should take 100. So the leads and the heavies wander off and come back with 20% easy, the next 10 harder, 6, 4, and then they declare that they've done all they can. So now we're around 180 units of time. That's all the customer gets. See this <flame chart precursor> chart? Everything is nice and flat. Our hands are tied now. Business is... not happy. They're glad they have something to show the customer, but it's really just a bone.
So between tasks I'd start poking around, because why not. A pinch here, an architectural tweak there, and maybe 25% just gruntwork, and I'd find another 40% out of the "we've done all we can" code, which is just a hair away from what the business asked for.
You can imagine that you invite a tremendous amount of well-deserved scrutiny on yourself doing stuff like this. But that's okay: the most important part of optimization is not the code speed but the bang for the buck. You can certainly interpret "Low hanging fruit" to fulfill that concern, in practice most people don't (for some, lack of imagination, for others, ego-protection). But once you consider everything, including, for instance, externalities on QA, or the misapprehensions of customers who expect every release to be faster... there are a lot of more complicated things to worry about than how to rearrange this block of code to improve branch prediction. Not touching it for 10% is just a heuristic.