EEVblog Electronics Community Forum
General => General Technical Chat => Topic started by: T3sl4co1l on April 16, 2019, 05:06:56 pm
-
Good analysis for anyone involved in projects -- engineers, managers, developers, contractors, etc.:
https://erikbern.com/2019/04/15/why-software-projects-take-longer-than-you-think-a-statistical-model.html
I've known this intuitively for a while, but it's nice to see data supporting it. :)
PSA when quoting a project for a client, say -- make sure it's clear what you're quoting, and that it may be much longer for unexpected reasons, and that they can take that into account if the need should arise.
Tim
-
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.
-
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.
That isn't working either.
The only way which works is splitting a project up in overseable tasks and then add the numbers. As the article T3sl4co1l has linked to pointed out you need to factor in some uncertainty as well. Another thing to watch out for is not to stray from the choosen path during implementation. For 9 out of 10 projects my initial estimate is just spot on.
-
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.
Hofstadter's Law:
It always takes longer than you expect, even when you take into account Hofstadter's Law. ;)
Douglas Hofstadter, "Goedel, Escher, Bach".
-
With project scheduling software, Gantt charts etc. common place in engineering, being even one day "behind" causes red squares on a screen and an uproar from management.
Endless meetings "why are you behind, how can we speed this up?" and then modifying the schedule to make it look like we are on time- instead of the real problem where things are taking too long due to a shortage of resources or a poorly organized project.
It's a common way to abuse engineers and make morale low.
I can only imagine Bob Pease and Jim Williams shaking their heads when filling out timesheets.
There's no category for exploring a neat idea that might be worthy of a patent. Obviously no category for cleaning one's workbench ;)
Careful of using time estimates where everything goes well.
I got in a fight with a boss- My estimate 4-5 days to get proto parts and he says 1 day.
Inputting the BOM, selecting parts, getting a purchase order issued, and shipping/receiving times- all add fat to the imagined time. Plus I am doing other tasks and meetings.
"I just use my personal credit card so no need for a P.O. and Digikey is next day" :palm:
So the Gantt chart was stripped of weeks worth of proto-build time. Sigh.
-
Great! I'm going to share this with the project manager... :-DD
-
To quote from the article:
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.
At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.
-
I'd agree with this overall assessment, and for engineering reasons that have nothing to do with corporate dysfunction. Let me quote a recent example.
Last week, a customer asked me to build up a prototype PCB from some left over parts I had in stock. I estimated 2.5 hrs, and I was actually done in just under 2. Score one small win for Andy.
They'd also asked me to make a minor software change to some code running on a PIC. This should have been about an hour's work.
Three hours later I'm still cursing the PIC. Occasionally, it works perfectly, but more often than not, an output which should be toggling is stuck low. Power cycle the board and it might work, but probably won't.
The fact that it sometimes works OK tells me I'm not doing anything fundamentally wrong. PIC registers all initialise to a known value, so even if I'm failing to set some essential register, that should mean it always fails.
Eventually I came to the conclusion that it's something to do with the phase relationships between different internal clock sources in the PIC. I'm 99% sure I've found a new, undocumented erratum in the device, and I don't make that claim lightly. With one small change to how a peripheral was clocked, it all started working perfectly.
Half a day wasted because of an unforeseen technical problem outside my control, and the half hour I'd saved on building the board was gone many times over.
-
To quote from the article:
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.
At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.
One thing to watch out for is the knowledge of the people doing the estimation. Having someone who doesn't understand the project break it down into subprojects they don't understand and estimate those separately should not be expected to improve their estimate.
In my view breaking things down into subprojects is mostly useful for identifying hidden unknowns that are swept under the rug in the top level project view.
-
To quote from the article:
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.
At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.
One thing to watch out for is the knowledge of the people doing the estimation. Having someone who doesn't understand the project break it down into subprojects they don't understand and estimate those separately should not be expected to improve their estimate.
In my view breaking things down into subprojects is mostly useful for identifying hidden unknowns that are swept under the rug in the top level project view.
This sounds very plausible, but it usually dead wrong. Most hidden unknowns are still hidden when you break things down. When a reasonably cautious person makes a gut feel estimate they tend to factor in a more general estimate for how many things typically go wrong on other projects of comparable size. This has a better chance of making a reasonable allowance for things going wrong.
-
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/
Tim
-
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/
Tim
If you compared that with a hardware development activity you would find the benefits of early problem detection have a much greater impact than with software, but in most other respects the picture is pretty much the same. If you are looking at chip design, the cost and time benefits of early problem detection often become really extreme.
-
This sounds very plausible, but it usually dead wrong. Most hidden unknowns are still hidden when you break things down. When a reasonably cautious person makes a gut feel estimate they tend to factor in a more general estimate for how many things typically go wrong on other projects of comparable size. This has a better chance of making a reasonable allowance for things going wrong.
It's very easy to overlook things, even when they are knowable. A lengthy itemized list looks comprehensive, but you can't spot what's not there...
Immediate example, I'm currently doing a little project that I quoted for two weeks design time. The major oversight was package size: I failed to note that I wrote a fairly high power level into the spec, which wouldn't have fit the customer's mechanicals with the original intended design. Consequently, I had to redesign the power section to take up less space. This not quite doubled the design time -- just using up the time I quoted for that stage.
This is otherwise a very ordinary project, very similar to things I've done before, so I felt confident in quoting as I did, and the project is still on schedule.
If it were a more speculative project, I'd want to perhaps quadruple the design or testing time (or both), to allow for challenges in correcting and tuning the design. A second round of prototypes may be recommended, too.
These are, of course, difficult things to sell a customer. That's the biggest reason to keep current with components, literature, technology and practice, and equally so, reason to charge fair rates which cover all the education and research that feeds into your process.
Tim
-
If you can estimate accurately then you have probably misunderstood the problem... that's why Agile development is trendy... it is a recognition that we don't understand the problem.
PS
I'm just finishing up a two week project... it's taken me three months. :-DD
-
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/
Tim
If you compared that with a hardware development activity you would find the benefits of early problem detection have a much greater impact than with software, but in most other respects the picture is pretty much the same. If you are looking at chip design, the cost and time benefits of early problem detection often become really extreme.
Indeed -- it's one thing to have just labor costs, it's another to have material costs in addition to that labor.
This is a huge driver behind simulators: when total cost can be reduced by exchanging some materials cost, or risk, with labor costs.
(I guess, since PCB-level circuit simulation has languished over the last three decades or so, that's proof of which side of the balance that's on...)
Tim
-
Q: - Why projects take longer than you think?
A: - 'Cause you're stupid!
;D
Q: - Why projects take longer than you think?
A: - 'Cause its too depressing to face how long most engineering tasks really take.
In my first job people used to keep calling my estimates pessimistic, even though things tended to take longer than I estimated. Apparantly the threshold for not being a pessimist is that things should take as least twice as long as you estimate. :)
-
Who is asking the questions about the project? What kinds of questions do they ask?
Usually those asking the questions are a collection of horizontal and vertical layers of management relative to the various engineering groups.
The questions they are asking are usually:
How long is it going to take?
When is it going to be done?
How long would it take to add this feature?
usually followed by:
Shouldn't that take just a couple days?
What questions are not being asked? How about:
Do we have a clear idea of what the product should be?
What can I do (as a manager) to clear the way? ("move furniture", as Joel Spolsky says)
Do I (at a management level) understand the existing product, the current technology, the new product, the market, etc.?
The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.
So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.
Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?
Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
-
Agile development is trendy...
Agile SW development.
Well, not trying to offend, but as a hardware developer I can testify I have never seen so much trendy bullshit in my entire life!
-
Agile development is trendy...
Agile SW development.
Well, not trying to offend, but as a hardware developer I can testify I have never seen so much trendy bullshit in my entire life!
I can see Agile being useful for something that requires a quick succession of updates, such as a web site.
I know of a hardware project that the Agile development process is being used on. I'm worried they will end up perfectly designing the wrong thing.
-
Agile works if you do it right. But it does mostly nothing to speed up the development. It is not meant to do that. Any speed ups are purely coincidental.
Agile reduces stress in the team and adds flow to the process. And for that it works perfectly. I was on a team that used it it was one the best times in my employment.
It will probably not work for hardware, I'm not sure.
-
Hardware and software are two totally different birds for time estimating and project management.
I've had S/W managers attempt to manage H/W and the result is a total disaster.
Understanding the huge differences is not easy or something you can pick up reading a paragraph on the Internet.
-
Who is asking the questions about the project? What kinds of questions do they ask?
Usually those asking the questions are a collection of horizontal and vertical layers of management relative to the various engineering groups.
The questions they are asking are usually:
How long is it going to take?
When is it going to be done?
How long would it take to add this feature?
usually followed by:
Shouldn't that take just a couple days?
What questions are not being asked? How about:
Do we have a clear idea of what the product should be?
What can I do (as a manager) to clear the way? ("move furniture", as Joel Spolsky says)
Do I (at a management level) understand the existing product, the current technology, the new product, the market, etc.?
The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.
So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.
Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?
Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
+1
Question asked about task schedule ... Just answer with even more complex question.
-
I forget the formula but there was a study of many industrial/commercial projects and how long they took vs what people expected at the start.
They came up with a formula where you stick in
- How long you expect the project to take if everything goes well.
- How long you expect the project to take if everything goes to shit.
- How many other projects are being done by the company at the same time.
- How hard the project is overall (easy, medium, hard).
etc.. And it spits out how long the project will likely take.
It's also a good idea to get some feedback into the project management cycle.
If all previous projects took on average 10% longer than expected then you add 10% to new project estimates.
Once you get a system that is accuracy estimating overall project time you can then start fine tuning individual processes to try make them faster.
The good thing is that the system self corrects. Any delays caused by management messing with processes causes the next project to pre-allocate more time to account for this.
-
Half a day wasted because of an unforeseen technical problem outside my control, and the half hour I'd saved on building the board was gone many times over.
This is what ends up killing otherwise well thought out estimates - the "unknown unknowns". Really the only things we can estimate reliably is work that we have done before, down to the last detail - and even that can go wrong!
-
The good thing is that the system self corrects. Any delays caused by management messing with processes causes the next project to pre-allocate more time to account for this.
And this is exactly how Agile development works on a high level. Not only managements screw ups, but also new hires, movements between teams and all this stuff gets factored in.
-
Half a day wasted because of an unforeseen technical problem outside my control, and the half hour I'd saved on building the board was gone many times over.
This is what ends up killing otherwise well thought out estimates - the "unknown unknowns". Really the only things we can estimate reliably is work that we have done before, down to the last detail - and even that can go wrong!
New Firmware from HW supplier.
-
The problem that typically cause overrun is "thought/planning process".
Most tend to think how things should go, then add up how long. How it should go isn't often the way things works. Unexpected turns in the corner, things doesn't react as expected...
Have a pessimist plan and manage the project progress, have an optimist manage the pessimist and the team.
-
When did the project schedule and deadlines become so absolute, hard-ass?
Estimates, ball park and jitter used to be acceptable. You could be behind or ahead a month, no biggie.
Now, I find employers bring out the Firing Squad if things are "late".
It's very bad for stressing young engineers that buy into it and work super hard to catch up to the dictated deadline. I see it burn some of them out. Very destructive, Mr. Musk.
-
When did the project schedule and deadlines become so absolute, hard-ass?
...
My sense is that shift occurs in any industry when it transitions from "high-tech" to "commodity". For instance, in successive waves appliances, radio, television, video recorders, computers, and now internet applications are making that shift. During the transition phase, the goal of being a market leader with some feature drives the whole "time to market" push, followed by cost-reduction.
We happen to discuss it now because this forum is centered on the electronics equipment and software industry.
It's just that now there really isn't a "high-tech" class of products anymore - everything is a commodity. An iPhone, which would have been mind-blowingly high-tech in the '80's, so much so that such things were almost completely missed by science fiction, is a commodity.
-
It's just that now there really isn't a "high-tech" class of products anymore - everything is a commodity.
That's an interesting observation. We have gotten used to new tech very quickly and take it 100% for granted. Probably because hardly anyone feels the innovations have threatened their livelihoods... Will this carry on with the next wave of "high tech", when it arrives (AI)?
-
Heh. I knew this. It's one of the reasons I retired early.
Because even _routine_ well understood development projects suffer from this effect, so tend to blow out and become destroyers of enjoyment of life.
But I don't want to develop ulcers over boring stuff like I used to do for a living. I want to stress out over projects I think are worthwhile. To be worthwhile they must be both cool, and things that have never been done before.
Unfortunately with _those_ types of things, the unforseen problems really blow out. Especially if a project is complex, with many interdependent strands, all independently tending to get stuck and hold up everything else. Such things cannot be pursued in a commercial context, because they are just impossible under any kind of realistic funding model. Not even crowd funding can work, because failure and absurd delay is the norm.
This is why I adopt the 'fanatical hermit scraping by in a cave' development model. More or less.
-
Engineering is always the bottle-neck doing product development. Up against impatient executives, managers, investors - it's been getting worse.
So much so that safety lapses are the only beacon saying "stop pushing and rushing us".
Not to self-blame our (engineer) estimating skills at all.
A lot of engineers are realizing "it's no fun anymore" because they started off doing fun hobby projects and then onto the corporate treadmill, which is all about doing things "as quickly as possible".
Creativity is gone, there is no reward for working harder or faster, and you get abused by being chastised for "late" efforts.
The trolls in Accounting can take weeks to issue a P.O. to Digi-key, but engineering can't waste a day.
-
I spent my entire career in construction and there was a lot of estimating and project management (software or not). Whenever I had to estimate something, I created as many line items as I could think of and estimated them separately. Then I buried 20% inside every line item. It didn't really show up as each number was still 'reasonable'.
Then the magic! I would add a 20% contingency line item entry to the subtotal knowing full well that finance was going to remove it. I never got that contingency passed the bean counters! They felt good about constraining the budget and I still had contingency in every task item.
As this was all 'internal' money, it was really just a game. Everybody had to play and I didn't want to lose. The company never did post-mortems anyway so once a job was done it was time to move on. Unless the project doubled in cost, nobody really cared.
-
Adding fat to project time estimates is really dangerous. It's not "self-correcting" doing this because you are not fixing the delays and inefficiencies within the corporation which will remain or worsen.
People can also think the time estimates are "negotiable" as if we have some strange command over space-time.
When I add a little fat to time estimates, people squawk over how long it will take and then issue an ultimatum:
Instead of hiring more people because the project is resource-limited, they just create a fake schedule to please the executives.
The engineers know the project schedule is all a fantasy, nobody listened to them, but they are then flogged when things "fall behind".
I've seen it so many times, products that are years behind schedule with engineering looking like a inept business unit.
It's actually a failure of the project management basics - listen and honour the engineer's time estimates, and add staff if required.
-
The reason for breaking a project down into as many line items as possible is to get as close to understanding the real 'scope of work' as possible. There are so many projects where the total scope is unknown that it is just better to try to get a handle on it right up front. In my business it was things like asbestos abatement, code changes since the original construction, handicap access, and on and on. Not necessarily obvious at first glance.
Break it all down, understand the true scope of work.
The reason for padding the estimate of each line item is that it will all be contracted. I couldn't control the incoming quotes, all I could do was try to use the cost of previous work to forecast this project. Sometimes we would go out to contractors and get bids for various trades (general construction, electrical, HVAC, painting, engineering, etc) but not always. Sometimes it was design/build and most sometimes we would just wing it.
In a perfect world, there would be plans and specs on the work before it went out to bid. Alas, the world isn't perfect and we didn't have time to serialize the project. It's OK to wing it! Just don't crash and burn.
-
Engineering is always the bottle-neck doing product development. Up against impatient executives, managers, investors - it's been getting worse.
Creativity is gone, there is no reward for working harder or faster, and you get abused by being chastised for "late" efforts.
The trolls in Accounting can take weeks to issue a P.O. to Digi-key, but engineering can't waste a day.
In your first sentence, you forgot to include marketing. Their favorite spiel: Our main competitor ALREADY HAS gone into production with their product, and they will steal ALL of our customers.
In your second sentence, my experience is: if somehow you were able to perform a miracle, upper management will expect that you perform similar feats routinely. That will eventually burn you out.
In your third sentence, it appears that you and I work for the same company!! What I will do, if I really require something urgently, I'll charge it to my personal CC, and then getting reimbursed from the petty cash account.
-
I find the general mechanism in the OP credible, and it simplifies the problem enough to study the other end of the problem. Now that you know some projects are going to blow up, and have a statistical estimate for it, you can evaluate risk in proposals better. You can't just bid the worst case, you will never get work. You can't bid the best case, you will always lose money. Now you can get a justifiable guess for the point at which you will on average come out ahead.
Which only leaves one problem. There will almost always be someone more desperate or ignorant than you who will bid below the break even line. Leading to the management despair and pressure described in many of the posts here.
-
Catalina;
It can actually become worse....if the design is for a cost-sensitive, high volume product. The design engineers, faced with relentless cost and time pressures will take shortcuts, which will translate into daily pain for the manufacturing facility.
-
Catalina;
It can actually become worse....if the design is for a cost-sensitive, high volume product. The design engineers, faced with relentless cost and time pressures will take shortcuts, which will translate into daily pain for the manufacturing facility.
That MAY be just a variant of what I said. The owners of the manufacturing facility could in principal know that those short cuts will lead to pain, in fact that they will lead to financial losses on the project unless they get lucky. But the alternative is to skip the shortcuts, develop a solid product, and get none of the market. It is a choice between the possibility of making money (if you are lucky) and the assurance of losing money if you do it right. Both choices suck, and the engineers and manufacturing guys don't like the better one.
-
On the subject of generating elapsed-time effort estimates. No other part of the project preparation or execution process is fraught with as much self-deception, inaccuracy and even blatant horsehockey as this part is.
I am sure that most, if not all engineers, have at some time been suddenly asked by a superior to make a prediction of performance in one circumstance or other. Many times when one is put into such a position the temptation to give a very pleasing response cannot be wholly avoided. One wants to say that it can be done in two weeks. Only later when it actually turns out to be two months is the truth revealed. This occurs because at the moment we are asked, an astonishing thing happens: we become super-engineers.
We start to think that we can do the hardware design almost as fast as it takes to scribble it down on paper or get it captured to CAD...
We think that any meeting will take only 5-10 minutes because nobody will argue...
We think that debugging will be short because nothing will go wrong...
Design reviews will not turn up any unexpected problems...
Software will work first time as hoped...
We will be at best efficiency for the duration of the project...
Nobody will get sick..
We will not wake up with a hangover from the bad weekend before...
Why does this happen?
It happens for many reasons some of which are given below but most importantly, it happens because we choose to allow it to happen. If we can delude ourselves into thinking that nothing bad will happen as a result then we may not correct ourselves as we should.
It happens because at the time we are asked to quote on the task, we are faced with the unpleasant possibility that our audience (the client or supervisor) may not like what he hears. We don't want to face the reaction that may be forthcoming if s/he doesn't like it. So we tend to bend our belief in ourselves in order to deliver a more pleasing (if completely unrealistic) picture.
I have seen people actually break out in a sweat and stammer at this point.
Has anybody used the Delphi technique or the "Wide-Band" Delphi" technique to job estimation. I hear it is supposed to be somewhat accurate. At least more accurate than just asking the doers what they think because it forces those whose estimate disagree with the groups average estimate to justify their answers. This is how it is described (my words):
The Delphi Technique:
The task or task list is then shopped around to each member of the group for their opinion on the work effort required for each task. They each should make their best estimation of the time needed, in days/weeks, not hours, for each task and report it to s/he who is responsible for collecting them. The results are tabulated and labelled as the “First Approximation” or some such. As with most things these results will show most guesses for each task grouped around a mean with a few which fall significantly away from the mean value. Those group members who submitted the latter sort are asked, in the presence of the entire group, to explain their guess. After listening to all of the arguments each group member is asked to guess again on the entire list of tasks. This new set of values is tabulated. Lo! And Behold! The new “Second Approximation” values fall in more closely collected sets around new mean values with fewer anomalous values. A second discussion is held. If these new values still do produce anomalous values then a third set of guesses may be asked for. This is usually as far as the technique will be useful (and as far as the patience of the group will allow).
The "Wide-Band" Delphi" technique is the Delphi Tech combined with some arithmetic based on the Three-Point (Optimistic, Pessimistic, Most Likely).
-
The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.
So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.
Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?
Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
Imagine this.... (I lived it)
The year is 2013. I'm the lead industrial engineer for the largest rift and quartered hardwood sawmill in the United States. We're in the middle of trying to turn the mill around after some crazy mismanagement by previous owners -- the bank had taken over the mill, and an investment firm had since bought it from the bank. We needed production of at least 61,000 BFD per day. The mill was at 14,000 - 18,000 BFD when I was brought in to turn it around.
I started going through all of the operations to see what was going on. An entire planing mill was down, but why? They were out of $0.11 bulbs for the Optical Comparator used to sharpen the planer knives and nobody cared enough to let someone higher up know. An 11 cent bulb was costing $120k / day in lost production. I ordered a box of 100, stocked 10 in the supply cabinet by the comparator, and put the remaining 90 in the designated area in maintenance storage. Total cost: $11.70 delivered.
Next, the sawmill was having trouble keeping blades sharp. It was basically a giant bandsaw mill -- the blades were about 30 feet in diameter, about 1/8" thick, and 8-10" across. What was the problem? Apparently some of the control boards for the automatic sharpener had let out their magic pixies and so the machine had to be manually operated because there was no limit function or reset working. The part was obsolete, but I managed to find one and had it overnighted. Total cost: $271.86 shipped.
Meanwhile, I had the dead board sent to my office. I rebuilt it, made a new version in Eagle, and was ready to order new PCBs to have on stand-by.
So, for $283.56 I brought mill production from 15,000 BFD to 45,000 BFD. In other words, that investment made back $90,000 in lost productivity.... PER DAY.
I put together a massive SQL system to make it easier to run reports and track production, and had brought production back to 58,000 BFD when they sacked me for "spending too much money". I had spent just over $800 to get to this point. That $800 total earned back $129,000 in losses per day. But hey, I was spending too much money. They're virtually bankrupt now. Production is only a few thousand BFD a month now.
-
Some manufacturing workplaces don't want everything working.
They like a leisurely pace and I've seen crews sabotage equipment so they get to do very little work on their shift, with the line down. Or sabotage anyone's work to fix things.
Or they are so cheap and the rule is to spend absolutely nothing. Not a penny. They flip out if you splurge and buy paper towels or a screwdriver. $800 is outrageous ;)
-
...nobody cared enough...
There's the problem, right there in a nutshell.
-
The problem that typically cause overrun is "thought/planning process".
Most tend to think how things should go, then add up how long. How it should go isn't often the way things works. Unexpected turns in the corner, things doesn't react as expected...
Have a pessimist plan and manage the project progress, have an optimist manage the pessimist and the team.
yup :D
-
I have a solution for this whole thing that works for me:
I charge 4x my initial (internal) estimate and give 10% back if I complete the work early. Surprisingly people piss their pants excitedly about being ripped off in this way, especially the second and third times :-DD
-
When my team planned the Sonoko project (http://www.downthebunker.com/reloaded/space/viewforum.php?f=37), we were not considering a lot of things do not react as expected, for example a big problem (https://bugzilla.kernel.org/show_bug.cgi?id=195933) with the Linux kernel, and ZERO collaboration from guys on the internet.
the mistake was: to assume that someone was still interested in legacy ppc405 chip
so we are still all alone with a lot of unsolved problems :-//
-
The trick is to go with all the popular shit even if you hate it because you can usually google your way out of a hole faster than having to shape a new wheel.
-
Very interesting read. I fully appreciate the investigation.
I am actually wondering how this applies to HW dev. Because I have to make estimations about my time, and sometimes I really have hard time explaining what is going on. Take for example, they want a widget for x amount of money, and they want it in 6 months. It would take something like a few days to actually make something that would do what the widget does, and then some time for the others to code something, however a bunch of things happen in real life.
A) I cannot spend 100% of my time on the project cause other things are as well important.
B) We need a second and a third prototype, because one little thing is broken
C) Because there is a price target, I have to have several designs made, organize meetings with suppliers, manufacturers, gather information about prices, availability, compare them, make several effort estimates. Even organizing the meetings is several weeks.
D) You need a custom thingamabob for it, where the tooling needs several months to be made.
-
C) Because there is a price target, I have to have several designs made, organize meetings with suppliers, manufacturers, gather information about prices, availability, compare them, make several effort estimates. Even organizing the meetings is several weeks.
D) You need a custom thingamabob for it, where the tooling needs several months to be made.
For these pair of items, it usually means having things made in China. This is where The Troubles begin:
From the "lost-in-the-translation" discussions, to the crazy meeting hours (in Chinglish, of course), to their holidays not aligning with ours, to the way the see things vs. your point of view...... and of course, receiving the first production batch which will be found to have a "minor" issue that will render them unusable, more crazy meetings attempting to figure out the issue, agreeing on sorting criteria, figuring out a correct rework procedure, dealing with the inevitable scrap, paying overtime fees to meet deadlines.... and finally having to air-freight the reworked material.
In the end, the cost will be several times the original quote.
-
The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.
So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.
Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?
Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
Imagine this.... (I lived it)
The year is 2013. I'm the lead industrial engineer for the largest rift and quartered hardwood sawmill in the United States. We're in the middle of trying to turn the mill around after some crazy mismanagement by previous owners -- the bank had taken over the mill, and an investment firm had since bought it from the bank. We needed production of at least 61,000 BFD per day. The mill was at 14,000 - 18,000 BFD when I was brought in to turn it around.
I started going through all of the operations to see what was going on. An entire planing mill was down, but why? They were out of $0.11 bulbs for the Optical Comparator used to sharpen the planer knives and nobody cared enough to let someone higher up know. An 11 cent bulb was costing $120k / day in lost production. I ordered a box of 100, stocked 10 in the supply cabinet by the comparator, and put the remaining 90 in the designated area in maintenance storage. Total cost: $11.70 delivered.
Next, the sawmill was having trouble keeping blades sharp. It was basically a giant bandsaw mill -- the blades were about 30 feet in diameter, about 1/8" thick, and 8-10" across. What was the problem? Apparently some of the control boards for the automatic sharpener had let out their magic pixies and so the machine had to be manually operated because there was no limit function or reset working. The part was obsolete, but I managed to find one and had it overnighted. Total cost: $271.86 shipped.
Meanwhile, I had the dead board sent to my office. I rebuilt it, made a new version in Eagle, and was ready to order new PCBs to have on stand-by.
So, for $283.56 I brought mill production from 15,000 BFD to 45,000 BFD. In other words, that investment made back $90,000 in lost productivity.... PER DAY.
I put together a massive SQL system to make it easier to run reports and track production, and had brought production back to 58,000 BFD when they sacked me for "spending too much money". I had spent just over $800 to get to this point. That $800 total earned back $129,000 in losses per day. But hey, I was spending too much money. They're virtually bankrupt now. Production is only a few thousand BFD a month now.
Good going Electromotive! Every ops manager should read this. Come to think of it, so should every developer. Sometimes, we forget how much it costs per hour when a production line is down.
-
Because if I get bored/annoyed with them, they have to sit in some dark corner until I get bored/annoyed with something else and need something else to work on ;D
-
Because if I get bored/annoyed with them, they have to sit in some dark corner until I get bored/annoyed with something else and need something else to work on ;D
Thanks for a refreshing perspective after all the whining... ;)
-
Engineering is always the bottle-neck doing product development. Up against impatient executives, managers, investors - it's been getting worse.
So much so that safety lapses are the only beacon saying "stop pushing and rushing us".
Not to self-blame our (engineer) estimating skills at all.
A lot of engineers are realizing "it's no fun anymore" because they started off doing fun hobby projects and then onto the corporate treadmill, which is all about doing things "as quickly as possible".
Creativity is gone, there is no reward for working harder or faster, and you get abused by being chastised for "late" efforts.
The trolls in Accounting can take weeks to issue a P.O. to Digi-key, but engineering can't waste a day.
These days it seams all about galley slaveing, rowing the corporate boat to the whip of the greedy masters:
(https://static1.squarespace.com/static/558dadc5e4b0813627c48032/t/56c85d0c07eaa0772e101abb/1487687317582/Roman+Galley+Slaves)
Why galley slaving when FED prints the fiat bills anyway! Picking lingonberries mending the gardening rather then the boat, life to short to bow to wannabe masters.
-
when you total up the responses to the thread the answer is simple: its completely random, making mangers gamblers. :-DD
-
also meetings, its a wild factor
-
And engineering turnover.
At the project’s eleventh hour, a key engineer who had been with the project since its conception, quits and a poor rookie is hired to lead the project to completion.
Barely.
-
The original article is about software. A confounding issue in software is that you are ALWAYS doing something that has never been done before. Or else you're an idiot.
In many other endeavors, your experience on the previous job helps you estimate the next one. If you've built ten houses of 1500 square feet, then you've probably got a pretty good idea of how long the eleventh one will take.
But in software, if you ever find yourself doing something that's similar to stuff you did last year (or last week), or stuff you expect to do next year/week, then you'll write a program to automate the task, and the time required for that repetitive task drops to near zero. There's nothing better than a computer to handle repetitive work.
These days, many software tasks consist of finding the right libraries and tools, figuring out how they work, and hooking them together. Uncommon tasks still may require writing complex custom software, but if you're writing complex custom software that's similar to what you wrote last time, you should put it into a library so that you can efficiently re-use what you previously wrote.
The result is that wise and productive programmers spend all their time doing stuff they've never done before. And doing new stuff is always hard to estimate.
That may also be true to some degree for some other fields, where people always find themselves doing new things. The thing about software, though, is that it is so easy to automate the repetitive parts, so as soon as there's repetition, it's automated and the time required for that part of the job goes away.
Contrast software with jobs like lawn mowing. I mow my own lawn, and I can always estimate the time required with exquisite precision, because the task is well understood and I don't get better at it.
-
The original article is about software. A confounding issue in software is that you are ALWAYS doing something that has never been done before. Or else you're an idiot.
In many other endeavors, your experience on the previous job helps you estimate the next one. If you've built ten houses of 1500 square feet, then you've probably got a pretty good idea of how long the eleventh one will take.
But in software, if you ever find yourself doing something that's similar to stuff you did last year (or last week), or stuff you expect to do next year/week, then you'll write a program to automate the task, and the time required for that repetitive task drops to near zero. There's nothing better than a computer to handle repetitive work.
These days, many software tasks consist of finding the right libraries and tools, figuring out how they work, and hooking them together. Uncommon tasks still may require writing complex custom software, but if you're writing complex custom software that's similar to what you wrote last time, you should put it into a library so that you can efficiently re-use what you previously wrote.
The result is that wise and productive programmers spend all their time doing stuff they've never done before. And doing new stuff is always hard to estimate.
That may also be true to some degree for some other fields, where people always find themselves doing new things. The thing about software, though, is that it is so easy to automate the repetitive parts, so as soon as there's repetition, it's automated and the time required for that part of the job goes away.
Contrast software with jobs like lawn mowing. I mow my own lawn, and I can always estimate the time required with exquisite precision, because the task is well understood and I don't get better at it.
Interesting point of view. But clearly not totally accurate. It is foolish to automate everything you have done before. There should be a trade between the non-zero costs of automating the task and the estimate of the number of times you will do the task again. And there you are again, predicting the future with inadequate data.
To use your house building analogy, obviously if you expect to build 1500 square foot houses with the same layout over and over again you will automate. Prefabricate sections. Build jigs and other tooling. If you have just built one and don't have a customer on the line for the second one you don't know whether to automate any part of the task. And in both houses and software the next customer will often want something "better". 1450 square feet. Or 2500 sq. ft. Now obviously that is a situation that begs for automating with a parameter. But the obvious problem is that a scale factor isn't enough. You probably have to add rooms as the size goes up. And maybe trim level options. Regional variations. (Ranch style for Californians, Salt Boxes for New Englanders...). All of which can be automated, but pretty soon the automation is a major project and you still haven't lined up the second customer.
Most design jobs have the qualities you ascribe to software. Less akin to lawn mowing where you are doing the same thing over and over. But in both software and hardware there are some jobs which pretty predictable. The heavens may fall on me for saying this but web page design is one. And I used to work for a power company "designing" new service installations. I was an intern between undergraduate years in college and had been over educated and over qualified for the job for several years when I took it.
-
Building the automation system will have its problems over and above the problems of building the base product. Subtle variations that an experienced human would just cope with without a second thought become intricate and sometimes very difficult to automate. For starters, you would have to know that you need to capture that information and then you would have to encode it for repetitive execution. Plus you need to include some mechanism for identifying cases that are outside those which have been encoded AND take an appropriate action - such as pausing for human intervention.
But, back on the original point:
The only way which works is splitting a project up in overseable tasks and then add the numbers.
I once had a software project where I was the team lead. The project was to create a new system based on an existing one, so a lot of the curly bits had already been addressed. I did the project estimate by breaking everything down to the basic components and then looking at each and asking myself "How long would it take me to do this on a good day?". I then doubled that number. I took the project total of 120 person-days to management and after an hour's "discussion" it got whittled back to 60. :palm: At the end of the project, I checked the actual time taken: 125 person-days - but I still didn't get the project in on time, did I? >:(
-
Less akin to lawn mowing where you are doing the same thing over and over.
Even lawn mowing is surprisingly complicated if you do a lot of it. Tools... tractor, riding mower, self propelled push, push, manual? Totally automated lawn robot? Strimmer? Electric? Battery? Two stroke? Four stroke? etc. -
Efficient lawn mowing depends on knowing the specific lawn in advance, or you end up like the garden companies, with a trailer full of different implements!
-
And when the lawnmower get an system add on like MCAS your in deep shit! And out the window goes everything!
-
C) Because there is a price target, I have to have several designs made, organize meetings with suppliers, manufacturers, gather information about prices, availability, compare them, make several effort estimates. Even organizing the meetings is several weeks.
D) You need a custom thingamabob for it, where the tooling needs several months to be made.
For these pair of items, it usually means having things made in China. This is where The Troubles begin:
From the "lost-in-the-translation" discussions, to the crazy meeting hours (in Chinglish, of course), to their holidays not aligning with ours, to the way the see things vs. your point of view...... and of course, receiving the first production batch which will be found to have a "minor" issue that will render them unusable, more crazy meetings attempting to figure out the issue, agreeing on sorting criteria, figuring out a correct rework procedure, dealing with the inevitable scrap, paying overtime fees to meet deadlines.... and finally having to air-freight the reworked material.
In the end, the cost will be several times the original quote.
Well, not really. We make high quality stuff, high margin. Local assembly house, sourcing and design. B2B.
But that doesn't mean that I can just go crazy with the designs whenever I want.
I still have to do that most of the time, because there just isn't time. We told our manager, that we could employ two extra engineers who could do just cost cutting, cause there is so much extra money left in the designs. You know, when you have only a few months to spit out a product, you over engineer.
-
I have been pondering this for about 30 years, ever since I started writing software professionally.
Of course, the cart is always put before the horse. People ask "why is the project taking longer than it should", rather than "why was the time estimate wrong". For some reason, I guess because it comes first, the estimate is regarded as gospel truth, and if the actual work didn't match the estimate then it must be due to bad developers.
This despite the fact it is pretty generally known you can't predict the future. Software projects simply contain too many "unknown unknowns" to come up with accurate, detailed estimates.
I have noticed that the unknown unknowns all lead to more work required, there are vanishingly few times when it is unexpectedly discovered that less work is required. Even the known unknowns are badly accounted for, e.g. a study found developers only spent 50% of the time coding, even though they are nominally 100% on a project. The "lost time" is all the workplace overhead, like meetings, training, holidays, sickness, supporting legacy projects etc.
On the flip side, all the drivers from business management are the wrong way. i.e. they want less time, less cost, less resources. There is a strong tendency for them to refuse to listen to anything they don't want to hear. If they get an answer that doesn't fit, they ask again until they get an answer they like. Even if the developers are not "yes" men, the management will always take their most optimistic estimate.
It's a classic case where the "wisdom of crowds" breaks down, because there is a strong bias towards the "least cost" answer, rather than the median answer.
Unfortunately, I have come to the conclusion that the naive and ignorant people in charge always win the argument, so the industry will never improve.
-
...
On the flip side, all the drivers from business management are the wrong way.
...
A running joke here is something like:
Manager: "When will that task be done?"
Engineer: "August."
Manager: "Marketing August, or Engineering August?"
Marketing August is the 1st. Engineering August is the 31st.
-
Of course, the cart is always put before the horse. People ask "why is the project taking longer than it should", rather than "why was the time estimate wrong".
Often things are underestimated for the "positive" reason that people want to do the project... because they think it is cool or find it satisfying for other reasons, so they tend to underestimate how hard it will be, and/or overestimate how good the payoff will be!
-
I recently had to estimate a number of tasks for a project, and assign cursory time estimates to the various tasks.
From a simple linear point of view, the time estimates usually take the form of BASE +/- ERROR or BASE +/- %ERROR. With this one gets estimates like:
40 hours +/- 8 hours.
5 weeks +/- 20%.
Or other such things.
However, with a very technical project that requires a lot of research to even understand the implications of some of the requirements, this really isn't adequate and presumes I understand much more about how to accurately estimate tasks than is possible at the early stages.
Therefore, when I presented my list of tasks with a time associated with each task, I included the disclaimer that the times in hours were accurate to a factor of 5. How can this be concisely stated? By using dB(t)!
So I get estimates like:
40 hours +/- 14 dB(h) for a range of 8 hours to 200 hours.
5 weeks +/- 14 dB(w) for a range of 1 week to 25 weeks.
Close enough for engineering work.
-
So I get estimates like:
40 hours +/- 14 dB(h) for a range of 8 hours to 200 hours.
5 weeks +/- 14 dB(w) for a range of 1 week to 25 weeks.
Close enough for engineering work.
And the best thing: Nobody will ask you for estimates again anytime soon! ::)