Author Topic: Why projects take longer than you think  (Read 4138 times)

0 Members and 1 Guest are viewing this topic.

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 15449
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Why projects take longer than you think
« on: April 16, 2019, 05:06:56 pm »
Good analysis for anyone involved in projects -- engineers, managers, developers, contractors, etc.:

https://erikbern.com/2019/04/15/why-software-projects-take-longer-than-you-think-a-statistical-model.html

I've known this intuitively for a while, but it's nice to see data supporting it. :)

PSA when quoting a project for a client, say -- make sure it's clear what you're quoting, and that it may be much longer for unexpected reasons, and that they can take that into account if the need should arise.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 
The following users thanked this post: SeanB, thm_w, a59d1

Offline alanb

  • Frequent Contributor
  • **
  • Posts: 344
  • Country: gb
Re: Why projects take longer than you think
« Reply #1 on: April 16, 2019, 06:38:35 pm »
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 19839
  • Country: nl
    • NCT Developments
Re: Why projects take longer than you think
« Reply #2 on: April 16, 2019, 06:42:46 pm »
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.
That isn't working either.
The only way which works is splitting a project up in overseable tasks and then add the numbers. As the article T3sl4co1l has linked to pointed out you need to factor in some uncertainty as well. Another thing to watch out for is not to stray from the choosen path during implementation. For 9 out of 10 projects my initial estimate is just spot on.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline ebastler

  • Super Contributor
  • ***
  • Posts: 3570
  • Country: de
Re: Why projects take longer than you think
« Reply #3 on: April 16, 2019, 06:55:30 pm »
Double the number and move to the next unit type.
a one hour task becomes 2 days, a one day task becomes 2 weeks, a one week task becomes 2 months etc.

Hofstadter's Law:
It always takes longer than you expect, even when you take into account Hofstadter's Law.  ;)

Douglas Hofstadter, "Goedel, Escher, Bach".
 

Offline floobydust

  • Super Contributor
  • ***
  • Posts: 3593
  • Country: ca
Re: Why projects take longer than you think
« Reply #4 on: April 16, 2019, 07:04:47 pm »
With project scheduling software, Gantt charts etc. common place in engineering, being even one day "behind" causes red squares on a screen and an uproar from management.

Endless meetings "why are you behind, how can we speed this up?" and then modifying the schedule to make it look like we are on time- instead of the real problem where things are taking too long due to a shortage of resources or a poorly organized project.
It's a common way to abuse engineers and make morale low.
I can only imagine Bob Pease and Jim Williams shaking their heads when filling out timesheets.
There's no category for exploring a neat idea that might be worthy of a patent. Obviously no category for cleaning one's workbench ;)

Careful of using time estimates where everything goes well.
I got in a fight with a boss- My estimate 4-5 days to get proto parts and he says 1 day.

Inputting the BOM, selecting parts, getting a purchase order issued, and shipping/receiving times- all add fat to the imagined time. Plus I am doing other tasks and meetings.
"I just use my personal credit card so no need for a P.O. and Digikey is next day"  :palm:
So the Gantt chart was stripped of weeks worth of proto-build time. Sigh.
 

Offline luismh

  • Newbie
  • Posts: 4
  • Country: ar
Re: Why projects take longer than you think
« Reply #5 on: April 16, 2019, 07:39:52 pm »
Great! I'm going to share this with the project manager... :-DD
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 5654
  • Country: gb
Re: Why projects take longer than you think
« Reply #6 on: April 16, 2019, 07:52:25 pm »
To quote from the article:
Quote
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.

At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.
 

Offline AndyC_772

  • Super Contributor
  • ***
  • Posts: 3631
  • Country: gb
  • Professional design engineer
    • Cawte Engineering | Reliable Electronics
Re: Why projects take longer than you think
« Reply #7 on: April 16, 2019, 07:55:45 pm »
I'd agree with this overall assessment, and for engineering reasons that have nothing to do with corporate dysfunction. Let me quote a recent example.

Last week, a customer asked me to build up a prototype PCB from some left over parts I had in stock. I estimated 2.5 hrs, and I was actually done in just under 2. Score one small win for Andy.

They'd also asked me to make a minor software change to some code running on a PIC. This should have been about an hour's work.

Three hours later I'm still cursing the PIC. Occasionally, it works perfectly, but more often than not, an output which should be toggling is stuck low. Power cycle the board and it might work, but probably won't.

The fact that it sometimes works OK tells me I'm not doing anything fundamentally wrong. PIC registers all initialise to a known value, so even if I'm failing to set some essential register, that should mean it always fails.

Eventually I came to the conclusion that it's something to do with the phase relationships between different internal clock sources in the PIC. I'm 99% sure I've found a new, undocumented erratum in the device, and I don't make that claim lightly. With one small change to how a peripheral was clocked, it all started working perfectly.

Half a day wasted because of an unforeseen technical problem outside my control, and the half hour I'd saved on building the board was gone many times over.

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 2158
  • Country: us
Re: Why projects take longer than you think
« Reply #8 on: April 16, 2019, 08:17:30 pm »
To quote from the article:
Quote
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.

At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.

One thing to watch out for is the knowledge of the people doing the estimation.  Having someone who doesn't understand the project break it down into subprojects they don't understand and estimate those separately should not be expected to improve their estimate.

In my view breaking things down into subprojects is mostly useful for identifying hidden unknowns that are swept under the rug in the top level project view. 
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 5654
  • Country: gb
Re: Why projects take longer than you think
« Reply #9 on: April 16, 2019, 08:43:02 pm »
To quote from the article:
Quote
I have two methods for estimating project size:
(a) break things down into subprojects, estimate them, add it up
(b) gut feeling estimate based on how nervous i feel about unexpected risks
So far (b) is vastly more accurate for any project more than a few weeks
Many years ago, the place I worked ran a 2 or 3 day session on project management, much of which was about estimating. As examples, they used projects the company had actually done, so they knew the resources these projects actually took. For each estimate they asked us to size it up on a gut feel basis, and then break it down and do a methodical estimate. In every case, for everyone on the course, the gut feel was closer to reality than the detailed estimate.

At the end of the day you still have to do the rather bogus detailed analysis, or people won't engage you for the project. Just don't trust them.

One thing to watch out for is the knowledge of the people doing the estimation.  Having someone who doesn't understand the project break it down into subprojects they don't understand and estimate those separately should not be expected to improve their estimate.

In my view breaking things down into subprojects is mostly useful for identifying hidden unknowns that are swept under the rug in the top level project view.
This sounds very plausible, but it usually dead wrong. Most hidden unknowns are still hidden when you break things down. When a reasonably cautious person makes a gut feel estimate they tend to factor in a more general estimate for how many things typically go wrong on other projects of comparable size. This has a better chance of making a reasonable allowance for things going wrong.
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 15449
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Why projects take longer than you think
« Reply #10 on: April 16, 2019, 08:52:31 pm »
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 5654
  • Country: gb
Re: Why projects take longer than you think
« Reply #11 on: April 16, 2019, 09:05:23 pm »
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/

Tim
If you compared that with a hardware development activity you would find the benefits of early problem detection have a much greater impact than with software, but in most other respects the picture is pretty much the same. If you are looking at chip design, the cost and time benefits of early problem detection often become really extreme.
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 15449
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Why projects take longer than you think
« Reply #12 on: April 16, 2019, 09:06:37 pm »
This sounds very plausible, but it usually dead wrong. Most hidden unknowns are still hidden when you break things down. When a reasonably cautious person makes a gut feel estimate they tend to factor in a more general estimate for how many things typically go wrong on other projects of comparable size. This has a better chance of making a reasonable allowance for things going wrong.

It's very easy to overlook things, even when they are knowable.  A lengthy itemized list looks comprehensive, but you can't spot what's not there...

Immediate example, I'm currently doing a little project that I quoted for two weeks design time.  The major oversight was package size: I failed to note that I wrote a fairly high power level into the spec, which wouldn't have fit the customer's mechanicals with the original intended design.  Consequently, I had to redesign the power section to take up less space.  This not quite doubled the design time -- just using up the time I quoted for that stage.

This is otherwise a very ordinary project, very similar to things I've done before, so I felt confident in quoting as I did, and the project is still on schedule.

If it were a more speculative project, I'd want to perhaps quadruple the design or testing time (or both), to allow for challenges in correcting and tuning the design.  A second round of prototypes may be recommended, too.

These are, of course, difficult things to sell a customer.  That's the biggest reason to keep current with components, literature, technology and practice, and equally so, reason to charge fair rates which cover all the education and research that feeds into your process.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline NivagSwerdna

  • Super Contributor
  • ***
  • Posts: 2110
  • Country: gb
Re: Why projects take longer than you think
« Reply #13 on: April 16, 2019, 09:08:06 pm »
If you can estimate accurately then you have probably misunderstood the problem... that's why Agile development is trendy... it is a recognition that we don't understand the problem.

PS
I'm just finishing up a two week project... it's taken me three months.  :-DD
« Last Edit: April 16, 2019, 09:10:21 pm by NivagSwerdna »
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 15449
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Why projects take longer than you think
« Reply #14 on: April 16, 2019, 09:12:02 pm »
Related -- specific to software, but reasonably generalizable in most fields:
https://stevemcconnell.com/articles/software-quality-at-top-speed/

Tim
If you compared that with a hardware development activity you would find the benefits of early problem detection have a much greater impact than with software, but in most other respects the picture is pretty much the same. If you are looking at chip design, the cost and time benefits of early problem detection often become really extreme.

Indeed -- it's one thing to have just labor costs, it's another to have material costs in addition to that labor.

This is a huge driver behind simulators: when total cost can be reduced by exchanging some materials cost, or risk, with labor costs.

(I guess, since PCB-level circuit simulation has languished over the last three decades or so, that's proof of which side of the balance that's on...)

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 5654
  • Country: gb
Re: Why projects take longer than you think
« Reply #15 on: April 16, 2019, 09:12:41 pm »
Q: - Why projects take longer than you think?
A: - 'Cause you're stupid!
;D
Q: - Why projects take longer than you think?
A: - 'Cause its too depressing to face how long most engineering tasks really take.

In my first job people used to keep calling my estimates pessimistic, even though things tended to take longer than I estimated. Apparantly the threshold for not being a pessimist is that things should take as least twice as long as you estimate.  :)
 

Offline Tomorokoshi

  • Frequent Contributor
  • **
  • Posts: 885
  • Country: us
Re: Why projects take longer than you think
« Reply #16 on: April 16, 2019, 09:13:20 pm »
Who is asking the questions about the project? What kinds of questions do they ask?

Usually those asking the questions are a collection of horizontal and vertical layers of management relative to the various engineering groups.

The questions they are asking are usually:
How long is it going to take?
When is it going to be done?
How long would it take to add this feature?

usually followed by:
Shouldn't that take just a couple days?

What questions are not being asked? How about:
Do we have a clear idea of what the product should be?
What can I do (as a manager) to clear the way? ("move furniture", as Joel Spolsky says)
Do I (at a management level) understand the existing product, the current technology, the new product, the market, etc.?

The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.

So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.

Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?

Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
 

Online RoGeorge

  • Super Contributor
  • ***
  • Posts: 2438
  • Country: ro
Re: Why projects take longer than you think
« Reply #17 on: April 16, 2019, 09:14:09 pm »
Agile development is trendy...

Agile SW development.
Well, not trying to offend, but as a hardware developer I can testify I have never seen so much trendy bullshit in my entire life!

Offline Tomorokoshi

  • Frequent Contributor
  • **
  • Posts: 885
  • Country: us
Re: Why projects take longer than you think
« Reply #18 on: April 16, 2019, 09:22:22 pm »
Agile development is trendy...

Agile SW development.
Well, not trying to offend, but as a hardware developer I can testify I have never seen so much trendy bullshit in my entire life!

I can see Agile being useful for something that requires a quick succession of updates, such as a web site.

I know of a hardware project that the Agile development process is being used on. I'm worried they will end up perfectly designing the wrong thing.
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 6615
  • Country: us
    • Personal site
Re: Why projects take longer than you think
« Reply #19 on: April 16, 2019, 09:27:36 pm »
Agile works if you do it right. But it does mostly nothing to speed up the development. It is not meant to do that. Any speed ups are purely coincidental.

Agile reduces stress in the team and adds flow to the process. And for that it works perfectly. I was on a team that used it it was one the best times in my employment.

It will probably not work for hardware, I'm not sure.
Alex
 
The following users thanked this post: nctnico, hendorog, newbrain, Jacon

Offline floobydust

  • Super Contributor
  • ***
  • Posts: 3593
  • Country: ca
Re: Why projects take longer than you think
« Reply #20 on: April 16, 2019, 10:32:49 pm »
Hardware and software are two totally different birds for time estimating and project management.

I've had S/W managers attempt to manage H/W and the result is a total disaster.
Understanding the huge differences is not easy or something you can pick up reading a paragraph on the Internet.
 

Offline Vtile

  • Frequent Contributor
  • **
  • Posts: 993
  • Country: fi
  • Ingineer
Re: Why projects take longer than you think
« Reply #21 on: April 16, 2019, 11:15:20 pm »
Who is asking the questions about the project? What kinds of questions do they ask?

Usually those asking the questions are a collection of horizontal and vertical layers of management relative to the various engineering groups.

The questions they are asking are usually:
How long is it going to take?
When is it going to be done?
How long would it take to add this feature?

usually followed by:
Shouldn't that take just a couple days?

What questions are not being asked? How about:
Do we have a clear idea of what the product should be?
What can I do (as a manager) to clear the way? ("move furniture", as Joel Spolsky says)
Do I (at a management level) understand the existing product, the current technology, the new product, the market, etc.?

The illustration above about using a personal credit card to expedite a Digi-Key order hit way too close to home, and it seems relatively common. To put some numbers on it, let's say an engineer with a burdened cost of $100,000 per year works on a project with an expected budget of $1,000,000. An order of $100 is necessary to move a key part of the design forward.

So suppose it's Wednesday the 1st at 5:30 PM and to expedite an issue you want to make a Next Day Digi-Key order for delivery on Thursday the 2nd. The boss left at 5:00 PM, and he as the one credit card for the group. So it's entered by 6:00 PM but it won't get processed anyway until the next day because Purchasing leaves at 4:00 PM. But then it doesn't get processed because the purchaser took Friday the 3rd off, so now it doesn't get ordered until Monday the 6th, but they did 3-day delivery instead of next day, making it arrive on Friday the 10th.

Did this save any money anywhere in the process? What is the value of expediting blocking processes like that? Would a $1000 credit card per engineer be worth while? The liability on a $1000 credit card is... $1000. With an engineer who is answerable to what is purchased, so ordering cookies or golf balls probably won't happen. What is the liability of a week of delay? $10,000?

Look at all the relative orders of magnitude here. The little tasks get in the way large responsibilities, like priority inversion in some embedded system.
+1

Question asked about task schedule ... Just answer with even more complex question.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 7632
  • Country: nz
Re: Why projects take longer than you think
« Reply #22 on: April 16, 2019, 11:21:28 pm »
I forget the formula but there was a study of many industrial/commercial projects and how long they took vs what people expected at the start.

They came up with a formula where you stick in
- How long you expect the project to take if everything goes well.
- How long you expect the project to take if everything goes to shit.
- How many other projects are being done by the company at the same time.
- How hard the project is overall (easy, medium, hard).
etc.. And it spits out how long the project will likely take.


It's also a good idea to get some feedback into the project management cycle.
If all previous projects took on average 10% longer than expected then you add 10% to new project estimates.

Once you get a system that is accuracy estimating overall project time you can then start fine tuning individual processes to try make them faster.

The good thing is that the system self corrects. Any delays caused by management messing with processes causes the next project to pre-allocate more time to account for this.


« Last Edit: April 16, 2019, 11:23:20 pm by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 

Offline SilverSolder

  • Super Contributor
  • ***
  • Posts: 2507
  • Country: 00
Re: Why projects take longer than you think
« Reply #23 on: April 16, 2019, 11:24:10 pm »
Half a day wasted because of an unforeseen technical problem outside my control, and the half hour I'd saved on building the board was gone many times over.

This is what ends up killing otherwise well thought out estimates - the "unknown unknowns".   Really the only things we can estimate reliably is work that we have done before, down to the last detail - and even that can go wrong!

 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 6615
  • Country: us
    • Personal site
Re: Why projects take longer than you think
« Reply #24 on: April 16, 2019, 11:25:23 pm »
The good thing is that the system self corrects. Any delays caused by management messing with processes causes the next project to pre-allocate more time to account for this.
And this is exactly how Agile development works on a high level. Not only managements screw ups, but also new hires, movements between teams and all this stuff gets factored in.
Alex
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf