agile methodology for hardware projects

[Morgoroth]

Someone have any experience with agile methodology for hardware projects ?, I'm involved on a project surrounded by software guys, without any experience on hardware development, preaching over the advantage of agile methodology for hardware... now, I like to keep my mind open and try to see how that methodology could improve my development, but I can't stop thinking that on a computer you always can fix a problem, but with hardware is different, is more complex since the software (OSs, drivers, etc...) are only a part of the project.

The problem that I have is probably this methodology could bring more problems than benefits given the other areas of the company measure future developments given a system of points and suppose to resume a future effort on that... I just don´t get it, probably my mind is too obstinate.

Any experience and/or comments welcome.

  JP

[singapol]

I'm just a hobbyist not a pro but your thoughts are normal.So I looked up what is Agile..whatever, it originates
from software development. Theory is just theory, while some report success in applying agile to hardware  development there are cons to it also. It's easy to apply a list of metrics to measure performance provided it's based on real life. I sympathise with you ..haha as a hardware man. Talk is easy as you say about doing it on a computer by simulation? hardware costs money and time so the thinking part is critical ( trying to get it right the first time) but sadly in real life it's not the case. Take a piece of wisdom from the engineer who designed the mars rover, he said " It does not it to be perfect but just GOOD ENOUGH" to do the job.

http://www.design-reuse.com/articles/32442/are-we-too-hard-for-agile.html

http://blog.grabcad.com/blog/2014/05/14/pros-cons-agile-hardware-product-development/

[Dago]

First you would need to define what "agile methodology" is. Most software shops do NOT follow the original agile method specified in the agile manifesto but some sort of weird bastardized version of it (usually bypassing any kind of benefits you could actually get from following the method).

Usual version of "agile" seems to be some variation of "have a quick meeting everyday and try to do shit quickly" (quick meeting meaning anything from 15 min to two hours) which is not that much related to actual agile...

As a whole I'm rather skeptical of the benefit of these methods for HW development because as you said you just cannot be so quick/flexible with things when ordering new round of prototypes can take a month and so forth.

[cat87]

We've tried to implement the  "agile" method in our SW test team, with some success. Unfortunately, I can tell you that the methods described  are just for software develoment.
Everything is based on a direct communication path between the SW test team and SW development team, so that feedback can be given  very fast and bugs ironed out on the fly.
I doubt the same can be dome with HW...I mean, OK, sure, it can, but if you have to actually spin a board before  you find out there's a glitch somewhere,  that will eat up a lot of time and resources. Not much point to that
Just go with the ordinary development cycle.

[ivaylo]

I always found hardware more susceptible to (what they call) Agile than software. In hardware you just naturally iterate over your prototype. Not to mention that some methodologies considered Agile today came entirely from hardware development - https://en.m.wikipedia.org/wiki/Kanban. Now, I wouldn't try and do Skrum and all that nonsense with HW. So yes, perfectly acceptable, existing for a long time, but probably under different names. Wonder what exactly were the software people saying you could be doing differently (in actuality they could probably learn something from you)?

[2N3055]

Agile is just fluff from marketing and management types...
In the end team adopts to a sort of internal workflow that does what needs to be done to make it work, and all the interruptions to satisfy management....

Classic waterfall process works because you need to freeze requirements. Requirements changes can have butterfly effects in the rest of the project.
So, in reality you cannot move the target and hit it.....

I know of one big public project that is being done with sort of agile development methodology... and it was a disaster... and still is.. F35 fighter jet..

Adding a feature to an almost finished product sometimes require redesign from the scratch.. All else is wishful thinking from people that don't do actual work..

[tggzzz]

Agile is a very ill-defined jumble of many concepts, many of which came from other engineering disciplines.

There is remarkably little difference between hardware and software. It can be a fun game over a glass of beer to try to get a newbie to define the difference between them - and then to point out examples where their defininition fails. Of course hardware and software do have some different characteristics - but exactly the same is true of just software technologies. Think of RDBMS, functional/procedural/logic/OOP/spreadsheet/constraint based languages, and the old joke "you can write FORTRAN in any language".

Arguably there is more difference between analogue or RF and digital electronics, than there is between digital electronics and software - particularly embedded / real-time software, and with distributed software systems.

And the problems that arise in projects are common to all projects, whatever the technology.

So yes, agile can be applied to hardware. And, just as with software, it can also be misapplied.

[MosherIV]

Hi

It looks like most here do not embrace 'agile'.

Someone have any experience with agile methodology for hardware projects ?

Yes, I have worked closely with my hardware colleagues.

I like to keep my mind open and try to see how that methodology could improve my development

As someone said "Most software shops do NOT follow the original agile method" - so what are the key concept to adopt and that work?

Think of Agile as lots of mini waterfall iterations or sprints.
Within each iteration - define a key development or a goal to achieve and concentrate to deliver that.
Have a rough plan of the whole project - order in which to tackle things.

It is tricky for hardware/electronics if the whole system is one BIG thing but even the biggest design had sub systems.
So set the 'sprint' (iteration period) to match typical period for delivering a sub-system.

Do not get distracted - only work on what is in each sprint.

At the end of each sprint - evaluate and refine how you work ie adjust the working practices in the next sprint to get better results.
Re-plan after each sprint.
This also allows for any change in requirements and any unforeseen problems that arise.

It can work for hardware, it does require a change in mind set though. Do not think of 1 long project - think of lots of little projects which deliver lots of little part of the whole system.

Hope this helps. 

[dferyance]

Unfortunately Agile has just become a buzzword. There are good ideas and merits behind it but they have all be overshadowed by nonsense. In the software world, "agile" has come to mean "scrum" which has come to mean having lots of status report meetings. It also is an excuse for doing no planning whatsoever and having lack of direction on a project. This isn't the intent behind agile or the problems it was meant to solve. It is just what it has become once project managers and people who like gantt charts and metrics take over. Something intended to give engineers more control over a project has become the tool for others to take it away.

Having worked at both a hardware company that understood that engineering changes take time and a bunch of software companies where the attitude was, "I don't know or care what the feature is, but you better get it out the door by the end of the week", software desperately needs to slow down and plan a lot. At least in hardware, you have a somewhat tangible product so people tend to be more understanding that engineering takes time ("tend" being completely relative, not absolute or universal).

So stay away from agile. Nothing wrong with the original ideas but now-days it is just an excuse for poor project management.

[ajb]

The first problem is that iterating hardware is expensive, so any methodology that requires many rapid iterations of hardware is going to have a much higher cost floor.  If the hardware is high complexity and has a niche application, even *one* board respin may be intolerable (think of some of the teardowns of (what used to be) ridiculously expensive kit with bodge wires all over the place, because it was cheaper to bodge the hundred or so boards they needed than to redesign the PCB).  It's really only financially feasible to repeatedly iterate hardware if it's some combination of simple and high volume.  But if it's simple, then why would you need so many iterations?

The first-and-a-half problem is that iterating hardware takes time.  So you finish a new 'sprint', order hardware, and then what?  Twiddle your thumbs for a few days while you wait for boards etc?  Have everyone on the team switch to another project, forcing a major mental context-switch that will have to be reversed in a few days?  Plow ahead on the next 'sprint' of this project, so that by the time you have this iteration in hand for testing your current design documents no longer reflect it?

The second problem is that it's almost impossible to implement a new hardware feature without affecting many other aspects of the design.  You're not talking about writing a handful of new functions and clicking "Compile", you're talking about re-evaluating your power budget, finding or making enough contiguous free space on the board to place new components, hopefully not breaking other functions as you shuffle things around to make room, running new signals, possibly changing connectors to add new signals in/out of the board (which may require changing other PCBs), possibly revising your mechanical design to accommodate new connectors and/or a larger PCB.  It's much easier to do all of that in CAD + design review meetings BEFORE you order PCBs or start fabricating enclosures.

I guess you could do something agile-ish in the schematic design stage with strict hierarchical schematic capture, but even then the tools don't necessarily lend themselves to propagating changes through interfaces between sheets in a well-controlled way, as far as I know, so you still need to have a well-defined structure from the beginning before you divvy up the work.

In short, each 'waterfall' in hardware is unavoidably expensive, so any methodology that requires more of them is going to be a tough sell.  Even if the waterfalls are smaller from a design point of view, the nature of hardware requires a large minimum amount of effort to implement it, based on the amount of effort required to spin the PCB or enclosure or whatever.  As a cost proposition it makes more sense to front-load all of the design so you get as much as possible out of the first prototype, do as much modification/bodging of the prototype as you can to verify any necessary hardware revisions, then do another, hopefully final, design.

Of course if when you said "hardware" you were including firmware/HDL, then okay, sprint away, but that's not really the same thing as hardware.

[tggzzz]

Nothing wrong with the original ideas but now-days [agile] is just an excuse for poor project management.

More accurately, different poor management.

I've seen agile improve company's products, but the danger is that religious zealots push the doctrine too far.

[tggzzz]

The first problem is that iterating hardware is expensive, so any methodology that requires many rapid iterations of hardware is going to have a much higher cost floor.

Only for some forms of hardware: consider FPGAs, for example.

The first-and-a-half problem is that iterating hardware takes time. 

Only for some forms of hardware: consider FPGAs, for example. There's nothing in agile that dictates the length of a sprint.

So you finish a new 'sprint', order hardware, and then what? 

Only religious zealots would argue/assume that agile removes the need for planning.

The second problem is that it's almost impossible to implement a new hardware feature without affecting many other aspects of the design.

Software is exactly the same, if not worse: start by considering why COBOLlers are still in demand, and why IBM mainframes are still essential.

I guess you could do something agile-ish in the schematic design stage with strict hierarchical schematic capture, but even then the tools don't necessarily lend themselves to propagating changes through interfaces between sheets in a well-controlled way, as far as I know, so you still need to have a well-defined structure from the beginning before you divvy up the work.

You need the same for software.

Of course if when you said "hardware" you were including firmware/HDL, then okay, sprint away, but that's not really the same thing as hardware.

Just so. The harder you look, the smaller the differences between hardware and software. That isn't a new phenomenon: I first noticed it in the 70s, and it has become more valid since then.

[Siwastaja]

Words like "agile" are problematic, because they are completely meaningless buzzwords. Anyone can and will use words like this for any purpose they like or don't like.

You should try to talk with them on a less abstract level. You could ask them what they actually want from the hardware development flow. It should be a bidirectional flow of ideas where you educate them about practical hardware design issues, and they give their out-of-the-box insight how it could be changed. Mostly, you would find their ideas completely unusable and ridiculous in a bad way, but every now and then there might be something important, exactly because they don't know anything about HW design.

[Nuzzler]

Think of Agile as lots of mini waterfall iterations or sprints.
Within each iteration - define a key development or a goal to achieve and concentrate to deliver that.
Have a rough plan of the whole project - order in which to tackle things.

It is tricky for hardware/electronics if the whole system is one BIG thing but even the biggest design had sub systems.
So set the 'sprint' (iteration period) to match typical period for delivering a sub-system.

Do not get distracted - only work on what is in each sprint.

At the end of each sprint - evaluate and refine how you work ie adjust the working practices in the next sprint to get better results.
Re-plan after each sprint.
This also allows for any change in requirements and any unforeseen problems that arise.

It can work for hardware, it does require a change in mind set though. Do not think of 1 long project - think of lots of little projects which deliver lots of little part of the whole system.

Good summary and how most software development shops, that I've worked with, implement the agile methodology. From the software side of things...one thing we try to do is reword RFE's and bug's summary statements in the form of "As a user, I should be able to do x". Once we have chosen the features/bugs we're tackling for a given sprint we try to keep focus on those as to avoid scope creep, etc...

As others have already noted, this may or may not translate well to hardware development depending on the type and maturity of the project.

- Nuzz

[Tomorokoshi]

I agree with the overall sentiment here.

Think about these situations for an Agile hardware project and how to deal with them using the Agile paradigm:

1. The product goes to the EMC lab and fails.

2. The product goes to an approval agency and fails.

3. The product goes to field test and fails.

4. A component goes EOL just after a PCB revision.

[filssavi]

Only for some forms of hardware: consider FPGAs, for example.

Well to be honest FPGA's are the only kind of hardware design where the agile (as in iterate quickly as many times as you can) is feasible, since FPGA developement is turning getting closer and closer to the CS side, devices are getting more and more powerfull and there will be less and less need to wirry about resources ( and on top of that High level synthesis is getting better and better, not I'm not saying you will be a great FPGA guy if you have no electronics background, but you don't need an EE major to use One for shure) that is because the build phase is generally very quick (generally seconda to few hours) and free,

But the majority of Electronics doesn't consist in programmino/configuring FPGA's most of the times (even when there an FPGA involved) there is a board to responsabile to iterate, and redoing a while board for all but the simplest stuff is inconvenient and expensive at best and downright impossible at worst think for example cellulare base station kind of design you will have a Digital board with fine pitch High pin count devices for the Digital baseband processing side and than you'll have rf sections with thightly controller impedances stackups etc, the manifacturing alone of a board that complex takes time it can't be done in 2 days

And that is the Easy stuff, if we take a look to IC design (even on a old node where the Mask set is not crazy expensive) the tapeout is a pretty log and convoluted process (for example mosis required thas some aspects of the design such as what devices are used and metal layer count is decised 11 weeks from scheduled manifacturing date. Now once the design has been sent to the fab you have to twiddle your thumbs (or work on another design), since the manifacturing processo will take another 8-9 weeks (not to mention the whole process will have cost you 15-20 k$ for a very cheap/simple devices)

As for the make a lot of small independent-ish subsystems that is not a silver bullet, it will be possible, but only in certain kind of design/markets, an industrial PLC/control system is a very good example of such a design, a modern smartphone on the other hand will definitely not be the case (try explaning to designers/marketing that their phones must be 2mm thicker to accomodate the board to board Interconnects) there are also High vibrations environements, where connectors will greatly decrease reliabiliy (aerospace or mining industry)

[Ice-Tea]

Some guy said something along these lines: "hardware is exactly the same as software. Except that for every time you push "compile", 25k goes down the drain". Sums it up, I guess.

Yes, FPGA code is different but is VHDL coding really hardware? More like something in between...

[rstofer]

Words like "agile" are problematic, because they are completely meaningless buzzwords. Anyone can and will use words like this for any purpose they like or don't like.

You should try to talk with them on a less abstract level. You could ask them what they actually want from the hardware development flow. It should be a bidirectional flow of ideas where you educate them about practical hardware design issues, and they give their out-of-the-box insight how it could be changed. Mostly, you would find their ideas completely unusable and ridiculous in a bad way, but every now and then there might be something important, exactly because they don't know anything about HW design.

Exactly right!  I hate the words 'paradigm shift'.  I have no idea what they mean in some particular context and neither does the person who uttered them.  They picked up the words at a management seminar and are trying to use them with engineers.  We tend to think in terms of 'step-wise refinement' (Niklaus Wirth) and not in terms of paradigms - whatever they are...

Disrupters can be helpful.  We get in a rut of doing things in a particular way using well established guidelines.  It is sometimes a good idea to throw out the rules and start from scratch.  That doesn't mean throw out F=ma, some rules are laws!

I have been in a lot of meetings where non-engineers were trying to 'help' me do my job with the best of intentions.  They were smarter than me, just ask them, and they just wanted to pass on their vast experience.  It was hard to keep from laughing...

[PChi]

I thought that "agile" was lets not bother to write a specification. Then wonder why it's taking so long. If you don't have a plan how can you estimate how long it's going to take? It's like setting off on a car journey with no clear idea of your destination.

[tggzzz]

And that is the Easy stuff, if we take a look to IC design (even on a old node where the Mask set is not crazy expensive) the tapeout is a pretty log and convoluted process (for example mosis required thas some aspects of the design such as what devices are used and metal layer count is decised 11 weeks from scheduled manifacturing date. Now once the design has been sent to the fab you have to twiddle your thumbs (or work on another design), since the manifacturing processo will take another 8-9 weeks (not to mention the whole process will have cost you 15-20 k$ for a very cheap/simple devices)

It isn't always that clear cut in the software world!

I have seen a software product where an incremental change from a customer took 6 months to get in the field. There was absolutely no justifyable reason for that duration; the reason was a ridiculously convoluted codebase and the customer/development/QA in separate silos.

[tggzzz]

I thought that "agile" was lets not bother to write a specification. Then wonder why it's taking so long. If you don't have a plan how can you estimate how long it's going to take? It's like setting off on a car journey with no clear idea of your destination.

Only if you are stupid (and there are plenty of idiots around).

Any process or methodology is doomed to fail if it applied inappropriately. Yes, some people believe that following a process is sufficient to guarantee a good result; they are usually disappointed.

[tggzzz]

I agree with the overall sentiment here.

Think about these situations for an Agile hardware project and how to deal with them using the Agile paradigm:
1. The product goes to the EMC lab and fails.
2. The product goes to an approval agency and fails.
3. The product goes to field test and fails.
4. A component goes EOL just after a PCB revision.

There are direct analogies for software, particularly distributed systems combining components from multiple other companies.

[filssavi]

And that is the Easy stuff, if we take a look to IC design (even on a old node where the Mask set is not crazy expensive) the tapeout is a pretty log and convoluted process (for example mosis required thas some aspects of the design such as what devices are used and metal layer count is decised 11 weeks from scheduled manifacturing date. Now once the design has been sent to the fab you have to twiddle your thumbs (or work on another design), since the manifacturing processo will take another 8-9 weeks (not to mention the whole process will have cost you 15-20 k$ for a very cheap/simple devices)

It isn't always that clear cut in the software world!

I have seen a software product where an incremental change from a customer took 6 months to get in the field. There was absolutely no justifyable reason for that duration; the reason was a ridiculously convoluted codebase and the customer/development/QA in separate silos.

I can understand that updates and changes can take a long time also in software, but there is a massive difference between the two worlds, in your case the delay was bacause of (granted indirect) a human error in the developer-QA-customer relationship and a horrible codebase (which we can also trace to a mix of errors from management who let that code slide out of hand and previous developers having written said code, had not these errors been committed the update would have been much faster

in the IC production case what i described is the best case scenario where the design is perfect (no bugs errors and such) there are no delays the Fab-management/engineering dept comunication is perfect the times are basically driven from the process length, the mask set has to be checked, produced and verified, one that is done the true manifacturing process can begin, here the times will vary with the design complexity (the finer the features the more steps it take, more metal layer more steps etc...) for a single metal layer digital only 0.35 um ,which is an unrealystically simple (for example only 1 metal layer no high/low Vth transistors etc) digital only process takes more than 50 different steps, and a lot of the steps require baking the wafer in ovens ( ok i'm oversimplifing but oxidations and diffusions in particular) and that is inherently slow (ramp up/down times) you get the idea...

So in hardware frequent respins are not possible because the process is just too long, there would be also a lot to say about the feasibility of just the engineering part of a theoretical ic agile developement methodology in fact for every change (however stupid and easy it might be) before manifacturing the whole design must be simulated and verified to work (no one likes to flush 10-20 k$ to few millions down the drain) and for complex designs (like say a leading edge microprocessor, be it arm intel amd etc) it might take months just for that

[alexanderbrevig]

As I work a bit with rapid prototyping I'd say that to be agile in the hardware world you need to be able to design for changes. This means maybe laying out more footprints than is needed (for alternate parts) using a lot of 0R jumpers so you can test blocks in isolation, be generous with test points (also easy to solder on hookup wires to those) and the list continues.

Agile is hard to define and indeed it's grown to something that it was not intended to be.
http://agilemanifesto.org/

Like most production runs I find that agility is the one true way for finding the right spec. When the spec is good - waterfall is the way to go because you can branch off and do things in parallel. I.e the firmware can be tuned while the molds are made, the DFM can start while procurement starts.

Things like an in-house PnP and a 3d printer are investments that helps with the agility for new ideas and prototypes. Agility in the face of going from a prototype to a production unit is, in my mind, a fallacy. For software though, I'm a strong believer in being agile at all times.

My 0b10

[tggzzz]

And that is the Easy stuff, if we take a look to IC design (even on a old node where the Mask set is not crazy expensive) the tapeout is a pretty log and convoluted process (for example mosis required thas some aspects of the design such as what devices are used and metal layer count is decised 11 weeks from scheduled manifacturing date. Now once the design has been sent to the fab you have to twiddle your thumbs (or work on another design), since the manifacturing processo will take another 8-9 weeks (not to mention the whole process will have cost you 15-20 k$ for a very cheap/simple devices)

It isn't always that clear cut in the software world!

I have seen a software product where an incremental change from a customer took 6 months to get in the field. There was absolutely no justifyable reason for that duration; the reason was a ridiculously convoluted codebase and the customer/development/QA in separate silos.

I can understand that updates and changes can take a long time also in software, but there is a massive difference between the two worlds, in your case the delay was bacause of (granted indirect) a human error in the developer-QA-customer relationship and a horrible codebase (which we can also trace to a mix of errors from management who let that code slide out of hand and previous developers having written said code, had not these errors been committed the update would have been much faster

in the IC production case what i described is the best case scenario where the design is perfect (no bugs errors and such) there are no delays the Fab-management/engineering dept comunication is perfect the times are basically driven from the process length, the mask set has to be checked, produced and verified, one that is done the true manifacturing process can begin, here the times will vary with the design complexity (the finer the features the more steps it take, more metal layer more steps etc...) for a single metal layer digital only 0.35 um ,which is an unrealystically simple (for example only 1 metal layer no high/low Vth transistors etc) digital only process takes more than 50 different steps, and a lot of the steps require baking the wafer in ovens ( ok i'm oversimplifing but oxidations and diffusions in particular) and that is inherently slow (ramp up/down times) you get the idea...

So in hardware frequent respins are not possible because the process is just too long, there would be also a lot to say about the feasibility of just the engineering part of a theoretical ic agile developement methodology in fact for every change (however stupid and easy it might be) before manifacturing the whole design must be simulated and verified to work (no one likes to flush 10-20 k$ to few millions down the drain) and for complex designs (like say a leading edge microprocessor, be it arm intel amd etc) it might take months just for that

I'm not going to argue your points because they are valid. But it rarely as bad as the example I gave, and there are many valid cases where software changes are very difficult and time consuming.

Consider anything where safety is important.
Consider anything where service interruptions/downtime are very expensive. (Think telecom systems, the airline reservation systems)
Consider anything where mistakes can be devastatingly expensive. (Think automated trading systems)

If you read comp.risks (and if not, why not?) you will become all too familiar with the effects.