Unnecessary Complexity

[T3sl4co1l]

Discuss: complexity of products.

I've seen, more and more, the trend towards higher complexity designs, no matter if it's warranted to do so, or not.

There are clear cases where it is: for example, the ever-onward-marching computing power of a typical computer.  That is complexity incarnate, intentionally so.  A powerful computer must be powerful at all levels, and so, too, you will (almost necessarily) encounter bugs at all levels.  Ffrom hardware errata to OS vulnerabilities to shitty user libraries, there's just too much state to manage, too many interactions, too many edge cases, for it to "simply work".

There are less clear cases.  For example, flashing an LED (or similarly mundane task) on a clearly overpowered Raspberry Pi.  This is a continuum.  The justification can span anywhere from the trivial (it's a familiar system?), to, say, poorly optimized code that pushes the design to a bigger CPU than should be necessary, but that CPU is otherwise more-or-less being fully utilized.

Other suboptimal examples: say, using an ARM when a PIC would do, or using an MCU at all when a few logic gates would do; using one MCU (that's more expensive) because it's familiar; etc.  These, too, are justifiable in low production quantities, where development labor is poorly amortized over the production run.  They're less justifiable over many runs, but cost reduction can come in many forms as products are pushed into the mass-market range.


What I don't get, is the trend towards baroque designs: those which are, not only needlessly complex for their functionality, but which are considerably more expensive than a more optimal solution would be, at all levels: from development to production to maintenance to disposal.

Is it the engineer, overambitiously creating a byzantine interface plagued with bugs?  Is it the manager, pushing for faster deadlines and telling the team to make it happen through whatever means they can -- if that means putting Python on an embedded system, so be it?  Or is it marketing, promising the newest, shiniest, most powerful and feature-rich kit, whether the customer will ever be able to make use of that functionality at all?


Marketing does seem to drive a lot of this; but that, too, is another justification.  Example: there fundamentally isn't much they can add to automobiles anymore, only having made incremental improvements to safety and comfort over the last few decades.  (Self-driving cars being the obvious -- and quite justified -- exception to this, now that computers, interfaces and code are powerful enough to make this problem finally tractable.)  But they must continue the tradition of rolling out a new model year, with some kind of improvement to show, whether it be subjective or objective.  To fail to do so, would be economic suicide.

What about products that truly do not add value to their customers, that do not make the manufacturer near as much profit as they could otherwise earn?

Juicero comes to mind.  The product was still created.  People still bought it.  It did flop as soon as everyone else expected.  Well, I guess it's unclear if it was only bought by fringe consumers, those who get value from doing the new and cool thing, or the green thing, or who believe the marketing wank, or whatever.  The product was extremely overbuilt from almost every angle; perhaps that is justifiable in terms of development (a hasty, but very-likely-to-work design), I don't know.  A more refined design could've saved the manufacturer a lot of production cost, though maybe not total (including amortized time-to-market + design labor), also unclear.

I should probably not read too much into transient examples; better to look at products with a strong market presence over multiple generations of product.

What examples come to your mind?  In particular: cases that probably can't be explained through any of these reasons.

Hmm, I can think of a few examples from the medical market.

Software is absolutely fair game.  A rich backend is understadable from a user and data management perspective, but there are so many examples, even just web pages say, let alone apps and so on, that are just so heavily bloated that they can't possibly be a net value to the provider or the user.  And again, not just transient one-offs, but among top-100 players too.

Tim

[Mr. Scram]

Complexity in one area may be optimisation in another. That complex processor performing a mundane task may be more flexible or more accessible to the user or tool chain or even just simplifying inventory management. The Juicero obviously optimised for a high end market and even though that didn't work out, it does for many other products. Even if the engineering doesn't make sense, the marketing very well could. Sometimes products are simply optimised to deliver a fat bill, without adding much added value to the customer.

[unitedatoms]

You are defending the Beauty. But it is in the Eye of Beholder.

For some, the is no Beholder. As they confuse themselves to Him

[rx8pilot]

I am guilty of using an MCU in place of a few logic gates.....but generally for space savings. A 4x4 QFN is smaller than 3 discreet logic chips and easier to assemble. On top of that, it provides some logic flexibility downstream that will not trigger a re-design.

It is using a tiny fraction of the MCU - but who cares, I am saving in other areas.

Another area that I have been accused of pursuing unnecessary complexity is with products that serve a quickly changing market. I have laid the groundwork for additional features that I can roll out easily over time. This, of course, comes with the expense of a delayed release but has given me product lifecycle extensions and the ability to respond to requests for custom configurations. It takes some consideration to decide if this is a good approach for a given product. In many cases, getting anything to the market first is better than being late. In my case, playing the long game seems to pay off.

[T3sl4co1l]

Complexity in one area may be optimisation in another. That complex processor performing a mundane task may be more flexible or more accessible to the user or tool chain or even just simplifying inventory management. The Juicero obviously optimised for a high end market and even though that didn't work out, it does for many other products. Even if the engineering doesn't make sense, the marketing very well could. Sometimes products are simply optimised to deliver a fat bill, without adding much added value to the customer.

Indeed, and I gave examples of this.

I am guilty of using an MCU in place of a few logic gates.....but generally for space savings. A 4x4 QFN is smaller than 3 discreet logic chips and easier to assemble. On top of that, it provides some logic flexibility downstream that will not trigger a re-design.

Another good example.

But what about cases where these justifications (and the various ones I gave, and others), simply don't seem to exist?

Tim

[SiliconWizard]

As unitedatoms hinted (even if maybe not completely sober), complexity is sometimes in the eye of the beholder.

Is a single, small MCU more complex than three discrete logic ICs? It all depends on your perspective. On a transistor level, it obviously is. Tremendously. But on an overall design level by current standards? Maybe not.

Complexity also has two faces (at least). That could be using parts that are overkill for the task at hand, but basically doing the same thing as would simpler parts (as in your example). You may have reasons for choosing this, and the advantages, short- as well as long-term, may well overweigh the intrinsic added complexity. And then there is the design that does too much for no useful reason, which can only hinder the robustness of the basic functionality.

[T3sl4co1l]

You are defending the Beauty. But it is in the Eye of Beholder.

For some, the is no Beholder. As they confuse themselves to Him

Perhaps.  Beauty works for the personal project, often needlessly ornate -- but I've seen several projects where the build is simply a mess, even the developers building it agreeing it isn't beautiful!

Tim

[T3sl4co1l]

And then there is the design that does too much for no useful reason, which can only hinder the robustness of the basic functionality.

Exactly.  It's the latter case that I don't get.  I've seen a few examples myself (which I can't really talk about here unfortunately), and I just have to wonder, whose idea was it, who signed off on this thing?  All aspects point to it being bad (long time-to-market; high development cost; riddled with bugs; etc.), even given the often perverse demands of marketing (a fragile product, that's probably confusing for the customer; bug fixes throughout the product lifetime; etc.).

Tim

[SiliconWizard]

Unfortunately, many companies do not have a clear sign-off process. Responsabilities are unclear, and ultimately given to people that don't have the necessary background.

[rx8pilot]

But what about cases where these justifications (and the various ones I gave, and others), simply don't seem to exist?

Tim

A long time ago, I was working with a rigging technician on a movie set. He was laughing at one of the younger guys trying to tie-down something with rope. The mess of knots he created was indeed pretty funny. His remark was simply "If you don't know a knot, tie a lot"

I have certainly over-complicated something purely because I did not understand it very well. A fairly natural reaction is to take the more is better approach when you want it to work. Also, it could be from the lack of time/interest in doing the engineering fundamentals to simplify the design.

[rstofer]

I think there has been quite a bit of incremental improvement of automobiles, notably in the electronics.  Linking to cell phone for hands free operation, navigation on digital dashboard, tire pressure monitoring, electric cars in general, self-driving as a side issue, user selectable dashboard layout, remaining mileage calculation, front and rear cameras, lane changing warning, lane following, automatic braking, traction control and so on.

I didn't include engine management because that is pretty old stuff.  Same for speed control - old news.

I think working for the instrumentation part of the automotive industry would be very interesting.

[ajb]

I'm having a hard time thinking of examples of unnecessarily complicated products, but perhaps that's because I'm not a gadget freak and my car is almost ten years old.  Maybe smart TVs?  But those have a marketable use case in not requiring a bunch of little boxes hanging off your TV taking up HDMI ports to access your preferred media sources.

Even thinking of some of the rental cars I've driven while travelling, the new features are usually (at least when they work properly) meaningful improvements to safety and/or convenience.  I think there's too much pressure for cost optimization in that domain for features that aren't marketable.  Whether or not they work as intended is another matter, and there are some pretty horrid user experiences in some of the instrument clusters/entertainment panels of cars out there.

Another area that I have been accused of pursuing unnecessary complexity is with products that serve a quickly changing market. I have laid the groundwork for additional features that I can roll out easily over time.

Same here.  I often design in hardware capabilities that I would like to take advantage of in the future, and sometimes they never get used.  Other times, the boss asks me what it would take to implement a capability, expecting an answer in weeks, and I can be like "oh that?  Yeah, I saw it coming, just give me a day to turn that on in the firmware.  " Sometimes it means we throw away a few bucks on the BOM we could have done without, but we're small enough in volume that it doesn't add up to much.

I designed a simple power/analog board recently that needed overtemp protection and to enforce some timing limits.  I could have used a 6-pin MCU for $0.50, but a 32-pin ARM with USB is $1, and then we can monitor and record a bunch of other internal parameters while the modules are tested and configured (95% of the firmware already exists for this, as does a test application that can talk to it, so added development cost is minimal).  The total cost is increased, but accountability, consistency, and therefore overall quality is improved.  We're trending this way for a lot of projects, where yes, we COULD use a bunch of trim pots and well-trained techs to calibrate various modules, that would be simple in terms of hardware and development time, but it comes with additional costs in time to do the calibration, possible lack of consistency from system to system, and more complicated service.  Throwing an MCU and some DACs onto the board probably looks like needless expense and complexity to an outsider, but it makes the initial setup and later service much simpler and more consistent and lowers the lifecycle cost of the system. 

[tggzzz]

I should probably not read too much into transient examples; better to look at products with a strong market presence over multiple generations of product.

What examples come to your mind?  In particular: cases that probably can't be explained through any of these reasons.

Absolutely anything in the "enterprise software" domain.

Start by considering asynchronous protocols on top of synchronous protocols on top of asynchronous protocols on top of.....

And very few people are aware of the entire protocol stack, let alone have a basic comprehension of the layers.

Telecom software is a little better.

[LapTop006]

Two common causes (at least in my experience at a giant software company).

1. Deliberately making things complex to get promoted. Not always done intentionally, but often obvious from outside. This also covers cases where something existing *should* be reused but isn't, usually with little to no justification.

2. Many incremental changes done to be as small a change as possible. Laudable on their own, but after a while they can end up building up into an utter mess that can take longer to untangle than it took to get into the state.

Both can come from environments that push for speed at all costs, often taking just as long (or often far longer) to achieve a solid product as a slow & careful process would have.

[Berni]

Modern TVs are a good example of feature bloat.

These days TVs are capable of playing back movies from USB or the internet, open web pages, run apps, even make skype calls. Yet every time i tried using these features i found out that while they work, they are not actually very usable. Sure it plays movies from a USB drive but has problems with some video formats, or drops frames if the video bit rate is too high. Then try to add subtitles and again has trouble reading them at times, subtitles embedded in MKV? Like they are not there, but it plays the MKV video fine. The internet browser also works but its slow as heck and navigating anything with a IR remote is like trying to do brain surgery with boxing gloves on. Yes the features are there but they are so poorly implemented that you don't even want to use them.

All of this made the need for a big linux OS behind it, but even that is quickly thrown together and un-optimized. The TV takes 30+ seconds to boot. Once it starts showing the TV channel there is an additional 5 seconds before it starts accepting button presses from the remote and going trough the menus every press has about 500ms of latency before the menu actually moves.

So to solve that problem i hooked up small PC running Kodi/XBMC and now it plays back any format under the sun and runs Firefox at full speed. But when i hooked up a new bigger Philips TV to it i found that first of all the mouse movement lagged a good half second, i eventually found game mode buried deep in the menus and that brought the latency down a lot (But i could still feel some slight lag, tho not enough to be distracting). However the image on the screen looked absolutely horrendous. It was like my PC was hooked up with 50m of VGA cable. To fix that i had to dig trough the menus to turn off any image enhancement crap i could, yet some of it still had sliders. On some sliders setting them to zero did not help. So i had to bring up various test patterns on the screen and tweak the sliders to find where on the slider that enhancement is neutral/off.

Just make a TV that's good at being a TV rather than pile on so many features that it actually ruins the core functionality of it being a TV.

[tggzzz]

Just make a TV that's good at being a TV rather than pile on so many features that it actually ruins the core functionality of it being a TV.

... and sell your viewing habits to anybody that will pay.
... and sell your speech to anybody that will pay. (Now, where's my tinfoil hat gone?)

[LapTop006]

Just make a TV that's good at being a TV rather than pile on so many features that it actually ruins the core functionality of it being a TV.

... and sell your viewing habits to anybody that will pay.
... and sell your speech to anybody that will pay. (Now, where's my tinfoil hat gone?)

The former at least isn't speculation, sadly.

[AntiProtonBoy]

The TV takes 30+ seconds to boot. Once it starts showing the TV channel there is an additional 5 seconds before it starts accepting button presses from the remote and going trough the menus every press has about 500ms of latency before the menu actually moves.

Holy shit, you pretty much hit the nail on the head re. unnecessary complexity. TVs are absolute shite these days. It's rather ironic that we have tech better than ever before, and yet we make TVs that perform worse in terms of latency than old analogues.

I remember a hilarious John Carmack tweet pretty much summing up how screwed up the situation is.

[Rick Law]

Taking the last two replies (quoted below) together, it actually shown the added complexity added a new source of revenue  - thus, it is not an unnecessary complexity.  It is a complexity there for a reason (even if we don't like the reason).


First  reply:

The TV takes 30+ seconds to boot. Once it starts showing the TV channel there is an additional 5 seconds before it starts accepting button presses from the remote and going trough the menus every press has about 500ms of latency before the menu actually moves.

Holy shit, you pretty much hit the nail on the head re. unnecessary complexity. TVs are absolute shite these days. It's rather ironic that we have tech better than ever before, and yet we make TVs that perform worse in terms of latency than old analogues.

I remember a hilarious John Carmack tweet pretty much summing up how screwed up the situation is.

Second Reply:

Just make a TV that's good at being a TV rather than pile on so many features that it actually ruins the core functionality of it being a TV.

... and sell your viewing habits to anybody that will pay.
... and sell your speech to anybody that will pay. (Now, where's my tinfoil hat gone?)

The former at least isn't speculation, sadly.

[tggzzz]

Taking the last two replies (quoted below) together, it actually shown the added complexity added a new source of revenue  - thus, it is not an unnecessary complexity.  It is a complexity there for a reason (even if we don't like the reason).

That's not a helpful argument, since it would mean that any complexity arising from lack of care/understanding/time is "necessary" because it increases profits by reducing upfront costs.

To put it another way, all "complexity is there for a reason (even if we don't like the reason)".

[Rick Law]

Taking the last two replies (quoted below) together, it actually shown the added complexity added a new source of revenue  - thus, it is not an unnecessary complexity.  It is a complexity there for a reason (even if we don't like the reason).

That's not a helpful argument, since it would mean that any complexity arising from lack of care/understanding/time is "necessary" because it increases profits by reducing upfront costs.

To put it another way, all "complexity is there for a reason (even if we don't like the reason)".

I don't mean to be argumentative, I too don't care for using my TV to browse the net...

But if additional revenue is not considered a good reason, what would be a reason for a business to add any feature?  It may end up being a bad revenue generator.  I rather they "keep it clean", but much as I hated it, I don't think it is wrong for them (marketers) to try and test the market appeal of potentially worthy/worthless features.

[ajb]

These days TVs are capable of playing back movies from USB or the internet, open web pages, run apps, even make skype calls. Yet every time i tried using these features i found out that while they work, they are not actually very usable. Sure it plays movies from a USB drive but has problems with some video formats, or drops frames if the video bit rate is too high. Then try to add subtitles and again has trouble reading them at times, subtitles embedded in MKV? Like they are not there, but it plays the MKV video fine. The internet browser also works but its slow as heck and navigating anything with a IR remote is like trying to do brain surgery with boxing gloves on. Yes the features are there but they are so poorly implemented that you don't even want to use them.

Poor implementation or a poor user experience is orthogonal to device complexity, though.  Sure, scope creep or requirements that otherwise outscale development resources can lead to poor implementation, but they're still two separate parameters.  Even if the complex feature set is a major cause of the poor implementation, that doesn't mean the complexity is unwarranted.  I was shocked when I glanced at TV prices in a big box store recently--huge, high resolution panels from major brands are stupid cheap now!  (Granted, my TV is a plasma, because at the time I purchased it they were still widely considered to be superior to LCDs, so I'm probably a bit out of touch!)  So if prices have dropped so sharply on the panels, then clearly there must be more pressure to compete on features, which is where all of the smart stuff comes in--it's also probably meant to help persuade people to upgrade from earlier flat panels, which are still perfectly good otherwise.

To put it another way, all "complexity is there for a reason (even if we don't like the reason)".

This is kind of the problem with the core question, I think.  Obviously there is a reason for every design decision, so we're just arguing over whether or not we like the reasons--or at least whether we find those reasons, erm, reasonable, in their context.

[tggzzz]

Taking the last two replies (quoted below) together, it actually shown the added complexity added a new source of revenue  - thus, it is not an unnecessary complexity.  It is a complexity there for a reason (even if we don't like the reason).

That's not a helpful argument, since it would mean that any complexity arising from lack of care/understanding/time is "necessary" because it increases profits by reducing upfront costs.

To put it another way, all "complexity is there for a reason (even if we don't like the reason)".

I don't mean to be argumentative, I too don't care for using my TV to browse the net...

But if additional revenue is not considered a good reason, what would be a reason for a business to add any feature?  It may end up being a bad revenue generator.  I rather they "keep it clean", but much as I hated it, I don't think it is wrong for them (marketers) to try and test the market appeal of potentially worthy/worthless features.

I didn't mean to imply you are being argumentative!

Using "to maximise profits" as a justification implies it is reasonable (in some sense) not to strip out surperfluous bloat from a system - since stripping out crap takes time and expertise. Consequently baroque bloat and complexity becomes regarded as "reasonable".

I don't think either of us regard that as a justifiable chain of reasoning.

[TerraHertz]

Any time you are confronted with something that seems inexplicably stupid, you should always give consideration to two potential explanations:
1. Yes, everyone in a position of authority was in fact, inexplicably stupid.
2. You're just not seeing the reasons for the stupidity. Perhaps because you are failing to consider subterfuge, lies, greed, and outright conspiracy.

Take the Intel architecture and MS Windows for example. Why is it so ridiculously over-complex and frequently churned?
The answer is that this situation has strong benefits for certain parties, who are in positions where they are able to direct development.
Microsoft benefits from the complexity because they make most of their income from running high cost 'professional qualifications' courses. The more complex the OS, and the more often the GUI and internal details churn, the better for MS.
Also they now want to impose their 'subscription model' and the only way they can do that is if the OS is complex and byzantine enough that they can hide enforcement features within it. Stuff that makes freeware alternatives 'mysteriously just not work', that prevents users opting out of auto-updates, or worst, installing some other OS on their hardware. (That's what UEFI and 'secure boot' is about.)

Intel benefits because they have to keep the US Deep State happy, which means there have to be universal State-accessible backdoors hidden in the hardware. And to even hope to achieve this and keep it secret, requires a lot of obfuscatory complexity.

Plus both MS and Intel management are ideologically aligned with those who wish for an all-powerful surveillance state. Thus corporate profit is not the only motive. They only pretend it is.

[james_s]

I was reading somewhere recently that the "smart" TVs are actually cheaper than they would be without the smarts because they're subsidized for their ability to gather marketing data if you use the "smart" features. Like a lot of people I just want a display with a great picture, lots of inputs and a usable remote. Huge bonus points if it has buttons on the front bezel where you can actually use them rather than hidden away on the back or top or something. It makes absolutely zero sense for the consumer to bake an underpowered poorly implemented streaming device with a 2 year live cycle into a product with a 10+ year life cycle. The *only* reason this is done is for data gathering and to deliberately drive the sales of new TVs.

As far as complexity in general, there is a trend of technology for the sake of technology. Some of this may be engineers wanting an excuse to play with new tech, more I suspect is pushed by marketing. Some may be desperate attempts to turn mundane mature products into something new.