food for thought: code bloat

[tellurium]

Yep and it’s getting worse each release. Half the ancillary crap that runs behind the scenes and the remote host for remote machines eat a ton of RAM and resources. I’ve dumped it and gone back to vim recently.

Likewise. I try a couple of editors every year for the last 20 years, and always go back to vim.

The nice feature of vscode / sublime I've missed was multiple selection with Ctrl-D then simultaneous edit. Useful for quick renames / refactorings. Though, vim's search + cgn + replacement + .   does the same thing, as I've found out.

[bd139]

I tend to use

Code: [Select]

:%s/foo/bar/g for that.

[SiliconWizard]

Yep and it’s getting worse each release. Half the ancillary crap that runs behind the scenes and the remote host for remote machines eat a ton of RAM and resources. I’ve dumped it and gone back to vim recently.

I've never used it.
Well, I have tried VSCodium (which is VSCode without telemetry) and was unimpressed. It's slowish, eats up a lot of RAM and I don't like the text editor part itself much (at least compared to what I'm used to.)
The only thing it has going for it is the extension system and the amount of extensions. Which makes it look like any popular software: it is popular because it is popular (so sure the ecosystem grows exponentially). Not because it's particularly good.

The only occasional use I make of it is to edit markdown files. It has an extension for previewing the result, which is handy. If you can point me to a markdown viewer that is not web-based and that works well, I'll be glad to switch!

[bd139]

Yea I was using it for markdown as well to maintain internal documentation in GitHub.

I just use plain text files now

[Berni]

The reason that VSCode, VisualStudio, Sublime..etc use so much more resources is also due to the extensive autocomplete functionality they have.

They are basically halfway compiling the code in the background as you type, so it understands what you are referencing in the code and only suggests valid things about that item. This means it has to hold an up to date picture of what every namespace,class,structure..etc looks like in not only your code but also whatever is is exposed by the included libraries.

I personally like code editors that provide good autocomplete and Microsofts Intelisense does a rather good job. These autocomplete features help make me more productive by not making me remember the exact name of everything in my code and saves time with me having to look up what a structure contains or getting it wrong and finding out the name typo at compile time. This is especially true when using outside libraries that i never used before and don't know any of the functions from by head (the whole function prototype appears right there as i type it). Since you can just hit a key and have it autocomplete for you also speeds up typing, reduces chances of a typo and removes the discouragement of using longer names. At the end of the day it helps me get my coding done faster. Computers are better at remembering things than i am. If this comes at a cost of a gig or two of memory so be it, my machine has 32GB.

Could these features be implemented in a more optimized resource efficient way? Yes they certainly could. But until someone actually does it this is the best we got. But you will always have those Arch linux users rocking there good'ol vim on there trusty IBM Thinkpad from 2005

[bd139]

I used to have working autocomplete on a 512MB machine. There is no reason for it to be as painfully large as it is now.

[Berni]

Just so.

But it also needs the right language. Good luck with autocompletion with, for example, Forth or Lisp!

Autocompletion has one other virtue that newbies don't understand. Too often you see statements to the effect that "language X requires s too much typing" and that my language is better because it uses duck-typing. Autocompletion removes the "problem" of too much typing. (Personally I'd rather the compiler told me I'm wrong, rather than have a run-time error.)

Yep this sort of functionality needs a lot of effort to implement than just syntax highlighting. But i tend to end up using the more popular languages so they are usually supported(C, C#, Python etc..). I don't tend to learn new languages until i find something that my existing set of languages doesn't do well.

If strong typing or dynamic typing is better is a topic for another day. My own preference is to have types defined wherever possible. I even use type hinting in Python just so that the IDE can warn me when i try to stuff something into the wrong spot. Even tho the Python language tends to be designed to not give a damn about data types. The autocomplete feature of the PyCharm IDE even tries to be helpful even without type hints by tracking the movement of data trough the code, so that it can do a best guess about the data type, or when accessing the members of a object in the code it assumes the member is part of the object and suggests it for autocomplete (even tho it has no idea what kind of object it is actually dealing with). So it still kinda works but nowhere near as well as in strong typing.

I do admit the strong typing of VHDL did feel annoying to me, always tended to prefer Verilog instead. But both of those languages are pretty terrible and i never found any decent IDE for them. But i just deal with it for when i want to do stuff with FPGAs

[SiliconWizard]

autocomplete crap freaking annoys me to no end, so that's definitely not for me. I don't need
Give me good syntax highlighting, and sensible auto-indentation (which VSCode doesn't have IMHO, but that will of course be highly subjective).

As to a language needing "too much typing"? As I say on a regular basis, if, when developing software, the amount of typing needed takes any significant amount of time/energy compared to the design effort, then you're either writing trivial stuff or you're doing something really wrong. Just my 2 cents.

[bd139]

That’s where YAML needs to fuck off and die 

[SiliconWizard]

I don't particularly care for YAML, there are better structured formats out there.

[bd139]

There are indeed.

My pet pieve is templated YAML which is just pain and a definite incarnation of Greenspun’s Tenth Rule. Helm I’m looking at you there.

[brucehoult]

autocomplete crap freaking annoys me to no end, so that's definitely not for me. I don't need

I *hate* things flashing up when I'm trying to think and type. If you have to hit a key to get the suggestions then fine.

As for reducing typing, emacs' M-/ command works fine for me. It searches first backwards in the current file, the from the end of the current file, and then other open files, for a word starting with the characters to the left of the cursor, and completes it. That's all you need for not repeatedly typing long and descriptive names.

Give me good syntax highlighting, and sensible auto-indentation (which VSCode doesn't have IMHO, but that will of course be highly subjective).

I'm not sure what is "good" syntax highlighting. String literals and comments different to other things maybe. I don't need my freaking editor to remind me which things are keywords and type names. I'd rather have plain black text than an editor that looks like a Fisher-Price toy.

As to a language needing "too much typing"? As I say on a regular basis, if, when developing software, the amount of typing needed takes any significant amount of time/energy compared to the design effort, then you're either writing trivial stuff or you're doing something really wrong. Just my 2 cents.

100%.

[Nominal Animal]

I'm not sure what is "good" syntax highlighting. String literals and comments different to other things maybe. I don't need my freaking editor to remind me which things are keywords and type names. I'd rather have plain black text than an editor that looks like a Fisher-Price toy.

(I prefer gray text on black for whatever reason, and as little of the editor showing as possible; only the text I'm editing.)

I have noticed that when browsing through code new to me, syntax highlighting reduces the cognitive load: it makes it faster for me to understand the code.
When writing my own code, it's probably just eye candy.

I only use colors for highlighting, though.  (Well, with Pluma I use the defaults which does apply italics to certain parts of Doxygen comments, and only my own theme/color palette.)

One quirk in my typing is that AltGr+Space must map to plain space, and not to non-breaking space (U+00A0), because in the keyboard layout I use, { is AltGr+7, [ is AltGr+8, ] is AltGr+9, and } is AltGr+0, and those are very often followed by a space by a finger in my other hand, and I occasionally release the AltGr key too late.  I used to have to modify the layout to do this, but in Linux Mint 20.04 it was this way from the get go.

As to a language needing "too much typing"? As I say on a regular basis, if, when developing software, the amount of typing needed takes any significant amount of time/energy compared to the design effort, then you're either writing trivial stuff or you're doing something really wrong. Just my 2 cents.

I fully agree!  And I often refactor/rewrite my own code.

[Berni]

In almost all the IDEs you can set up syntax highlighting or indenting or autocomplete to work the way you like it.

Syntax highlighting is another thing that is a little productivity boost. I also don't like it when it is too colorful, so i sometimes take a bit to find a good color theme i like. But just gentle color hints around the code make it so that i don't actually need to read everything when just looking over the code. I don't need to look for where a string literal begins or ends, or when those brackets begin or end, what is part of a comment what is not. My vision has a finite amount of text bandwidth it can read, so if i can avoid wasting it to read syntax details that means there is more bandwidth left over for actually reading the code i want to read.

I know computers are better than me at doing some of these things, so i let them do those things for me so that i can focus on actually thinking about the code i want to write. Not on the typing or reading it, those are just a barrier between my brain and the source code file where i want to get my code into.

Maybe i am just really bad at reading quickly or typing quickly, so that having to do more of those slows down my process of coding. Or maybe my memory is so terrible that i can't remember enough of a libraries documentation in my head to avoid having to have to look trough it too often while i code(as this is time spent not coding). I have no idea, all i know is the modern bells and whistles make me get the job done faster in comparison to a bare bones editor.

Try using an advanced editor for a week or so on a project (to get used to the new visual queues) and then switch back to a bare editor to see the difference. It is much like how i started out using Eagle for PCB design in the old days and it was fine, now that i use AltiumDesigner going back to Eagle makes me surprised i got anything actually done in it.

[PlainName]

The only occasional use I make of it is to edit markdown files. It has an extension for previewing the result, which is handy. If you can point me to a markdown viewer that is not web-based and that works well, I'll be glad to switch!

https://typora.io/

https://github.com/marktext/marktext

[Kjelt]

Software bloat follows the hardware bloat 

You can ask the same non relevant question for hardware:
In the 80's we only needed 29,000 transistors for an Intel 8088, why do we now need 57,000,000,000 transistors for a mobile M1 processor ?
How much energy can we safe if we go back to 29000 transistors and 4 bit color graphics with a 320x200 resolution ?
Who needs those 4K super real games and movies ?

Totally irrelevant questions IMO. Real software bloat are unused bytes that can be thrown out without compromising any of the end results.
No company will ever invest millions of $ for 20-30% in code reduction, they invest in the next thing. Why ? Because new things have a ROI, while code reductions and cleaning only costs money.

https://en.wikipedia.org/wiki/Transistor_count

[madires]

And companies writing software don't care about code bloat since the customer has to pay for the faster PC needed. Hopefully this will change with rising energy costs. And software has a CO2 footprint too.

[tellurium]

Totally irrelevant questions IMO. Real software bloat are unused bytes that can be thrown out without compromising any of the end results.
No company will ever invest millions of $ for 20-30% in code reduction, they invest in the next thing. Why ? Because new things have a ROI, while code reductions and cleaning only costs money.

Totally agree on the ROI point.

Not fully agree with the last statement, that code reductions/cleaning only cost money. If the software is a one-off, then yes. But if the software gets reused by a company, and gets bloated, then the technical debt increases with time significantly. I've seen cases when development teams cannot even do anything new - they got busy with fixing bugs that appear faster they can fix them; also the bloat slows down everything. In such cases, it is cheaper to do a cleanup or redesign rather than to throw more bloat.

But that is again, the ROI / money issue, not a "high standards" issue. If you're a head of development presenting on a management meeting, the "high standards" argument does not work. The money argument does.

[AkiTaiyo]

I recently took it upon myself to completely rewrite the processing engine in one of our imaging software products because the inefficiency and bloat was really bugging me. 

Most of our software is developed by an offshore team who have the approach that a PC has infinite resources, so there is no need to manage them well.

Our software is written in C# so they were leaving the garbage collector to tidy up frames received from the cameras (30fps 3MP cameras).  The garbage collector was running at least once a second and collecting up to 2GB of unused object each time.

I rewrote the core operation with a strong focus on resource management and making use of processor extensions such as AVX and SSE, as well as ditching a bunch of external libraries. 

The improvements are not just noticeable in how responsive the software is, but it was actually measurable at the socket on the test computer. 
Using the old software, the test PC draws 96W from the wall, with the new software it draws 44W. That’s a considerable reduction in power consumption!

[Kjelt]

Not fully agree with the last statement, that code reductions/cleaning only cost money.

Ofcourse as a SDev I know that and you know that. The problem I have encountered the last 22+ working years is that management that has to decide where to spent the fte's on does not see it like that.
They think engineers always want to make things more perfect than necessary, or that SDevs suffer from the "not written by me" syndrome and sometimes that is the case, but fixing bloated software stacks or redesigning them so they become more maintainable costs money but earns itself back on the long term.

Most managers nowadays get rated by short term objectives. Time periods of 1 yr max. I never saw a manager that would get a bonus if their software stack would be improved over 5 years time.
And in my work experience the last 8 years in a SAFe wow it was always what can you demonstrate us you did the last sprints or PI ? It is very difficult to demonstrate you cleaned up the software bloat or architecture and then show the product behaves exactly the same way.

[tellurium]

Most managers nowadays get rated by short term objectives. Time periods of 1 yr max. I never saw a manager that would get a bonus if their software stack would be improved over 5 years time.
And in my work experience the last 8 years in a SAFe wow it was always what can you demonstrate us you did the last sprints or PI ? It is very difficult to demonstrate you cleaned up the software bloat or architecture and then show the product behaves exactly the same way.

Exactly. There is also a career/money dimension to it. Many (all?) developers want nice salaries and be promoted, and there is way easier to get more "karma points" and get promoted by developing something new and "impactful" rather than cleaning up old shit.

For example, if Joe is  the only developer in a company, that wouldn't be a problem. But usually there are many developers. If Joe is the one who cleans the bloat, and Bill from the neighbor team actually produces the bloat ; and then Bill gets promoted, and not Joe.. That is a big demotivator for Joe to do any refactoring/cleanup in the future.

[Bruce Abbott]

They think engineers always want to make things more perfect than necessary,

Which is true. This thread is proof of that.

People who complain about bloat wasting resources and reducing performance don't understand the goal of the system. It's not to make computing more efficient or better for the environment, and it's certainly not to make engineers feel they have done a better job. Our capitalist society is based on convincing others to pay you for the goods and services you produce. So long as they do that it doesn't matter how efficient you are. In fact the more 'make-work' you can convince them is necessary the more it benefits you.

Moore's law says computing power increases exponentially. But without a matching increase in bloat the computer industry would soon be reduced to the same status as toilet paper. Actually worse, because with efficient software a 10 year old computer can still do everything users need it to even after many years of use, whereas toilet paper can only be used once.

Imagine if the engineers got their way and broke the vicious cycle of bloat. Companies like Intel and Microsoft would become shadows of their former selves as sales dried up. Millions of people (including engineers) would lose their jobs and have to do menial work instead. Consumers would discover they didn't need bloat in other things too, like 4k TVs and oversized SUVs. The ripple effect on the economy would be devastating. 

The next markets to crash would be consumer electronics, motor cars and the oil industry, followed by advertising and the news media. Then the internet would crash because content providers would have to charge users rather than relying on dodgy advertising. Unemployment would skyrocket, bitcoin would drop to zero and all our investments would become worthless as the banks failed. All because a few engineers didn't understand the real purpose of bloat.       

[PlainName]

because with efficient software a 10 year old computer can still do everything users need it to even after many years of use

I think you may be conflating bloat with features. If you compare, say, Windows 7 and Windows 2000 there is quite the difference in size - two order of magnitude just for the system drive. Some of it will be bloat but most of it is usability and features that aren't in W2K. Aero, which I'd class as a useful feature rather than bloat, isn't even a dream for W2K.

New features don't depend on just having the resources to waste on them. Mostly they come about because of previous features and someone seeing that and thinking, "Nice! but wouldn't it be cool to..." and off you go.

The contrast of Windows versions is also illustrative of software engineering itself. If you add features for robustness - perhaps bounds checking on arrays - they that requires more code which looks to all the world like bloat, but it has made the software better. Except for a few cases, you don't get owt for nowt, so if you want the fancy high-level language stuff you pay in resources. And you'll be wanting a bigger, faster machine to run it all.

I am not saying that, uh, 'relaxed' coding ability doesn't lead to bloat and thus the need for faster stuff, but I don't see it as the only, or even major, driver.

[madires]

Moore's law says computing power increases exponentially. But without a matching increase in bloat the computer industry would soon be reduced to the same status as toilet paper. Actually worse, because with efficient software a 10 year old computer can still do everything users need it to even after many years of use, whereas toilet paper can only be used once.

Actually, less e-junk and wasted resources would be much better for us to survive on the long term. Based on your reasoning we should limit the lifetime of cars to 5 years, instead of using them for 15 to 20 years or even longer.

[Nominal Animal]

They think engineers always want to make things more perfect than necessary,

Which is true. This thread is proof of that.

People who complain about bloat wasting resources and reducing performance don't understand the goal of the system.

Which is to gamble that the inevitable crash does not occur while you have a stake in the game, even when it means that fixing the system is out of the question and the ensuing crash will be harsher and more violent to the people who have to live through it.

Who cares about them, as long as it is not us that have to suffer, right?

This is exactly what we're doing to Africa right now, and have done so for almost half a century.  Anyone who objects, is obviously a right-wing racist nutjob.


Thing is, there is nothing capitalistic about that.  It's just short term gambling mentality.  Sometimes, you gotta take a risk or a hit, because the alternative is just that much worse.  The way open source has changed the computing industry –– even though some members here hate it from the bottom of their hearts because they feel it is stealing profit opportunities away from them, and they do not have any idea how to profit from it legally –– shows that changes are possible without inducing such a crash, especially if it is done gradually enough.

In particular, I'm not interested in refactoring everything to be better.  But I am for doing it to the key systems, which is the gradual approach that does not lead to a systemic crash because grifters no longer can shave their living off of the fat.