EEVblog Electronics Community Forum

General => General Technical Chat => Topic started by: legacy on October 25, 2019, 12:46:35 pm

Title: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 12:46:35 pm
Facts: everything starting from the Linux kernel to u-boot is usually a mess of code which is continuously full of bugs, the documentation is usually something devs do not produce, and when you find yourself in trouble because source is a complete mess, things completely unclear, and what you get from a git clone behaves like a deranged dog on speed ... well ... do you find reasonable to subscribe yourself to a mailing list asking for a piece of advice?

Be prepared to a shock: nobody will ever reply to you if  - what you ask is not mainstream - and - you are not willing to pay 50 USD / h -

Stefan Roese at Denx replied this way, and  I might accept it, but what about my time spent signaling bugs? tests? etc? I have to contribute for free, and when I need help ... do I have to pay for it?

it is not fair  :-//
Title: Re: sometimes the open-source sucks
Post by: mansaxel on October 25, 2019, 01:01:41 pm
I should know better than feeding trolls.  :palm:
Title: Re: sometimes the open-source sucks
Post by: madires on October 25, 2019, 01:12:19 pm
OMG! If I'm writing open source software people will report bugs and even demand support for free. How horrid! >:D
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 01:33:11 pm
I should know better than feeding trolls.  :palm:

First, learn respect. "Troll" is not appropriate, and rather offensive.

I am a bit shocked by his response. Let me clarify what I mean. I asked a bit of clarification regarding a board of which he is the maintainer. Something like no more of ten min of his time which could have addressed in the right direction. But, the only thing he specified is that he could reply only if paid. Therefore I spent two months on this, and now he wants the patch back, I have spent hours and hours of my time fixing stuff, and I have never asked a penny for this.
Title: Re: sometimes the open-source sucks
Post by: senso on October 25, 2019, 01:35:26 pm
Define wants the patch back.
Title: Re: sometimes the open-source sucks
Post by: rjp on October 25, 2019, 01:35:39 pm
Open source is what people are prepared to do for free, on their own time, for their own amusement.
If you cant deal with that premise, its best you dont participate in it.
Personally, sometimes i fix things, sometimes I walk away, their is literally nothing to complain about, their is noone who cares.
 
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 01:37:46 pm
demand support for free

One thing is "full support", one thing is "a piece of advice" to address someone in the right direction.

Would you ask me 50 USD/h to give advice WHICH will help your software to be fixed for a board of which YOU are the maintainer?

He said that he is the maintainer, but he is no more interested in the board, in fact, he dropped everything in 2017. And ok, I might understand it, but ... now the problem is

he asks money for advice, and then he wants my patch back (to resume the board-support into the 2019-2020 tree?) for free: precisely this is the not fair point.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 01:42:40 pm
Define wants the patch back.

Code: [Select]
diff -u OriginalFile MyFixedFile > myPatchFile
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 01:56:19 pm
Open source is what people are prepared to do for free, on their own time, for their own amusement.
If you cant deal with that premise, its best you dont participate in it.
Personally, sometimes i fix things, sometimes I walk away, their is literally nothing to complain about, their is noone who cares.

OK. Therefore, my perception of open-source was a naive bug, since I give it the meaning of the immortal words of Bill and Ted, “Be excellent to each other”  :palm:


My fault, sorry.
Title: Re: sometimes the open-source sucks
Post by: Monkeh on October 25, 2019, 01:59:07 pm
I am a bit shocked by his response. Let me clarify what I mean. I asked a bit of clarification regarding a board of which he is the maintainer. Something like no more of ten min of his time which could have addressed in the right direction. But, the only thing he specified is that he could reply only if paid. Therefore I spent two months on this, and now he wants the patch back, I have spent hours and hours of my time fixing stuff, and I have never asked a penny for this.

Perhaps he does not have months of time to address a board he considers a dead end?

Or perhaps it's just the tone of your emails.
Title: Re: sometimes the open-source sucks
Post by: SiliconWizard on October 25, 2019, 02:26:53 pm
Facts: everything starting from the Linux kernel to u-boot is usually a mess of code which is continuously full of bugs, the documentation is usually something devs do not produce, and when you find yourself in trouble because source is a complete mess, things completely unclear, and what you get from a git clone behaves like a deranged dog on speed ... well ... do you find reasonable to subscribe yourself to a mailing list asking for a piece of advice?

You sound in a very bad mood, so I think you should let it go before a discussion can be fruitful. ;D

That said, even though you're inflating all this probably because of the above, I agree with that on a general level. But if you don't want to be bothered by that, just write your own software from scratch, or use commercial solutions only.

If, like many others, you're using/exposed to open source because doing without it would either take you way too much time, or cost too much (which in the end is the same), well... you know why you're doing that.

it is not fair  :-//

Yes it is. It's free and open. Drawback is that you're not entitled to expect anything including support.
I don't know if you've released open source software before, but if you have, and it has gotten any kind of traction, how did you feel about being constantly bothered with support you'd have to give away for free? That's just not how things work. As to documentation... yes that's often very lacking in many open source projects. But again, how much documentation do you yourself produce during your developments? Why expect more from others (and for free!) than you'd expect from yourself?

Open source doesn't "suck" any more than gifts do. You may not like what you're given, but you can't complain.

And yes, when designing products, open source is to be used knowing fully well what you're getting into.
Title: Re: sometimes the open-source sucks
Post by: free_electron on October 25, 2019, 02:44:04 pm
Facts: everything starting from the Linux kernel to u-boot is usually a mess of code which is continuously full of bugs,

To paraphrase one of the most common heard things about 'open source': "You have the source ! You are now all powerful, can examine and modify anything you don't like ..."

More commonly encountered as Open source coded by broken sours. Half finished stuff in perpetual state of flux encumbered by religious wars between vi vs emacs and gnome vs kde. Total anarchy reigned by (pitch)forks.

Title: Re: sometimes the open-source sucks
Post by: magic on October 25, 2019, 03:17:28 pm
That's sadly true. We should just agree to settle on vi and xfce already.

Define wants the patch back.

Code: [Select]
diff -u OriginalFile MyFixedFile > myPatchFile
I'm sure the question was "define wants" rather than "define patch" :-DD

And yes, Linux sucks. Anything other than the latest and greatest that every hipster is salivating to nobody gives a damn. Sometimes even working patches take some whining to get accepted because maintainers who weren't around when the code was originally written treat it as "here be dragons" ;D
Title: Re: sometimes the open-source sucks
Post by: newbrain on October 25, 2019, 03:37:57 pm
Open source is what people are prepared to do for free, on their own time, for their own amusement.
Looks at payslips of last 3-4 years: :bullshit:.

And, though not always the case, many of Legacy's complaints are sadly true, especially where no payslips are involved.
Title: Re: sometimes the open-source sucks
Post by: Marco on October 25, 2019, 03:38:51 pm
he asks money for advice, and then he wants my patch back (to resume the board-support into the 2019-2020 tree?) for free: precisely this is the not fair point.

He asks 50$/h to consult on open source software ... have you tried to negotiate a fee for consulting on a closed source version? He might be willing to do that for more.

If he no longer is an active user of the code but integrates patches and gets the project release ready as charity, ie. be the maintainer, your patch has no value to him. The patch might have value in general, that might even be approximately equal to the value of his consultation ... but it still has no value to him. What you're essentially doing is saying that if you do some charity you get the moral right to tell other people how to be charitable.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 04:03:40 pm
I'm sure the question was "define wants" rather than "define patch" :-DD

Yup  :D :D :D

Code: [Select]
def do_patch is ... #previous def
def do_wants is
{
    apply do_patch to each modified file
    tarball modified files
    email.attach tarball
    email.send "attaching patches"
}

Quote
Linux sucks.

Starting from Linux 2.6.26, the PowerPC branch went more complex due to a lot of facts.
PPC and PowerPC were merged into one, the PCI stuff was merged too, and ... whole "platform" stuff was definitively merged into "DTS-ONLY".

Hence, during this process, and starting from 2.6.27, a lot of stuff started to be broken, removed, confused, screwed up, and not documented at all.

Anyway, you see his name in a lot of source: I supposed his name was there as the person to ask to. Yes, he is, but wants money, even for a piece of a piece of enlightenment regarding what he and his collegued did on the firmware and on the kernel.

Quote
Anything other than the latest and greatest that every hipster is salivating to nobody gives a damn. Sometimes even working patches take some whining to get accepted because maintainers who weren't around when the code was originally written treat it as "here be dragons" ;D

Yup. For MIPS this is very frustrating. I have direct experiences.
But, PowerPC was (note the past verb) a bit better years ago.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 04:09:43 pm
Perhaps he does not have months of time to address a board he considers a dead end?

He motivated this reason: EOL board. So why did he request my patches?
Title: Re: sometimes the open-source sucks
Post by: nctnico on October 25, 2019, 04:17:42 pm
Open source is what people are prepared to do for free, on their own time, for their own amusement.
You couldn't be further from the truth. The serious open source projects are backed by large companies who join forces to create a piece of software which they need but isn't part of their core product. Open source (or better put: the various GPL licenses) are very effective frameworks to share the work load in an informal, no so structured way.
Title: Re: sometimes the open-source sucks
Post by: Monkeh on October 25, 2019, 04:18:43 pm
Perhaps he does not have months of time to address a board he considers a dead end?

He motivated this reason: EOL board. So why did he request my patches?

Because if someone is using it and you happen to have improvements which are of reasonable quality, why not take them?
Title: Re: sometimes the open-source sucks
Post by: tunk on October 25, 2019, 04:40:34 pm
Was the original source released under GPL (or similar license)?
If so, you may legally be obligated to release your patches etc.

Also, you're using words like "team" and "we", suggesting you're
a commercial business. If that's the case, then I'm surprised you
expects others to work for free.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 05:01:08 pm
Drawback is that you're not entitled to expect anything including support.
I don't know if you've released open source software before, but if you have, and it has gotten any kind of traction, how did you feel about being constantly bothered with support you'd have to give away for free? That's just not how things work.

I have never promoted my products/projects (for which I do not get a penny) telling "oh, OpenSource is great, we are all friends, everything is for free, come, relax, enjoy it, ..."

During 2005-2007 DENX did it with several PPC eval boards promoted to universities (Do you remember old XILINX boards? They had a PPC405GP core aboard, therefore u-boot was the promoted firmware), therefore, I believe they should thank students and volunteers.

how much documentation do you yourself produce during your developments?

Enough :o

My "crew" is composed of four heads including me, and none of us talk talk the same language native, but we do comment on each critical section, at least. In English. This helps us, as well as it helps people outside the squad.

Open u-boot, and you will wonder what a lot of code is  written for. We spent three weeks just for understanding why the SDRAM was set in a certain way (and, in the end, it was a bug). There is no comment about anything. Open Linux, and you will wonder the same. And I have the feeling this happens for a precise reason: because this way you have to pay to have the answer.

you can't complain.

I do complain because I feel taken fool.


Title: Re: sometimes the open-source sucks
Post by: newbrain on October 25, 2019, 05:09:39 pm
Was the original source released under GPL (or similar license)?
If so, you may legally be obligated to release your patches etc.

It depends very much on what the code will be used for.
And, in any case, code needs not be made public but only available to the users (for a limited time). You have no obligations "upstream".

In my direct experience, the users have never cared!
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 05:11:16 pm
Was the original source released under GPL (or similar license)?
If so, you may legally be obligated to release your patches etc.

Precisely my feeling. I am afraid we will probably delete the whole folder and abandon it for an home made firmware. This way there won't be any problem.

Also, you're using words like "team" and "we", suggesting you're a commercial business.

Good point, but in order to avoid any confusion I had added in my email a link to our website, where we declare the area as no-profit.

Isn't enough?  :-//

Group and team mean "hobbyists" and "students", nothing done for the business. We do not sell anything, and the only things we report on our page is a voluntary donation, which is more for "hardware" donation than "money" donation  ;D
Title: Re: sometimes the open-source sucks
Post by: mansaxel on October 25, 2019, 05:31:38 pm
I should know better than feeding trolls.  :palm:

First, learn respect. "Troll" is not appropriate, and rather offensive.


And perhaps fitting, still, because the first post in this thread was all frustration (that we, now, understand quite well) and neither explanation nor background. It was a good example of flamebait.  Had you counted to 10 before you wrote, and supplied some context, I'd nodded in agreement. (I know this, because I've made this mistake too, frequently.)

On a more general note, FOSS community is not as easy to navigate as one might think. When one is only paid in reputation, recognition becomes a hard currency, and feuds tend to escalate, since people can't move along and earn money somewhere else with the same ease as one can (must!) drop a consultancy gig in the more traditional IT environment. 

Some projects are better than others, and it indeed seems that the PPC part of Linuxland is not a happy place. Not surprising since all the world is an Intel or an ARM. According to some.
Title: Re: sometimes the open-source sucks
Post by: madires on October 25, 2019, 06:51:33 pm
he asks money for advice, and then he wants my patch back (to resume the board-support into the 2019-2020 tree?) for free: precisely this is the not fair point.

I fully understand your frustration. But that's a typical business model of open source businesses. They offer some fancy open source software for free and earn money by selling support. Back in the Amiga days I've written a few tools for FTS networks (Fidonet) for my BBS and published them as shareware. I knew that my tools were used by others but I didn't make any money. Instead I received messages like "I like your tool and I will pay the shareware fee if you add such and such feature". Lovely, isn't it?
Title: Re: sometimes the open-source sucks
Post by: German_EE on October 25, 2019, 06:58:53 pm
In Open Source you get what you pay for, I downloaded my copy of Linux Mint for free and I'm therefore not entitled to any support. If I had purchased a copy of Red Hat Linux with a support contract then I could get a number to call if something stopped working.

Open Source is however fixable. It's possible for someone to go into the source code of a program and correct an issue, the patches are then reviewed (the principle of many eyes) and then distributed amongst the community. Try doing this with MS Windows and see how far you go!
Title: Re: sometimes the open-source sucks
Post by: legacy on October 25, 2019, 07:36:42 pm
I fully understand your frustration.

Thanks. Appreciated  :D

I am seriously having bad days with opensource recently.


But that's a typical business model of open source businesses.

Life is full of new experiences. This is a sort of red-pilling on the real nature of some aspect of the opensource. Something completely different from expectation, hence frustating.

But! we have learned a lot, anyway. Which is always good.

They offer some fancy open source software for free and earn money by selling support.

Yup, exactly. Now I know how they charge for consulting. I supposed this only applied to Windriver when your business is about automotive, but, it seems that certain software companies applies the same rule to everything, including educational.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 26, 2019, 03:21:05 am
Was the original source released under GPL (or similar license)?
If so, you may legally be obligated to release your patches etc.
No.

If the codebase is licensed under GPL, and you provide binaries to someone outside your organization, you need to provide the sources to them as well.

There is no need to provide the sources to anyone else, not even the original developers (although that is considered poor manners of roughly similar magnitude as asking money for answering a simple question over email).

You can use modified GPL code inside your own organization, without releasing the sources to anyone.

If you disagree with this, go read what GNU says about using GPL (https://www.gnu.org/licenses/gpl-faq.html), to see that they agree.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 26, 2019, 03:36:15 am
Be prepared to a shock: nobody will ever reply to you if  - what you ask is not mainstream - and - you are not willing to pay 50 USD / h -
Oh man, I can feel your frustration.  I've been there too.

it is not fair  :-//
No, and a lot of people are assholes, even/especially when working on free/open source stuff.

I wish I knew of a solution, but I don't.

I do know that things exacerbate as the hardware becomes rarer; I guess that is one reason why I don't use legacy hardware much.

Before the first Raspberry Pi came out, I posted on their forum that I have lots of experience putting OSes together (I was quite active on Linux from Scratch, and was very up to date on the inner workings of Linux distributions), and would like to put together a minimal Linux distro for those who wish to integrate their own solutions on top -- and got promptly banned from those forums as a dirty greedy leech; an ugly, dirty, smelly Linuxie.
I only have a Raspberry Pi Zero, and that only because I know the foundation makes a loss on each sale.
If one knows the history of the founders, and the history of Qualcomm wrt. Linux -- basically, if Linux was a person, Qualcomm would probably secretly pay criminals to rape them daily, just because they simply utterly hate the rules Linux operates under --, it is nothing surprising.

The only way to deal with this that I know, is to keep a personal secret shit list of stuff you won't help with, ever again, and why; and to try and keep oneself to forums and groups where you can contribute positively, and where your contributions are valued.  :-//
Title: Re: sometimes the open-source sucks
Post by: Siwastaja on October 26, 2019, 09:00:32 am
Note that while spending "just 10 minutes" to help you in the right direction sounds trivial, multiply that by 100 similar requests from all the others.

They likely are not evil or trying to monetize your needs. They just simply don't have time to work for free, even for "just 10 minutes", because such requests accumulate.

In that situation, all they can do is pretty much copy-paste a standard reply.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 26, 2019, 10:15:20 am
Note that while spending "just 10 minutes" to help you in the right direction sounds trivial, multiply that by 100 similar requests from all the others.

They likely are not evil or trying to monetize your needs. They just simply don't have time to work for free, even for "just 10 minutes", because such requests accumulate.

In that situation, all they can do is pretty much copy-paste a standard reply.

This makes sense, but so why they want my patches back? Frankly this sounds rather sneaky to me.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 26, 2019, 10:19:59 am
I do know that things exacerbate as the hardware becomes rarer; I guess that is one reason why I don't use legacy hardware much.

Have you already seen the change-log made between OpenBSD 6.5 and 6.6?
Retired support for the SGI/MIPS platform.

PowerPC is the next on the death note, and my friend commented "End of line. It looks like most people don't really like to work with legacy hardware, and everybody seems to be ditching most machines like SGI MIPS and IBM 4xx PowerPC these days because they just couldn't find anyone willing to maintain them"

Title: Re: sometimes the open-source sucks
Post by: legacy on October 26, 2019, 10:50:55 am
I wish I knew of a solution, but I don't.

Regarding u-boot, my friend commented "PowerPC isn't our problem, that ancient crappy Promise chip in combination with a firmware that doesn't setup PCI in a sane way are", therefore, since modern Linux Kernels relies on the firmware for the correct initialization, it seems u-boot is the problem, and it will be less frustrating writing the firmware from scratch than having something to deal with Denx's people.
Title: Re: sometimes the open-source sucks
Post by: Karel on October 26, 2019, 12:03:06 pm
Open-source sucks... but the alternative sucks more. So, I'll stick with open-source...
Title: Re: sometimes the open-source sucks
Post by: GlennSprigg on October 28, 2019, 11:46:28 am
Is it just me?... or do I sense that 'someone'  likes 'everything' for FREE, and thinks that
there's a whole support team waiting on-line just to help 'them' ???

I wholeheartedly support & use Open-Source software, with the peace of mind that the
VAST MAJORITY of such 'quality' ware is 'Globally' supported/followed, and with virtually
unlimited help from numerous sources/people. OK... They may not be Microsoft, or the
like, but I would not be as trusting of them for help/solutions if it was !!

Look at 'Open-Office', or 'Inkscape' etc. Totally Free, and put together by a vast conglomeration
of Global experts, with no other interest than sharing knowledge. Questions are fine. However,
making demands of them, without Contributing anything, is a slap in the face to them.
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 11:57:02 am
For me I find it sucks when:

1. I find a bug
2. I respectfully submit a bug report as requested in the README
3. Nothing happens for 6 months
4. I pop on the mailing list asking if I can submit a patch.
5. I get told yeah go ahead
6. I submit the patch via PR or directly.
7. Nothing happens.
8. I give them a nudge.
9. My PR and ticket gets closed by someone on the team.
10. Issue not solved
11. Ask what happened politely.
12. Don't get a response or at worst get a fucking mouthful back.

This is why I don't submit bugs any more to projects which don't have commercial backing.

One of the above bugs ended up a CVE
Title: Re: sometimes the open-source sucks
Post by: Syntax Error on October 28, 2019, 12:45:17 pm
 |O Yep, Github/Slack in a nutshell.

Closure of a ticket (Point 9) is often because there is "not a bug" as the maintainer meant the bad behaviour "by design". Frustrated, the bug hunter, tells them to go fork their project, and moves on.

Open source can often mean a restrictive bro-clique on Reddit.
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 12:46:33 pm
Yeah been there with the "go fork it" thing.

The answer was "fuck you comic book guy from the simpsons".  :-DD

Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.
Title: Re: sometimes the open-source sucks
Post by: Jeroen3 on October 28, 2019, 12:48:49 pm
I don't know what you all expect, but it's above all files and in all installers:
Quote
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
Or something similar.

Thinking it is not fair is also not fair. You got 28 years of development for a kernel, for free.
Then you think 50 USD/h for a few hours of work is wrong.

Go try and license something with IBM or Oracle, or even better... Microsoft.
And then ask them to do something not mainstream.

 :-//
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 01:17:42 pm
I had a neighbour once. He had a nice garden. Let the kids play on it, with no warranty expressed or implied. Then he bought a dog which left dog eggs all over it. People volunteered to pick the dog eggs up but he shouted "no, I want the dog eggs as they are" from his ivory tower. Some people spoke to him personally and he called them all the names under the sun. Everyone slowly left the garden because their kids were not wanting to step in dog shit. The grass yellowed as the heaps of shit piled up. Turns out he wanted people to know he was such a good person but no he wanted an ivory tower to speak from so people could see him in a different light to reality.

And that's where about 75% of open source is. Not through altruism, not through ideology but to promote ones ego via the ideology. This serves no purpose but implosion, self destruction and merely contributes to the heat death of the universe and the destruction of the ideology.

I don't mind picking up the shit myself. I don't mind paying the guy to pick up the shit. Having shit everywhere is bad. I go out of my way to help people with things I have done and released into the wild.

There are a lot of piles of shit brewing.
Title: Re: sometimes the open-source sucks
Post by: mansaxel on October 28, 2019, 02:13:08 pm
Yeah been there with the "go fork it" thing.

The answer was "fuck you comic book guy from the simpsons".  :-DD

Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.

Was it the "username starts with digit" one? Anyway, you're my hero for arguing with the Lennart.
Title: Re: sometimes the open-source sucks
Post by: madires on October 28, 2019, 02:22:32 pm
Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.

Neither do I like poetteringware. Unfortunately it's popular despite being a PITA. I call it "the dark side of open source".
Title: Re: sometimes the open-source sucks
Post by: jadew on October 28, 2019, 02:33:43 pm
Don't you just love it?

When someone complains that opensource sucks, you get a ton of people setting that person straight because: "Well, it's free, so what do you expect?"

When someone says commercial software is better, you get a ton of people setting that person straight because: "That is not true, opensource is the best."
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 03:47:43 pm
Yeah been there with the "go fork it" thing.

The answer was "fuck you comic book guy from the simpsons".  :-DD

Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.

Was it the "username starts with digit" one? Anyway, you're my hero for arguing with the Lennart.

Not that one. It was "DBus crashed and now I can't reboot my fucking server" related.
Title: Re: sometimes the open-source sucks
Post by: james_s on October 28, 2019, 03:57:51 pm
Some software is garbage, some is quite good. I have seen very little correlation in recent years between cost paid and quality.
Title: Re: sometimes the open-source sucks
Post by: magic on October 28, 2019, 04:14:38 pm
Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.
Should have posted it on medium together with screenshots of revelant correspondence and submit it to lamer news :popcorn:
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 04:18:50 pm
Edit: I got into a massive argument with Lennart Poettering once over systemd behaviour and was shouted down. So when they opened the CVE "not a bug" thing I wrote an I told you so essay on github. The fucking cowards deleted it.
Should have posted it on medium together with screenshots of revelant correspondence and submit it to lamer news :popcorn:

He was too busy getting a Pwnie award for lamest vendor. Didn't think I needed to rub it in any further.  :-DD

https://www.theregister.co.uk/2017/07/28/black_hat_pwnie_awards/ (https://www.theregister.co.uk/2017/07/28/black_hat_pwnie_awards/)
Title: Re: sometimes the open-source sucks
Post by: Sal Ammoniac on October 28, 2019, 04:30:10 pm
More commonly encountered as Open source coded by broken sours. Half finished stuff in perpetual state of flux encumbered by religious wars between vi vs emacs and gnome vs kde. Total anarchy reigned by (pitch)forks.

You forgot to mention that there's often no documentation at all. If there is any, it's poorly written and often assumes the user already is familiar with the usage of the software. Looking at the code is frequently frustrating because much of it is poorly commented (I've seen files with thousands of lines of C with zero comments). What comments there are are often out of date or flat out wrong. It's chaotic anarchy.  :palm:
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 04:39:08 pm
Even worse, if there is documentation, it's in GNU info!
Title: Re: sometimes the open-source sucks
Post by: Karel on October 28, 2019, 06:30:11 pm
When someone says commercial software is better, you get a ton of people setting that person straight because: "That is not true, opensource is the best."

Which is more true than false:

Quote
Based on static analysis defect density, in the 2013 report, we found that open source code outpaced commercial code in quality.
This trend continues in 2014;

Defect density (defects per 1,000 lines of code) of open source code and commercial code has continued to improve since 2013:
When comparing overall defect density numbers between 2013 and 2014, the defect density of both open source code and commercial
code has continued to improve. Open source code defect density improved from 0.66 in 2013 to 0.61 in 2014,
while commercial code defect density improved from 0.77 to 0.76.

Linux continues to be a benchmark for open source quality. By leveraging the Scan service,
Linux has reduced the average time to fix a newly detected defect from 122 days to just 6 days.
Since the original Coverity Scan Report in 2008, scanned versions of Linux have consistently achieved
a defect density of less than 1.0. In 2013, Coverity scanned more than 8.5 million lines of Linux code
and found a defect density of .61

https://news.synopsys.com/2014-04-15-Coverity-Scan-Report-Finds-Open-Source-Software-Quality-Outpaces-Proprietary-Code-for-the-First-Time (https://news.synopsys.com/2014-04-15-Coverity-Scan-Report-Finds-Open-Source-Software-Quality-Outpaces-Proprietary-Code-for-the-First-Time)

https://news.synopsys.com/2015-07-29-Coverity-Scan-Open-Source-Report-Shows-Commercial-Code-Is-More-Compliant-to-Security-Standards-than-Open-Source-Code (https://news.synopsys.com/2015-07-29-Coverity-Scan-Open-Source-Report-Shows-Commercial-Code-Is-More-Compliant-to-Security-Standards-than-Open-Source-Code)

http://go.coverity.com/rs/157-LQW-289/images/2014-Coverity-Scan-Report.pdf (http://go.coverity.com/rs/157-LQW-289/images/2014-Coverity-Scan-Report.pdf)
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 28, 2019, 07:08:33 pm
Synopsys is poncing their static analyser there. Amazing the last thing I touched that coverity had given all clear I found a trivial privilege escalation hole in it. YMMV but you can’t measure let alone define quality in a complex system defined by static rules, only the compliance with the rules. Simply you can write a turd that’ll green light a static analyser easily.

Edit: all the steamers in OpenSSL are a good example for ref.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 28, 2019, 08:50:40 pm
Regarding donations

I have randomly known a guy who represents another example of how open source can go wrong - "you want me to fix PCI code, and on top of spending days if not weeks of work on it I should pay for the privilege of buying a PCI XIO bridge from you?" - I had contacted him with the idea of collaboration in mind, and he asked hardware donation like if you are obliged to pay someone.

That's crazy, a donation is a voluntary decision that you make if you like to do it, and it cannot be imposed by anyone for any reason!

He complained that he cannot find hardware on eBay without the need to sell an arm, two legs, and both eyes (the XIO_PCI bridge still cost 300 euro), so I offered to mediate with a couple of sellers from whom my squad buy things simply because we are in a good touch with them hence they can apply a discount, even to him if we persuade them to do so.

And there are also a couple of tricks for the purchase if you know a couple of hardware hacks. I offered to explain, of course for free and this way he would have only paid 50-60 euros rather than 300 euros.

But, in this case, the problem is not opensource, but rather that you cannot deal with a donkey head ( I want to be polite). In fact, You know what? This dude claims and asserts that this is the case where he MUST (must?!?) receive the hardware entirely (entirely!?!) for free, as a donation, because he has been around for the last twenty years and bla-bla-bla as if he was the most contributor to the opensource ever.

I have not even seen his name in any patch, but this is not the point, but rather sometimes open source really looks rotten and sick to me with certain persons.
Title: Re: sometimes the open-source sucks
Post by: Monkeh on October 28, 2019, 09:07:44 pm
Regarding donations

I have randomly known a guy who represents another example of how open source can go wrong - "you want me to fix PCI code, and on top of spending days if not weeks of work on it I should pay for the privilege of buying a PCI XIO bridge from you?" - I had contacted him with the idea of collaboration in mind, and he asked hardware donation like if you are obliged to pay someone.

That's crazy, a donation is a voluntary decision that you make if you like to do it, and it cannot be imposed by anyone for any reason!

He complained that he cannot find hardware on eBay without the need to sell an arm, two legs, and both eyes (the XIO_PCI bridge still cost 300 euro), so I offered to mediate with a couple of sellers from whom my squad buy things simply because we are in a good touch with them hence they can apply a discount, even to him if we persuade them to do so.

And there are also a couple of tricks for the purchase if you know a couple of hardware hacks. I offered to explain, of course for free and this way he would have only paid 50-60 euros rather than 300 euros.

But, in this case, the problem is not opensource, but rather that you cannot deal with a donkey head ( I want to be polite). In fact, You know what? This dude claims and asserts that this is the case where he MUST (must?!?) receive the hardware entirely (entirely!?!) for free, as a donation, because he has been around for the last twenty years and bla-bla-bla as if he was the most contributor to the opensource ever.

I have not even seen his name in any patch, but this is not the point, but rather sometimes open source really looks rotten and sick to me with certain persons.

Indeed, it is very hard to deal with someone who expects you not only to spend your time working on their problem for nothing, but also to spend your own money on the hardware.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 02:32:45 am
Do note that the problems discussed here are not with the software, but with the developers.

Sometimes the two are related, as in systemd being technically crap because its developers lack in design and programming skills; but sometimes they are completely separate, as in systemd folks mostly ignoring bugs and fixes because they think they are better than anyone else, and do not believe their code can have bugs.

The GCC project used to have the latter to the hilt a decade or so ago, but seems to be going in a better direction now.

A lot of open source projects suffer from bugs, because most developers aren't interested in fixing code that does not affect themselves, and will instead spend their time developing new features they think they might like.  Only projects with discipline -- either the company paying the devs, or a project leader who won't accept new features from developers who don't respond to bug reports -- can avoid this.  But few people have the necessary discipline.  (It is also the reason why democracy does not work for software projects: nobody wants to do the dull unglamorous bug-fixing work.)

It is often useful to distinguish code issues from developer issues.  That way, you put the right thing on your shit list.
Title: Re: sometimes the open-source sucks
Post by: hermit on October 29, 2019, 02:44:09 am
Kinda sucks that you have to wait for a project to mature to see how they will treat users.  I don't know if source forge still exists but it used to be frustrating going there and finding 10 abandoned or poorly maintained projects because everyone was duplicating efforts because THEY wanted to be the head god.  People wanted to imitate Torvalds right down to his obnoxious behavior because they were under some stupid delusion that you had the right to be an axehat as project lead.
Title: Re: sometimes the open-source sucks
Post by: rjp on October 29, 2019, 02:46:52 am
Its always funny with conversations that attempt to turn hundreds of thousands of different developers and software systems into one "true"  opinion.

I could offer anacdotes of terrible systems and great systems all day, from both sides of the fence.

Open source is 2 seperate worlds as far as I can tell so its meaningless to lump them - things like linux are no longer community driven and most of the code comes from professional coders working for the big boys, ditto things like postgres and other core web applications.

then we  have 100's of thousands of projects done by random indivuduals and small teams  ,and with those its highly variable and you get what you get, and yes, you need to read the source and fork if required.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 03:00:07 am
I have randomly known a guy who represents another example of how open source can go wrong - "you want me to fix PCI code, and on top of spending days if not weeks of work on it I should pay for the privilege of buying a PCI XIO bridge from you?" - I had contacted him with the idea of collaboration in mind, and he asked hardware donation like if you are obliged to pay someone.

That's crazy, a donation is a voluntary decision that you make if you like to do it, and it cannot be imposed by anyone for any reason!
No, you're in the wrong here, legacy.

Now, I happen to know your intent was not to get a new "customer", but simply to invite a new person to participate in the development.  Yet, that is not what it looks and feels to anyone on the other side.

You see, I've been on both sides, and I can tell you that being on the other side, treated like you treated that person, hurts.

What you did, is akin to asking someone to come to an art exhibit, but at the door, tell them they need to pay for their own ticket.  Or asking someone for a bicycle trip, and when they say they don't have a bike, you offer to sell them one.  It does not work that way; you look like someone wishing to exploit others if you do that.

You need to learn to examine your interactions from the other side, to understand how your actions and communications are perceived, because others cannot perceive your intent directly.  I am telling you this, because this is exactly what I myself had had to learn to do.

If you look up invitation (https://en.wiktionary.org/wiki/invitation) in a dictionary, you can see some of the associated expectations.  The associated term solicitation (https://en.wiktionary.org/wiki/solicit#English) should tell you there is a negative side.  Consider yourself being invited for brunch, for it to turn out being about you buying something from that person.

When we talk about building a community, inviting new members with technical ability is delicate.  It is very easy to be seen as looking for free labour to exploit.  A lot of companies and clients do that -- either as "skill tests", or as "you'll get so much kudos for this!".  So, the community organizer needs to prove their good intentions first.  I am sure all developers have found themselves in a situation they have been exploited for free or cheap labour, and most hate it as much as I do.

The way you should have approached that developer person, is to offer to lend them one, explaining that as a small community of volunteers, you cannot yet afford to buy them one.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 03:13:40 am
I don't know if sourceforge still exists but it used to be frustrating going there and finding 10 abandoned or poorly maintained projects because everyone was duplicating efforts because THEY wanted to be the head god.
That's exactly a people-problem, and nothing to do with open source per se.  You see that even more in nontechnical non-profit organizations.

Open source is 2 seperate worlds as far as I can tell so its meaningless to lump them - things like linux are no longer community driven and most of the code comes from professional coders working for the big boys, ditto things like postgres and other core web applications.
I don't see a distinct difference between commercially-backed and community-driven projects; but I do see a difference in well lead (benevolent dictatorship by a capable person) and poorly managed projects.

Companies can pay developers to fix bugs, and that is a major factor; it makes management simpler and much easier.  Bounties only work for small minority of devs, often the younger ones; not everyone enjoys a gig economy.

You see, there are a lot of commercially-backed projects with poor leadership/management/dev control.  Commercial backing won't guarantee anything, it just makes management easier, because money is a powerful incentive.  Consider PHP: a lot of money has been sunk into it, and it is used by a lot of people, but it is a poor, insecure tool for the things it is used for.  All that money has given it additional warts and publicity, but hasn't made it technically any better.

My little hate button is the forked Linux distros manufacturers provide with single-board computers, with very little support, and finite lifetime.  They are an awful example of how commercial backing is no guarantee.
Title: Re: sometimes the open-source sucks
Post by: rjp on October 29, 2019, 03:22:01 am

My little hate button is the forked Linux distros manufacturers provide with single-board computers, with very little support, and finite lifetime.  They are an awful example of how commercial backing is no guarantee.

Definately, but then again, thats a one time bitten, 10 times shy situation, I think the initial flurry of excitement over those things has died down and now the market is much more mature and focused on the long term support and lack of binary blobs.

I know I look for a publically maintained kernel, onboard emmc and confirmation that the features on the chipset I want are actually supported before I purchase now.


I don't see a distinct difference between commercially-backed and community-driven projects; but I do see a difference in well lead (benevolent dictatorship by a capable person) and poorly managed projects.

Companies can pay developers to fix bugs, and that is a major factor; it makes management simpler and much easier.  Bounties only work for small minority of devs, often the younger ones; not everyone enjoys a gig economy.

You see, there are a lot of commercially-backed projects with poor leadership/management/dev control.  Commercial backing won't guarantee anything, it just makes management easier, because money is a powerful incentive.  Consider PHP: a lot of money has been sunk into it, and it is used by a lot of people, but it is a poor, insecure tool for the things it is used for.  All that money has given it additional warts and publicity, but hasn't made it technically any better.
 

I do make a distinction in the sense that many of the posters rant about lack of documentation, or snarky "fork it and fix it responses" .. I expect those from the random chaos of small teams and individuals, they literally owe you nothing and are perfectly allowed to be busy on something else and not care how useful your contribution might have been.

Those code bases are take it or leave it, be glad if something good happens and you form a relationship, move on if you dont.
Title: Re: sometimes the open-source sucks
Post by: bsfeechannel on October 29, 2019, 05:17:37 am
It is interesting to note that after all these years of free software people still struggle with the concept.

Richard Stallman regrets that the word free in English means free as in free beer and free as in freedom. And also disapproves of the fact that to avoid this ambiguity people chose to call it open source, as it is misleading, since you can actually have open source software with restrictive licenses.

Free software is free as in freedom. It has nothing to do with your agreements to have access to it or have it modified. Once I have it, what can I do with it? Can I use it the way I want? Can I modify it to my needs? Can I correct its bugs? Can I copy it? Distribute it?

So free software is not about get your software for free, have a say in its development for free, have support for free, tell the developer what to do for free, or the like.

Free software is not about software quality either. Although we have very high quality free programs around, there are crappy free programs out there, just like high quality and crappy proprietary ones.

People also confuse democracy with anarchy and use other inconvenient political metaphors to describe how free software is developed.

When you report a bug, you do just that. When you have a patch that solves your problem, you have just that.

When I report bugs, I don't do it because someone has an obligation to solve it, but to raise awareness about the problem. Maybe there's someone out there who knows the solution, or gets interested in solving it. I don't know. It is just an announcement.

If I really want the problem solved, I obviously will have to hire someone to do it. Or tackle the problem myself.

When I find a solution, I also report it because I think it is a retribution for all the voluntary work that made it possible for me to have access to the software. I wouldn't have a bug if the program had not been written in the first place.
Title: Re: sometimes the open-source sucks
Post by: techman-001 on October 29, 2019, 05:25:49 am

My little hate button is the forked Linux distros manufacturers provide with single-board computers, with very little support, and finite lifetime.  They are an awful example of how commercial backing is no guarantee.

Mine too!

Commercial involvement is often a guarantee of a slapped together Linux distro, poorly understood by the seller and only provided to give the illusion of a finished hardware product while they offload their cheap crappy stock before closing their doors and moving on to the next quick buck.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 05:48:53 am
When I find a solution, I also report it because I think it is a retribution for all the voluntary work that made it possible for me to have access to the software. I wouldn't have a bug if the program had not been written in the first place.
One of my personal flaws is that I severely overreact when completely ignored.  I can take negative feedback, but being ignored I do not handle well.  I understand how difficult it is for a developer to consider patches (be they fixes or added features) coming out of the blue, without knowing the context or the submitting developer; and I understand why so many patches get overlooked (not examined by developers).. but it still is a major problem for me.

Code of Conduct people usually do not see how they are in fact making it risky for developers to respond to bug reports, fixes, or patches out of the blue -- because anything can be considered offensive --, and are in fact strongly encouraging devs to completely ignore submissions from unknown people.  This drives tribalisation and projects turning inwards, excluding "outsiders", even if the original intent was completely opposite.

Note that I do not intend to derail this thread by politics; I am pointing all this out to show how complicated it is to manage a development community, to get the effects/results/response you want.  My own flaws and quirks exclude me from several, and I've had had to learn a lot about interpersonal interaction to get even this far.  In the same sense, I wish legacy understands that the point here is to get them to change their behaviour, because it is not congruent with their intent.  That is, to get the results and interaction they want, they need a different approach.  Hopefully, I've at least roughly outlined why and how.
Title: Re: sometimes the open-source sucks
Post by: bsfeechannel on October 29, 2019, 06:52:17 am
Note that I do not intend to derail this thread by politics

The mention of politics was not aimed at you specifically. My intention was exactly to dispel this kind of analogy.
Title: Re: sometimes the open-source sucks
Post by: thinkfat on October 29, 2019, 07:48:03 am
Let me see if I can piece this story together:

You take a piece of free software to give you a headstart on a hardware project. You run into trouble for lack of understanding of the hardware and the software structure. You ask the maintainer of the software package for advice. He replies that the particular hardware you're using is obsolete and he cannot help you right away because he worked on that hardware ages ago and doesn't recall the details, but he would do it if you paid for his time and offers you a reasonable rate. You decline and fix the problem yourself.

Then you go and complain loudly on a public internet forum about how unfairly you have been treated, because the maintainer would like to integrate the patch you developed so that others don't run into the same problem.

This demonstrates a level of entitlement that is just mindboggling.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 08:03:23 am
The mention of politics was not aimed at you specifically. My intention was exactly to dispel this kind of analogy.
I understand; my point was more that team management, or project leadership, is completely different to politics in general, and that what one considers politically effective, does not mean it is applicable to development projects.  I would not want to be ruled by a benevolent dictator in real life, but as a project leadership method, it works orders of magnitude better than democracy.  That sort of thing.

Project leadership and community building is a separate art, but one that can be learned.

This demonstrates a level of entitlement that is just mindboggling.
I disagree: entitlement (https://en.wiktionary.org/wiki/entitlement) is not the right word.

It is a matter of not understanding how project management and inviting new members into a collaboration works.

You can ascribe it to malice or entitlement or lack of empathy, but from offline discussions with legacy, I do not believe it is so.  Legacy has even shown that they are willing to work hard to fix an existing somewhat related problem previously, so I believe there is nothing negative or selfish in the intentions here; only a severe, but fixable, bug in the implementation, related to a misunderstanding of how project communities work.

Some people have an instinctual grasp of that.  Some of them become natural leaders, the rest of them happily exploit other people for their own gain.  People like me have to learn it the hard way, but it can still be done well that way too.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 29, 2019, 01:22:33 pm
"you want me to fix PCI code, and on top of spending days if not weeks of work on it I should pay for the privilege of buying a PCI XIO bridge from you?"

This sentence was sneaky because he had already told me that he wants to fix PCI code, and he already told me that he wants to buy the PCI-XIO. Here he was only trying to make me feel wrong, in order to persuade me to donate the hardware for free.

Oh, let me tell you what happened in the past with similar dudes.

One year ago we offered hardware for free under the agreement that the receiver would have not sold it on eBay but rather used it to collaborate.

Don't you guess what happened then?  :palm:

That feeling and belief that one can rely on someone went down hell because certain human beings do suck. But humans cannot be all bad, ain't it?
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 29, 2019, 02:13:20 pm
What you did, is akin to asking someone to come to an art exhibit, but at the door, tell them they need to pay for their own ticket. 
No. This is wrong.
Perhaps, but based on the information you posted at the beginning of the thread, I disagree.

This is not the moment to be defensive, either.  Admitting error, no matter how painful, allows one to reassess, adjust, and learn.  If there is no pain, there is nothing to overcome, nothing on the line.  I do not know all the details so I do not know if I am right or not, but I have been where you are now, and I tell you, you should take this as a learning opportunity, and analyze the sequence of events to the fullest of your ability.  But be brutally honest to yourself about the ways your own communications may have been (mis)understood; examine the events from all sides you can, avoiding any preconceptions or assumptions: do not make it easy for yourself, or you won't learn.  There is no need to explain anything to anyone else; only to learn from this. 

The simple fact is, you and your community is suffering because of communication issues like this one.  Yes, the world of humans is an unfair and quite crappy place, but we cannot force or even entice others to change.  All we can do is change the ways we ourselves interact with others.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 29, 2019, 02:39:20 pm
Admitting error, no matter how painful, allows one to reassess, adjust, and learn.

if I should consider what those guys said, it would mean my squad is useless and inadequate for everything related to the opensource, and we'd better close the forum and all the related projects.

This hurts, but if going ahead this way is a mistake, then I will for sure close everything.
Title: Re: sometimes the open-source sucks
Post by: Siwastaja on October 29, 2019, 04:22:37 pm
Admitting error, no matter how painful, allows one to reassess, adjust, and learn.

if I should consider what those guys said, it would mean my squad is useless and inadequate for everything related to the opensource, and we'd better close the forum and all the related projects.

The problem is, you didn't originally provide this information. The picture changed quite a lot with your latest posts.

Still, even if they were total manipulative assholes, there usually is something to learn.
Title: Re: sometimes the open-source sucks
Post by: jadew on October 30, 2019, 01:07:34 am
I'm not sure I understand the situation anymore, maybe because I'm tired, but am I getting this correctly that the guy asked for a board, so he can add support for it and you refused?

If yes, then that was your first mistake. I don't know what kind of board it is, but I doubt its price wouldn't have been covered by even just one day of work.
Title: Re: sometimes the open-source sucks
Post by: m98 on October 30, 2019, 11:04:01 am
And this is one of the reasons why I'm never going to bother using complex free software for anything productive if there's a commercial alternative.
Title: Re: sometimes the open-source sucks
Post by: GlennSprigg on October 30, 2019, 11:09:56 am
I guess we can all have good & bad experiences from either Open-Source or Commercial Vendors
when/if we try to offer help or 'fixes' etc.....  Once I found a major bug in a JavaScript Module, being part
of a quite powerful and very popular 'WebSite Creation' package, causing errors due to certain parameters.

I re-wrote the section, & submitted it to them via email. By the very next day, they emailed me back,
confirmed my diagnosis/rewrite, thanking me for my efforts, and informed me they have now included it
in the very next revision Update coming out shortly !   8)

Maybe I was just 'lucky' ?
P.S.   In 'payment', they even gave me a free license for one of their 'paid' software modules.  :)
All I wanted, was to simply help...
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 30, 2019, 11:38:01 am
Maybe I was just 'lucky' ?
No, you just contacted a well-managed project.

And this is one of the reasons why I'm never going to bother using complex free software for anything productive if there's a commercial alternative.
That is fairly idiotic.  You are associating "open source == badly managed project" and "commercial == well managed project".

Instead, look at how the project developers deal with bug reports and patch submissions.  It usually takes about fifteen minutes for me to get a good handle on an open source project, by looking at the mailing list (and list rules) and bugzilla.  With some commercial projects, you either need to find someone who has experience with their support, or take the plunge.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 11:46:59 am
He replies that the particular hardware you're using is obsolete and he cannot help you right away because

Re-read email, he said he has the documentation in his drawer, just he has no time to open the drawer, search for his notes under the letter "W", find the board (for which he is the mainteiner), and give a reply.

Unless you pay for this.

he worked on that hardware ages ago and doesn't recall the details, but he would do it if you paid for his time and offers you a reasonable rate

Checked again the email, and calculated the "resonable rate" ... he said 50 Euro / h with a minimal commit of 5-10 hours, it means 250-500 euro.

complain loudly on a public internet

On the internet, "loudly" means you write in CAPS LETTERS. Did I?

This demonstrates a level of entitlement that is just mindboggling.

This is nothing but simply inappropriate.
Title: Re: sometimes the open-source sucks
Post by: m98 on October 30, 2019, 12:06:21 pm
That is fairly idiotic.  You are associating "open source == badly managed project" and "commercial == well managed project".

Instead, look at how the project developers deal with bug reports and patch submissions.  It usually takes about fifteen minutes for me to get a good handle on an open source project, by looking at the mailing list (and list rules) and bugzilla.  With some commercial projects, you either need to find someone who has experience with their support, or take the plunge.
It might be idiotic, but I've never encountered any notable problems that way. Of course my statement is oversimplified, there are a lot of other indicators I base a decision on if it matters.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 12:11:04 pm
It usually takes about fifteen minutes for me to get a good handle on an open source project, by looking at the mailing list

We are having problems with the router node (http://www.downthebunker.com/reloaded/space/viewtopic.php?f=79&t=515) now. If observed within a time window of a few hours it appears "stable" and "solid" like a stone.

However, within 48h, the probability of a crash is also solid true, and this means there is some "bastard" bug somewhere.

Now, the latest component added is a pair of Oxford Semiconductor Ltd OX16PCI954 UARTs, which are capable of DMA, therefore it's conceivable they could corrupt something, but it's difficult to confirm because the doc is unclear (yes, it might support, but yes/no/implemented?configured?WTF?), and the code ... is weird, even the PCI enum looks weird.

Weeks later, digging deeper finally we observed that the OX16PCI954 doesn't have DMA enabled, hence it cannot be the guilty. There the problem seems now related to the VIA-Rhine chip, which is DMA capable, and stress-tested during our burn-in.

I don't know how accurate Microtik's tests were , but I know that the board is still mainstream with Linux, and nobody noticed that starting from kernel 4.4.* the router does not even bootstrap kernels.

I do find it funny  :-DD

Do you happen to know to whom I should write? I am in the MIPS mailing list, but ... they seem to be not very interested because kernel 4.4 is about for years old, and, in general, people are not very interested in debugging kernels that old.
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 30, 2019, 12:14:02 pm
Ahh microtik. Have you tried using a cricket bat to hit it over a fence?
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 12:19:10 pm
I re-wrote the section, & submitted it to them via email. By the very next day, they emailed me back,
confirmed my diagnosis/rewrite, thanking me for my efforts, and informed me they have now included it
in the very next revision Update coming out shortly !   8)

We spent two months testing PCI_SATA controllers for HPPA and PPC. In the end my team found a bug with the HPPA Linux support, hence we informed the mailing list, and we worked together to get the bug fixed. Then a dude merged the patch mainstream so it was included directly in Linux.

I remember I wrote to a retired engineer who worked for HP during the gold age of PA-RISC, and he said that he didn't remember details regarding the hardware but he had his personal notes in a drawer and he would have given an eye for us.

His help contributed to bug-fix, and it took only a couple of days this way.

Great!!!! That was a super very great experience  :D
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 12:32:54 pm
Ahh microtik. Have you tried using a cricket bat to hit it over a fence?

... that board has an evil story on its shoulder.

I bought it online years ago from a company. The board arrived perfectly packaged and brand new.

Something like two years after the purchase I received a legal letter from a lawyer claiming that from the business budget resulted that my router not paid. He said there was an open business invoice, and he intimated me to give him the money.

At that point, since the router was paid online, for sure it was paid via Paypal, hence I wrote to them in order to have written confirmation, but I had to hire a lawyer to validate the document and to talk with the other lawyer in order to close this story.

In the end, we understood what happened: the person from whom I bought was an employee of a company that was selling Mikrotik products, but he tried to sell a product like if it was of his personal property. Therefore there was an open business invoce ... to me.

Hence he got the money, but not the company. This is something like if you steal something from the company where you are employed.


I spent 200 euro to fix this story, the router cost me 140 euro, therefore now, it's a personal matter, and IT MUST work  :D
Title: Re: sometimes the open-source sucks
Post by: bd139 on October 30, 2019, 12:36:24 pm
Oh nice. Had one of those nice people before. Issue turned out to be a SAP bug in their system but I had to diagnose it remotely from their idiocy  :palm:

I understand the position now :)
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 30, 2019, 12:38:03 pm
However, within 48h, the probability of a crash is also solid true, and this means there is some "bastard" bug somewhere.
Kernel panic? Silent lockup? Anything in the logs? Do you monitor the key component temperatures to verify they stay within sane values?

I don't know how accurate Microtik's tests were
532A is already quite old, so even thermal issues are possible (bad caps?).  Plus, Mikrotik is Mikrotik; I would not expect any support from them.

Do you happen to know to whom I should write? I am in the MIPS mailing list, but ... they seem to be not very interested because kernel 4.4 is about for years old, and, in general, people are not very interested in debugging kernels that old.
No, unfortunately not.  OpenWRT folks might know more, but they seem to hang out on IRC, which I don't use.
Title: Re: sometimes the open-source sucks
Post by: Siwastaja on October 30, 2019, 01:03:27 pm
And this is one of the reasons why I'm never going to bother using complex free software for anything productive if there's a commercial alternative.
That is fairly idiotic.  You are associating "open source == badly managed project" and "commercial == well managed project".

Indeed. Altium Designer, for example, is a very classical example of a commercial project where the management refuses to have bugs fixed, despite ever-growing bug tracker records with thousands of unfixed issues, with reports of customer reps fighting against the customers, claiming that the well-known bugs are caused by the customer's hardware. Instead, they focus on ticking more feature boxes. Fixing trivial bugs is reported to take years, even a decade.

It actually sounds very much like a typical badly managed open source project, and has many similarities to KiCad community.

Which is unsuprising - it's people, after all, who make the management decisions. It's to be expected to see similar management and support problems in both open source, and commercial software.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 30, 2019, 01:07:53 pm
Exactly.  It is not a coincidence that people-managers call managing developers cat herding.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 01:44:41 pm
Kernel panic? Silent lockup? Anything in the logs? Do you monitor the key component temperatures to verify they stay within sane values?

Code: [Select]
do_page_fault(): sending SIGSEGV to minicom for invalid read access from 00000000
epc = 77b867b8 in libc-2.9.so[77ad4000+160000]
ra  = 00414b18 in minicom[400000+2f000]

Code: [Select]
CPU 0 Unable to handle kernel paging request at virtual address 1040ff60, epc == 8049543c, ra == 80495fe0
Oops[#1]:
CPU: 0 PID: 6395 Comm: in.tftpd Not tainted 4.4.197-BlurryFishButt-rb532 #2

Events like these, logs are silent, but we see symptoms of memory corruption in the tty structure, and ethernet structures.

During the burn-in, minicom is used to monitor one of the serial channel bombarded by high-streams, while the TFTP server is also bombarded by requests.

And these two randomly crash, but only under heavy load, probably because there is a concurrency in the PCI bus regarding the two ethernet units and the two uart modules.

I don't know 532A is already quite old, so even thermal issues are possible

Anyway, we have two RB532A with the same behavior. They are running in our computer room, with ventilation and 14 C controlled temperature.

Not sure, but it looks more a software problem; kernel 2.6 are all unable to operate when uart-modules are installed (the kernel crashes immediately), kernel <=4.4 are OK, but it also maybe caps, which haven't been investigated yet.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 30, 2019, 03:21:55 pm
And these two randomly crash, but only under heavy load, probably because there is a concurrency in the PCI bus regarding the two ethernet units and the two uart modules.
Is this RB532A, a Mikrotik board that allows one to stack "daughtercards", each providing mini-PCI slots?
If so, this could just be a Mikrotik hardware design issue also.

But yeah, I do agree it is most likely a software issue.

Have you ran some sort of memtest on it?  If not, that might be a good idea.  (I would personally run it as init, allocating suitable amount of RAM (most of it), mlock() the RAM, an then repeatedly fill it with a Xorshift generator verifying with the same initial data that the data is unchanged.  I do not believe the RAM is at fault here, but making sure, excluding it as a possible reason, could be useful.)

The minicom SIGSEGV looks like a NULL pointer dereference, though.

Also, do you have two mini-PCI UART modules installed?  The driver may not be designed to do that correctly.  A lot of PCI devices assume the system has exactly one of those.  If you have an online link to the sources (gitweb or similar, I'm lazy and don't want to untar tarballs) of that particular driver, I could take a look and make an educated guess.

Not sure, but it looks more a software problem; kernel 2.6 are all unable to operate when uart-modules are installed (the kernel crashes immediately), kernel <=4.4 are OK, but it also maybe caps, which haven't been investigated yet.
No, I agree; I am just pointing out some of the possible causes just in case.

Me, I'd verify the RAM using some sort of quickly written memory test, so I could exclude the RAM from the list of possible reasons.  Then, I'd modify it a bit, to use a few rounds of the Xorshoft PRNG per word of memory, to also pin the CPU, to get a RAM+CPU stress test.  If that does not trigger any crashes or unexpected memory changes, you can basically exclude userspace from the list of possible reasons, leaving just the kernel drivers.  While it can seem like superfluous work, being able to subdivide the problem space like this, excluding possible by unlikely causes one by one, works.

That said, if this indeed is the multi-miniPCI-daughter-card board, and you have more than one daughter card installed, you could test them one-by-one, using the same kernel configuration if possible.  This too takes time, but if they are stable alone, and only unstable when both in use -- perhaps the stack/slot order matters too? --, then you have pinpointed the problem to the interaction between the two drivers.

Then, adding some kind of PCI spinlock in the driver (as you said, the devices do not use DMA), to ensure "logical transactions" to the two devices are not interspersed, might "fix" the issue.  If it does not, it is a hardware issue.  If it does, then one can contact the Linux PCI people, and ask for advice on how to properly implement that feature.

Or this is how I would go about it.  There is probably a much better way; I just don't know it.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 03:56:03 pm
Is this RB532A, a Mikrotik board that allows one to stack "daughtercards", each providing mini-PCI slots?
If so, this could just be a Mikrotik hardware design issue also.

description and a few pics are here (http://www.downthebunker.com/reloaded/space/viewtopic.php?f=79&t=271). It's a rb532A + rb502-daughtercards.

Have you ran some sort of memtest on it?

The firmware comes with a diagnost that test the physical ram: done, all passed.

Besides as a "wild" test, we compiled "cmake" (which strongly uses C++ on large chunks) directly on the board. It took something like two weeks no stop, and it used all the physical ram (64Mbyte) plus 200Mbyte of the stack on the microdrive plus 200Mbyte of the stack over NFS.

This way we tested
- the ram
- the Microdive
- the built-in Ethernet

This test was performed without any additional miniPCI hardware, just the router compiling stuff in a corner.

Insanely crazy, but greatly funny  :D

Also, do you have two mini-PCI UART modules installed?  The driver may not be designed to do that correctly. 

Yup, your plan about testing only one quad-uart sounds good  :D
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 04:00:32 pm
The minicom SIGSEGV looks like a NULL pointer dereference, though.

yes, but an application in userspace usually doesn't crash the kernel like it happened a few seconds later. I do believe it was a synthom that something corrupted both the user and kernel space.

I mean
If something corrupts the memory in userspace -> the app crashes, the kernel does not crash
If something corrupte the memory in kernel space -> weird behaviors manifest followed by kernel crash.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 04:08:15 pm
multi-miniPCI-daughter-card board, and you have more than one daughter card installed

You can only have one daugther-card "rb502" installed on the top of the router "rb532A", and it gives two extra miniPCI slots.

Currently we have
-A- one quad-uart miniPCI module installed on the router rb532A
-B- one quad-uart miniPCI module installed on the daugther-card rb502

Crashes randomly happen the same way, no matter if we test a uart channel on A or B. This probably means the daugther-card rb502 is OK. More tests are need to confirm.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 30, 2019, 04:33:00 pm
Yup, agreed; the stress testing you've done clearly indicates the issue is probably the kernel driver (some PCI detail overlooked, maybe tiny race window), or (unlikely but possible) the PCI hardware implementation on that board.

I would definitely test with just A, and RB502 disconnected, for say eight hours (or until crash).
Then, with just A, but RB502 connected (B not connected), for another say eight hours (or until crash).
Then, without A, but RB502 connected and B connected, for another say eight hours (or until crash).
Title: Re: sometimes the open-source sucks
Post by: legacy on October 30, 2019, 05:48:13 pm
If you have an online link to the sources (gitweb or similar, I'm lazy and don't want to untar tarballs) of that particular driver, I could take a look and make an educated guess.

Thanks for your help. Mure more than appreciated  :D

I have just created some pages here (http://www.downthebunker.com/chunk_of/stuff/public/projects/sonoko-x11/router-board/kernel/k4.4.197) and uploaded text files, only those involved with the serial driver, so you can consult them online.

(http://www.downthebunker.com/chunk_of/stuff/public/projects/sonoko-x11/router-board/kernel/k4.4.197/menu-serial.png)

Code: [Select]
00:0a.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (rev 01) (prog-if 06 [$
        Subsystem: Oxford Semiconductor Ltd Device 0000
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 140
        Region 0: I/O ports at 18800840 [size=32]
        Region 1: Memory at 50012000 (32-bit, non-prefetchable) [size=4K]
        Region 2: I/O ports at 18800860 [size=32]
        Region 3: Memory at 50013000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: serial
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 31, 2019, 07:49:10 am
I'm still just looking the source over, but something poked me in the eye, so although very unlikely (to be a fix), here's my first theory:

The quirk in fixup-rc32434.c disables prefetch and sets PCI cache line size for bus 0, slot 6. It does not have an IRQ, so it is not an active device; perhaps it is the RB502 daughter card with the miniPCI slots?  The quirk looks suspiciously like a lazy dev making a minimal tweak to make their test machine work, seeing as there is no comment explaining the quirk at all.  This makes my paranoia itch.

Granted, the lspci listing shows that the two 4K memory regions on the quad-uart shows that it has those exact settings (prefetch disabled, 32-bit PCI cache lines), so the quirk seems to work (assuming the other quad-UART device has the same memory region features), and I am probably wrong... but I still have that itch.  Best to verify, methinks.

The existence of that quirk is odd to me, considering the fact that it only targets one exact slot in such a modular hardware design.  It is an early fixup too..

Okay, so my first suggestion is to modify the fixup to cover basically all PCI devices.  Essentially, replace arch/mips/pci/fixup-rc32434.c:rc32434_pci_early_fixup() with something like
Code: [Select]
static void rc32434_pci_early_fixup(struct pci_dev *dev)
{
const int  slot = PCI_SLOT(dev->devfn);
const int  bus = dev->bus->number;

/* Force no prefetch and 32-bit cache line size for all external PCI devices.
   The reason for the necessity of this quirk is unknown. */
if ((bus == 0 || bus == 1) && (slot >= 0 && slot < 12)) {
/* disable prefetched memory range */
pci_write_config_word(dev, PCI_PREF_MEMORY_LIMIT, 0);
pci_write_config_word(dev, PCI_PREF_MEMORY_BASE, 0x10);
/* set 32-bit PCI accesses */
pci_write_config_byte(dev, PCI_CACHE_LINE_SIZE, 4);
}
}
I don't recall seeing the entire lspci output, so you might wish to change the bus and slot selection, though.  The idea is to cover all PCI devices outside the processor and PCI bus chip.

Recompile, and stress-test with both quad-UART modules installed to see if it fixes the issue.  If not, revert.

If this did fix the issue (which I only give a small chance), it would indicate a bug/failure/weakness in the RB502 design, or perhaps the PCI bus chip on the main RB532A board.  Perhaps an early quirk is needed to ensure ALL mini-PCI devices on the RB502 won't do PCI prefetches -- remember, the Oxford Semiconductor chip is just a PCI-UART bridge; it is rather obvious why it might go bonkers if prefetches occur, although the 0:a PCI device seems to have the quirked properties  --, or perhaps the entire PCI subsystem on the RB532A tolerates no PCI prefetches.  I think the latter is more likely, and the dev writing the quirk was just too lazy to think about it: the absence of a comment supports this.  Usually devs are rather cross with hardware needing quirks, and like to leave comments explaining why a quirk is needed.  Not having the comment might be an indicative detail.

In particular, if the PCI subsystem hardware architecture is such that PCI prefetches interleaved with driver access can yield the incorrect results, it would certainly explain the random kernel memory corruption.  Prefetches occur very soon before/after an actual/intended access, so such a hardware bug would essentially be like a short race window, during which separate PCI accesses may "swap" their data, or something similar.  Again, I consider this unlikely to yield a fix, but it is certainly something that I would want to exclude before trying anything else.
Title: Re: sometimes the open-source sucks
Post by: legacy on October 31, 2019, 11:47:01 am
In particular, if the PCI subsystem hardware architecture is such that PCI prefetches interleaved with driver access can yield the incorrect results, it would certainly explain the random kernel memory corruption. 

Code: [Select]
--- .config     2019-10-30 21:54:05.297488563 -0000
+++ k4.4.197-rb532.config       2019-10-29 00:11:45.102878924 -0000
@@ -1105,7 +1105,7 @@
 # CONFIG_NET_VENDOR_TOSHIBA is not set
 CONFIG_NET_VENDOR_VIA=y
 CONFIG_VIA_RHINE=y
+# CONFIG_VIA_RHINE_MMIO is not set
-CONFIG_VIA_RHINE_MMIO=y
 # CONFIG_VIA_VELOCITY is not set
 # CONFIG_NET_VENDOR_WIZNET is not set
 # CONFIG_FDDI is not set

On our second board under testing, my friend The Halloween unset the MMIO mode of the VIA-Rhine Ethernet modules, and this "seems" to be an improvement: not yet seen any memory corruption, not yet a panic in the last 36 hours, and the machine seems stable without any glitch  :D

A synthom of memory curruption is perceived as glitch, I mean you suddenly lost the ssh connection to the board, and/or see garbage on the serial console. This usually anticipated a crash.


I will report your message to him, and try to prepare a new patch. Thanks for the hint.
Title: Re: sometimes the open-source sucks
Post by: Nominal Animal on October 31, 2019, 12:15:31 pm
In particular, if the PCI subsystem hardware architecture is such that PCI prefetches interleaved with driver access can yield the incorrect results, it would certainly explain the random kernel memory corruption.
[Disable VIA Rhine memory-mapped IO]
Yes, that patch is extremely interesting, because it stops the driver from using memory-mapped access to the driver, essentially avoiding the PCI prefetch issues as well.  (They are only done for memory and memory-mapped I/O.)  If you do encounter a crash, my suggested patch is not going to work either, because the error is not caused by PCI accesses.
(Unless, of course, the underlying issue is with memory-mapped I/O accesses only.  Could be, but very, very unlikely.)

If disabling the memory-mapped IO works, I am not sure whether it makes sense to try my patch, unless you can measure a significant drop in throughput with memory-mapped IO disabled.  I believe the difference between the two should be small, since no DMA is involved in any case.

I will report your message to him, and try to prepare a new patch. Thanks for the hint.
Do let us know here after testing.

And, to get back to the topic of this thread, hopefully we are in agreement that it is not exactly the open source software that often sucks, but more that development communities often suck, because people in general often suck.  The hardest part is not the software itself, but dealing with the developers, IMHO.