Author Topic: Commercial product source code  (Read 33697 times)

0 Members and 1 Guest are viewing this topic.

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #25 on: April 15, 2014, 11:37:24 pm »
heartbleed was discovered by pen testing it, not by looking at the code.

At work we code review our peer's work and we probably take more time reviewing it than it took to code it.

Oh, the hilarity.  Someone who is on record displaying an utter lack of understanding of the security implications of memcpy and an unwillingness to learn is lecturing others on how to avoid security issues involving -- get this -- memcpy.

Not following you, I never mentioned we even use memcpy, we have custom classes with overflow protection for what is worth. Also we custom make all kinds of analysis tools for our code base. Including a ton of unit tests for all our systems.

And yeah, maybe it's an overstatement about taking longer reviewing code than writing it. But the reviews are in person and line by line from the changelist and always done by peers that know the system.

Edit: having said that, I do use memcpy for my projects at home, if that is what you are referring to. But I know how to use it and on microcontrollers with limited address space I do abuse it as well.
« Last Edit: April 15, 2014, 11:48:16 pm by miguelvp »
 

Offline GiskardReventlov

  • Frequent Contributor
  • **
  • Posts: 598
  • Country: 00
  • How many pseudonyms do you have?
Re: Commercial product source code
« Reply #26 on: April 16, 2014, 12:14:01 am »
Your question makes a lot of sense to me. Learning a language requires seeing (or hearing) it in use and usage examples.  You are asking for software that was/is in a commercial product but the source code is available to the public.  And specifically MCU code. 

I would check:
github
archive.org
avrfreaks

There must be some other good places to check/ask. I wonder about IEEE and other orgs like that. They may be able to provide pointers (pun).
I may do some looking too because I had pretty much the same thought since I recently embarked on MCU programming.

What I'll eventually do and what I always do in a new langauge after "Hello World!" is to come up with something useful and then implement that.  For example when I tried python I wrote an anagram maker and a news aggregator. I learned a lot. 

My problem right now is to come up with an idea.  I am chewing on an idea to make a passive solar room heater.
The MCU will control a small solar power fan that will run based on an algorithm. The fan will move warm air into the room. The heating part is passive and just uses black tubing.
 

Offline SL4P

  • Super Contributor
  • ***
  • Posts: 2318
  • Country: au
  • There's more value if you figure it out yourself!
Re: Commercial product source code
« Reply #27 on: April 16, 2014, 12:55:10 am »
Examples only will never guide you in ow to make your software safe or secure.

First suggestion - make conditions fail-safe.
Second - write modular code where possible - and berify each module in itself.
Third - get a peer to  break it
Fourth - wait for issues & bugs to appear... they will
Don't ask a question if you aren't willing to listen to the answer.
 

Offline GiskardReventlov

  • Frequent Contributor
  • **
  • Posts: 598
  • Country: 00
  • How many pseudonyms do you have?
Re: Commercial product source code
« Reply #28 on: April 16, 2014, 02:13:29 am »
Examples only will never guide you in ow to make your software safe or secure.

First suggestion - make conditions fail-safe.
Second - write modular code where possible - and berify each module in itself.
Third - get a peer to  break it
Fourth - wait for issues & bugs to appear... they will

Show us the code then! I think you're missing the point. You can't learn english or german by reading about it. Or having someone tell you the rules of the language.
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: Commercial product source code
« Reply #29 on: April 16, 2014, 02:31:35 am »
Im interested in this purely from an educational point of view. I find much of the published source code and educational examples are full of errors, Id like to see how the pros do it.

The pros write code that's full of errors too. One reason that commercial code is not readily available is that the owners would often be embarrassed for people to look at it.

There's a simple reason for this. If you make physical devices with bad designs it becomes rather visible. They have gaps in the joins, they vibrate, they squeak, they rattle. You can look at them and see the bad quality.

On the other hand, if you make code with bad designs and poor quality, it is harder to see the problems. If you work hard enough, you can make badly designed code appear to work OK, such that few problems are visible. You can test it a lot and patch over most of the bugs and problems that turn up so they are trapped before they surface. So even if the interior is ugly, the surface can look smooth. Much commercial software is like this. (Which is why, for example, the display driver often crashes on my PCs.)

This is the nature of software. It is often not as good as you would like to hope it is.

The best way to write good software is to learn how to engineer good systems. It doesn't really matter whether the systems you engineer are electrical, mechanical, or software. The underlying design principles are the same. Code is just the material you work with, like metal or wood or circuitry. Learn systems engineering and you will write good software.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #30 on: April 16, 2014, 02:31:59 am »
You learn a language by using it, not by hearing it or seeing it.

Same applies to code.

Edit: but since IanB replied in between, yeah, software, hardware, etc... "engineering" faults in the premise that "it's good enough" that doesn't mean we all do that.
« Last Edit: April 16, 2014, 02:34:32 am by miguelvp »
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: Commercial product source code
« Reply #31 on: April 16, 2014, 02:34:25 am »
You learn a language by using it, not by hearing it or seeing it.

Same applies to code.

Key factors to writing good software, in order of importance:

1. ...
2. ...
3. ...
4. ...
5. ...
6. ...
7. ...
8. ...
9. ...
10. The language you use to code in.
 

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4317
  • Country: us
  • KJ7YLK
Re: Commercial product source code
« Reply #32 on: April 16, 2014, 02:37:46 am »
You learn a language by using it, not by hearing it or seeing it.

People learn by different methods. Making an absolute statement like that discounts a large portion of learners.
I learned several programming languages by studying examples of the code, with the language reference at hand.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #33 on: April 16, 2014, 02:55:41 am »
You learn a language by using it, not by hearing it or seeing it.

Same applies to code.

Key factors to writing good software, in order of importance:

1. ...
2. ...
3. ...
4. ...
5. ...
6. ...
7. ...
8. ...
9. ...
10. The language you use to code in.

True that, I guess, the same applies to spoken languages. But practice and errors (and learning from them)  is key for both.

You learn a language by using it, not by hearing it or seeing it.

People learn by different methods. Making an absolute statement like that discounts a large portion of learners.
I learned several programming languages by studying examples of the code, with the language reference at hand.

Although you are correct on the learning part of being visual, abstract, etc. You only can master them by using them. No teacher or method can give you that.
 

Offline hamster_nz

  • Super Contributor
  • ***
  • Posts: 2803
  • Country: nz
Re: Commercial product source code
« Reply #34 on: April 16, 2014, 03:51:07 am »
Not so sure I agree... for most products it is easy to engineer in safety margins in most products - use more steel, exceed minimum requirements, thicker plastic, more decoupling caps, designed in redundancy and resilience. The tolarance on caps is like +/- 30% or so.

In the most part, building software is more like building in tempered glass. One little defect can cause the whole thing to shatter. It is more like math - for the most part it is either correct or not.
Gaze not into the abyss, lest you become recognized as an abyss domain expert, and they expect you keep gazing into the damn thing.
 

Online Mechatrommer

  • Super Contributor
  • ***
  • Posts: 11534
  • Country: my
  • reassessing directives...
Re: Commercial product source code
« Reply #35 on: April 16, 2014, 03:53:15 am »
Will someone please kill the C language ? Or at least make a runtime lib that does boundary checking so that this kind of crap can never occur ?
you have it for the longest time... VB, Delphi or probably most of other higher level language did this for you. the irony is they are condemned by the "community". if you want my suggestion, i suggest go for VB (the older one) because its the condemned most for its poorest performance, but handle whats your need pretty nicely.

talking about memcpy or alloc or whatever that do not boundary or zero check, its not the security developers fault, they just use features where provided. it should be mr Ritchie's fault. but no, when looking deeper its Intel's fault and then RAM manufacturers fault (for not providing in circuit exception fault or hardware level memory clearing function), looking deeper no, its actually scientists' and professors' fault. if you want to look even deeper down to the ultimatum, sure you can but i wont mention it here (in my post at least). yes, its easy to finger point when we dont intimately understand the purpose of one thing at hand or just take it for granted.

I find much of the published source code and educational examples are full of errors, Id like to see how the pros do it.
The pros write code that's full of errors too.
people always asking the wrong wish... they thought "pros" is all about robustness.they forget the root word of it... "profession", nothing got to do with robustness or zero error. you just need to add a little word to it... "experienced" pros, or "good" pros, but then you dont need the "pros" behind. otoh a "good" "pro" is the one that can take the burden of "given the shortest or limited time" to solve a problem, if you are in the business you know how its done, thats why its kept closed sourced, IanB hinted it pretty well, its no difference with how a kid or developer try to fix others people's code, the only difference with the pro is he is assumed to be expert or full of knowledge in brain about the field.
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline vvanders

  • Regular Contributor
  • *
  • Posts: 124
Re: Commercial product source code
« Reply #36 on: April 16, 2014, 03:56:37 am »
Im interested in this purely from an educational point of view. I find much of the published source code and educational examples are full of errors, Id like to see how the pros do it.

The pros write code that's full of errors too. One reason that commercial code is not readily available is that the owners would often be embarrassed for people to look at it.

...

This is the nature of software. It is often not as good as you would like to hope it is.

The best way to write good software is to learn how to engineer good systems. It doesn't really matter whether the systems you engineer are electrical, mechanical, or software. The underlying design principles are the same. Code is just the material you work with, like metal or wood or circuitry. Learn systems engineering and you will write good software.
I think you're largely over-emphasising the reason commercial code isn't released. It has more to do with competitive advantage and less with how warty it looks. There's even some cases like data oriented design where something might look "messy" to someone who's not familiar with memory access patterns and caches.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #37 on: April 16, 2014, 04:04:50 am »
people always asking the wrong wish... they thought "pros" is all about robustness.they forget the root word of it... "profession", nothing got to do with robustness or zero error. you just need to add a little word to it... "experienced" pros, or "good" pros, but then you dont need the "pros" behind. otoh a "good" "pro" is the one that can take the burden of "given the shortest or limited time" to solve a problem, if you are in the business you know how its done, thats why its kept closed sourced, IanB hinted it pretty well, its no difference with how a kid or developer try to fix others people's code, the only difference with the pro is he is assumed to be expert or full of knowledge in brain about the field.

Don't even know how to respond to this one, other than saying that I'm paid probably 5 times or more than a junior. I guess companies are stupid and pay us lazy old folk more just because  :-DD
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4195
  • Country: us
Re: Commercial product source code
« Reply #38 on: April 16, 2014, 04:32:15 am »
All other arguments aside, it would probably be *really good* to look at ANY OSSW project of significant size on code.google.com, or github, or any other source code management framework that would allow you to see a history of edits, problem reports, and email discussions, as well as just the current code.
A project will multiple maintainers would be more interesting than a single-maintainer project.

Bug/"Issue" tracking is a skill of its own, and a pretty important one.
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: Commercial product source code
« Reply #39 on: April 16, 2014, 04:40:59 am »
I think you're largely over-emphasising the reason commercial code isn't released.

Yes, of course. I didn't say there aren't other more important reasons, I just picked one of the more interesting ones that's not talked about so much.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #40 on: April 16, 2014, 05:11:44 am »
Yeah, that's why there was such a fuss about letting professional athletes compete in the Olympic Games.

Sure I play some ..... and I'm better than ....

You really don't think professional whatever you choose are not better qualified than people developing on their spare time? and not for a living?

But yeah, if it make you happy thinking that way, go for it. You'll be amazed at the low level of competency kids from University come to the workforce, and they do think they know everything, but that's no new.

But yeah, we do hire the bright ones we can find and develop them, their views in life tend to change drastically.
 

Offline andersm

  • Super Contributor
  • ***
  • Posts: 1198
  • Country: fi
Re: Commercial product source code
« Reply #41 on: April 16, 2014, 06:30:50 am »
You really don't think professional whatever you choose are not better qualified than people developing on their spare time? and not for a living?
That's a bit of a non-sequitor. There are excellent hobbyist coders, and there are people who program for a living that are absolutely terrible at it (look no further than The Daily WTF for plenty of evidence of the latter).

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #42 on: April 16, 2014, 06:36:58 am »
You really don't think professional whatever you choose are not better qualified than people developing on their spare time? and not for a living?
That's a bit of a non-sequitor. There are excellent hobbyist coders, and there are people who program for a living that are absolutely terrible at it (look no further than The Daily WTF for plenty of evidence of the latter).

As there are excellent hobbyist musicians, artists, basketball players, football players, insert here. If they are excellent they wouldn't be hobbyists. Or maybe they don't care but most do.
 

Offline tjaeger

  • Regular Contributor
  • *
  • Posts: 101
Re: Commercial product source code
« Reply #43 on: April 16, 2014, 07:16:21 am »
Oh, the hilarity.  Someone who is on record displaying an utter lack of understanding of the security implications of memcpy and an unwillingness to learn is lecturing others on how to avoid security issues involving -- get this -- memcpy.

Not following you, I never mentioned we even use memcpy, we have custom classes with overflow protection for what is worth. Also we custom make all kinds of analysis tools for our code base. Including a ton of unit tests for all our systems.

And yeah, maybe it's an overstatement about taking longer reviewing code than writing it. But the reviews are in person and line by line from the changelist and always done by peers that know the system.

Edit: having said that, I do use memcpy for my projects at home, if that is what you are referring to. But I know how to use it and on microcontrollers with limited address space I do abuse it as well.
You claimed in the other thread that it is fine and dandy use strcpy and memcpy with overlapping memory regions if you have some control over the standard library implementation despite what the C standard has to say about this.  Not only that, but you viciously attacked a code analysis tool that points out that programs that do this exhibit undefined behavior.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #44 on: April 16, 2014, 07:42:13 am »
yeah, but look at what I said.

I did mention that for some projects you really would want that, but not for microcontrollers where space is limited.

Think about it, if you have over 100 people and over 10 years of code base, you don't want to trust anyone. If you have a little project and you have full control then you don't have to stress that much over other people's usage of your code not knowing what it really does.

There are different beasts all together.

At home and on limited space projects I might even do self modifying code. On big projects with over one million lines of code and with many people touching the code everywhere, there is no way I would do that. Experience will let you know when to use certain techniques and when you don't.

So, we professionals know the limits on what to use and when.

What I said about that project you mentioned is that I rather have our team make the policies (and we have many) than some code randomly telling us what to do. We have to do a lot of crap at 60fps, that means 16.666 ms per frame cycle and it can never hitch and always pushing the boundaries of what the hardware can drive, but with reliability and quality assurance on top of all that.

But if I only have 2K of program space on an MCU for some home project, believe me, I'm going to abuse the instruction set because I will make a point to fine tune it to the max. And I always have been an advocate for assembly for MCU programming, C is good for kernels not MCUs
« Last Edit: April 16, 2014, 08:08:24 am by miguelvp »
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4195
  • Country: us
Re: Commercial product source code
« Reply #45 on: April 16, 2014, 07:58:34 am »
Quote
there are people who program for a living that are absolutely terrible at it
Tthere are also "legitimately professional" EEs, designers, professors, scientists, managers, hobbyists who have made good, and so on, who THINK they can program.  Expert Coders who don't know much about algorithms, expert theoreticians who don't know much about coding.   And lots and lots of people who understand a lot less about "security" (or safety) than they think they do.  All you have to do is go look at the initial sets of RFCs that created the internet, to come up with all sorts of fundamental flaws (mail transfer protocols with no authentication at all, eh?  What a great idea!  Loose source routing?!)  Getting paid doesn't automatically make you an expert, and it especially doesn't make you an expert in ALL of bits and pieces that actually go in to making a product.
 

Offline tjaeger

  • Regular Contributor
  • *
  • Posts: 101
Re: Commercial product source code
« Reply #46 on: April 16, 2014, 08:32:23 am »
Think about it, if you have over 100 people and over 10 years of code base, you don't want to trust anyone. If you have a little project and you have full control then you don't have to stress that much over other people's usage of your code not knowing what it really does.
You never have full control unless you know exactly what your compiler does.

Quote
There are different beasts all together.

At home and on limited space projects I might even do self modifying code. On big projects with over one million lines of code and with many people touching the code everywhere, there is no way I would do that. Experience will let you know when to use certain techniques and when you don't.

So, we professionals know the limits on what to use and when.
No, you clearly don't.  Undefined behavior is never okay. Never ever. Ever... Ever.

And experience is worthless here.  As compilers are getting better, they optimize more aggressively and all of a sudden code that used to work fine doesn't work anymore or turns into a gaping security hole.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #47 on: April 16, 2014, 08:45:34 am »
Think about it, if you have over 100 people and over 10 years of code base, you don't want to trust anyone. If you have a little project and you have full control then you don't have to stress that much over other people's usage of your code not knowing what it really does.
You never have full control unless you know exactly what your compiler does.

Quote
There are different beasts all together.

At home and on limited space projects I might even do self modifying code. On big projects with over one million lines of code and with many people touching the code everywhere, there is no way I would do that. Experience will let you know when to use certain techniques and when you don't.

So, we professionals know the limits on what to use and when.
No, you clearly don't.  Undefined behavior is never okay. Never ever. Ever... Ever.

And experience is worthless here.  As compilers are getting better, they optimize more aggressively and all of a sudden code that used to work fine doesn't work anymore or turns into a gaping security hole.

Bah, you play your safe game with mediocre performing code and I'll keep on playing my top game. And for what is worth, I didn't have to contribute a patch for the last 10 years (knock on wood) and yeah one day I had all kinds of managers around me because of some third party problem but I had a fix for that (without a patch) and never became an issue and when the third party fixed the problem we reactivated the code. But yeah never think outside the box and you'll be safe, but going nowhere.


 

Offline tjaeger

  • Regular Contributor
  • *
  • Posts: 101
Re: Commercial product source code
« Reply #48 on: April 16, 2014, 08:58:01 am »
No, you clearly don't.  Undefined behavior is never okay. Never ever. Ever... Ever.

And experience is worthless here.  As compilers are getting better, they optimize more aggressively and all of a sudden code that used to work fine doesn't work anymore or turns into a gaping security hole.

Bah, you play your safe game with mediocre performing code and I'll keep on playing my top game. And for what is worth, I didn't have to contribute a patch for the last 10 years (knock on wood) and yeah one day I had all kinds of managers around me because of some third party problem but I had a fix for that (without a patch) and never became an issue and when the third party fixed the problem we reactivated the code. But yeah never think outside the box and you'll be safe, but going nowhere.

This is getting absurd.  Are you really saying that you need to rely on undefined behavior to get good performance in C?  It's really not that hard:  Undefined behavior means the compiler can do whatever it wants -- which can have disastrous consequences.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Commercial product source code
« Reply #49 on: April 16, 2014, 09:30:42 am »
No, what I said is that compilers shouldn't care about libraries that are not part of the actual programming language as per the thread you are talking about. I can make my own strcpy and I don't want the compiler to police me. So big NO to kcc.

Also what I said also is that if I know the instruction set of the targeted processor, and it can do overlapping memory copies with expected behavior and I would take advantage of that on small MCU.

Also what I said is that I rather program in Assembly because C does a poor job on MCU's.

On large projects tailored to big systems and with a lot of people collaborating, I'll go to safer standards.

I also said that C belongs to kernels, but I wouldn't do overlapping functions there.

If I know that for a particular MCU the memcpy listing behaves like I want it to behave then I might use the memcpy function, but most likely I would use assembly and if I have to support other micros then I will use macros. But only when a program doesn't fit the chip.

There are many aspects to programming. For example displaying a hex number from a 4 line input. I approached it with boolean logic using CPLD like features in a chip, someone else mentioned using an LUT. But using my method I could use the same chip to drive three 7 segment displays, using the "simpler" LUT approach only allowed for two 7 segment displays. So I saved one 3rd of the cost with my more complicated approach.

There are fewer and fewer kids out of University that  know assembly, "Because you don't need it anymore", compilers are crap on what they do and waste a bunch of resources, normally nobody cares because we just throw more resources to it, but on MCU's you just can't do that.

More than your concern with C or C++ or standards I would be more worried about the lack of programmers to embrace functional programming. Multicore everywhere and no program can take advantage of the latest hardware advances. Learn at least some Erlang. I might do you some good.

Cloud computing same thing. C is old, so it's C++ and the funny thing is that Erlang is as old but way different. F# go for it, even Smaltalk, Haskell fine, whatever. C as it is is going to loose it's ground soon because the code will be inefficient in the current hardware.

The thing is, that I do keep up with current trends in both software and hardware, so your thought on what I can do or I can't do are pretty much irrelevant. I will keep on doing what I'm doing because I know myself better than you do, and I know where things are going as well.

One thing is for sure, in programming you just can't stop learning or you'll loose your edge, but the same is true everywhere.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf