EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: RoGeorge on July 10, 2023, 07:39:53 am

Title: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 10, 2023, 07:39:53 am
Did an inventory of my unfinished tinkering projects last weekend, and most are stalled without firmware/software.

The goal is to write fast enough and good enough code, so to get functional projects, not to become a programmer able to work in a team.  Environment is Linux and FOSS tools.

I've wrote small programs occasionally, starting decades ago.  Fortran was probably the first I read a book about in the high school, but didn't practice, later practiced some Basic on home computers, then at university we did some Turbo Pascal, then read K&R which later applied to programming microcontrollers, with some other languages and algorithm books/classes in between (i.e. the Knuth's 'Algorithms' books, about half of the 4-5 volumes).  Latest language I've dealt with is Python.  I know the basics of OOP, but not able to take advantage of it.  I'm stuck with the procedural writing style.  Never tried functional programming or other fancier stuff.

My guess is I might have to unlearn the procedural programming style and move to OOP.  Or maybe I'm really bad at programming.  I don't know what I'm doing wrong, but I know my projects are eventually abandoned because the software part became overwhelming.

What to do to become more productive in writing code (for small/hobby projects)?  Some programming exercises/books/classes to follow?  Something else to learn?  :-//

Most of the hobby projects require to write some MCU firmware, usually in C, and sometimes a PC interface, usually in Python.  Any directions of how to write that faster is welcome.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: Karel on July 10, 2023, 07:52:53 am
Just practise and learn from your mistakes. There's no way to speed up programming without introducing more bugs.
Regarding microcrontrollers, stick with C.

Let the flamewar begin  :popcorn:
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DavidAlfa on July 10, 2023, 08:08:48 am
It's all about practice, same like a language.
I was good at talking english until 18, went several times to Scotland, with 12 I could fully go on my own, book things at the library, go shopping... Stopped  going there and 15 years later, while it's not fully lost, your mind want to say something with perfect pronunciation and the output is, at best, like a phone call from an Indian scammer.

Programming needs everyday use to keep it fresh, stop for few months and you'll need at least several days to recover the agility.
10 years ago I would do a flowchart and implement the code lighting fast, buy I was on it 10hr daily.
Stopped for 8 years, it took quite some work, months, to get everything back.
It's overwhelming at first, you feel like you wasted the entire day for nothing, but your brain is slowly training up.
You might got to bed confused, but after a good sleeping, next day you'll notice it's clearer now, by magic, it's the brain settling things down.

Don't overload yourself too much, take a small nap when you feel it's too much, even sometimes it's better to let it sit until tomorrow.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 10, 2023, 09:18:40 am
Just practise and learn from your mistakes. There's no way to speed up programming without introducing more bugs.
Regarding microcrontrollers, stick with C.

I spent 10 years of my life forgetting the bad programming style I learned in college because it was all wrong.
As a personal opinion based on my personal experience, the biggest contribution here for me is my job, because you can both learn and be motivated.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: brucehoult on July 10, 2023, 10:04:22 am
My guess is I might have to unlearn the procedural programming style and move to OOP.  Or maybe I'm really bad at programming.  I don't know what I'm doing wrong, but I know my projects are eventually abandoned because the software part became overwhelming.

I don't think the language matters much, as long as it gives you access to what you need. Asm and C (and Rust, Pascal, and similar) by definition give you easy access to anything the machine can do.

Python doesn't. Not without calling out to C/asm, which you can do but it adds friction.

OOP or not doesn't matter.

You need to practice SOME form of generalisation and abstraction in your code, so you can design and write small modules that do one thing well, and test them in isolation to the rest of the program.

But it matter basically not at all whether that is structs with functions that do things with them, or "objects" with "methods". It's the same thing.  Also virtual functions vs a type enum and switch statements is basically the same thing and neither is better until you get into programs with tens of thousands of lines of code.

Don't ever write 1000 lines of code and hope it all works first try.

Write and test maybe 10 lines of code at a time. Maybe 20 or 30 if it's completely straightforward code. Maybe only 1 line if it's tricky.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: Nominal Animal on July 10, 2023, 10:13:08 am
Design and document.  Do not trust your thoughts are precise enough or practical; test.  Document your intent, what you want the code to achieve, and not what the code does.

When writing code, write in small units that you can test.  The smaller the amount of code a problem can hide in, the easier it is to find.  Test often.  You probably should spend at least as much time testing your code, as you spend writing it.

I often write tiny test programs to test a single data structure and its associated function interfaces, just so I can check it really works.  I don't make them into libraries, because I don't need generic stuff; I need stuff that does the task I want it to.  Perhaps learn to write such test cases so efficiently that you won't wonder if spending the time writing one is worth it.  (I'm tempted to claim they always are worth it, you see.)

Investigate the different ways of accomplishing things.  This means knowing about different abstract data structures (hash tables in particular, hash algorithms), even though you are not absolutely certain how to implement one, so that you can consider the different ways you can solve a given task.
I see programmers ask questions like "How do I do X with Y?" very often, with Y the completely wrong tool for X.  So, you must also keep asking yourself whether you are trying to solve a problem you need to solve, or are you trying to solve the problem only because a previous tentative decision made you arrive at this.  Very often, such possible solutions form a tree, and you need to consider many alternatives, possibly several steps ahead (or more often, backwards into things you assumed but are not actually axiomatically true), to find the good solutions.
Rarely is there only a single answer or way, and "the best" always depends on the context.

Always consider what the code does if something unexpected happens.  For example, you might have tested your potentiometer readings are always between 15 and 997, except that when the ambient temperature and humidity changes you get an occasional 14 or 998 or something, and your careful scaling routine shits itself.  (My particular favourite of this is when using floating-point numbers, and ranges that exclude one or both boundaries: 99999999.0f/100000000.0f = 1.0f, for example, because 99999999.0f = 100000000.0f using IEEE-754 single precision floating point, float AKA Binary32.)

Learn to read others' code.  At one level, this is understanding what the code does.  At the next level, it is deriving the intent of the programmer from that.  At the next level, how the code interacts with other code.  (Locking, mutexes, and parallel execution interactions tend to be hard on us humans, but fortunately on microcontroller targets we only need to consider interrupts: no locking, only atomicity or detecting interrupted access to non-atomic data.)

Learn to ask good questions, and ask them.  Even if you only ask the question from yourself, do put it into words.  This works, because it forces your mind to organize the idea/question, and format it in a linear fashion.  I use English for all my software development, including my experiment/test codes and their documentation, although my native language is Finnish.  For some odd reason, this helps.  A good question digs into the problem itself, describing the underlying problem, and does not impose unnecessary restrictions (like already selected solution methods).
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 10, 2023, 11:12:27 am
Probably my complaining was too generic.  Let's take an example.  Say the project is to make a datalogger.  I know there are many data loggers already, will only use this theme as a learning example, to identify the pitfalls.

My approach would be like this:

1.  Write some goals/specs for the logger:
    1.1.  Analog inputs:  4
    1.2.  Digital inputs:  8
        1.2.1  (nice to have) Digital outputs (e.g. for turning on/off the DUT, or to trigger a timelaps for a standalone photo camera, etc.):  4
    1.4.  Use a common MCU for its ADC and I/O pins, say an Arduino-like devboard, MCU firmware written in C

    1.5.  Sampling speed - from 1 sample/second to 1 sample/hour
    1.6.  Data is timestamped, then pushed to a COM port at the specified sampling interval
        1.6.1 (nice to have) - store data on a local SD card
    1.7.  Apart from interval sampling, samples can be taken at arbitrary time by a command through the COM port
        1.7.1 (nice to have) - wireless COM port, e.g. a nRF24 or alike

    1.8.  On the PC side, store the timestamped samples in a CSV file, plain text, preferably using Python
    1.9.  A live view window on the PC, to display each sample(s) after reading them from the COM port, text only, no plots
        1.9.1 (nice to have) - a command window able to send a request for an extra sampling commands to the same COM port, while keep logging the incoming periodic samples



The idea is to tell what/how I am tempted to do, then to learn if/what is wrong, and what should be done instead.  This is not code yet, didn't write it yet, but will try to in the next steps, then ask for corrections.

Please keep in mind that I'm an electronist by profession, so I'm tempted to (over)think it as if a program must fully work from the first try.  I'm tempted to design all before writing the first line, while professional programmers tend to rather grow their design (or at least that is how it looks to me), meaning many iterations with more and more complete code at each version.  Not requesting to make it in one go, just telling that it is not about coding advice only, more about software design, or how to write some code to get a job done (don't know if "software design" is the proper term).

- Is this first step a reasonable start?  What should be changed?
- How would you proceed to make/write a logger?

Most of all, how many hours should take to write this?  More like 2, or more like 200 hours?



To highlight again, this is a request for online debugging of what am I doing wrong.  Can not pinpoint that alone, need your help with it.  The intent with this topic is to say at first how I would usually do each step (including my own written code later), then to get feedback for what wrong habits do I have, and how to fix that, or how/what would you do differently.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: MK14 on July 10, 2023, 11:27:10 am
More like 2, or more like 200 hours?

You'll get the software project, around 90% complete in the first 2 hours.  It only needs the other 198 hours, to finish the last 10%.

Edit: Not a joke.
I can't easily find a good source, but here is one example:

https://www.reddit.com/r/webdev/comments/gxcq78/ah_after_you_have_completed_almost_90_percent_of/ (https://www.reddit.com/r/webdev/comments/gxcq78/ah_after_you_have_completed_almost_90_percent_of/)
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 10, 2023, 02:25:47 pm
after you have completed almost 90 percent of/ (https://www.reddit.com/r/webdev/comments/gxcq78/ah_after_you_have_completed_almost_90_percent_of/)

Yup, it's true!
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 10, 2023, 03:11:45 pm
For embedded the learning doesn't only happen inside the person but also in their equipment. To start with something simple but useful, one may get a MCU development kit/setup to become familiar with. Then try to find existing software examples and libraries related to what you want to do and modify/supplement with your own inventions.
Some basic tools for embedded projects are:
- Implement a serial, USB or ethernet link with your linux host.
- Implement a LCD or TFT display
- Implement a sensor with SPI or I2C
- Implement a RTOS
- others..
People in a hurry might prefer Arduino.

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 10, 2023, 04:36:54 pm
Might have done similar projects, the problem is they were painfully slow to write.  For each of these I've put online (somehow incomplete), there are countless other unfinished.  Even the published ones are in an unbaked state, such that not even myself can make a use of them after I forget what I've put there.  A few examples:

Quote
- Implement a serial, USB or ethernet link with your linux host.
Oscilloscope screen capture over LAN (either Linux or Windows)
https://github.com/RoGeorge/DS1054Z_screen_capture (https://github.com/RoGeorge/DS1054Z_screen_capture)

Quote
- Implement a LCD or TFT display
LCD display packed as an Arduino library (only attached as a zip in EEVblog, not pushed to the online Arduino libs repositories):  https://www.eevblog.com/forum/projects/a-2-wires-lcd-with-hd44780-and-3-x-74ls74-(put-2-shift-registers-on-same-2-pins)/ (https://www.eevblog.com/forum/projects/a-2-wires-lcd-with-hd44780-and-3-x-74ls74-(put-2-shift-registers-on-same-2-pins)/)
(https://www.eevblog.com/forum/projects/a-2-wires-lcd-with-hd44780-and-3-x-74ls74-(put-2-shift-registers-on-same-2-pins)/?action=dlattach;attach=1704070;image)

Quote
- Implement a sensor with SPI or I2C
https://hackaday.io/project/2823-how-are-you-errr-are-you-alive (https://hackaday.io/project/2823-how-are-you-errr-are-you-alive)
https://www.youtube.com/watch?v=8PXV2ejueOE (https://www.youtube.com/watch?v=8PXV2ejueOE)

Quote
- Implement a RTOS
Not exactly an RTOS, the closest I might have wrote is a CP/M BIOS, written in Z80 assembler long time ago, it was about 4kB of assembled code, and took a couple of years to finish, didn't count the hours:
https://www.youtube.com/watch?v=wdahLHINNcM (https://www.youtube.com/watch?v=wdahLHINNcM)

Quote
- others..
Some Python simulation of an inflating universe:
https://www.youtube.com/watch?v=BjbNZ7SLzwM (https://www.youtube.com/watch?v=BjbNZ7SLzwM)
Code posted in the YouTube video description:  https://youtu.be/BjbNZ7SLzwM (https://youtu.be/BjbNZ7SLzwM)

Another Python example  (in Jupyter Notebooks) to control a power supply to make a MOSFET curve tracer:
https://www.eevblog.com/forum/testgear/rigol-dp832-power-supply-as-automated-curve-tracer/ (https://www.eevblog.com/forum/testgear/rigol-dp832-power-supply-as-automated-curve-tracer/)
(https://www.eevblog.com/forum/testgear/rigol-dp832-power-supply-as-automated-curve-tracer/?action=dlattach;attach=1297009;image)

Some multi channel PDM (pulse density modulation) demo in C:
https://hackaday.io/project/6356-delta-sigma-versus-pwm (https://hackaday.io/project/6356-delta-sigma-versus-pwm)
https://www.youtube.com/watch?v=jwq-51bcTuw (https://www.youtube.com/watch?v=jwq-51bcTuw)



Please note that they are all very low complexity programs.  Anything more elaborated ends up as never finished.

Can not write small programs fast enough, can not write larger programs at all.  :-\
How to fix this?
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 10, 2023, 05:29:54 pm
Well, my-fs took several years just to be usable.

Don't worry about your difficulties with complexity, I can assure you are not alone.

Just, mind that complexity needs pianification, abstraction and design.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 10, 2023, 05:35:03 pm
Quote
https://hackaday.io/project/2823-how-are-you-errr-are-you-alive
Isn't that a Kinetis board? I would try to run that Kinetis project on the bluepill in order to not lose the effort. Porting things you already know is a good exercise. When i do these things i usually write a work protocol into a text file, so others (or me) can later understand and recover what happened.
Did you get the USB data link between the bluepill and your host running? You could try and use STM32CubeMX and the STM32 HAL libraries to make it happen. If you learn to reuse software components it will be less tedious.

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: cfbsoftware on July 10, 2023, 10:08:45 pm
What to do to become more productive in writing code (for small/hobby projects)?  Some programming exercises/books/classes to follow?
Read the book Programming Pearls by Jon Bentley.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: james_s on July 10, 2023, 11:24:33 pm
It depends on your style of learning, there are some decent programming courses on Udemy although the quality varies, they regularly have sales so you can get them pretty cheap sometimes.

The language doesn't really matter, and neither does OOP or not, the biggest challenge in programming, at least for me is just coming up with the optimal organization and algorithms. If you have a specific project you're having trouble with you might try posting the code on the forum along with a description of the difficulty and see if anyone has ideas. Most of the various building blocks you'll find in any program have been done by somebody before.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 11, 2023, 12:17:52 am
What to do to become more productive in writing code (for small/hobby projects)?  Some programming exercises/books/classes to follow?  Something else to learn?  :-//

Choose simple tools that make it easy to create and understand your implementation. Avoid tools that are so complex that the "learning curve" for their applicability, "edge-cases" and "gotchas" is a problem in itself. Examples: C++, and to a slightly lesser extent, C :(

Create an abstract model of the problem and of your implementation's structure. Abstract => not technology specific. Example: use FSMs, which can be implemented in hardware or software.

Structure your implementation using layered abstractions, with the bottom level being very close to hardware, and the top level looking remarkably similar to your abstract model. Example: bottom level grabs an input when it arrives, top level has an event causing a state change. Implement and test each abstraction on its own.

Define the minimum possible set of functions/operations that allows you to test your design and implementation concepts. Delay elaborating all the variations until later, when convenient. Structure your implementation so that is possible.

And, of course, persevere.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 11, 2023, 07:07:33 am
I think you are asking for social encouragement  :D

if so: open a Youtube channel(1), and post projects there.

I think, it's Good for these points:

(1) it's better and simpler than Hackaday, which can be used as "mirror"
(2) rubberducking
Quote
In software engineering, rubber duck debugging is a method of debugging code by articulating a problem in spoken or written natural language. The name is a reference to a story in the book The Pragmatic Programmer in which a programmer would carry around a rubber duck and debug their code by forcing themselves to explain it, line by line, to the duck
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 11, 2023, 07:55:22 am
Sorry, but no.  I'm asking for debugging/review of what I am doing wrong.  Not asking for encouragement.

Since others can write complex programs while I keep failing at it over and over, for decades, I guess I might be doing something wrong.  Or I might have developed bad programming habits.  Sure, there's always a possibility that I'm not suitable for programming, would not mind if so, though I would like to know why.

At some point complexity seems to explode in my programs, while other programmers seem to somehow keep that under control in their projects.  Or at least that's what I suspect as one of the causes.

Don't know what is it, and it's not the first time I'm asking.  So I thought this time to take a small project, the MCU data logger, and write here what I usually do, step by step (will also try to keep track of the time spent at each step), including the C and Python code, so others can point what I am doing wrong, and if something took me way longer than it should.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 11, 2023, 08:15:45 am
Maybe it's about structured programming? Ever refactored something?
One way to attack this: Break down any source file larger than 400 lines into pieces. Invent meaningful interfaces between the components and a meaningful name for each component. Think about globals. 400 lines of C is the maximum i can write and debug on a good day. The number may depend on the person.
Same thing: Whenever modifying/extending a working application, o good way to modify it's code is to reference new interfaces implemented in additional source files.

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 11, 2023, 08:36:12 am
Sorry, but no.  I'm asking for debugging/review of what I am doing wrong.  Not asking for encouragement.

ok.




Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 11, 2023, 09:16:16 am
Since others can write complex programs while I keep failing at it over and over, for decades, I guess I might be doing something wrong.  Or I might have developed bad programming habits.  Sure, there's always a possibility that I'm not suitable for programming, would not mind if so, though I would like to know why.

Can you design and implement electronic hardware without such problems?

If so, use the same concepts and techniques on software. The details will differ, but the approach won't.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 13, 2023, 11:48:11 am
Can you design and implement electronic hardware without such problems?

That's an interesting question, I never thought about programming that way.  I guess even complex hardware projects (i.e. when I designed a full Z80 Spectrum from scratch, with video and disk controller included https://hackaday.io/project/1411-xor-hobby-a-vintage-z80-computer-prototype ), are still low complexity when compared to software projects.

For the hardware projects, the most agonizing tasks to me are about how to design something using only parts I already had in the inventory, without ordering anything extra.  Another problem (with hardware) is to commit between something very basic, or something very generic.  For example, I need to measure the I/V curve for a lightbulb, but then I want to extend it a little over other ranges, or over other types of components, and so on, adding more, but then becomes too bloated and thinking again about a minimalist project, and so on.  So, for hardware is more of a problem about choice, than a problem about braking a big project into small blocks.
 
Your question made me think I might have a problem with decision making and committing to fixed specs.  Could be.  In hardware such problems are smaller, because the choices are limited by the available parts, and other limitations from the physical world, while in software the possibilities are endless.  In software it is always possible to want/add more, or to switch between a completely different programming ways.

A side effect to this is that even the nicest working program still feels unfinished.  There is always something more you could add, or you could improve to a program.  Very frustrating.



Ever refactored something?

Usually that's where I lose control, so I need to re-arrange/redesign some parts.  I've read a refactoring means better structured code, while retaining the same functionality from before.  I usually end up with buggy code after a refactoring attempt, or with a broken project that doesn't work any more.



When writing code, write in small units that you can test.  The smaller the amount of code a problem can hide in, the easier it is to find.  Test often.  You probably should spend at least as much time testing your code, as you spend writing it.

I often write tiny test programs to test a single data structure and its associated function interfaces, just so I can check it really works.

This is another thing I don't do.  Sometimes I test (manually), for example when I doubt about the limit cases of a loop, or of some calculation, never wrote automated tests, and never had regression tests associated with the code I write.  Testing only very brief, and the tests are always manual tests, with some inputs I suspect might be troublesome.

I've read once some book that was in fashion, about TDD (Test Driven Design).  Write a test for every line seems nonsense to me.  Who's testing the test, then?  I recall even from one of the working places where a software team tried TDD.  Very often the tests were failing because of bugs in the tests, and after struggling with the TDD paradigm for a couple of weeks, they ended up being less religious about TDD, and they only kept a few test that were making sense.  It was a huge software project, and those guys were top programmers at a multinational, not amateurs.

I'm confused what should I test, and how.  Many of my projs have some hardware, so I guess it will take considerable effort to mock the hardware signals.



Only replied here to a few specific question, but I want to thank you all for the help.

Sorry for the late reply, and didn't start the logger exercise because I've found a book with explained project examples, and avidly reading from it:  https://zjnu2017.github.io/OOAD/reading/Object.Oriented.Analysis.and.Design.with.Applications.3rd.Edition.by.Booch.pdf  So far this book seems very close to what I'm trying to learn.  Will see how much I'll be able to apply in practice.

Saying this because in teaching examples, programming seems always easy.  In practice it's never as easy as in the "hello world" seen in the books.  ;D
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 13, 2023, 01:56:47 pm
Can you design and implement electronic hardware without such problems?

That's an interesting question, I never thought about programming that way.  I guess even complex hardware projects (i.e. when I designed a full Z80 Spectrum from scratch, with video and disk controller included https://hackaday.io/project/1411-xor-hobby-a-vintage-z80-computer-prototype ), are still low complexity when compared to software projects.

I disagree. There are many similarities between hardware and software at the requirements, design and testing levels, even if the implementation techniques are significantly different.

Key strategies in designing hardware or software or a combination of them are:

Quote
For the hardware projects, the most agonizing tasks to me are about how to design something using only parts I already had in the inventory, without ordering anything extra.  Another problem (with hardware) is to commit between something very basic, or something very generic.  ... So, for hardware is more of a problem about choice, than a problem about braking a big project into small blocks.

The same is true for software:
.

Quote
Your question made me think I might have a problem with decision making and committing to fixed specs.  Could be.  In hardware such problems are smaller, because the choices are limited by the available parts, and other limitations from the physical world, while in software the possibilities are endless.  In software it is always possible to want/add more, or to switch between a completely different programming ways.

A side effect to this is that even the nicest working program still feels unfinished.  There is always something more you could add, or you could improve to a program.  Very frustrating.

Again, the same points can be made about hardware!

Quote
When writing code, write in small units that you can test.  The smaller the amount of code a problem can hide in, the easier it is to find.  Test often.  You probably should spend at least as much time testing your code, as you spend writing it.

I often write tiny test programs to test a single data structure and its associated function interfaces, just so I can check it really works.

The same is true of hardware. It is a natural fit with manhattan prototyping styles.


Quote
I've read once some book that was in fashion, about TDD (Test Driven Design).  Write a test for every line seems nonsense to me.  Who's testing the test, then?  I recall even from one of the working places where a software team tried TDD.  Very often the tests were failing because of bugs in the tests, and after struggling with the TDD paradigm for a couple of weeks, they ended up being less religious about TDD, and they only kept a few test that were making sense.  It was a huge software project, and those guys were top programmers at a multinational, not amateurs.

I'm confused what should I test, and how.  Many of my projs have some hardware, so I guess it will take considerable effort to mock the hardware signals.

TDD has its virtues, but as is often the case in the software world, zealots rapidly expand good ideas so far they become stupid.

The quality of the tests is something zealous softies often ignore. Too often they think something works because it passes their tests.

Works for hardware and software:
Title: Re: Familiar with programming, not prolific, any advice?
Post by: madires on July 13, 2023, 01:57:12 pm
Can not write small programs fast enough, can not write larger programs at all.  :-\
How to fix this?

A large part of programming is the architecture of the software, i.e. how to split a big problem into smaller ones and how to orchestrate everything. Embedded software adds another level of complexity as the software has to match the resources available and make the best use of the hardware (in the ideal case, side note: the quick and dirty way is to use libs available and go for an oversized MCU). You can find a ton of literature on that, but for hobbyist softdev I'd recommend to get just an overview of common methods and to find out which approach works best for you. It can take some time and effort to find your way.

Another large part is about algorithms, i.e. how to solve the small problems from above. Again, tons of literature available. Reading code written by others helps when it's good code (most isn't). Be careful with coding speed. Sometimes a few lines of a bespoke algorithm can take hours and is worth the effort, instead of typing down a standard solution in minutes but running much slower. Especially with MCUs this can make a difference (or you'll have to go for the oversized MCU again).

And the most inportant part is talent. If you have it things will be much easier and faster. Otherwise it will be a tough ride.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 13, 2023, 02:20:42 pm
Formal/automated testing is important. Otherwise you lose a lot of time doing similar manual tests over and over again. There will be new bugs whenever working on a (almost) mature application. Think of tests as redundant implementation of the same application. Sometimes the application itself gets written twice in order to discover errors..

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 13, 2023, 05:19:40 pm
the number of lines of code in the project reduces. (Explodes the brains of idiots that count LoC as a measure of productivity, though!)

too many lines is also a serious problem for ICEs and AI-assisted automatic testing as the time to analyze the number of linearly-independent paths through a module grows exponentialy with the number of module lines, that's why in my-C I imposed a finited number of arguments passed to a function and a finited number of lines themselves of the body of function.

my-c is the first C-like compiler that cares about expressing the confidence level in a module as 1/(number of linearly-indipendent paths), and since an application is composed of multiple modules, the total confidence level is the sum of the confidence level of each module, the lower the inverse of that final number is, the easier to understand and less risky to change the whole application is.

           { dev , test } = | L(App, confidence level) | < eps
 
Considering "software life cycle", it should it be great for both the development and testing squads as it reduces the number of iteractions (measured by "i0")
Code: [Select]
i0=0;
test_report = {}
is_ok=False
while(is_ok)
{
     dev (in_out: it, in: test_report); /* development team */
     test(in:     it, out:test_report); /* testing     team */
     nbugs=test_report.nbugs;
     is_ok=(nbugs <= eps);
     i0++;
     test_report.is_ok=is_ok;
}
but if you take developers of non-military stuff, worse still home automation developers, and put my-c in front of them (I have already tried this social experiment), they perceive it as a "hostile and too stupidly limiting language" (their words), and they think it is more hostile than a xenonomorph alien for nothing but "for some stupid reasons tied to some stipid abstract ideals", as if its architect were nothing but a fanatic.

That's the human-factor problem in computer science  :-//


p.s.
this is also an example of how a tool can help measure complexity and keep it under control.


edit:
is_ok=(nbugs isEqualTo 0);
is_ok=(nbugs <= eps);
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 13, 2023, 08:14:19 pm
... AI-assisted automatic testing ...

You are living in a state of sin. If you ask the AI whether it has supplied sufficient tests, it will "hallucinate" the answer "yes".

Analogous example: the prosecution lawyer which used AI to generate case references, only to have the defence point out that none of the cases actually existed....
Quote from: https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html
In the declaration Mr. Schwartz filed this week, he described how he had posed questions to ChatGPT, and each time it seemed to help with genuine case citations. He attached a printout of his colloquy with the bot, which shows it tossing out words like “sure” and “certainly!”

After one response, ChatGPT said cheerily, “I hope that helps!”"

That's become a widely know cautionary tale :)

Quote
as the time to analyze the number of linearly-independent paths through a module grows exponentialy with the number of module lines, that's why in my-C I imposed a finited number of arguments passed to a function and a finited number of lines themselves of the body of function.

Those problems are best kept under control by having a design strategy based around the concepts of locality, cohesion and coupling. The language is of secondary importance.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: brucehoult on July 13, 2023, 10:55:54 pm
Your question made me think I might have a problem with decision making and committing to fixed specs.  Could be.  In hardware such problems are smaller, because the choices are limited by the available parts, and other limitations from the physical world, while in software the possibilities are endless.  In software it is always possible to want/add more, or to switch between a completely different programming ways.

In software, do the simplest thing that could possibly work. You can always change it later. Don't add generality "just in case" -- "You aren't gonna need it" (YAGNI)

Quote
Ever refactored something?

Usually that's where I lose control, so I need to re-arrange/redesign some parts.  I've read a refactoring means better structured code, while retaining the same functionality from before.  I usually end up with buggy code after a refactoring attempt, or with a broken project that doesn't work any more.

HUGE red flag.

Refactoring should be a simple mechanical code transformation. It is so mechanical that people make IDEs with the common refactorings built in: "rename variable/function", "move code out of loop", "convert global to function argument", "extract code block into a function", "inline function into its callers", and so on.

Test after every small refactoring. Commit each individual change to git so that you can always backtrack if you break something.


Quote
I've read once some book that was in fashion, about TDD (Test Driven Design).  Write a test for every line seems nonsense to me.  Who's testing the test, then?

That's very simple.

You write the test for a feature (or bug) before you implement the feature (or fix the bug).

Check that the test fails.

Implement  the feature or fix the bug.

Check that the test now passes.

It's not a test for every line of code. It's tests for every feature.

Quote
I recall even from one of the working places where a software team tried TDD.  Very often the tests were failing because of bugs in the tests, and after struggling with the TDD paradigm for a couple of weeks, they ended up being less religious about TDD, and they only kept a few test that were making sense.  It was a huge software project, and those guys were top programmers at a multinational, not amateurs.

I've worked in multinationals. They always have a lot of very ... "mid" ... programmers and only a few stars to keep an eye on the rest. And they obviously were not familiar with TDD, so probably over- or under-applied it, didn't get the idea, etc.

Quote
I'm confused what should I test, and how.  Many of my projs have some hardware, so I guess it will take considerable effort to mock the hardware signals.

Sure, but that's better than abandoned projects.

And it's not THAT much effort. Hardware is just some combination of combinatorial logic and some state machine, right?
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 13, 2023, 11:50:01 pm
Your question made me think I might have a problem with decision making and committing to fixed specs.  Could be.  In hardware such problems are smaller, because the choices are limited by the available parts, and other limitations from the physical world, while in software the possibilities are endless.  In software it is always possible to want/add more, or to switch between a completely different programming ways.

In software, do the simplest thing that could possibly work. You can always change it later. Don't add generality "just in case" -- "You aren't gonna need it" (YAGNI)

That's the dogma, and there is value in it - but it should be applied judiciously and with good taste. The latter is often missing, and the dogma is used to excuse and even to justify it.

My attitude is to identify where enhancements might reasonably be expected to occur, and to ensure that component[1] boundaries are in place. The first functionality that should be plugged into the boundaries is the absolute minimum sufficient to show that all the components are connected. The next should be the minimum necessary in a minmally useful product.

[1] "component" can be anything relevant, e.g. an algorithm in a procedural/functional/OO language, or a "software IC" in a larger bean-type ecosystem, an external computer in a distributed system, or a meter/siggen/scope/etc in an ATE system.

Quote
Quote
Ever refactored something?

Usually that's where I lose control, so I need to re-arrange/redesign some parts.  I've read a refactoring means better structured code, while retaining the same functionality from before.  I usually end up with buggy code after a refactoring attempt, or with a broken project that doesn't work any more.

HUGE red flag.

Refactoring should be a simple mechanical code transformation. It is so mechanical that people make IDEs with the common refactorings built in: "rename variable/function", "move code out of loop", "convert global to function argument", "extract code block into a function", "inline function into its callers", and so on.

That is easy and very successful with strongly typed languages, but far more difficult and less successful with dynamically typed or untyped languages. Think Java/C# for the former, Lisp/C/C++ for the latter.

Quote
Test after every small refactoring. Commit each individual change to git so that you can always backtrack if you break something.


Quote
I've read once some book that was in fashion, about TDD (Test Driven Design).  Write a test for every line seems nonsense to me.  Who's testing the test, then?

That's very simple.

You write the test for a feature (or bug) before you implement the feature (or fix the bug).

Check that the test fails.

Implement  the feature or fix the bug.

Check that the test now passes.

It's not a test for every line of code. It's tests for every feature.

That's OK so far as it goes, but isn't sufficient.

Try writing a test to
... and when you've achieved that, don't forget to write up your proof for publication in, say, the ACM. Then sit back and bask in the adulation you will receive :)

The fundamental point is missed by newbies: test can only show the presence of a fault - tests can never show the absence of a fault. Experienced engineers live by the old motto "you can't inspect (by doing a test) quality into a product: it has to be designed in".
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 14, 2023, 05:25:23 am
Software systems are based on assumptions. One can include health tests into an application to verify basic assumptions, like the famous memory test, checksum test, stack/heap space, clock source, presence of peripherals etc. The application can watch other invariants. This is standard practice when it comes to a certain complexity.
In this realm there is a difference between hardware and software. With hardware one can "see" whether there is a short, a burned fuse or a connector got unplugged. Nowadays many devices have a LED to indicate the powered state. Something like that.

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: thermistor-guy on July 14, 2023, 06:45:43 am

...My guess is I might have to unlearn the procedural programming style and move to OOP.  Or maybe I'm really bad at programming.  I don't know what I'm doing wrong, but I know my projects are eventually abandoned because the software part became overwhelming.

What to do to become more productive in writing code (for small/hobby projects)?  Some programming exercises/books/classes to follow?  Something else to learn?  :-//

Most of the hobby projects require to write some MCU firmware, usually in C, and sometimes a PC interface, usually in Python.  Any directions of how to write that faster is welcome.

What are you like at writing functional specifications, and defining h/w and s/w interfaces for your design? Can you write a clear spec., or define an interface, and then design to it?

The less you write down, the more you have to store in your head, and that quickly gets overwhelming.

I remember one of my past team leaders saying, "If it's easy to do, it gets done." His corollary was, when faced with something difficult, find ways to make it easier.

When writing embedded code, I used to write comment sections stating what the code has to do, and briefly why. The I'd write code to do exactly that.

I'd use this as a go/no-go test - if I couldn't write that comment section, it meant I didn't really know what I was trying to do. So I'd stop, generate supplementary
design documentation that nailed down that particular aspect, then go again. That supplementary document might be something small - a decision table, a timing
diagram, a pinout, a sketch of data flows. But it was something I couldn't keeo in my head, and needed to see in front of me, while I wrote the code for it.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 14, 2023, 07:31:11 am
When writing embedded code, I used to write comment sections stating what the code has to do, and briefly why. The I'd write code to do exactly that.

I'd use this as a go/no-go test - if I couldn't write that comment section, it meant I didn't really know what I was trying to do. So I'd stop, generate supplementary
design documentation that nailed down that particular aspect, then go again. That supplementary document might be something small - a decision table, a timing
diagram, a pinout, a sketch of data flows. But it was something I couldn't keeo in my head, and needed to see in front of me, while I wrote the code for it.

That's a useful technique for all software, and hardware for that matter.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 14, 2023, 08:47:50 am
Quote
as the time to analyze the number of linearly-independent paths through a module grows exponentialy with the number of module lines, that's why in my-C I imposed a finited number of arguments passed to a function and a finited number of lines themselves of the body of function.

Those problems are best kept under control by having a design strategy based around the concepts of locality, cohesion and coupling. The language is of secondary importance.

my-c is simply a "meter" to "measure" (well, to "estimate") the complexity, used in design strategy based around the concepts of locality (per function, per module), cohesion and coupling (useful during system integration, again to "measure" the final complexity).



Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 14, 2023, 08:57:44 am
You are living in a state of sin. If you ask the AI whether it has supplied sufficient tests, it will "hallucinate" the answer "yes".

AI means a lot of things. AI-assisted means we use statistically based predictive algorithms to create test vectors, things that usually humans(1) would not think of doing, things that potentially could not be tested.


(1) note that in the military sector it is absolutely forbidden for the development team to talk to the testing team, so even that they don't even have to greet each other, that they are sitting in two different openspaces, that they go to two different toilet rooms and different canteens (same physical canteen room, but different timetables), and the only allowed form of communication is the test_report, as I indicated in the loop above.
Code: [Select]
     ...
     dev (in_out: it, in: test_report); /* development team */
     test(in:     it, out:test_report); /* testing     team */
     ...
Why'? because if you make the testers talk to the developers, you will pollute the test vectors, since the testers will be influenced by what the developers say, and vice versa.

Humans have bias problem.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: brucehoult on July 14, 2023, 09:23:13 am
(1) note that in the military sector it is absolutely forbidden for the development team to talk to the testing team, so even that they don't even have to greet each other, that they are sitting in two different openspaces, that they go to two different bathrooms and different canteens (same physical canteen room, but different timetables), and the only allowed form of communication is the test_report, as I indicated in the loop above.

Ha!

There have been a couple of jobs at medium-large companies where I have ended up dating the QA person responsible for torturing my code!
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 14, 2023, 09:38:55 am
in the most extreme case, they even relocate to two different buildings in two different geographical areas, several thousand kilometers away, and forbidding to hire human with personal relations (husband, wife, boyfriend, girlfriend) between those in the development area , and who is in the testing area.

-
as corollary, this seriously means you cannot test your own software with the same efficiency (in finding bugs) that you can have if other people test it and report to you.

So, for my personal projects I adopt these contromeasures:


(1) in practice it means having functions with fewer lines of code to test, and much fewer test vectors if(...){}else{}: that's how complexity is measured by my-c, it has a precise algorithm to measure the number of linearly independent paths, but but to make it immediately understandable, we can simplify and say that it more or less means this.

Commercial Software like Cantata have similar "meters", but they are very expensive.

Of course, that it will then be a system integration verfication. You have functions that individually work well, you put them together ... and you have to see how they behave, but since you have a lower total complexity, it is very likely that you also have fewer flaws.

my-c, when integrated into Geany or Nedit, can be queried live, and you can ask to measure the complexity of the function, or of the module, you're currently working on, then you can iteratively try to reduce it.

I love this! It helps a lot! Refactoring is a pleasure this way  :D
Title: Re: Familiar with programming, not prolific, any advice?
Post by: Nominal Animal on July 14, 2023, 09:39:31 am
I've read once some book that was in fashion, about TDD (Test Driven Design).  Write a test for every line seems nonsense to me.  Who's testing the test, then?  I recall even from one of the working places where a software team tried TDD.  Very often the tests were failing because of bugs in the tests, and after struggling with the TDD paradigm for a couple of weeks, they ended up being less religious about TDD, and they only kept a few test that were making sense.  It was a huge software project, and those guys were top programmers at a multinational, not amateurs.

I'm confused what should I test, and how.  Many of my projs have some hardware, so I guess it will take considerable effort to mock the hardware signals.
I'm a pragmatist.  I use tests to find out.  Mad scientist with his experiments, perhaps.

For example, in another thread (https://www.eevblog.com/forum/beginners/mcu-atx-project-how-to-control-fan-speed/msg4960495/), I'm discussing with Sparkydog about how to control one or two 12V 4-pin PWM fans at constant RPM using minimal circuitry.  As typical fans age, the PWM duty does not stay constant wrt. airflow/RPM, so using a small MCU to give a startup pulse and then control the fan RPM via its tachometer output (two pulses per rotation), with a FAN-FAULT signal to the rest of the board in case one or both stall, makes a lot of sense.

Let's say I decide to write such a firmware for some 8-bit AVR, perhaps ATtiny85.  (I do that sort of stuff for free, only asking them to pass the help forwards.)
How do I go about it?

The circuitry needed is simple, and I've already simulated it.  I only need two cheap N-channel MOSFETs (NX138AKR is my favourite, assuming hand-soldered board) and a couple of resistors, and a bypass capacitor for the microcontroller.  An MCU with a timer peripheral with two output channels (most AVRs!) is needed, so that the 25kHz PWM output is handled by the peripheral.

The "trick" is how the tachometer inputs are handled.  The pulse interval varies between say 0.1s (100ms, 300 RPM) and 0.01s (10ms, 3000 RPM).  Not only are the two fans completely unrelated, but you want to find out if either one or both are stalled, i.e. pulse interval exceeds the maximum limit.
We can do this using input edge interrupts, or using an innermost busy loop counting loop iterations as "ticks".  If we use an innermost busy loop, each tachometer measurement can take up to 0.3s (because we need to detect a pair of pulse edges from each fan).  That is more than sufficient for the PWM update rate, as the fan blades have significant inertia, and the response to PWM changes isn't instantaneous anyway.

I would start by testing the two method of two-fan tachometer reading: the interrupt method, and the counting loop iterations method.
Both have their upsides and their downsides.  Also recall that to get say 8-bit precision at the 25 kHz PWM, we need the PWM timer base to be at least 6.4 MHz; or say 4 MHz, for duty cycle control between 48 (30%) and 159 (100%-1).  Thus, the AVR MCU clock will have to be 4 MHz or higher in any case.

(The simulations I've already done to find out the electrical requirements and suitable components could be counted as the first "testing" I did.)

As the controller wants to provide an initial start pulse to get the fan moving, I must then consider how the overarching logic of controlling the target tachometer reading, towards which the fan PWM will be adjusted.  In this case, it is straightforward: it is done after each tachometer sampling, as an outer main loop, iterating at least once every 0.3 seconds, but possibly as often as once every 0.01 seconds.  So, this outer loop does need some kind of way to reliably measure time intervals under half a second, so it can tell when PWM adjusting needs to be done or not.  It does not need to be precise, however, so even knowing the inner loop iteration count or similar will suffice; also so would an 8-bit one-second timer (with 4ms resolution), as long as it is under the outer main loop control and not related to PWM.

Next, I would separately implement the PWM control, and test that.  Note that some AVRs have only one 16-bit timer/counter (with two PWM outputs at different duty cycles but the same frequency), and no cycle counter, so Arduino-style millis() or micros() functionality requires a separate timer/counter, or interaction with the PWM clock.  I would rather avoid having an interrupt fire 25000 times a second on a small AVR, though.

Next, I would combine the above, and check the interactions.  At every stage, I make sure there are no bugs (so no "I'll fix that later"!), and no compiler warnings.  This minimizes the regions where bugs may lurk at, and makes it easier and faster to handle the interactions.

Along the way, as I "test" things, I record my assumptions and findings as documentation.  This means that when I get to this point, I should only need to collect the documentation, write some glue text (introduction, licensing, etc.), and the project is done.

I'd probably also create a variant of the firmware that instead of the FAN-FAULT signal, uses a potentiometer to control the fan RPMs (with each fan having their minimum and maximum RPMs hardcoded in the firmware).  That would be useful for others, too, which is a large part of why I'm considering writing the above code.

I have absolutely zero idea how long doing the above should take.  I know I should know, as I definitely have the experience, but I just do not: it is one of my very significant faults.  At least I'm now old enough to report my progress correctly (and only because I learned to write out my plan like this above, and describe how much of each stage has been completed, and what unexpected stuff I discovered); I used to fail a lot at that, too.

While this may or may not help, the above is what I meant by "testing".  It is as much exploration, as it is verification; and I use it not just to test the code, but my own understanding of the problem and the behaviour and limitations of the solution model I'm proposing.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 14, 2023, 09:42:20 am
There have been a couple of jobs at medium-large companies where I have ended up dating the QA person responsible for torturing my code!

the theory, the practice ... the problem is that in the QA department there was a beautiful Swedish girl with green eyes and long legs ... how can you not invite her to lunch? maybe outside the base so as not to point out that you have the yellow card (from the development department) and she has the red one (from the QA department), and therefore you shouldn't even say hello to her?

We are human after all...   :-//
Title: Re: Familiar with programming, not prolific, any advice?
Post by: brucehoult on July 14, 2023, 09:54:46 am
There have been a couple of jobs at medium-large companies where I have ended up dating the QA person responsible for torturing my code!

the theory, the practice ... the problem is that in the QA department there was a beautiful Swedish girl with green eyes and long legs ... how can you not invite her to lunch? maybe outside the base so as not to point out that you have the yellow card (from the development department) and she has the red one (from the QA department), and therefore you shouldn't even say hello to her?

We are human after all...   :-//

Well, yes. The one in Redmond in 1998 was blonde. And intelligent. First date, she took a taxi from work to the Bellevue TGIFs (her suggestion), I drove there. Drove her home on the completely opposite side of Redmond, after parking up beside a lake for an hour or two.  Better not to start the rumour mill in any company prematurely, or at all.

Funny story: in Moscow I *twice* was in some random metro wagon late at night or in the weekend when my (married w/ children) department head entered the same wagon together with a programmer from another department [1]. When my stop came I casually said "Hi Sveta, Sergey" as I went out the door.

[1] the woman who starts off nearest to me in this https://www.youtube.com/watch?v=yBOahTgmys4 (https://www.youtube.com/watch?v=yBOahTgmys4)
Title: Re: Familiar with programming, not prolific, any advice?
Post by: SiliconWizard on July 14, 2023, 09:19:41 pm
When writing embedded code, I used to write comment sections stating what the code has to do, and briefly why. The I'd write code to do exactly that.

I'd use this as a go/no-go test - if I couldn't write that comment section, it meant I didn't really know what I was trying to do. So I'd stop, generate supplementary
design documentation that nailed down that particular aspect, then go again. That supplementary document might be something small - a decision table, a timing
diagram, a pinout, a sketch of data flows. But it was something I couldn't keeo in my head, and needed to see in front of me, while I wrote the code for it.

That's a useful technique for all software, and hardware for that matter.

Yes, well probably more than just useful.
Generally speaking, what it comes down to is, if you don't know what you are doing while engineering something, things are not going to end well.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: dietert1 on July 14, 2023, 09:50:32 pm
Also another example for redundancy: Describing the same thing first in natural language, then in a programming language. The 400 lines in one day i mentioned above are possible only after having a problem absolutely clear. Just coding and debugging.

Regards, Dieter
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 15, 2023, 10:12:24 am
After the first 4 chapters (Concepts) from the book I've found couple of days ago, https://zjnu2017.github.io/OOAD/reading/Object.Oriented.Analysis.and.Design.with.Applications.3rd.Edition.by.Booch.pdf turns out I don't know all the basic mechanisms already available in OOP languages.

For example, at page 144 of 717, one of the bullet points reads
Quote
The three kinds of relationships include association, inheritance, and aggregation.

I knew only about inheritance, not about the other two.  ;D



Will try again, this time starting from square one:
Any book recommendation to learn the basics about objects, please?

Would prefer a book where the teaching style is from bottom to top, to start from some examples and counter-examples, then to deduce the abstractions, so to see how the ideas came into existence, then some exercises.  I don't like the style where some generic/abstract idea is stated first (apparently pulled out of thin air), then justified later by telling how to apply it to whatever situation the abstraction was meant for in the first place.

Preferred language for exercises would be Python.  Ideally would be a book with only the OOP basics, and language agnostic, if such a language agnostic OO book exists (doubting about existence because I've read some languages have more OO features than others, so I guess the OO basics might be different between different languages).
Title: Re: Familiar with programming, not prolific, any advice?
Post by: brucehoult on July 15, 2023, 10:23:36 am
Note that OOD is not the same thing as OOP.

Not to mention that, yes, there are many interpretations of OOP.

- inheritance or prototypes?

- multiple-inheritance, or single-inheritance plus interfaces/mixins/traits?

- inheritance of implementation or only inheritance of interface?

- internal methods or external?

- selection of virtual function based on the dynamic type of only the first argument (and maybe static types of the others), or based on the dynamic types of all arguments?

- mutable state in objects, or immutable? YES! You can do OOP perfectly well in pure functional languages, returning a new object rather than modifying the original one.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 15, 2023, 11:04:11 am
Note that OOD is not the same thing as OOP.

I've noticed in those 4 chapter I've just read that, indeed, apparently generic words I might have used interchangeably, might in fact have a very specialized meaning in programming.

At this moment, to me inheritance means a child class will have (by default) the same attributes and methodes as the parent class has.  As for the rest of the questions, I can only assume what they are about, but I don't know the exact meaning.

Just to be sure I'm not accidentally misusing dedicated terms, asking in plain English:
- would like to write programs based on objects, as apposed to programs based on procedures/functions only.
- what book to start with?

Assume zero knowledge about programming with objects.  I only program occasionally, for my hobby projects, procedural programming style + common sense, just as a singer that sings by the ear and never went to a proper school to learn how to read the musical notes (by profession, electronist specialized in hardware/digital electronics, not software).

The final goal is to write with ease, and faster, the software part for my hobby projects.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: RoGeorge on July 15, 2023, 02:23:24 pm
Searched all morning browsing books that teach programing with objects.  Browsed 5-10 top titles that come up from a Google search.  Very disappointed.  Talking about the form, can't judge the content myself.

Most authors can't write concise sentences.  "Basically", "obvious", "actually", "clearly" and alike emphasizing or opinionated words, comics rather than diagrams, long idioms instead of a single word, etc.

George Carlin - Flying [Live from NYC '92]
https://www.youtube.com/watch?v=46fOtLfYC4Q (https://www.youtube.com/watch?v=46fOtLfYC4Q)

Eventually, the only decent book I could find was "The C++ Programming Language Fourth Edition" of Bjarne Stroustrup.  http://chenweixiang.github.io/docs/The_C++_Programming_Language_4th_Edition_Bjarne_Stroustrup.pdf (http://chenweixiang.github.io/docs/The_C++_Programming_Language_4th_Edition_Bjarne_Stroustrup.pdf)

It has 1300+ pages, which is a bummer, but at least it's first hand info coming from the C++ author.

Never learned C++ before, because when I first encountered C++ I didn't like the "::" operator.  It was a childish reason, I know.  I'll commit to this book for now, better late than never.

Though, before proceeding, I want to ask if it would be OK to learn from this 4th edition, or from the 1st edition (which I assume might be shorter and easier)?
Title: Re: Familiar with programming, not prolific, any advice?
Post by: Nominal Animal on July 15, 2023, 04:10:12 pm
Most authors can't write concise sentences.
You have no idea how difficult it is to be precise, concise, and easily understood at the same time.  :'(

If you like Python 3, then extending and combining existing Qt UI widgets is one sure way to explore many OO concepts in practice.

The signal and slot mechanism should probably be understood first, though; it can throw people off.  It is Qt's way of implementing inter-object communication, or abstracting events and event handlers, so that the connections between the two can be controlled at run time.
Each object has a set of signals it can emit, for example clicked or textChanged.  Each signal (from a specific object) can be connect()ed to one or more slots; each slot being just a Python callable (method, function, etc.).  When the object emits (due to user interaction; or your code emit()s) a signal, the Qt runtime (verifies and) remembers the parameters.  Later on, when that code completes, and execution is returned back to Qt runtime, it causes each of the connected slots/callables to be called in turn, passing the information from one Qt object to another.

Just never, ever use the Designer to generate Python code for your UI designs.  Instead, save your UIs as XML .ui files, and load them at runtime.
The generated Python UI code is horrible to maintain, whereas properly written .ui-loading code is nice.  Also, if you use a specific pattern for naming your slots (event handlers), it is trivial to auto-connect them to the .ui elements' signals.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: tggzzz on July 15, 2023, 08:40:27 pm
Simple points about OOP which are often glossed over...

Inheritance: is-a relationship.
Composition: has-a relationship.

Inheritance is two things. Externally it is extending the interface that defines what a component can do; the subclass is more specialised than the superclass. Internally it is a code sharing mechanism.

Inheritance can be often overused at the application level; usually composition is preferable. Example: a Money class could be considered as a specialisation of a FloatingPoint class, but it is preferable that instances of Money are composed of a FloatingPoint object. That prevents you from even considering that you could take the square root of money:)

Having said that, inheritance is invaluable when your client says "that's great, but in addition I also need other things just like that but which also does X"

EDIT: understand the Liskov Substitution Principle. Absolutely key concept.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: cfbsoftware on July 15, 2023, 11:16:12 pm
Searched all morning browsing books that teach programing with objects.  Browsed 5-10 top titles that come up from a Google search.  Very disappointed.  Talking about the form, can't judge the content myself.
I recommend Object-Oriented Programming in Oberon-2 by Hanspeter Mössenböck, published by Springer-Verlag.

Quote
The goal of this book is to convey the fundamentals of OOP, namely classes, inheritance and dynamic binding. The emphasis is on the concepts rather than on the specifics of a particular programming language. In addition, readers should learn to determine for which problems OOP is most suitable, and which problems would be better solved with conventional means.

This book is out of print and is made available as PDF with the friendly permission of Springer‐Verlag:

https://ssw.jku.at/Research/Books/Oberon2.pdf
Title: Re: Familiar with programming, not prolific, any advice?
Post by: nctnico on July 16, 2023, 12:41:10 am
Did an inventory of my unfinished tinkering projects last weekend, and most are stalled without firmware/software.

The goal is to write fast enough and good enough code, so to get functional projects, not to become a programmer able to work in a team.  Environment is Linux and FOSS tools.

I've wrote small programs occasionally, starting decades ago.  Fortran was probably the first I read a book about in the high school, but didn't practice, later practiced some Basic on home computers, then at university we did some Turbo Pascal, then read K&R which later applied to programming microcontrollers, with some other languages and algorithm books/classes in between (i.e. the Knuth's 'Algorithms' books, about half of the 4-5 volumes).  Latest language I've dealt with is Python.  I know the basics of OOP, but not able to take advantage of it.  I'm stuck with the procedural writing style.  Never tried functional programming or other fancier stuff.

My guess is I might have to unlearn the procedural programming style and move to OOP.  Or maybe I'm really bad at programming.  I don't know what I'm doing wrong, but I know my projects are eventually abandoned because the software part became overwhelming.

What to do to become more productive in writing code (for small/hobby projects)?  Some programming exercises/books/classes to follow?  Something else to learn?  :-//

Most of the hobby projects require to write some MCU firmware, usually in C, and sometimes a PC interface, usually in Python.  Any directions of how to write that faster is welcome.
Use an MCU that supports Micropython and don't care about the costs especially if you want to get a project finished quickly. Spare time is valuable! I have been venturing into implementing the high level parts of MCU firmware using scripted languages like Lua and Micropython. I'm currently working on a project for a customer that uses Micropython to implement the logic. It is a whole lot less tedious compared to using C.
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 16, 2023, 05:32:18 pm
Most authors can't write concise sentences.
You have no idea how difficult it is to be precise, concise, and easily understood at the same time.  :'(

To be honest, I too find this attitude very annoying, and it bothers me that we are here to give free advice just as some people give free advice, and he doesn't even put "thanks" to posts because he wants "books" or "simple answers" written in precise, concise, and easily understood at the same time sentences and what else? also a piece of cake? of which, worse still, then he allows himself to read what is convenient for him, ignoring the same advice he is asking for, and, worse still, by labeling what he does not understand as "cult".
Title: Re: Familiar with programming, not prolific, any advice?
Post by: DiTBho on July 16, 2023, 05:38:43 pm
I have been venturing into implementing the high level parts of MCU firmware using scripted languages like Lua and Micropython. I'm currently working on a project for a customer that uses Micropython to implement the logic. It is a whole lot less tedious compared to using C.

When working near Playstation2 game developers, I noticed this approach but with "eLUA" instead of "LUA"

"e" = embedded.

They said that you save a lot of times, and it was true by taking a look at the project work tables.
(I was there as temporary project manager, even if got hired to support MIPS-specific things)
Title: Re: Familiar with programming, not prolific, any advice?
Post by: nctnico on July 16, 2023, 09:04:11 pm
eLUA and LUA are pretty much the same because LUA by itself is pretty lightweight already. There are actually several forks of LUA for embedded (microcontroller) use so eLua as-is doesn't really exist. I'm working on-and-off on a LUA fork myself that is easy to use with a round-robin (OS less) scheduler. I still need to improve ways to do single stepping through the Lua script.