Author Topic: Hitting a wall while writing software  (Read 5653 times)

0 Members and 1 Guest are viewing this topic.

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Hitting a wall while writing software
« on: February 26, 2021, 02:16:38 pm »
It happens to me all the time when building something new.  The hardware is nice and fun to make, then writing the software turns the project into a chord.  Software is many times more time consuming, never finished, and tend to grow exponentially in complexity until it all turns into a mess, needs at least a refactoring if not rewriting everything, and so on.

Not sure how to deal with that.  I don't want to rewrite 2-3 times every program, end even when I rewrite something, I still hit a wall (at about a few hundreds lines) where it all became a huge mess and lose the control.  The language doesn't seem to be making much difference.  I'm not a software developer, but I had to write here and there, over many decades, all kind of software in all kinds of languages (mostly for personal use) i.e. BASIC, Z80 assembler, C, Pascal, VB, VBA, SQL, Python, etc.

Also familiar with techniques like versioning, automated testing, TDD, continuous integration, agile, etc. but I'm not really using any of that at home.  For example, I use git for home projects so I can easily share through github, but never roll back to previous versions, don't use branches, and when I try something new I rather make a manual "save as" before modifying.

I suspect my lack is in software design, maybe in OOP (I still use procedural programming style) or maybe it is because I was always "playing by ear".

TL;DR
What to learn, or what habit to change so my project will not end as abandoned because of unfinished software?
 :-//



LATER EDIT:  summarize all answers before
2021-03-05 16:34, Fri
=====================
https://www.eevblog.com/forum/general-computing/hitting-a-wall-while-writing-software/msg3495656/#msg3495656
« Last Edit: March 05, 2021, 02:36:36 pm by RoGeorge »
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 4125
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: Hitting a wall while writing software
« Reply #1 on: February 26, 2021, 02:26:55 pm »
Quote
Software is many times more time consuming, never finished, and tend to grow exponentially in complexity until it all turns into a mess, needs at least a refactoring if not rewriting everything, and so on.
You're going too fast  ;) Slow down and design first. Try some TDD, it's very easy in Visual Studio, you can choose a new project with google test and start testing C/C++, very useful for embedded.
At least write tests for the little functions, eg: maths, base64encode. So you don't have to chase bugs in them on target, because that is unbelievable slow. And if you do think you found one, you create more tests to cover that specific scenario.

Begin here:
https://refactoring.guru/design-patterns/book
https://docs.microsoft.com/en-us/visualstudio/test/how-to-use-google-test-for-cpp?view=vs-2019
or some books/keynotes of Uncle Bob, they are an easy read/watch.
« Last Edit: February 26, 2021, 02:29:31 pm by Jeroen3 »
 

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2906
  • Country: gb
Re: Hitting a wall while writing software
« Reply #2 on: February 26, 2021, 02:59:04 pm »
Worth knowing how big a typical project of yours is, what hardware it targets and what sort of functionality you generally need to build in.

A half decent programmer can certainly knock together a few hundred lines of code which is reasonably clear and maintainable whether it is in assembler or C. C++ might be overkill for small platforms (the standard runtime libraries being rather large, but you can avoid using them if you need to do that) and I (no longer) like it much because it's too easy to produce "write only" code.

I don't think I'd recommend Pascal, BASIC or VB for an embedded project, but assembler, C or C++ would all be good choices depending on the exact functionality needed.
 

Offline tszaboo

  • Super Contributor
  • ***
  • Posts: 7854
  • Country: nl
  • Current job: ATEX product design
Re: Hitting a wall while writing software
« Reply #3 on: February 26, 2021, 03:08:24 pm »
You are not doing anything wrong. Writing software IS boring, and most companies will hire specially bread people to do it. It only means that you don't find it interesting, just like me. I mean, yes, sometimes, to an extent. Usually, as you said, few hundred lines. Or few days.
You can achieve a lot more in that time by using higher level language. Micropython was more productive for me than C or C++.
Just move on, accept that you are a hardware guy. Nothing wrong with that. And then realize the power of soldering iron over the JTAG programmer.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 8089
  • Country: de
  • A qualified hobbyist ;)
Re: Hitting a wall while writing software
« Reply #4 on: February 26, 2021, 03:26:50 pm »
Note down the essential features and start small. Try to avoid feature creep while writing code. And don't force things.
 

Offline golden_labels

  • Super Contributor
  • ***
  • Posts: 1323
  • Country: pl
Re: Hitting a wall while writing software
« Reply #5 on: February 26, 2021, 04:19:06 pm »
Perfectionism will kill you in programming. I tell you that as both a perfectionist and a former software developer. :)

Unfortunately, one has to learn to stop worrying and love the bomb. That shouldn’t be treated as a permission to be lazy, to lack in knowledge or to avoid acquiring more exerience. But you have to understand that a perfect program can not be written and some sane limits of effort must be set.

A few tricks:
  • Take your time understanding, what you have to write before even running your favourite code editor. Use paper+pen a lot, make some drafts on paper, spot major problems there.
  • Write the primary logic first in a manner that it produces the desired effect for valid data. It may be sketchy, it may be imperfect, it may be ugly, it may be inefficient, but it must do what it is exected to do.
  • If you see any obvious issues, that can’t be fixed within 2 minutes, mark them as bugs in comments and fix immediately after the sketchy version is in somewhat working. Not before, but also do not leave that for unspecified “later”.
  • If you spot any potential problems that are unlikely, add them to some bugs list, describe them and leave them for being fixed later.

And now your main problem becomes not that you can’t write a program, but the ever-growing backlog of the “fix-later” issues ;). It is useful, though. First, the number of issues WONTFIX’d with no good reason is a measure of your laziness. Second: it tells you how many feature requests you may accept from other people (or yourself). Of course ideas are always welcome and you should always write them down, but often you can’t use them all. Third, probably most important, the number of issues WONTFIX’d with the reason “I prefer to spend some time with friend/family” or similar is a measure of your sane approach to developing programs. Really: set time limits.
People imagine AI as T1000. What we got so far is glorified T9.
 

Offline capt bullshot

  • Super Contributor
  • ***
  • Posts: 3033
  • Country: de
    • Mostly useless stuff, but nice to have: wunderkis.de
Re: Hitting a wall while writing software
« Reply #6 on: February 26, 2021, 04:36:28 pm »
I know pretty well what you describe.
For one particular project, it helped a lot I drew this diagram: http://wunderkis.de/pvbat/sm.pdf before I started to write code.
So the state machine worked nearly out of the box, still were enough nasty interesting and challenging details to take care of (esp. HW interaction at certain state transitions).

« Last Edit: February 26, 2021, 04:40:26 pm by capt bullshot »
Safety devices hinder evolution
 

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #7 on: February 26, 2021, 04:49:17 pm »
Worth knowing how big a typical project of yours is, what hardware it targets and what sort of functionality you generally need to build in.

Good point, without an example it will all be just talking words and generic advice.

It just happens that I was cleaning around, and stumbled upon a box with yet another unfinished project, a discharger for AA batteries.  It is a simple project, cobbled a minimal hardware in a couple of hours, with lots of corner cutting because "it should be possible to do that in software".

The goal was to sort out the health of AA rechargeable batteries by discharging them with a constant current:
- main function - measure the remaining capacity of the battery, C (mAh), stop when the voltage become too low
- must have - 4 independent discharging channels, and plot of voltage over time on the PC
- nice to have - measure internal resistance by changing the discharge current while observing the battery voltage variations
- nice to have - autodetect when another battery is plugged, and restart a new measuring process

The hardware was just a MOSFET with a shunt resistor for each battery, and a microcontroller to read each battery current and each battery voltage (ATmega48 from an Arduino nano).  Each MOSFET is controlled by a linear voltage (not by PWM) in order to set the desired current.  The constant current is stabilized by a PID loop for each battery.  The analogue voltage for MOSFETs is obtain by improvising a DAC from a PDM (Pulse Density Modulation) followed by an RC filter to average the pulses into an analogue voltage that will control the MOSFET gate.

Found a pic of it, this is channel 2 only, an AA battery socket with a MOSFET, an RC filter and a shunt resistor.  The rest is just wires connected to Arduino pins.  Will search for a schematic drawn, too.



I don't recall the exact state of the software.  The interrupts driven PDM DAC was working, oversampling ADC, too, digital PID kind of working but with some winding problems IIRC.  Battery change detection was a mess, and no serial commands were yet implemented (to control the whole thing for PC).  No PC side program for voltage plot or mAh calculation either.

Will search for the sources, and attach them as an example of unfinished software.
« Last Edit: February 26, 2021, 04:53:48 pm by RoGeorge »
 

Offline Caliaxy

  • Frequent Contributor
  • **
  • Posts: 301
  • Country: us
Re: Hitting a wall while writing software
« Reply #8 on: February 26, 2021, 05:09:47 pm »
“Hardware eventually fails. Software eventually works.”   ;)
 
The following users thanked this post: james_s

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #9 on: February 26, 2021, 06:04:18 pm »
Found the schematic and the software:



BattDisch.ino (main)
Code: [Select]
//------------------------------------------------------------------------------
// Hardware related definitions
//------------------------------------------------------------------------------
#define N_CH  4  // Number of PDM channels (Pulse Density Modulation)

#define PDOUT_MASK  B01100000 // Bits PD6 and PD5 are PDM out for CH4 and CH3
#define PBOUT_MASK  B00000110 // Bits PB2 and PB1 are PDM out for CH2 and CH1



//------------------------------------------------------------------------------
// Delta-Sigma related definitions
//------------------------------------------------------------------------------
#define MAX_ALL 2048         // PDM resolution

#define MAX_CH_1    MAX_ALL  // Distinct possible steps between 0-100%
#define MAX_CH_2    MAX_ALL  // Distinct possible steps between 0-100%
#define MAX_CH_3    MAX_ALL  // Distinct possible steps between 0-100%
#define MAX_CH_4    MAX_ALL  // Distinct possible steps between 0-100%



//------------------------------------------------------------------------------
// Global variables used for each software channel of a Delta-Sigma modulator
//------------------------------------------------------------------------------
volatile uint16_t  max[N_CH];    // Maxim level (resolution) for each channel
volatile uint16_t  req[N_CH];    // Requested levels, 0 <= req <= max
volatile uint16_t  sum[N_CH];    // Integrators value, 0 <= sum < 2*sum

volatile uint8_t   outBits;      // Each bit store the output value of one modulator



//------------------------------------------------------------------------------
// Global variables used for each battery channel
//------------------------------------------------------------------------------
volatile uint16_t battRead[N_CH];    // Store oversampled V_batt ADC value
volatile uint16_t sensRead[N_CH];    // Store oversampled I_batt ADC value

volatile uint16_t min[N_CH];           // Undervoltage level
volatile int8_t protStat[N_CH];     // Undervoltage protection status



//------------------------------------------------------------------------------
// Protection values
//------------------------------------------------------------------------------
#define Imin  3     // 1024 is for 1.1V on 0.15ohms,  1 unit ~ 7 mA
#define Vmin  100   // 1024 is for 5.0V,  1 unit ~ 5 mV



//------------------------------------------------------------------------------
// Desired discharging current
//------------------------------------------------------------------------------
volatile uint16_t dischI[N_CH];



//------------------------------------------------------------------------------
// Generic loops index reused variables
//------------------------------------------------------------------------------
int8_t i, n;



//------------------------------------------------------------------------------
// Function prototypes
//------------------------------------------------------------------------------
void init_all_CH_arrays();
void init_pins_dir();
void init_interrupts();

void init_prot();
void init_control();

void read_all_ADC();
void check_prot();
void control_I();



void init_all_CH_arrays() {
//------------------------------------------------------------------------------
// Initialize all 10 channels arrays with different periods and initial values
//------------------------------------------------------------------------------
  max[0] = MAX_CH_1;  // Set maxim value (resolution) for each PDM channel
  max[1] = MAX_CH_2;
  max[2] = MAX_CH_3;
  max[3] = MAX_CH_4;
 

  //  #define REQUEST MAX_ALL
  //#define REQUEST   3 * MAX_ALL / 4   // Initial PDM value after a HW reset
  #define REQUEST   1500   // Initial PDM value after a HW reset

  req[0] = REQUEST;      // Initial channels start values
  req[1] = REQUEST;
  req[2] = REQUEST;
  req[3] = REQUEST;

  // req[0] = MAX_ALL;      // Initial channels start values
  // req[1] = MAX_ALL;
  // req[2] = MAX_ALL;
  // req[3] = MAX_ALL;
 

  #define DISCH 70        // desired discharge current, 70 for aprox 0.5A

  dischI[0] = DISCH;
  dischI[1] = DISCH;
  dischI[2] = DISCH;
  dischI[3] = DISCH;
}



void init_pins_dir() {
//------------------------------------------------------------------------------
// Initialize pins direction
//------------------------------------------------------------------------------
  DDRB |= PBOUT_MASK;  // set PB2, PB1 as outputs
  DDRD |= PDOUT_MASK;  // set PD6, PD5 as outputs

  DDRC = 0;  // all port C as inputs for ADC
  PORTC = 0; // no initial pullups for ADC inputs
}



void setup() {
  // put your setup code here, to run once:

  // initialize serial communications at 9600 bps:
  Serial.begin(9600);
 
  init_all_CH_arrays();
  init_pins_dir();
  init_interrupts();

  init_prot();
  init_control();

  Serial.println("Initialized");

}

void loop() {
  // put your main code here, to run repeatedly:

  read_all_ADC();
  delay(10); // wait at least 5ms for the Vref to have time \
                to stabilize to 1.1V for the next measurement

  //check_prot(); // undervoltage protection

  control_I();
  delay(100);   // wait for Vgs to stabilize after PDM adjust



  //print only if a second has passed
  unsigned long time_s = 0;
  unsigned static long prev_s;

  time_s = millis() / 976;
  //time_s = millis() / (976 + 976);
  if (time_s == 0) prev_s = 0;

  // Serial.print(time_s);
  // Serial.print(" ");
  // Serial.println(prev_s);
 
  if (time_s > prev_s) {
    if (true) {
      send_to_serial_battRead();
      send_to_serial_sensRead();
      send_to_serial_req();
      send_to_serial_proStat();
      send_to_serial_dischI();
      Serial.println();
    }

    prev_s = time_s;
  }



  //convert_to_human_units();

  //Serial.println("loop ended");
}




ADCreadings.ino
Code: [Select]
void read_all_ADC() {
//------------------------------------------------------------------------------
// Read 2**OVERSAMPLINGs of ADC for each channel's V_batt and I_batt
//------------------------------------------------------------------------------

// !!! Switching Vref from 1.1V to 5V takes only 10us (Vcc has low impedance)
// !!! Switching Vref from 5V to 1.1V takes about 5ms because of the C on Aref
//     For default OVERSAMPLE 3 and DISCARDED 8, read_all_adc() takes ~10ms
//     For default OVERSAMPLE 5 and DISCARDED 8, read_all_adc() takes ~33ms

#define OVERSAMPLING    1   // must be 0...6, will measure 2^^OVERSAMPLING times
#define DISCARDED       4   // number of discarded conversions after Vref switch



    // set the ADC range to 1.1V and read all channels for I_batt
    analogReference(INTERNAL);              // set ADC Vref = 1.1V
    //delay(10);

    for (i = 1; i <= DISCARDED; ++i)           
        sensRead[0] += analogRead(N_CH);    // discarded batch of readings after changing the Vref
   
    for (n = N_CH - 1; n >= 0; n--) {

        sensRead[n] = 0;
        for (i = 1; i <= (1 << OVERSAMPLING); ++i)   {
            sensRead[n] += analogRead(N_CH + n);
        }

        sensRead[n] >>= OVERSAMPLING;   
    }



    // set the ADC range to 5V and read all channels for V_batt
    analogReference(DEFAULT);               // set ADC Vref = 5V
    //delay(10);

    for (i = 1; i <= DISCARDED; ++i)           
        battRead[0] += analogRead(0);       // discarded batch of readings after changing the Vref

    for (n = N_CH - 1; n >= 0; n--) {

        battRead[n] = 0;
        for (i = 1; i <= (1 << OVERSAMPLING); ++i) {
            battRead[n] += analogRead(n);
        }

        battRead[n] >>= OVERSAMPLING;
    }

    analogReference(INTERNAL);              // to have it stabilized on 1.1V next time
    analogRead(0);                          // dummy read to switch Vref to INTERNAL
}




Icontrol.ino
Code: [Select]
//------------------------------------------------------------------------------
// Stabilize the discharging current
//------------------------------------------------------------------------------
volatile static uint16_t start[N_CH];
volatile static uint16_t stop[N_CH];

void control_I() {
    for (int8_t n = 0; n < N_CH; n++) {
        if (dischI[n] > 0)          // if channel on (I desired > 0)
            control(n);
        else {
            noInterrupts();
                req[n] = 0;             // power off the channel
            interrupts();
            init_control_n(n);      // init control for future random value of I desired > 0
        }
    }
}



void init_control() {
    for (int8_t n; n < N_CH; n++) {
        init_control_n(n);
    }
}



void init_control_n(int8_t n) {
    start[n] = 0;
    stop[n] = max[n];
}



void control(int8_t n) {
    if ((stop[n] - start[n]) < 2) {         // if segment very small, enter in follower mode
        if (sensRead[n] < dischI[n]) {         // I DS too small, increment
            if (req[n] < max[n]) {
                noInterrupts();
                    req[n]++;
                interrupts();
            }
        };
        if (sensRead[n] > dischI[n]) {         // I DS too big, decrement
            if (req[n] > 1) {
                noInterrupts();
                    req[n]--;
                interrupts();
            }
        };
        start[n] = req[n];                      // follow
        stop[n] = req[n];
    }
    else {                                  // in search mode, half the searching segment
        if (sensRead[n] < dischI[n]) {         // I DS too small, pick the bigger half
            start[n] = req[n];
        }
        else if (sensRead[n] > dischI[n]) {                            // I DS too big, pick the smaller half
            stop[n] = req[n];
        }
        else {                                  // right on spot, switch to follower mode
            start[n] = req[n];
            stop[n] = req[n];
        }
        noInterrupts();
            req[n] = (start[n] + stop[n]) >> 1;
        interrupts();
    }
}




PDMinterupt.ino
Code: [Select]
void init_interrupts() {
//------------------------------------------------------------------------------
// Set Timer2 to generate interrupts at each 20us (TCCR2B = 2, OCR2A = 39)
//------------------------------------------------------------------------------
  TCCR2A = (1 << WGM21);    // CTC mode
  TCCR2B = (2 << CS20);     // 1..7 for 16MHz div 1, 8, 32, 64, 128, 256 or 1024
  OCR2A = 39;               // 0..255

  TIMSK2 |= (1 << OCIE2A);  // Set interrupt mask bit
}



ISR(TIMER2_COMPA_vect) {
//------------------------------------------------------------------------------
// Timer2 CTC interrupt rutine
//------------------------------------------------------------------------------
  // output last calculated
  PORTB = outBits & PBOUT_MASK;  // update value PB bits 1, 2
  PORTD = outBits & PDOUT_MASK;  // update value PD bits 5, 6
 

  // calculate the outputs status to be written at next interrupt
  //    mapping outBits bits order to output pins/channels
  //    bit 0 -> CH1 of the discharger (PB1)
  //    bit 1 -> CH2 of the discharger (PB2)
  //    bit 2 -> CH3 of the discharger (PD5)
  //    bit 3 -> CH4 of the discharger (PD6)

  // Sigma delta modulation algorithm using "synthetic division"
  outBits = 0;                  // Initialize all outBits for recalculation
  sum[3] += req[3];               // Update integrator value
  if (sum[3] < max[3])
    outBits++;                    // LSB = 1
  else
    sum[3] -= max[3];             // LSB = 0 (untouched) and adjust integrator

   
  // Sigma delta modulation algorithm using "synthetic division"
  outBits <<= 1;                  // Shift other bits and reset current LSB
  sum[2] += req[2];               // Update integrator value
  if (sum[2] < max[2])
    outBits++;                    // LSB = 1
  else
    sum[2] -= max[2];             // LSB = 0 (untouched) and adjust integrator


  // Sigma delta modulation algorithm using "synthetic division"
  outBits <<= 3;                  // Shift other bits and reset current LSB
  sum[1] += req[1];               // Update integrator value
  if (sum[1] < max[1])
    outBits++;                    // LSB = 1
  else
    sum[1] -= max[1];             // LSB = 0 (untouched) and adjust integrator

     
  // Sigma delta modulation algorithm using "synthetic division"
  outBits <<= 1;                  // Shift other bits and reset current LSB
  sum[0] += req[0];               // Update integrator value
  if (sum[0] < max[0])
    outBits++;                    // LSB = 1
  else
    sum[0] -= max[0];             // LSB = 0 (untouched) and adjust integrator


  // Align then invert the 4 calculated rezult bits
  outBits <<= 1;
  outBits ^= PDOUT_MASK | PBOUT_MASK;
}




printOut.ino
Code: [Select]
void send_to_serial_req() {
    for (n = N_CH - 1; n >= 0; --n) {
        Serial.print("req[");
        Serial.print(n);
        Serial.print("]=");
        Serial.print(req[n]);
        Serial.print(" ");
    }
    Serial.println();
}



void send_to_serial_battRead() {
        for (n = N_CH - 1; n >= 0; --n) {
        Serial.print("battRead[");
        Serial.print(n);
        Serial.print("]=");
        Serial.print(battRead[n]);
        Serial.print(" ");
    }
    Serial.println();
}



void send_to_serial_sensRead() {
        for (n = N_CH - 1; n >= 0; --n) {
        Serial.print("sensRead[");
        Serial.print(n);
        Serial.print("]=");
        Serial.print(sensRead[n]);
        Serial.print(" ");
    }
    Serial.println();
}



void send_to_serial_proStat() {
        for (n = N_CH - 1; n >= 0; --n) {
        Serial.print("protStat[");
        Serial.print(n);
        Serial.print("]=");
        Serial.print(protStat[n]);
        Serial.print(" ");
    }
    Serial.println();
}

   
   
void send_to_serial_dischI() {
        for (n = N_CH - 1; n >= 0; --n) {
        Serial.print("dischI[");
        Serial.print(n);
        Serial.print("]=");
        Serial.print(dischI[n]);
        Serial.print(" ");
    }
    Serial.println();
}




protection.ino
Code: [Select]
//void overdischarge_protection() {
//------------------------------------------------------------------------------
// Protection to trigger when V < Vmin or I < Imin, rearms when battery removed
// TO DO - reimplement it as a state machuine, this is just a D R A F T
//       - use PORTC = 255 (enable pullups for analog inputs)
//          to determine afsent battery (battRead wil be ~ 1020, sensRead ~ 70)
//------------------------------------------------------------------------------
//     for (n = N_CH - 1; n >= 0; --n) {
//         // if V <= Vprot thresh for long enough, disable channel
//         // if I is > 0 and V > Vprot for long enough, enable channel

//         if ((protStat[n] > 0) & (protStat[n] < 255))
//             protStat[n] += 1;
       
//         if (((battRead[n] <= Vmin) | (sensRead[n] < Imin)) & (protStat[n] > 0))
//             protStat[n] -= 1;

//         if ((protStat[n] == 0) & (battRead[n] < 10))
//             protStat[n] = 127;

//         if (protStat[n] == 0)
//             req[n] = 0;         // disable channel when protection has triggered
//     }
// }



#define INITPROTCOUNTER 10
#define LOWESTVOLTAGE   150 // 150 for minimum Vbatt = 0.73V



void arm_prot_ch(int8_t n) {
    protStat[n] = INITPROTCOUNTER;
    dischI[n] = DISCH;
}



void init_prot() {
    for (int8_t n = 0; n < N_CH; n++) {
        min[n] = LOWESTVOLTAGE;
        arm_prot_ch[n];
    }
}



void check_prot() {
    for (int8_t n = 0; n < N_CH; n++) {
        if ((req[n] > 0) && (dischI[n] > 0)) {                   // if channel is ON
            if (battRead[n] < min[n]) {
                protStat[n]--;
                if (protStat[n] < 0) protStat[n] = 0;
            }
            else {
                arm_prot_ch(n);
            }
        }
        if (protStat[n] == 0) {
            // req[n] = 0;                     // Undervoltage, turn channel OFF

            Serial.print("Channel ");
            Serial.print(n + 1);
            Serial.print(" turned OFF for battery voltage lower than ");
            Serial.println(min[n]);

            req[n] = 0;                     // Undervoltage, turn channel OFF
            dischI[n] = 0;


        }

        if (battRead[n] == 0) {
            arm_prot_ch(n);
        }       
    }
}

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5405
  • Country: us
Re: Hitting a wall while writing software
« Reply #10 on: February 26, 2021, 06:51:52 pm »
While this falls in the area of general words, I think one of your problems is a "software is simple" mindset.  My history is very similar to yours so my thoughts may provide you some insight.

1.  You have a very simple hardware implementation.  All of the hard parts have been dumped into software.  Think of how difficult your feature set would be if all of the functions on your list had to be done in hardware.  That should provide some reference for how hard the software is.  In this case I think software is easy is a correct assessment in the sense that once you figure out the correct physics and electronics required to detect the status and operation you are asking it is easier to implement in software than in hardware, but there is a lot of work involved.  You may be assigning this learning and design work to software while it would be there however you implemented the project.  If you had done all that work prior to the project start it probably would have made the actual software seem much simpler.

2.  Based on that thought there are some opportunities for software/hardware trades.  For example adding a pushbutton at each station for the operator to signal a new battery insertion would make the software side trivial (sort of, see next comment).

3.  You may want to revise some of your software architecture.  Interrupts are an incredibly useful tool.  But do introduce some problems in software.  I see no particular benefit to using them in your basic design.  A polling structure would meet all of the needs of a slowly changing system such as charging/discharging.  Generating nice time stamps with known intervals can even be handled using the Arduino provided ecosystem, or if you want to use the bare processor just by calibrating your loop speed.  But here is the feedback to comment 2.  An interrupt isn't a bad way to deal with a pushbutton.  But another trade presents itself - software or hardware debounce.  With an appropriate hardware debounce switch detection will also work in a polling environment.

4.  All of this comes down to doing a lot of thinking before you start, breaking the total project down to hardware and software chunks and making sure that each chunk is within your capabilities, or at least within what you will be willing to learn.  This last point may be the core of the problem.  I have a large project that is still not done after more than ten years of off and on progress.  One of the "hold ups" was that the project controls some potentially dangerous machinery and needs an emergency stop button that will shut everything down.  The software is done in VB and several subpanels are used, and this stop needs to be effective no matter where you are in the software.  It requires ability to write multi-threaded code in VB, which I can't, and haven't been able to learn.  I have implemented the emergency shutdown in hardware and foregone the software version.  I would still like to have the software version as a secondary path, but it just isn't happening. 

 

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 22274
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Hitting a wall while writing software
« Reply #11 on: February 26, 2021, 07:26:15 pm »
Agree, diagramming helps.  Sketch out the high level function, as tasks/events in a loop, or a flowchart, etc.  This can be implemented as a massive if-else in the main loop, or jumps between functions (make sure they don't loop forever of course...), or a more formal state machine structure (which in turn can be a big switch statement, or say, precalculated elsewhere using a state transition table).

Maybe you need to do something special-case, interrupting overall flow of the loop, and it's just not feasible to fold it into the loop normally -- that's fine, you can jump into another lengthy function and do your work there.  Don't do this lightly: you don't have access to all the clauses and functionality of the main loop, outside of it; more than a few special cases like this, will quickly lead to spaghetti.  See #1, diagram it out, try to integrate it better, maybe there's a pattern you're missing that would allow both operating modes to coexist, etc.

May also help to implement simple yet meaningful functions, and run them from a sort of command shell.  I've been doing this the last little while, and find it useful.  At least, because I don't have a debugger for my favored device family...  Example, figuring out a peripheral: well, start with the basics, write a command to PEEK/POKE its control registers.  This can be as basic as the MCU's own registers; which if they're memory mapped, simultaneously gives you insight into internal RAM, handy for inspecting the live program -- albeit from the shell's main loop, not at just any point along any particular function.  Maybe it's an SPI device, so you boot up the MCU's SPI peripheral, and implement a short data/command R/W function.  Maybe the next level up is an init command, or something that runs a sequence of values or commands or whatever.  Then commands to use specific internal functions, like uh, for an LCD display you'd certainly want some GRAM address and data commands so you can start drawing pixels.  Then maybe basic shapes, images, etc.

So you can start at a very low level, implementing bits and pieces, which don't go anywhere else in your program -- but you can still test them in this way (and yes, preferably with more detailed/formalized tests, as in TDD), meanwhile you can diagram the very highest of levels, and mark out what the overall program state goes through, what the highest level functions will be, etc.

And then finally, you can add in optimizations or customizations or whatever.  Maybe it's rather slow, so you need to reconsider some algorithms (accidentally quadratic?), or implement them in ASM.  Maybe you've been using it a while, and yeah it's usable, but it's still a bit of a pain and wouldn't it be great if it did X automatically?

Reminds me, I still need to add a filter calculator to my reverb effects box... It's usable, yes, but the filter coefficients are literally just right there on the menu, and good luck tweaking them just by eye!  That should be interesting by itself actually, I might do it in floating point (needs several trig functions; library is probably bloated and slow), or I might look into shaving a fixed-point-math-shaped yak... ;D


Oh, anyway, the source to that project is here if you're curious,
https://github.com/T3sl4co1l/Reverb
console.c and commands.c may be of interest (the command shell, and the commands which plug into it), or main.c and menu.c for the UI loop.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #12 on: February 26, 2021, 08:36:55 pm »
Try some TDD
TBH, I've tried TDD once after reading a book about it, and didn't like it.  Might worth using TDD for big projects and big teams of software developers, IDK, but does it worth for small single person projects?

I found TDD to be very impractical, if not impossible to follow.  Especially for embedded or some other software that controls real world machinery/hardware, I don't even know how to approach automatic testing when physical devices are involved, and even if it were to do that, it will probably take me 100 times more time writing the test cases then writing+debugging the program.

Is TDD still beneficial for small programs, like the example posted above?

https://refactoring.guru/design-patterns/book
Looks interesting, thank you.




Just move on, accept that you are a hardware guy. Nothing wrong with that.

Oh well, I move on, but then I found myself dedusting junk finished 2 years ago, yet still not usable because the software is not ready, like this battery discharger here (and most if not all of my other projects  ;D ).




Perfectionism will kill you in programming.
...
Really: set time limits.
I'll write down especially these two, to remember and apply in the future, thank you.

About the time limits, what would be a reasonable time limit for the goals in that discharger project?




drew this diagram: http://wunderkis.de/pvbat/sm.pdf before
...
state machine
Indeed, at some point I felt I should have drawn a state machine first for that discharger project (and for most of the other projects, too).  Usually I don't lay down a state machine before proceeding, and almost always end up with unwanted behavior of a device, or hard to spot bugs.  Will write down this one, too, thank you.

What did you used to make the one from your PDF?




3.  You may want to revise some of your software architecture.  Interrupts are an incredibly useful tool.  But do introduce some problems in software.  I see no particular benefit to using them in your basic design.

Indeed, I need to revise the project, and next time better use an opamp to stabilize the discharging current rather than dealing with that in a digital PID.

As a side note, the interrupts are a must have here.  They are used to generate PDM (Pulse Density Modulation) instead of PWM, so the design would be scalable to as many channels as I/Os and ADCs are available.

Software PDM is cheaper to implement, and works at much higher frequency than it is possible with software PWM.  For example, the yellow PDM trace shows much denser pulses than the blue trace of PWM pulses.



This is beneficial, for example, to avoid flickering at very low duty factors, when compared to PWM.  I've compared once PDM vs. PWM, and wrote a software PDM implementation for a MSP430, where all its 10 existing I/O pins were used to dim LEDs:



https://hackaday.io/project/6356-delta-sigma-versus-pwm
« Last Edit: February 26, 2021, 08:53:00 pm by RoGeorge »
 

Offline rstofer

  • Super Contributor
  • ***
  • Posts: 9921
  • Country: us
Re: Hitting a wall while writing software
« Reply #13 on: February 26, 2021, 09:41:47 pm »
I ran into the idea back in the early '80s:  Top Down Design, Bottom Up Coding.

Draw pictures of the blocks starting at the very top level and describe their interaction.  Keep drilling down with blocks until you hit the hardware.  In my case, the first bit of code I write will be a UART handler and the next bit will be conversion routines for outputting debug values.  To get the UART to run, I may need to add code for the various clock enables as well as any pre-scalers.

In theory, I would write all the hardware code first and then build up to the final user interface.  But the top level interface would already be designed so I know where I am heading as I work up.

 
The following users thanked this post: T3sl4co1l

Offline tszaboo

  • Super Contributor
  • ***
  • Posts: 7854
  • Country: nl
  • Current job: ATEX product design
Re: Hitting a wall while writing software
« Reply #14 on: February 26, 2021, 11:18:15 pm »
Just move on, accept that you are a hardware guy. Nothing wrong with that.

Oh well, I move on, but then I found myself dedusting junk finished 2 years ago, yet still not usable because the software is not ready, like this battery discharger here (and most if not all of my other projects  ;D ).

So partner up with someone who doesnt do hardware and likes software.
Or make the hardware, open source it and then, if it is interesting someone will write the software for you.
Do projects, where the most work is in the hardware. Analog stuff, volt nut, time nut and so on. You want to discharge a battery? Replace the microcontoller with a ten turn potentiometer.

All that code that you wrote there. All you need is a current setpoint, a voltage setpoint, and one or two off the self current and voltage panel meter. You can make a PI controller from an opamp.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15148
  • Country: fr
Re: Hitting a wall while writing software
« Reply #15 on: February 26, 2021, 11:59:22 pm »
Well, the OP may not be very fond of writing software... but even if they were. My humble experience in industrial settings is that software ALWAYS takes longer than hardware to develop, except in very specific cases with very involved analog design. I've almost never seen the opposite. Except with the design of complex integrated circuits...

Now of course liking it will make the process less painful (except for those that are waiting for the software to be done!)
Good methodology will also help. But still, don't expect software to be done in less time than the hardware counterpart in a typical project when software is required.

 

Offline rstofer

  • Super Contributor
  • ***
  • Posts: 9921
  • Country: us
Re: Hitting a wall while writing software
« Reply #16 on: February 27, 2021, 12:51:03 am »
Software is hard.  Many years ago I worked for one of the mainframe manufacturers.  It was said that it cost us more money to ship the OS/compilers than it cost to ship the machine.  And we're talking big iron, not those Itty Bitty Machines.

Not only is software hard, it is expensive.
 

Offline KE5FX

  • Super Contributor
  • ***
  • Posts: 1985
  • Country: us
    • KE5FX.COM
Re: Hitting a wall while writing software
« Reply #17 on: February 27, 2021, 01:07:22 am »
It happens to me all the time when building something new.  The hardware is nice and fun to make, then writing the software turns the project into a chord.  Software is many times more time consuming, never finished, and tend to grow exponentially in complexity until it all turns into a mess, needs at least a refactoring if not rewriting everything, and so on.

Not sure how to deal with that.  I don't want to rewrite 2-3 times every program, end even when I rewrite something, I still hit a wall (at about a few hundreds lines) where it all became a huge mess and lose the control.  The language doesn't seem to be making much difference.  I'm not a software developer, but I had to write here and there, over many decades, all kind of software in all kinds of languages (mostly for personal use) i.e. BASIC, Z80 assembler, C, Pascal, VB, VBA, SQL, Python, etc.

Also familiar with techniques like versioning, automated testing, TDD, continuous integration, agile, etc. but I'm not really using any of that at home.  For example, I use git for home projects so I can easily share through github, but never roll back to previous versions, don't use branches, and when I try something new I rather make a manual "save as" before modifying.

I suspect my lack is in software design, maybe in OOP (I still use procedural programming style) or maybe it is because I was always "playing by ear".

TL;DR
What to learn, or what habit to change so my project will not end as abandoned because of unfinished software?
 :-//

After 10 years of experience, you will be able to maintain 10,000 lines of crufty C code piled up in one place, and adapt it successfully when new requirements emerge.

After 15 years of experience, 20000 lines.

After 20 years of experience, 30000 lines.

After 25 years of experience, you will be able to (barely) maintain 40000 lines of insanity in one C file.  That's pretty much where I am.  |O

The only good news is that your competitors are having just as hard a time as you are.  And the only good advice is John Carmack's: don't get attached to your code.  Be ready to throw everything out and rewrite it, multiple times, the moment it starts to seem like that might be a good idea. 

Of course, John Carmack isn't going to pay your mortgage in the meantime...
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 27656
  • Country: nl
    • NCT Developments
Re: Hitting a wall while writing software
« Reply #18 on: February 27, 2021, 01:59:36 am »
In general when writing software: start with a good description and diagrams / flow charts. I can spend weeks on just figuring out how a (complicated) piece of software needs to be structured and put it on paper. Even if it is a re-incarnation of an existing piece of software.

The only good news is that your competitors are having just as hard a time as you are.  And the only good advice is John Carmack's: don't get attached to your code.  Be ready to throw everything out and rewrite it, multiple times, the moment it starts to seem like that might be a good idea. 
He is right. Software is build upon a base structure -call it a chassis- and at some point that chassis can no longer support all the addons. But I have an addition to John's statement: let someone else do the re-write and work on something more interesting yourself! You'll see that other people find the uninteresting stuff very interesting.
« Last Edit: February 27, 2021, 02:03:59 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline capt bullshot

  • Super Contributor
  • ***
  • Posts: 3033
  • Country: de
    • Mostly useless stuff, but nice to have: wunderkis.de
Re: Hitting a wall while writing software
« Reply #19 on: February 27, 2021, 09:06:45 am »



drew this diagram: http://wunderkis.de/pvbat/sm.pdf before
...
state machine
Indeed, at some point I felt I should have drawn a state machine first for that discharger project (and for most of the other projects, too).  Usually I don't lay down a state machine before proceeding, and almost always end up with unwanted behavior of a device, or hard to spot bugs.  Will write down this one, too, thank you.

What did you used to make the one from your PDF?


So your question is a bit ambiguous to me, may one part of my answer is superflous:
I've got no special tools, neither to draw the state machine nor to convert it to code.
I've used inkscape to draw the diagram, and my brains and fingers to convert it to a big switch  () { case } statement in plain C, like this:

Code: [Select]
while (1) {
switch (spwm) {
case SPWM_OFF : do_pwm_off(); break;
case SPWM_START: do_pwm_start(); break;
case SPWM_INV: do_pwm_inv(); break;
case SPWM_INVON: do_pwm_invon(); break;
case SPWM_SOLSTART: do_pwm_solstart(); break;
case SPWM_SOLMPP: do_pwm_solmpp(); break;
case SPWM_SOLREG: do_pwm_solreg(); break;
case SPWM_SOLMPPC: do_pwm_solmppchg(); break;
case SPWM_SOLREGC: do_pwm_solregchg(); break;
case SPWM_SOLRION: do_pwm_solregion(); break;
case SPWM_SOLREGI: do_pwm_solreginv(); break;
case SPWM_SOLCION: do_pwm_solchgion(); break;
case SPWM_SOLREGCI: do_pwm_solregchginv(); break;
case SPWM_DISC: do_pwm_disc(); break;
case SPWM_DISCSOL: do_pwm_discsol(); break;
case SPWM_CHGSTART: do_pwm_chgstart(); break;
case SPWM_CHGMPP: do_pwm_chgmpp(); break;
case SPWM_BKUP: do_pwm_bkup(); break;
case SPWM_BKUPSOL: do_pwm_bkupsol(); break;
case SPWM_TB: do_pwm_tb(); break;
case SPWM_FAIL: do_pwm_fail(); break;
default: stoprq=1; break;
};
...


I've attached the source file containing this particular statement, just in case you're interested. There's still a lot of other stuff around this state machine.
« Last Edit: February 27, 2021, 09:14:30 am by capt bullshot »
Safety devices hinder evolution
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6762
  • Country: fi
    • My home page and email address
Re: Hitting a wall while writing software
« Reply #20 on: February 27, 2021, 11:39:16 am »
For diagrams, I recommend trying out Dia as well as Inkscape.  Both are available for Windows, macOS, and Linux at least.

For state diagrams and graphs where you already know (that is, can list as text) the relationships, I recommend trying out Graphviz.  It is a tool that takes in simple textual descriptions of graphs (using the DOT language), and creates the graph images based on that description.  One of the supported output formats is SVG, so you can further finesse the output in e.g. Inkscape – although I've found that it is faster for me to just redraw the graph in Dia, looking at the Graphviz-generated graph.

The Graphviz Gallery page on Bazel Build System shows a pretty good example of both the output and the DOT language input you need for this kind of graphs.

As to the topic at hand, I'm afraid I'm so oddball my suggestions would probably not work for others.  I'm addicted to problem-solving, and because many problems are better solved by solving a different, underlying problem that causes the problem at hand to vanish, not all solutions I create involve me writing code.  I treat the projects as tools to solve problems, deconstructing it into sub-problems I can attack in any order.  I'm most productive if I solve the tedious sub-problems first; like what kind of communications format would work best between a computer program and a microcontroller, by implementing both ends in a simulated real-world scenario.  I learned rewrites are my friend, and nowadays by default try to create code I can refactor (rewrite in parts) easily whenever I like, without dreading it: it is par for the course.  I do write more code than I end up using, but every part I write tells me something about the problems I'm solving so is fundamentally useful.  Code is never "finished", never "perfect"; it is only "usable" or fragile agglomerated crap that needs a rewrite or at least a refactoring.
 

Offline Syntax Error

  • Frequent Contributor
  • **
  • Posts: 584
  • Country: gb
Re: Hitting a wall while writing software
« Reply #21 on: February 27, 2021, 12:55:39 pm »
As a 'softie', all I can add is that I agree with @NominalAnimal. Software is never done. At some magic state/threshold a "stable release" or "release candidate" is achieved. Which is immediately followed with "minor upgrades", "major upgrades" or "service packs" to correct some "stability issue" or a "zero day security flaw".

So then you're forever tweaking the codebase before the hardware becomes obsolete. And then beyond that, when the code achieves "legacy support".

Imagine being able to upgrade a PCB by adding and subtracting parts and tracks? But then every product would be made of those white plastic breadboards. Such is the beauty of software.

Contrasting SW to HW, HW is about fix it, sell it. But far too many SW developers have grown up with the luxury culture of  sell it, fix it. HW guys have "development prototypes", SW guys have "recursive normalization", "bug fixes" and a state of "perpetual beta".

Code: [Select]
namespace RefactorFactory
while(true) { /* add more code here */ }

Tip: Just add lots of comments. It will make your life so much easier... in (20)five years time :)
« Last Edit: February 27, 2021, 01:02:34 pm by Syntax Error »
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 4217
  • Country: gb
Re: Hitting a wall while writing software
« Reply #22 on: February 27, 2021, 01:02:55 pm »
Over the past 10 years I have rewritten my C source something like 5 times at least.
Use to add a suffix to the folder to manage it and avoid confusion.

What I find really difficult is syncing sources between multiple machines when I have to travel, for instance when I clone a repo on my personal laptop, and then forget it for a while, with some probability of forking the same code revision on a workstation when I get home but without a push. I use Git Merge + Vim Diff, but I sometimes make mistakes during the merge process.

Managing software is an hard job, not only it's difficult to design it, but also to manage revisions.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline retiredfeline

  • Frequent Contributor
  • **
  • Posts: 572
  • Country: au
Re: Hitting a wall while writing software
« Reply #23 on: February 27, 2021, 01:22:54 pm »
Quote
Hitting a wall while writing software

Well there's your problem. If you use one or both hands to hit then you won't be able to type. If you use your head to hit you won't be able to see the screen.  |O :-DD

No, I don't have any suggestions because it's all been said above. I've been writing software for decades and I'm still dissatisfied with my results, even though I get most of my projects working. Software is hard, and the best I can do is improve it until it's not worth my effort anymore.
 

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #24 on: February 27, 2021, 02:05:18 pm »
It seems I was falling for most (if not all) of the mistakes highlighted in this thread.  Very useful advises so far, thank you all.   :)

My biggest mistake, I guess, is the habit of start writing code without designing a state machine first.

I've stumbled upon this tool, QM, that not only can draw the diagram of a HSM (Hierarchical State Machine), but can also generate C/C++ code out of the drawing.





Will try to rewrite the battery discharger software using QM, and see how it goes.   :-DMM

Offline madires

  • Super Contributor
  • ***
  • Posts: 8089
  • Country: de
  • A qualified hobbyist ;)
Re: Hitting a wall while writing software
« Reply #25 on: February 27, 2021, 02:09:05 pm »
Basically it's about finding your way to code, i.e. the way working best for you. There are tons of programing concepts, methods and processes, and every few years there's a hype about some new concept which is promoted to be better and solving all issues. However, we still get a lot of crappy software. >:D
 

Offline rstofer

  • Super Contributor
  • ***
  • Posts: 9921
  • Country: us
Re: Hitting a wall while writing software
« Reply #26 on: February 27, 2021, 03:39:27 pm »
And the only good advice is John Carmack's: don't get attached to your code.  Be ready to throw everything out and rewrite it, multiple times, the moment it starts to seem like that might be a good idea. 

Of course, John Carmack isn't going to pay your mortgage in the meantime...

Way back around '70 and the days of FORTRAN and card decks, we used to joke that the best programs were written after the programmer dropped the box of cards.


 

Offline KE5FX

  • Super Contributor
  • ***
  • Posts: 1985
  • Country: us
    • KE5FX.COM
Re: Hitting a wall while writing software
« Reply #27 on: February 28, 2021, 03:14:24 am »
Way back around '70 and the days of FORTRAN and card decks, we used to joke that the best programs were written after the programmer dropped the box of cards.

Reminds me of the story of the Zen monks who were renowned for their elaborate mandalas.  They would spend all year crafting a beautiful sand painting on the monastery floor, only to sweep it all away an instant after completion.  Visitors from all over the world would flock to the monastery on that day to admire the mandala and join the monks in their contemplation of impermanence.

One year, a toddler broke loose from the surrounding crowd at the last minute before the ceremony began.  The child darted across the floor, giggling and whooping, gleefully flinging sand everywhere. 

The monks were pissed.

Don't be like the monks. 
 

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #28 on: February 28, 2021, 12:22:52 pm »
Me monk?!  No.
Me monkey! ;D

So far I've managed to install that QM tool, and find out how to compile those HSM diagrams on Linux (the examples and tutorials are mostly for Windows).  Made a HSM based blinky to run on a Kubuntu desktop.  Looks cumbersome, mostly because the QM tool uses classes and objects like it would be C++, while the generated code is plain C.

Next step is to understand how to do the same, but using specific board support file, so the compiled state machines will run on the given microcontroller.  There is a demo for Arduino from many years ago, will see if it still works.

Just to be sure everything will still run 10 years from now (in case I decide to keep using QM), the toolchain setup is made in a Kubuntu 20.04 LTS virtual machine kept always offline.

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5405
  • Country: us
Re: Hitting a wall while writing software
« Reply #29 on: February 28, 2021, 06:18:12 pm »
And the only good advice is John Carmack's: don't get attached to your code.  Be ready to throw everything out and rewrite it, multiple times, the moment it starts to seem like that might be a good idea. 

Of course, John Carmack isn't going to pay your mortgage in the meantime...

Way back around '70 and the days of FORTRAN and card decks, we used to joke that the best programs were written after the programmer dropped the box of cards.

Must have been an inexperienced programmer.  Those who had already dropped a card deck used columns 76-80 for sequence numbers.  Initially stepped by five or more to allow for insertion of cards.  Then a few passes through a collator would put a dropped deck back in order.  This was also useful for debugging because compile errors could flag the card number in which the error was identified. 
 

Offline hamster_nz

  • Super Contributor
  • ***
  • Posts: 2812
  • Country: nz
Re: Hitting a wall while writing software
« Reply #30 on: March 01, 2021, 04:57:36 am »
I find that unless I am familiar with the project from past experience that most projects need to be written three times.

Once to understand the problem

Again to understsnd a workable solution

And a third time to do it implement it properly.
Gaze not into the abyss, lest you become recognized as an abyss domain expert, and they expect you keep gazing into the damn thing.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Hitting a wall while writing software
« Reply #31 on: March 01, 2021, 05:09:19 am »
For diagrams, I recommend trying out Dia as well as Inkscape.  Both are available for Windows, macOS, and Linux at least.

I've always preferred a pencil and a pad of paper. I find it easier to focus on the problem when I don't have the distractions of software and trying to make a perfect presentable diagram. I scribble up a flowchart and decide on how I want it all to work, then I start building one part at a time.
 

Offline MIS42N

  • Frequent Contributor
  • **
  • Posts: 516
  • Country: au
Re: Hitting a wall while writing software
« Reply #32 on: March 01, 2021, 08:53:51 am »
Start with a pile of blank sheets, a pencil and an ERASER (the most important tool). Map out the logic. I am still a "lozenge is a decision (with only Y or N exit), a rectangle is something to do, a squashed circle is an exit or entry" type person. Keep it simple. It can end up with a whole lot of IF ... THEN but that will clean up when writing code. Rectangles might be (usually are at the top level) complex and end up being another piece of paper with entry and exit and internal logic. This process can take quite a while and end up with lots of pieces of paper, but it makes one think of the process rather than the problem. Express the logic in general terms, "count until complete" rather than "count up to 9", focus on what to achieve rather than the detail of how to achieve it.

Some of the logic will lend itself to interrupts. They become little programs all of their own with hardware inputs and logical outputs.

Once the logic is worked out, consider the variables needed to make it work. Maybe rough out algorithms to achieve the purpose. Can it be integer, boolean, fractional etc. I quite often start a list of variables like they will be in the program, before I start to cut code.

Where to go from here might be personal preference. I look at hardware to see how to get those variables (or if it is even possible). That may lead to researching other peoples' solutions, maybe pinch some code snippets. Hardware interfaces are usually bricks of code with a few shared variables. I have many enlarged prints of Integrated Circuits with the signals they are going to deal with. Processors are particularly annoying as they have pins that can do many things, so the diagram pins down what is the particular function in this project.

Where a bit of code looks like it will be "difficult" I resort to writing a test bed in VBS (visual basic scripting) that can be run at the command line (cscript). When it works in VBS, it can be recoded in the target language. But that's me.

Time to start writing. Quite often I go for quick and dirty methods, rather than looping four times I'll cut and paste four copies. Or a string of if.. else instead of case. The aim is to get something that does something (anything). The first attempt is usually a steaming heap of ... but if it works in any fashion, it can be cleaned up. I find this stage usually results in a lot of "it could be done better this way" ideas. Maybe they can be incorporated, maybe it's back to the paper. But keep working versions until a better version eventuates.

At some stage during development, the whole thing may need a rewrite. It usually is restructuring logic, putting the code for similar functions together (or even combining them), cleaning up variable names, setting up #define for variables hard coded in version 0, etc. I expect to do this as a matter of course, it allows me to be very dirty in getting a working version.

Even V0.1 may not be the end although it will be close. Consider the 'nice to haves' and add if convenient.

And above all, DOCUMENT. Assume you will come back to this code in a year with amnesia. I was told many years ago the program IS the documentation. One should be able to strip out the comments and find what each part of the program is doing and how it is doing it. If there's more lines of code than comments, it probably isn't documented well enough.

That's my 2¢ worth.
 

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 22274
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Hitting a wall while writing software
« Reply #33 on: March 01, 2021, 01:19:29 pm »
Another thing that careful design helps with, is excessive internal state.

It's relatively easy to sit down and start writing code, implementing various edges of the problem, and tacking on more bits until it meets or exceeds the full scope.

The problem is that "exceeds".  This is... maybe hard to give a good motivating example for, but uh, say you have one state machine, it has its state variable, maybe it has 27 states, stored in a byte variable (256 possible states).  What happens in the other 229 states?  Do they do weird things, like are some of the states done by matching numerical value and some by bitmasking and so on?  It's the software version of the Karnaugh map "do not care" value -- maybe you get nothing in the undefined range, maybe it's random, maybe it does strange and interesting (or even exploitable) stuff?

And then say you grew the program organically, and now it has two state machines.  Maybe they affect each other in weird ways, sometimes twiddling bits, sometimes adding or subtracting values from each others' state variables, or other state in the program.  What happens in all combinations?  It's a combinatorial explosion...

The more responsible approach might be to integrate both state machines into one; this doesn't solve the combinatorial problem, but fixing one part (the basic skeleton of the combined state machine) may lead to a more responsible design that abstracts away the combinatorial problem (e.g. offloading the additional state to dedicated counter registers, which are only accessed via arithmetic, so the range is well defined and easily checked).

Another general (if even more abstract, or difficult to analyze?) approach, is to write down the edge cases explicitly, and work to those.  Like, it's trivial to multiply two numbers together -- your average case is hardly any thought or effort at all.  But when does that process fail?  If that product needs to be larger than a few ten thousand, and you forgot that your platform does 16-bit arithmetic (because you were lazy and assumed C's "int" was big enough -- it's 64 bits on modern desktops and phones, 32 bits on much anything else, but 16 bits on 8086(?), AVR, etc.!), you're going to have a bad time!  Easy enough to check the edges here: put in min/max values of arguments, maybe signs as well, only a handful of tests are needed to verify result.  Now, obviously this gets much trickier in a nontrivial function.

And overflow, the result is undefined in C; though there are few platforms where anything but a modular result happens.  Maybe modular overflow is desired anyway, but maybe it should be saturating arithmetic instead -- who knows?  Does the program specification say anything about it?  If not, it's probably worth escalating that question...

And yet another very general approach: testing all possible inputs.  You can take a function as a complete black box, and in principle, check its behavior for all possible inputs.  Sure, it's not much, functions with hundreds of bits of arguments are common; but it's helpful when it can be done.  These days, this is very tractable (if rather slow in the upper range) for inputs up to 40, even 48 bits.  And random sampling of that space, may give useful insights for even more bits in less time (but easily misses things, too -- see Pentium FDIV bug!).  Obviously, forbidden or redundant states (like random pointers on a protected-mode architecture) can be excluded from search, easily enough, and pointer targets can be controlled, at least when the pointer is testing within allocated memory.  Guided random sampling can produce interesting results even faster, i.e., fuzzing.  For example, this is powerful enough to elucidate undocumented CPU instructions.

And related, one may be able to control the combinatorial explosion of parameter space, by carefully minding the type of that (collective) parameter.
https://en.wikipedia.org/wiki/Type_system#Specialized_type_systems
Supplying parameters independently, is a product type; the number of states to check is exponential with the number of parameters.  A sum type, uses the same bits for different purposes, by context.  The typed C implementation is, a function which takes a union as parameter; there may be unused bits in some of the subtypes, while the number of states only goes as the largest subtype.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6762
  • Country: fi
    • My home page and email address
Re: Hitting a wall while writing software
« Reply #34 on: March 01, 2021, 01:52:50 pm »
For diagrams, I recommend trying out Dia as well as Inkscape.  Both are available for Windows, macOS, and Linux at least.
I've always preferred a pencil and a pad of paper. I find it easier to focus on the problem when I don't have the distractions of software and trying to make a perfect presentable diagram. I scribble up a flowchart and decide on how I want it all to work, then I start building one part at a time.
Oh, I also sketch out everything on paper first.  I have several pads, and even one fancy pad with thicker rougher paper for gel ink pen sketching when I feel fancy or particularly snazzy idea bubbling.  Probably TMI, but I don't bring an electronic device to the toilet, I have a pen and sketchpad there.

As is my wont, I usually sketch those small details I need to attack before integrating the whole.  Looking at my previously filled sketchpads, it looks like I also use them as temporary swap space, jotting up important realizations and conclusions of unit tests/simulations/thought models.  For me, a simple glance seems to be enough to recall the important parts, but that is secondary; the primary function is the writing down process itself, which forces me to organize my thoughts, similar to the rubber duck debugging method: the act of properly expressing the problem as speech or written text or diagram is often enough to organize ones thoughts enough to solve the problem.

A good example that I recently laughed at, it being basically a mix between a chicken walking on the page and madman's rants, was in an earlier full sketchpad, about escape character methods.  Basically, if the escape sequence does not contain the escaped character itself – meaning to escape | you use \!, to escape < you use \(, and to escape \ you use \/, and so on – you avoid most problems associated with escaping; my notes being a scattered collection of reasons and conclusions.  For example, you can process and de-escape such both forwards and backwards, and reprocess/re-escape such input without adding additional de-escaping iterations.

(If you consider the C string "/\\\\\t", you cannot determine what it represents unless you find the leftmost backslash first.  The equivalent "/\/\/\t" on the other hand is unambiguous regardless of which direction you parse it in, because a backslash character is always the first character in a two-character escape sequence.)

That is, my initial on-paper sketches are a working tool for myself; when I arrive at understanding a subproblem, I document the conclusion, basis, and if mathematical the key expressions in Maple/Maxima/Sagemath notation for further manipulation and verification, in text files; and graphs and 2D and 3D illustrations in SVG files or DOT language descriptions.

I have literally hundreds of directories with small example programs in C/Bash/Awk/Python, descriptive text files, SVG diagrams, and Maple/Maxima/Sagemath "workbooks", testing/simulating/brute-forcing a solution to some subproblem.  The programs, when run with --help, always describe the program usage and a simple outline of the problem it works on.  When I'm trying to recall a past subproblem, I often use find and grep to find the subset of candidates, then run the programs in those to find the exact one I'm looking for.  I'm not even sure of the exact number, because when a directory sits for a long time unmodified and unaccessed and therefore no longer that interesting, I tend to move it to backup storage.  Otherwise I'd have to start making a topic hierarchy for the directories, and I don't want to do that yet.
« Last Edit: March 01, 2021, 02:07:03 pm by Nominal Animal »
 

Offline RoGeorgeTopic starter

  • Super Contributor
  • ***
  • Posts: 6629
  • Country: ro
Re: Hitting a wall while writing software
« Reply #35 on: March 05, 2021, 02:22:16 pm »
Quote
Jeroen3
- try TDD

grumpydoc
- take an example of a typical project
- use C/C++ for embedded

NANDBlog
- try a higher level language e.g. Micropython
- use the soldering iron rather than the JTAG
- maybe offload more software work into hardware and analog
- maybe partner up with someone who doesn't do hardware and likes software

madires
- avoid feature creep

golden_labels
- avoid perfectionism
- understand the problem first, with only pen and paper
- write the primary logic first, even if it's just sketchy at first
- note down issues and solve them later
- note down corner cases, leave them as bugs for later
- note down feature requests, put them in the won't fix list
- set time limits

capt bullshot
- drew a state machine diagram before start coding

Caliaxy
- “Hardware eventually fails. Software eventually works.”

CatalinaWOW
- avoid the "software is simple" mindset
- hardware in the example project might be too simple, don't try to do everything in software
- software/hardware trades, e.g. add a pushbutton to signal a new battery
- polling style might lead to simpler software than interrupts
- break down the project into smaller parts for both hardware and software

T3sl4co1l
- draw diagrams, top down first
- implement meaningful standalone functions and test them from a custom shell-like environment
- leave optimizations and fine tuning at the end
- avoid excessive internal state
- treat all states in a state machine, even the invalid ones
- do not skip data validation and treating errors
- test for invalid inputs, too

rstofer
- Top Down Design, Bottom Up Coding
- draw top blocks and describe their interactions
- drill down with the blocks until you hit the hardware
- write the hardware code first then build up to the final interface, e.g. get the UART running at first
- software is usually the hard and expensive part to develop, not the hardware
- the best programs were written after the programmer dropped the box of cards

SiliconWizard
- set the proper mindset/expectations:  software ALWAYS takes longer than hardware to develop

KE5FX
- don't get attached to your code
- be ready to throw everything out and rewrite it, multiple times

nctnico
- start with a good description and diagrams / flow charts
- software is build upon a chassis structure, at some point that chassis can no longer support all the addons
- eventually let other people rewrite or finish what put you down, someone else might enjoy doing that

Nominal Animal
- treat the projects as tools to solve problems
- solve the tedious sub-problems first, e.g. define a comm format between the MCU and the PC
- refactor (rewrite in parts) easily, without dreading it
- code is never "finished", never "perfect";  only "usable"
- sketch out everything on paper first
- use the rubber duck debugging method

Syntax Error
- treat software as a breadboard, not as a final PCB
- add lots of comments for understanding the code years later

DiTBho
- it's normal to rewrite 5 times at least
- software is a hard job to manage, design and keep revisions

retiredfeline
- software is hard
- improve it until it's not worth the effort anymore

madires
- it's about finding your way to code, i.e. the way working best for you, there are tons of programing concepts

hamster_nz
- most projects need 3 rewrites:  first to understand the problem, the solution, then to implement it properly

james_s
- use pencil and paper for diagrams, rather than a drawing software

MIS42N
- start with pencil and paper and an ERASER
- map out the logic
- keep it simple
- use hierarchical blocks with described inputs and outputs
- use plain English rather than i++ < 9, focus on what to achieve and not on how to achieve
- clean up spaghetti logic when writing code
- write difficult pieces of code in a higher language first, test then translate into desired language
- at some stage it may need to rewrite
- document, usually need to have more comment lines than code lines

Very useful inputs for me so far, thank you all!
Will try to apply them by re-tinkering that discharger project this weekend.   :D

Offline SilverSolder

  • Super Contributor
  • ***
  • Posts: 6126
  • Country: 00
Re: Hitting a wall while writing software
« Reply #36 on: March 05, 2021, 03:30:30 pm »

Adopt minimalism as a philosophy when writing software.  As in, "What does it actually have to do, at its essence?" - and get that working first, before even considering any embellishments.

Pause what you are doing every once in a while, and ask yourself - "WTF am I doing right now?"  - if the answer is:  "I'm getting to the most minimal possible solution that will demonstrate it works", keep going.  If the answer is anything else...  backtrack, and get back on the holy path!

This way, you always end up with a project that at least works...  even if it isn't pretty, or lacks convenience features -  in a minimum amount of time.

Most of the time, I find the minimal solution is "good enough for Australia" and live with that - life is short, cool ideas are many! 
 

Offline radar_macgyver

  • Frequent Contributor
  • **
  • Posts: 720
  • Country: us
Re: Hitting a wall while writing software
« Reply #37 on: March 05, 2021, 03:59:06 pm »
My 2c:

Make a list of all the features you want the project to have, and then check off each one as you finish implementing them. I find it helps me avoid the feature creep/perfectionism mentioned earlier, as otherwise I've found myself forever tinkering with the software and never finishing it.

Thank you to everyone who posted on here, these are good answers to something I've had trouble with too, recently.
 

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 22274
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Hitting a wall while writing software
« Reply #38 on: March 05, 2021, 05:38:30 pm »
Adopt minimalism as a philosophy when writing software.  As in, "What does it actually have to do, at its essence?" - and get that working first, before even considering any embellishments.

The counterargument to this is: take the time to make something general enough that it can be expanded later.

Well, when you can afford to do that, and when there is a strong expectation (or primary need) to have it expandable -- it's good, yes.

The counter-counterargument to this is: making the simplest working model (the essence) probably isn't all that much work, and even if you have to rewrite the whole damn thing to make meaningful expansions to it -- it's not a big cost on the whole.

In a similar vein, I think there is value in discounting sunk costs -- obviously, the "sunk cost fallacy" is a pretty strong statement by itself.  In this situation specifically, the value may not be so much in the direct implementation (which is being thrown away!), as in the experience gained from having done it once.  Taking a more zen attitude, allows you to discard earlier stepping stones, while crafting a better, more elaborate final version.

Consider what a lot of computing consists of: testing a bajillion things, essentially all of which fail; then one passes, and you win.  Do you consider all the bajillion-minus-one cases a sunk cost, or productive output? :)

This probably isn't great advice to managers, who sooner or later need to balance the direct cost of that labor, with actual productivity.  Or for paid projects, where the same applies to oneself of course.
:P

I have this experience, when I'm ripping up a PCB layout for example.  It's helpful to start with known-working component placement and routing.  Rearranging everything is faster the second time.  Mind, it's still more time -- this isn't economical as a primary design path, the ripup is still added cost -- but it does affect subsequent rework, say for major component or circuit changes, or size optimization.

Now, software is considerably more complex than PCB layout; it may well prove economical to use this kind of strategy, where the first attempts end up too rough and special-cased and should be discarded in favor of a cleaner (and more optimal to boot) final version.  That is to say: when the early versions are relatively cheap, and when fixing all the edge cases of those versions, would end up more expensive than redoing the whole thing (or substantial parts) from scratch.

Presumably, the number of cases where this happens, will go down over time, as one gains familiarity with programming in general, with the languages and features used, and the contours of the problem itself.  But I can see it might still pay off from time to time.  (Indeed, very productive programmers, tend to be very prolific.  Being able to conceive of a solution, and immediately -- within say a few hours -- bang out an implementation of it, allows one to do this very often indeed!)

Tim
« Last Edit: March 05, 2021, 05:41:37 pm by T3sl4co1l »
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline rstofer

  • Super Contributor
  • ***
  • Posts: 9921
  • Country: us
Re: Hitting a wall while writing software
« Reply #39 on: March 09, 2021, 06:16:14 pm »
Taking a more zen attitude, allows you to discard earlier stepping stones, while crafting a better, more elaborate final version.

A more elegant way of saying "drop the box of Fortran punched cards".
 
The following users thanked this post: SilverSolder

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 22274
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Hitting a wall while writing software
« Reply #40 on: March 09, 2021, 08:00:27 pm »
:-DD
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline bobcat2000

  • Regular Contributor
  • *
  • Posts: 209
  • Country: us
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf