Reducing RTOS complexity on micro, "never global vars"

[jnz]

By all means read my second example carefully, I am happy to draw a diagram or answer questions about it. You are definitely missing the point here though.
The pool is created only once at start up and the buffers are created only once at start up then used to populate the pool. The technique relies on the flow of buffers round an imaginary loop. From the pool to the data source, through the processing stages to the data sinks and then back to the pool. It’s what makes sharing the same data across multiple threads possible and it facilitates flow control if that is required.

Trying to follow this here...

If I have a struct like:

Code: [Select]

typedef struct {
mutex_t lock;
uint8_t data[BUFF_SIZE];;
} NOT_A_poolBuffer_t;

And I make that global and extern it into my other threads/files, so now I have my data global and an attached mutex in the struct. I lock that up during writes and reads. Let's assume I don't cause any crazy hangups with the mutex, this is a fairly simple application.

How is this effectively different than using the pool features built into RTX?

[SimonR]

You are using the pool built in to the RTX, but you still have to define the objects that you put in the pool. In this case its buffers.

I'll write some some example code to illustrate but it might take a couple of days.

[SimonR]

First let me thank you for creating this topic as I wasn't aware of the CMSIS RTOS api before and it looks like its worth futher invesigation. I've also learned a lot from trying to explain things. I had forgotten how very quickly things can get complicated if you are not careful. You can end up explaining the structure of your program and not the technique you are using.
You disn't actually say that you are using  CMSIS RTOS, I just assumed from the way the replies are worded that you are and from the fact that you used mailboxes. Anyway I've done my examples assuming that you are.
There are however a couple of issues with using CMSIS RTOS for my solution to be aware but I'll explain those seperately so as not to confuse things.

I've done 2 versions. One that uses Global variables as its simpler and and one reduces the number of globals.

[SimonR]

In reading the CMSIS RTOS documentation I realised that the Mailbox is actually using the technique I am about to describe, it’s just that all the detail is hidden. The down side of it is that is doesn’t solve the problem of sending the same data to more than one thread as far as I can see.
I started with the code example for osMessageCreate() about half way down the message queue documentation here:-

https://www.keil.com/pack/doc/CMSIS/RTOS/html/group___c_m_s_i_s___r_t_o_s___message.html

Code: [Select]

#include "cmsis_os.h"
 
osThreadId tid_thread1;                          // ID for thread 1
osThreadId tid_thread2;                          // for thread 2
 
typedef struct {                                 // Message object structure
  float    voltage;                              // AD result of measured voltage
  float    current;                              // AD result of measured current
  int      counter;                              // A counter value
} T_MEAS;
 
osPoolDef(mpool, 16, T_MEAS);                    // Define memory pool
osPoolId  mpool;
osMessageQDef(MsgBox, 16, T_MEAS);               // Define message queue
osMessageQId  MsgBox;
 
void send_thread (void const *argument);         // forward reference
void recv_thread (void const *argument);         // forward reference
                                                 // Thread definitions
osThreadDef(send_thread, osPriorityNormal, 1, 0);
osThreadDef(recv_thread, osPriorityNormal, 1, 2000);
 
//
//  Thread 1: Send thread
//
void send_thread (void const *argument) {
  T_MEAS    *mptr;
 
  mptr = osPoolAlloc(mpool);                     // Allocate memory for the message
  mptr->voltage = 223.72;                        // Set the message content
  mptr->current = 17.54;
  mptr->counter = 120786;
  osMessagePut(MsgBox, (uint32_t)mptr, osWaitForever);  // Send Message
  osDelay(100);
 
  mptr = osPoolAlloc(mpool);                     // Allocate memory for the message
  mptr->voltage = 227.23;                        // Prepare a 2nd message
  mptr->current = 12.41;
  mptr->counter = 170823;
  osMessagePut(MsgBox, (uint32_t)mptr, osWaitForever);  // Send Message
  osThreadYield();                               // Cooperative multitasking
                                                 // We are done here, exit this thread
}
 
//
//  Thread 2: Receive thread
//
void recv_thread (void const *argument) {
  T_MEAS  *rptr;
  osEvent  evt;
   
  for (;;) {
    evt = osMessageGet(MsgBox, osWaitForever);  // wait for message
    if (evt.status == osEventMessage) {
      rptr = evt.value.p;
      printf ("\nVoltage: %.2f V\n", rptr->voltage);
      printf ("Current: %.2f A\n", rptr->current);
      printf ("Number of cycles: %d\n", rptr->counter);
      osPoolFree(mpool, rptr);                  // free memory allocated for message
    }
  }
}
 
void StartApplication (void) {
  mpool = osPoolCreate(osPool(mpool));                 // create memory pool
  MsgBox = osMessageCreate(osMessageQ(MsgBox), NULL);  // create msg queue
   
  tid_thread1 = osThreadCreate(osThread(send_thread), NULL);
  tid_thread2 = osThreadCreate(osThread(recv_thread), NULL);
  :
}
It works exactly how the mailbox works except the message queue is visible. I took this example and modified it to share data across 3 threads without the need to copy. I also change the names of variables to match my buffer concept.

Code: [Select]

#include "cmsis_os.h"
 
osThreadId tid_threadSource;                     // ID for main thread (data source)
osThreadId tid_thread1;                          // for thread 1
osThreadId tid_thread2;                          // for thread 2
osThreadId tid_thread3;                          // for thread 3
 
typedef struct poolBuffer_s {
    osMutexId  RefCountLock;
    uint32_t RefCount;
    uint32_t DataCount;
    uint8_t Data[BUFF_SIZE];
} poolBuffer_t;
 
osPoolDef(buffer_pool, 16, poolBuffer_t);       // Define a pool of buffers
osPoolId  buffer_pool;
osMessageQDef(MsgBox1, 16, poolBuffer_t);        // Define message queue1
osMessageQId  MsgBox1;
osMessageQDef(MsgBox2, 16, poolBuffer_t);        // Define message queue2
osMessageQId  MsgBox2;
osMessageQDef(MsgBox3, 16, poolBuffer_t);        // Define message queue3
osMessageQId  MsgBox3;
 
void send_thread (void const *argument);         // forward reference
void recv_thread1 (void const *argument);         // forward reference
                                                 // Thread definitions
osThreadDef(send_thread, osPriorityNormal, 1, 0);
osThreadDef(recv_thread1, osPriorityNormal, 1, 1000);
osThreadDef(recv_thread2, osPriorityNormal, 1, 1000);
osThreadDef(recv_thread3, osPriorityNormal, 1, 1000);

//
//  Thread 1: Send thread
//
void send_thread (void const *argument) {
  poolBuffer_t    *buffptr;
 
    for(;;)
    {
        if(checkDataReady == true)
        {
            buffptr = osPoolAlloc(buffer_pool);           // Allocate memory for the message
            buffer->RefCount = 3;                         // we are sending to 3 threads don’t need a mutex here as we haven’t set it yet
                // Set the message content here maybe a video frame
            osMessagePut(MsgBox1, (uint32_t)buffptr, osWaitForever);  // Send Message to tread 1
            osMessagePut(MsgBox2, (uint32_t)buffptr, osWaitForever);  // Send same Message to tread 2
            osMessagePut(MsgBox3, (uint32_t)buffptr, osWaitForever);  // Send same Message to tread 3
            osThreadYield(); 
        }                             // Cooperative multitasking
    }
}

void myBufferFree(poolBuffer_t *buffer)
{
    osMutexWait(buffer->RefCountLock, osWaitForever);
    buffer->RefCount--;
    if(buffer->RefCount ==0)
    {
        osPoolFree(buffer_pool, buffer);
    }
    osMutexRelease(buffer->RefCountLock);
}
 
//
//  Thread 1: Receive thread1
//
void recv_thread1 (void const *argument) {
  poolBuffer_t  *bufferReference;
  osEvent  evt;
   
  for (;;) {
    evt = osMessageGet(MsgBox1, osWaitForever);  // wait for message
    if (evt.status == osEventMessage) {
      bufferReference = evt.value.p;
      // process the buffer here eg give data to video card
      myBufferFree(bufferReference);                  // free memory allocated for message
    }
  }
}
//
//  Thread 2: Receive thread2
//
void recv_thread2 (void const *argument) {
  poolBuffer_t  *bufferReference;
  osEvent  evt;
   
  for (;;) {
    evt = osMessageGet(MsgBox2, osWaitForever);  // wait for message
    if (evt.status == osEventMessage) {
      bufferReference = evt.value.p;
      // process the buffer here eg save videodata to a file
      myBufferFree(bufferReference);                  // free memory allocated for message
    }
  }
}
//
//  Thread 3: Receive thread2
//
void recv_thread3 (void const *argument) {
  poolBuffer_t  *bufferReference;
  osEvent  evt;
   
  for (;;) {
    evt = osMessageGet(MsgBox3, osWaitForever);  // wait for message
    if (evt.status == osEventMessage) {
      bufferReference = evt.value.p;
      // process the buffer here eg send video date to ethernet
      myBufferFree(bufferReference);                  // free memory allocated for message
    }
  }
}


void StartApplication (void) {
  buffer_pool = osPoolCreate(osPool(buffer_pool));                 // create memory pool
  MsgBox1 = osMessageCreate(osMessageQ(MsgBox1), NULL);  // create msg queue
  MsgBox2 = osMessageCreate(osMessageQ(MsgBox3), NULL);  // create msg queue
  MsgBox3 = osMessageCreate(osMessageQ(MsgBox4), NULL);  // create msg queue
   
  tid_threadSource = osThreadCreate(osThread(send_thread), NULL);
  tid_thread1 = osThreadCreate(osThread(recv_thread1), NULL);
  tid_thread2 = osThreadCreate(osThread(recv_thread2), NULL);
  tid_thread3 = osThreadCreate(osThread(recv_thread3), NULL);
}
As you can see its almost exactly the same except that we send the same message to 3 queue instead of 1. Each thread decrements the reference count before returning the buffer to the pool. And that’s it. I have written a higher level buffer free function so that the mutex code only appears in one place.
There is an issue in this code to do with initialising the pool but I don’t want to complicate things yet.

[SimonR]

Considerations when using CMSIS RTOS API

CMSIS RTOS API is supposed to be an implementation independent API to sit on top of whatever RTOS you choose to use. As a result C macros are used to create the structure for a given application.
For instance to create a message queue you would use the #define osMessageQDef like this, as per the example in the documentaion.

Code: [Select]

osMessageQDef(MsgBox, 16, T_MEAS);               // Define message queue
osMessageQId  MsgBox;
void main(void)
{
}

The #define is implementation dependant. Which means that if you wanted to include one as part of another structure for example you will find that it may or may not compile. It effectively means that every RTOS component has to be defined as global.
My requirement for a mutex is defined like this.

Code: [Select]

typedef struct poolBuffer_s {
    osMutexId  RefCountLock;
    uint32_t RefCount;
    uint32_t DataCount;
    uint8_t Data[BUFF_SIZE];
} poolBuffer_t;
But using CMSIS RTOS should if be defined like this

Code: [Select]

typedef struct poolBuffer_s {
    osMutexDef( RefCountLock);
    uint32_t RefCount;
    uint32_t DataCount;
    uint8_t Data[BUFF_SIZE];
} poolBuffer_t;
We can’t guarantee that the macro will result in correct code unless we know that the implementation underneath is correct.

Why is this important?
Well in my buffer pool example the buffers need to be initialised when they are put into the pool.

In my pthreads application the pool creation code looks like this.

Code: [Select]

Generic_Pool_t * CreateBufferPool(void)
{
    Generic_Pool_t *bufPool = PoolCreate();
    for(int count = 0; count < 10; count++)
    {
        Buffer_t * newBuffer = BufferCreate(bufferSize);
        newBuffer->Lock = MutexCreate();
        PoolAdd(newBuffer);
    }
    Return buffPool;
}
The problem with the MemoryPool osPoolCreate is that it creates the pool and allocates the space for each object in the pool but the objects are not initialised. As you can see it’s important to for the buffers to be initialised before they are added to the pool because they each have their own mutex.
To initialise them in CMSIS RTOS you would have to create the pool then take every single object out of the pool, initialise them and then put them back in.