Author Topic: using int32 to store everything from uint8 no int32 with values below 2 billion  (Read 1351 times)

0 Members and 1 Guest are viewing this topic.

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
So I am working with CAN Open and the faff is that I need to store and send values in 8, 16 and 32 bit signed or unsigned. This makes it impossible to say put them all in an array that would be very useful for say stepping through with easy to write code to send the setup values one after the other.

but it occurs to me. with all the memory I am already using to store pointers to variables that hold the relevant value I am already using 32 bits of memory anyway. And from
what I understand of two's complement representation a negative 8 or 16 bit number stored in a signed 32 bit space looks the same as when stored in an 8 or 16 bit space once the 2 or 3 leading bytes are removed.

Therefore providing where I have an unsigned 32 bit value I do not need to go above the halfway mark of a little over 2 billion then sending the number as a signed or unsigned when I represent an unsigned number in the signed space it makes no difference. This means that I can now have an array of signed 32 bit values that will hold any data type I am using. as I use memcpy() to move the data about the actual storage type is irrelevant. I know what size each variable is anyway as I have to store the information about the data size along with my record of the object dictionary (also in an array), so bingo.

This makes it easier to setup devices when I have to send a sequence of values as I can now while index is has not reached this value ect. In my actual program everything is a signed 32 bit number and the world is a better place for me  :-DD
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4118
  • Country: nz
Umm .. this is more a comment than a question ... but ...
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
It is a question. To be honest to me it looks like it is made to be so. In canopen land as part of your configuration you have to send the index (16 bits), subindex (8 bits) and number of bits (8 bits) to the devices. so I already have 32 bit values that define the complete "location" of the data and the length. But there is no way to identify signed or unsigned. But if I just store everything as 32 bit signed then the bit length field specifies how much of the 4 bytes to send.

-2 is 0x FF FF FF FE in 32 bit
-2 is 0x         FF FE in 16 bit

so if I need to use this value as 16 bit I can take the 32 bit value and truncate it. Providing I do not have a positive number that is so high it breaks into the negative range a positive signed 32 bit number is the same as an unsigned 32 bit number, 32 bits signed allows me to store 2'147'483'648. The most granular number I have come across so far is mA to define my motor current. I really don't think I will be using a motor that runs on 2'147'483 A (2 MA). Same for voltage in mV. even µV.

So am I nuts? or was this all meant to be?
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5092
  • Country: ro
  • .
You could probably have a separate array of bytes where you use 3 or 4 bits for every number : 2 bits or 3 bits for size (1 byte, 2 byte, 3 byte , 4 byte, more if you use 3 bits) and one for signed/unsigned

Then store the numbers as a series of bytes. If you want to parse the numbers, you just keep reading 4 bits at a time from the first array, then read up to n bytes from the second array and advance position in both arrays.

If you want to get more fancy, you could use a 32 bit value for every (up to) 8 numbers : 4 bits for the number of values, 4 bits for whatever and 24 bits for the 8 sets of 3 bits (data length and sign) ... or 4 bits for number of values, 28 bits for 7 x 4 bits

if you use uint32 for everything 8 numbers would use 8 x 4 = 32 bytes. with the idea above, 8 uint8 numbers could use 4 bytes + 8 x 1 byte = 12 bytes.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Well you see the problem is that I need to be able to cycle through an array. At the moment I have an array of pointers to the variables. So if I had say at array of single bytes I don't see how I would know the difference between data and the byte/signedness of the data.

So I have to send various variables to devices, at the moment I can cycle through them if I rearrange their pointers in my pointer array, but if I have the pointer array then I may as well use those 4 bytes to store the data, the problem is getting at different data types as though in an array. There is a parallel array that carrys the standard locations like 0x60410010, the index of these is the same as the index for my array of pointers. It means that first I have to setup a variable and assign it a value, then put a pointer t this variable into the pointers array. As all pointers are the same type the array works.

But if I just call everything int32 then the individual variables go away and free up memory and I just use the values at the locations and save pointery stuff.
 

Offline eutectique

  • Frequent Contributor
  • **
  • Posts: 417
  • Country: be
Or you can use a TLV thing, which is type-length-value. The generic entity is declared as
Code: [Select]
typedef struct {
    uint8_t type;
    uint8_t length;
    uint8_t value[];
} tlv_t;

type is an enum of int8, uint8, int16, uint16, MAC48, whatever.
length is the length of value in bytes
value is the payload
 
The following users thanked this post: Smokey

Offline Colt45

  • Contributor
  • Posts: 29
  • Country: ca
Yes, better to keep the payload as 8bit array in a struct.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
The array of bytes cannot be used as is as a value. This is not about generating the can message payload, it is about storing the data locally. So currently there is:

uint32 OD_entry["i"] this is the list of object dictionary items.
void * OD_value["i"] This contains the pointers to the variables of 8, 16 or 32 bit signed or unsigned.

"i" is replaced by definitions of the object name so for example
#define STATUS_WORD 0x03
This way the name of the object can be used instead of the number.

At the start of the program I have to assign the dictionary location and bit count to the OD_entry array and the pointer to the variable to OD_value.
So for example

uint16 STATUS_WORD_val ;

So:

OD_entry[STATUS_WORD ] = 0x60400010 ; // index 0x6040, subindex 0x00, number of bits 16.
OD_value[STATUS_WORD ] = &STATUS_WORD_val ;

So now I can use the object name to index the array of its definition or it's value. Obviously I can only memcpy() to and from the pointer but I have access to the variable.

This means that when I receive a SDO message which contains the index, sub index and data length I can reconstruct the definition and find it in the OD_entry array, when I find a match I know which index of the OD_value array will hold the pointer to the variable.

So lots of messing about and up to 12 bytes per value, most are 10. So if instead I use int32 variables all round I can put them all into one OD_value array that now is not of the void * type but just int32. This means that I can use the values directly and 8 bytes will be used all round.
« Last Edit: February 23, 2024, 06:09:07 pm by Simon »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6470
  • Country: fi
    • My home page and email address
I don't like language lawyerism, but since we are talking about integer casts, perhaps some consideration for the C standard is appropriate here.
TL;DR: Your approach works fine.

The key part is in C99-C17 6.3.1.3:
Quote from: ISO C Standard, 6.3.1.3
When a value with integer type is converted to another integer type other than _Bool, if the value
can be represented by the new type, it is unchanged.

Otherwise, if the new type is unsigned, the value is converted by repeatedly adding or subtracting
one more than the maximum value that can be represented in the new type until the value is in the
range of the new type.

Otherwise, the new type is signed and the value cannot be represented in it; either the result is
implementation-defined or an implementation-defined signal is raised.
This means that conversion from signed to unsigned uses modular arithmetic, and conversion to a signed integer type is exact and lossless as long as the value is representable in the target signed type; otherwise it is implementation-defined.  (GCC and Clang both support modular rules for signed integer target types also, but may have (now in the future) options to change this.)

The other important thing is what the standard says about exact-width integer types (intN_t, uintN_t), in C99 7.18.1.1 and C11-C17 7.20.1.1:
Quote from: ISO C standard, 7.18/20.1.1
The typedef name intN_t designates a signed integer type with width N, no padding bits, and a
two's complement representation. Thus, int8_t denotes such a signed integer type with a width of
exactly 8 bits.

The typedef name uintN_t designates an unsigned integer type with width N and no padding bits.
Thus, uint24_t denotes such an unsigned integer type with a width of exactly 24 bits.
It goes on to say that the types for N = 8, 16, 32, 64 are optional, but if supported, these types (typedef'd) will be supported.  For practical purposes, all widely-used C and C++ compilers do now support these (synthesizing them even when the hardware does not natively support them) from tiny C compilers to flagship commercial C++ compilers, so we can basically assume they (and the related least-width and fast types) are supported, possibly excepting the N = 64 when dealing with 8-bit targets and older compiler versions.

Thus, based on the above observations from the text of the standard:

While there may be standards-compliant C and C++ compilers that do not support these types (<stdint.h>/<inttypes.h> are provided by the compiler, not by the standard C library per se;), they are completely impractical, and can be discounted.  On all architectures, the compilers can synthesize the support for these types.

When supported, it is absolutely safe to use an int32_t variable to hold any unsigned value between 0 and 232-1 = 4294967295, inclusive; and any signed value between -231 = -2147483648 and 231-1 = 2147483647, inclusive; as long as you know whether the value is supposed to be signed or unsigned.  The cast to the target type will never fail, or produce an incorrect value.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
as long as you know whether the value is supposed to be signed or unsigned.  The cast to the target type will never fail, or produce an incorrect value.

But this is the main crux of the problem. When I receive data I know how many bytes the variable is, but I do not know if it is signed. So dumping everything into a signed 32 bit integer kind of simplifies it as I know that i will never use values above 2 billion.

I guess this still raises problems as I do not know if a 16 bit value is signed or not, if I cast a high 16 bit unsigned value to 32 bit as though signed I have a problem. i can't distinguish, if I simply pad out a negative 16 bit value with 16 bits set to 0 I now generate an incorrect positive 32 bit number.

OK, so all is not lost. I will have to update my Object definition to include a sign marker.

I was using the same format CAN Open wants me to use to send object definitions when I set up tho PDO stuff, but this is a small part of using CAN Open so converting back to this format is not a problem. I will need to change 0x60400010 to something like  0x60400090 so just add 0x80 for signed numbers. I can then put the number into a signed or unsigned holding value.

When I send I simply truncate any int32 down to the size of the variable.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6470
  • Country: fi
    • My home page and email address
What do you need such a generic associative array for CANopen, anyway?

For 11-bit CAN-IDs, there are 2048 possible IDs. 0x000 (NMT), 0x580–0x67F (SDO), 0x700–0x77F (guard/heartbeat), and 0x7E4–0x7E5 (layer setting services) are reserved and fixed, and only 1600 or so (0x001–0x580, 0x680–0x6DF, 0x780–0x7DF) are really available for PDOs (default four for receiving are 0x200+node per object dictionary key 0x1400, 0x300+node per key 0x1401, 0x400+node per key 0x1402, and 0x600+node per key 0x1403; default four for transmitting are 0x180+node per key 0x1800, 0x280+node per key 0x1801, 0x380+node per key 0x1802, and 0x480+node per key 0x1803), SYNC (default 0x080 per key 0x1005), TIME (default 0x100 per key 0x1012), and EMCY (default 0x080+node per key 0x1014) messages.

Thus, initially, a slave only listens for less than a dozen CAN-IDs, with the rest configured dynamically.  For the receive side, I'd use an array containing at least the CAN-ID and a function pointer to the message handler, sorted by decreasing CAN ID, so a binary search will locate the correct message handler very, very fast, without wasting memory.  (Adding a new receive CAN-ID is then an insertion operation, moving any entries at or after the insertion point one entry outwards; and removing a CAN-ID is a corresponding delete operation, moving any entries at or after the insertion point one entry inwards.)

The object dictionary index 0x0000 is never used, so can be used as an "unused" or "end of list" sentinel.
Indexes 0x0001–0x001F define fixed data types (code DEFTYPE), and optionally, when read yield the length (in bits) of each type.
Indexes 0x0020–0x003F define fixed structure types (code DEFSTRUCT).
Indexes 0x0040–0x005F define manufacturer specific structures (code DEFSTRUCT).
Indexes 0x0060–0x007F (+ (D-1)*0x0040) define types (code DEFTYPE) for logical device D = 1..8.
Indexes 0x0070–0x008F (+ (D-1)*0x0040) define structures (code DEFSTRUCT) for logical device D = 1..8.
Indexes 0x0260–0x0FFF are reserved.
Indexes 0x1000–0x11FF (0x1000–0x1029) belong to the standard communication profile, having specific meanings.  (If logical devices are used, index 0x1000 links to indexes 0x67FF, 0x6FFF, 0x77FF, 0x7FFF, 0x87FF, 0x8FFF, 0x97FF, and 0x9FFF, for each logical device respectively.)
Indexes 0x1200–0x127F define the SDO server parameters, including the CAN-IDs (data type specified by index 0x0022).
Indexes 0x1280–0x12FF define the SDO client parameters, including the CAN-IDs (data type specified by index 0x0022).
Indexes 0x1400–0x15FF define the PDO receive parameters, including the CAN-IDs.
Indexes 0x1600–0x17FF define the PDO receive structures (or mapping), with sub-indexes specifying the components in key index:sub-index:numbits 32-bit unsigned integer format.
Indexes 0x1800–0x19FF define the PDO transmit parameters, including the CAN-IDs.
Indexes 0x1A00–0x1BFF define the PDO transmit structures (or mapping), with sub-indexes specifying the components in key index:sub-index:numbits 32-bit unsigned integer format.
Indexes 0x1FA0–0x1FCF define object scanner lists for MPDOs, each being an array of up to 254 32-bit unsigned integers.
Indexes 0x1FD0–0x1FFF define object dispatch lists for MPDOs, each being an array of up to 254 64-bit unsigned integers.
Indexes 0x2000–0x5FFF define manufacturer-specific objects.
Indexes 0x6000–0x67FF (+ (D-1)*0x0800) define standardized objects for logical device D (per device profiles, CiA 40x).
Indexes 0xA000–0xAFFF define standardized network variable objects.
Indexes 0xB000–0xBFFF define standardized system variable objects.
Indexes 0xC000–0xFFFF are reserved.

Thus, it seems to me it would make sense to implement the object dictionary facilities in distinct subsets, instead of a single big associative array.
One for DEFTYPEs, one for DEFSTRUCTs, one for SDO parameters (indexes 0x1200–0x12FF), one for PDO parameters (0x1400–0x15FF, 0x1800–0x19FF), one for PDO structures (0x1600–0x17FF, 0x1A00–0x1BFF), and so on.

One interesting approach for an associative array you might consider (especially for object dictionary keys 0x6000–0x67FF) is an array of index:type:description structures in RAM, sorted by index in descending order, and populated at run time.  This means you can use a binary search to look up any index very efficiently.  As index is 16-bit and description is a pointer, that leaves 16 bits for type, which should encode both the object-code (VAR, ARRAY, RECORD) and the C type of the structure description points to.

VAR structures contain a single value, and ARRAY contains 1 to 255 values of the same type; in both cases the type specified by a single 16-bit unsigned integer (referring to a DEFTYPE object dictionary entry).  For these, you could use a structure (in Flash or RAM) that specifies that type, the C/C++ type of the values, and a pointer to the value or array of values in RAM.  The number of values would be encoded in type, so these would take up 256 of the 65536 possible values.  If you also encode the C/C++ type in it (uint8_t, int8_t, uint32_t, int32_t, float), about 1280 of the 65536 possible type values would be used.

Optionally, an additional 256 type values could specify the same, except instead of only the pointer to the first element, it would have a pointer to each variable; the variables all having the same C type.

Optionally, an additional 256 type values could specify the same, except instead of only the pointer to the first element, it would have a pointer to each variable, and a separate array of C type identifiers (maybe unsigned char) to define the type of each variable.

For VISIBLE_STRING, and UNICODE_STRING VARs I'd use one types for immutable strings, and one for each possible mutable/run-time set string maximum length (255).  For OCTET_STRING, I'd use 256 types for immutable strings, and 256 for mutable strings, depending on the length.  All together, these would take about 768 type values.

RECORD structures contain 1 to 254 values of different types.  For these, you could encode the number of values in type also, with the description consisting of an array of pointer:ctype:deftype tuples, where pointer points to the value in RAM, ctype is its type, and deftype is the 16-bit DEFTYPE dictionary index for that sub-index.  These would take 254 of the 65536 possible values of type.

Note that you do not need to define thousands of structures, as you can use variably modified types inside local scopes.  That is,
Code: [Select]
unsigned char get1(int h, int w, int y, int x, unsigned char *m) { return m[y*w+x]; }
unsigned char get2(int h, int w, int y, int x, unsigned char m[h][w]) { return m[y][x]; }
unsigned char get3(int h, int w, int y, int x, void *m) {
    typedef  unsigned char  mtype[h][w];
    return (*((mtype *)m))[y][x];
}
all typically compile to the exact same machine code. get2() uses a variably-modified type as a parameter, and get3() a variably-modified type in its body.

Clang does not like variably-modified arrays in structures, so if you want to have two different arrays consecutively, use
Code: [Select]
// Better:
unsigned char get1(int n, int i, void *data) {
    typedef   int16_t  atype[n];
    typedef   int8_t   btype[n];

    atype *const aptr = data;
    btype *const bptr = (void *)((char *)aptr + sizeof (atype));
    return (*aptr)[i] * (*bptr)[i];
}

// Not the GCC-specific:
unsigned char get2(int n, int i, void *data) {
    struct {
          int16_t a[n];
          int8_t b[n];
    } *ab = data;
    return ab->a[i] * ab->b[i];
}
« Last Edit: February 25, 2024, 02:08:50 am by Nominal Animal »
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
i am completely confused and it feels like I end up with more memory and stuff to set up than I have. PDO's are dealt with already, I have a stucture for them and the message receiving code will just copy the data from the message to the correct jstructure location. What I really need to do with is sort out SDO communication so basically everything.

Now the simple solution would be to just create an array as big as the number of possible indexes and sub indexes. This would mean that the index and sub index would simply address the array as the index, small problem 2^24 = 16 million and a bit, worse case each one is 4 bytes be that whatever, value or pointer it's 64MB, so not going to happen in 16kB of RAM or even the 128kB of ROM.

OK so I would just map the sections I use but this would still be excessive and wasteful. What I need to do is remap the 24 bits that make up the index aand sub index of the objects I use but I cannot use the indexes or sub indexes themselves as locators.

So the names of the objects I use are defines that translate to a unique number stating at 0. This number can be used as a unique identifier in any array. the goal is to type the object name in the program and that this works to make all the bits fall into place. as I have 6 types of data I can't create any structs if I want to use that data in my program. Alternatively when I search the array for an objects dictionary location definition by cycling through the indexes of an array that lists all of the index's and sub indexes I am interested in that array index number will be able to locate a pointer in a second array or the variable itself in the second array.

The object dictionary definition is complete when the type of data is identified, CAN open already uses this sort of scheme for telling the node what objects each PDO should transmit.

So:

index : 2 bytes
s index : 1 byte
variable type: 1 byte

A nice 32 bit variable. Or yea I could create an array of structs so that I can separately refer to index, sub index and data type without doing bit shifting. If i were using a lot of objects that use sub indexes (most just have an index with the sub at 0) this makes it easier to look through the indexes first then sub indexes. Worse case you have all 255 sub indexes used and now that is a long time spent searching.

When it comes to the data itself I have two options. find a way of unifying it to one type or use pointers that in an array that the same index number will call upon. I currently do the latter. But here I use 4 bytes for the pointer, then there is the data. So if 4 bytes will hold every type (it will) then I may as well have that array of int32 and forget the pointers.

So when I need to send a sequence of values for example to set up the slaves, instead of what I do now which is to write a line of code per item to send, I put all those objects consecutively in the array, this way I just step through the array incrementing the index until the last index.

When data is received (SDO) I simply take the first 4 bytes and extract index, sub index and number of bytes, then I run through the array comparing the indexes, if an index matches I then compare the sub index, if successful I can compare the number of bytes received although this may be superfluous at this point so I look at my definition of the data type and copy the data into the correct data type before storing it in an int32 that is capable of being used directly in my program.

For writing I simply send the index and sub index identified by the name that is a definition of the index in the array that contains the information along with the correct amount of bytes from the int32. total bytes used for any value sent 8.

Now often I send the same object to more than one node, but often the value is the same for both sometimes different. When I receive data I need to have one variable per node. This does start to lead me into having an array of objects that are specific for say setting up and an array of the working objects that has an array of values so that unique values can be sent to nodes or received.

Essentially here I am writing the CAN Open master that deals both with the communication DS301 and maybe others that are secret plus the DSP402 standard that relates to the motors I am using.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6470
  • Country: fi
    • My home page and email address
i am completely confused and it feels like I end up with more memory and stuff to set up than I have. PDO's are dealt with already, I have a stucture for them and the message receiving code will just copy the data from the message to the correct jstructure location. What I really need to do with is sort out SDO communication so basically everything.
I'm sorry if I confused you :-\.

Put simply, my point was that what CAN and CANopen call an "object dictionary" is actually a structured collection of entities that are interrelated in ways defined in the device profiles.  So, in my opinion, it would make sense to create functions that implement such ways, as accessor functions (get, set), and associate those with the "object dictionary".  For example:
Code: [Select]
// SPDX-License-Identifier: CC0-1.0  (i.e. Do Whatever You Want, just don't blame me)
#include <stdint.h>

// Basic types using CANopen numbering.
#define  TYPE_I8      0x02  /* int8_t   */
#define  TYPE_I16     0x03  /* int16_t  */
#define  TYPE_I32     0x04  /* int32_t  */
#define  TYPE_I64     0x15  /* int64_t  */
#define  TYPE_U8      0x05  /* uint8_t  */
#define  TYPE_U16     0x06  /* uint16_t */
#define  TYPE_U32     0x07  /* uint32_t */
#define  TYPE_U64     0x1B  /* uint64_t */
#define  TYPE_REAL32  0x08  /* float    */
#define  TYPE_REAL64  0x11  /* double   */
#define  TYPE_FLOAT   TYPE_REAL32
#define  TYPE_DOUBLE  TYPE_REAL64
// Note: A different scheme would allow simpler conversion implementation
//       between numerically identified types, as when copying objects
//       to or from the object dict.

// Accessor functions need some kind of *context*.  It is a constant, but could have many forms.
// This assumes __WORDSIZE == 32, i.e. sizeof (void *) == 4.
typedef const struct {
    union {
        void      *ptr;
        uint32_t   u32;
        int32_t    i32;
        uint16_t   u16[2];
        int16_t    i16[2];
        uint8_t    u8[4];
        int8_t     i8[4];
    };
} ctx;

// Object dictionary operations are stored in Flash, and describe accessor functions and their context.
typedef const struct {
    ctx   const context1;   // This is usually a pointer.
    ctx   const context2;   // This usually uses the integer components.  Say, a size of the array.
    int (*const get)(void *into, uint32_t index_intotype_subindex, ctx context1, ctx context2);
    int (*const set)(void *from, uint32_t index_fromtype_subindex, ctx context1, ctx context2);
} object_dict_ops;

// Each object dict entry takes (2+sizeof(void *)) bytes of RAM.
#ifndef  OBJECT_DICT_ENTRIES
#define  OBJECT_DICT_ENTRIES  500
#endif

// Recursive mutex/spinlock/interrupt locking.
uint32_t    object_dict_lock(void);
void        object_dict_unlock(uint32_t);

// Object dictionary itself.
const object_dict_ops *object_dict_entry[OBJECT_DICT_ENTRIES];  // In same order as object_dict_index
uint16_t               object_dict_index[OBJECT_DICT_ENTRIES];  // In sorted order, largest index first
int32_t                object_dict_entries = 0;
const int32_t          object_dict_entries_max = OBJECT_DICT_ENTRIES;

// Each object dict index is handled by a set of operations.
// These operations are added to the dictionary at run time, but defined at compile time.
int object_dict_attach(const object_dict_ops *ops, int index);

// Although only the index is needed for detaching operations,
// it may be useful to keep the other parameters for verification.
int object_dict_detach(const object_dict_ops *ops, int index);

// Individual values can be copied to and from the object dict, per the accessor functions.
int object_dict_get(void *into, uint_fast16_t index, uint8_t subindex, uint8_t into_type);
int object_dict_set(void *from, uint_fast16_t index, uint8_t subindex, uint8_t from_type);
Here is an example of accessor functions that can be used to access arrays of int32_t values:
Code: [Select]
int get_i32(void *into, uint32_t index_intotype_subindex, ctx context1, ctx context2)
{
    const uint8_t  into_type = (index_intotype_subindex >> 8) & 0xFF;
    const uint8_t  subindex = index_intotype_subindex & 0xFF;
    int32_t *const array = context1.ptr;
    const uint8_t  length = context2.u8[0];

    // Length requested?
    if (subindex == 0)
        return copy_i32_into(into, length, into_type);

    // Actual value?
    if (subindex <= length)
        return copy_i32_into(into, array[subindex-1], into_type);

    return -1;
}

int set_i32(void *from, uint32_t index_fromtype_subindex, ctx context1, ctx context2)
{
    const uint8_t  from_type = (index_fromtype_subindex >> 8) & 0xFF;
    const uint8_t  subindex = index_fromtype_subindex & 0xFF;
    int32_t *const array = context1.ptr;
    const uint8_t  length = context2.u16[0];

    // Cannot set length.
    if (subindex == 0)
        return -1;

    // Setting an actual value?
    if (subindex <= length)
        return copy_into_i32(array + subindex, from, from_type);

    return -1;
}
They assume that function
    int copy_i32_into(void *dst, int32_t src_val, uint8_t dst_type);
copies the specified src_val into *dst of dst_type type; and that function
    int copy_into_i32(int32_t *dst, void *src, uint8_t src_type);
copies the value of *src_val of type src_type into an int32_t at *dst.
They should also do any necessary type conversions.
(Note that for copy_into_i32(), dst is always aligned.  The void * can point to anything, so may be misaligned for the specified type.)

In general, you'd need different accessor functions for the complex structures (like the one specified in index 0x0022), for device-profile STRUCTs (if any!), for ARRAYs (separate ones for each type you use in RAM), and for VARs (separate ones for each type you use in RAM).

For example, to implement an array of 6 auxiliary variables for digital inputs at index 0x2050, you could do
Code: [Select]
int32_t  aux_variables[6];

const object_dict_ops  aux_variables_ops = {
    .context1 = { .ptr = aux_variables },
    .context2 = { .u8 = { sizeof aux_variables / sizeof aux_variables[0], }},
    .get = get_i32,
    .set = set_i32,
};
At runtime, you would need to associate the ops with the desired object dict index:
    if (object_dict_attach(&aux_variables_ops, 0x2050)) { /* failed */ }
Let's say you then have an int16_t v.  You can use
    if (object_dict_get(&v, 0x2050, 0x01, TYPE_I16)) { /* failed */ }
to get the first variable from 0x2050:0x01 to v, or
    if (object_dict_set(&v, 0x2050, 0x01, TYPE_I16)) { /* failed */ }
to copy the value of v to 0x2050:0x01.

The same functions can be used for any number of arrays of int32_t, each array only needs their own const object_dict_ops entry, to specify also the array location and size.

Note that object_dict_attach() should keep the entries in object_dict_index[] and object_dict_entry[] –– object_dict_entry[k] describes index object_dict_index[k] –– in decreasing order of index, so that all four functions can use a very fast binary search to find the target entry (or insertion index for _attach).  All four functions also must internally use lockval = object_dict_lock(); and object_dict_unlock(lockval); to protect the accesses to the object dictionary (via a mutex, spinlock, or interrupt mask), so that it is safe to get and set the object dict values from any context.

In your SDO receive and transmit functions, you'll use object_dict_set() and object_dict_get(), and if either returns nonzero/failure, you send the EMCY error message.

Because the indexes are compact in memory (a single array of uint16_t), a binary search will find them in less than 17 iterations (9 if you have less than 512 entries in use), and each iteration only involves an addition, a substraction, divide-by-two (one bit shift right), loading a 16-bit word from memory, and two comparisons.  It will be fast.  (Not as fast as directly referencing variables in memory, of course; but fast enough to use even in an interrupt context.)

I am writing [a CANopen] master
Okay, that simplifies things.  Slaves must be modifiable by the master, but the master only needs to support different kinds of slaves.  The above certainly applies, but you are unlikely to need the setters and only a single getter (as all your memory variables are int32_t's.)

Something like
Code: [Select]
// SPDX-License-Identifier: CC0-1.0
#ifndef   OBJECT_DICT_H
#define   OBJECT_DICT_H

// If you modify the state associated with an object dict entry non-atomically,
// you need to lock access to the object dictionary.
uint32_t object_dict_lock(void);
void     object_dict_unlock(uint32_t);

// Supported CANopen types
#define  TYPE_I8    0x02
#define  TYPE_I16   0x03
#define  TYPE_I24   0x10
#define  TYPE_I32   0x04
#define  TYPE_U8    0x05
#define  TYPE_U16   0x06
#define  TYPE_U24   0x16
#define  TYPE_U32   0x07

typedef union {
    uint32_t        key;
    struct {
        uint16_t    index;
        uint8_t     type;   // CANopen type; entry is always int32_t
        uint8_t     length;
    };
} index_type;

// Return the index:type:len corresponding to an index, or 0:0:0 if not found.
index_type  object_dict_find(uint_fast16_t index);

// Copy a value from object dict into local storage, applying format conversions.
// Returns the number of bytes copied, or negative for error.
int  object_dict_get_into(void *into, uint_fast16_t index, uint8_t subindex);

// Copy a value from local storage to object dict, applying format conversions.
// Returns the number of bytes copied from, or a negative error code.
int  object_dict_set_from(void *from, uint_fast16_t index, uint8_t subindex);

// Associate an object dict index with a variable (len=0) or an array of variables (len=1..253).
// Returns 0 if success, a negative error code otherwise.
int  object_dict_new(int32_t *mem, uint_fast16_t index, uint8_t length, uint8_t type);

// Error codes; must be negative
#define  OBJECT_DICT_FULL               -1
#define  OBJECT_DICT_INDEX_IN_USE       -2
#define  OBJECT_DICT_INVALID_INDEX      -3
#define  OBJECT_DICT_INVALID_SUBINDEX   -4
#define  OBJECT_DICT_INVALID_TYPE       -5
#define  OBJECT_DICT_INVALID_VALUE      -6  // Value exceeds type limits

#endif /* OBJECT_DICT_H */
and the corresponding implementation (with a fixed maximum number of object dict entries):
Code: [Select]
// SPDX-License-Identifier: CC0-1.0
#include <stdint.h>
#include "object_dict.h"

#ifndef  OBJECT_DICT_ENTRIES
#define  OBJECT_DICT_ENTRIES  512
#endif

static index_type       object_dict_index[OBJECT_DICT_ENTRIES];
static int32_t         *object_dict_entry[OBJECT_DICT_ENTRIES];
int                     object_dict_entries = 0;
const int               object_dict_entries_max = OBJECT_DICT_ENTRIES;

uint32_t object_dict_lock(void)
{
    // Recursive interrupt/spinlock.
    return 0;
}

void object_dict_unlock(uint32_t lock)
{
    // Recursive interrupt/spinlock.
    (void)lock;
}

// Internal binary search lookup.  Object dict must be locked.  Returns -1 if not found.
static int  index_at(uint_fast16_t index)
{
    if (index < 1 || index > 65535)
        return OBJECT_DICT_INVALID_INDEX;

    if (object_dict_entries < 1)
        return OBJECT_DICT_INVALID_INDEX;

    if (object_dict_index[0].index < index)
        return OBJECT_DICT_INVALID_INDEX;
    if (object_dict_index[0].index == index)
        return 0;

    if (object_dict_index[object_dict_entries - 1].index > index)
        return OBJECT_DICT_INVALID_INDEX;
    if (object_dict_index[object_dict_entries - 1].index == index)
        return object_dict_entries - 1;

    int  i = 0;
    int  imin = 1;                          // inclusive
    int  imax = object_dict_entries - 1;    // exclusive
    while (imin < imax) {
        i = imin + (imax - imin) / 2;
        if (object_dict_index[i].index > index) {
            imin = i + 1;
        } else
        if (object_dict_index[i].index < index) {
            imax = i;
        } else {
            return i;
        }
    }

    return OBJECT_DICT_INVALID_INDEX;
}

// Internal: Copy an int32_t to (unaligned) CANopen data buffer using the specified CANopen type.
//           Returns the number of bytes copied, or negative error code.
static int copy_i32_into(void *into, const uint8_t type, const int32_t value)
{
    if ( (type == TYPE_U8  && (value < 0 || value > 255)) ||
         (type == TYPE_U16 && (value < 0 || value > 65535)) ||
         (type == TYPE_U24 && (value < 0 || value > 16777215)) ||
         (type == TYPE_U32 && value < 0) ||
         (type == TYPE_I8  && (value < -128 || value > 127)) ||
         (type == TYPE_I16 && (value < -32768 || value > 32767)) ||
         (type == TYPE_I24 && (value < -8388608 || value > 8388607)) )
        return OBJECT_DICT_INVALID_VALUE;

    unsigned char *dest = into;
    unsigned char  data[4] = { ((uint32_t)value),
                               ((uint32_t)value) >> 8,
                               ((uint32_t)value) >> 16,
                               ((uint32_t)value) >> 24 };

    if (type == TYPE_I32 || type == TYPE_U32) {
        dest[0] = data[0];
        dest[1] = data[1];
        dest[2] = data[2];
        dest[3] = data[3];
        return 4;
    }

    if (type == TYPE_I24 || type == TYPE_U24) {
        dest[0] = data[0];
        dest[1] = data[1];
        dest[2] = data[2];
        return 3;
    }

    if (type == TYPE_I16 || type == TYPE_U16) {
        dest[0] = data[0];
        dest[1] = data[1];
        return 2;
    }

    if (type == TYPE_I8 || type == TYPE_U8) {
        dest[0] = data[0];
        return 1;
    }

    return OBJECT_DICT_INVALID_TYPE;
}

// Internal: Copy a CANopen data buffer value of the specified CANopen type to an int32_t.
//           Returns the number of source bytes copied, or negative error code.
static int copy_i32_from(void *from, const uint8_t type, int32_t *const to)
{
    const unsigned char *const src = from;
    uint32_t                   val;
    int                        len;

    switch (type) {

    case TYPE_I32:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8)
            + ((uint32_t)(src[2]) << 16)
            + ((uint32_t)(src[3]) << 24);
        len = 4;
        break;

    case TYPE_U32:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8)
            + ((uint32_t)(src[2]) << 16)
            + ((uint32_t)(src[3]) << 24);
        if (val & UINT32_C(0x80000000))
            return OBJECT_DICT_INVALID_VALUE;
        len = 4;
        break;

    case TYPE_I24:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8)
            + ((uint32_t)(src[2]) << 16);
        if (val & UINT32_C(0x00800000))
            val |= UINT32_C(0xFF000000);
        len = 3;
        break;

    case TYPE_U24:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8)
            + ((uint32_t)(src[2]) << 16);
        len = 3;
        break;

    case TYPE_I16:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8);
        if (val & UINT32_C(0x8000))
            val |= UINT32_C(0xFFFF0000);
        len = 2;
        break;

    case TYPE_U16:
        val =  (uint32_t)(src[0])
            + ((uint32_t)(src[1]) << 8);
        len = 2;
        break;

    case TYPE_I8:
        val = src[0];
        if (val & UINT32_C(0x80))
            val |= UINT32_C(0xFFFFFF00);
        len = 1;
        break;

    case TYPE_U8:
        val = src[0];
        len = 1;
        break;

    default:
        return OBJECT_DICT_INVALID_TYPE;
    }

    // Compiler will optimize this to a simple assignment, but this way we're standards compliant.
    if (val & UINT32_C(0x80000000))
        *to = -(int32_t)(~val) - 1;
    else
        *to = val;

    return len;
}

// Return the index:type:len corresponding to an index, or 0:0:0 if not found.
index_type  object_dict_find(uint_fast16_t index)
{
    index_type  result = { .index = 0, .type = 0, .length = 0 };

    if (index < 1 || index > 0xFFFF)
        return result;

    uint32_t  lock = object_dict_lock();

    int i = index_at(index);
    if (i >= 0)
        result = object_dict_index[i];

    object_dict_unlock(lock);
    return result;
}

// Copy a value from object dict into local storage, applying format conversions.
// Returns the number of bytes copied, or negative for error.
int  object_dict_get_into(void *into, uint_fast16_t index, uint8_t subindex)
{
    if (index < 1 || index > 0xFFFF)
        return OBJECT_DICT_INVALID_INDEX;

    uint32_t  lock = object_dict_lock();

    int i = index_at(index);
    if (i < 0) {
        object_dict_unlock(lock);
        return OBJECT_DICT_INVALID_INDEX;
    }

    int32_t  value;
    uint8_t  type = object_dict_index[i].type;

    if (subindex == 0) {
        if (object_dict_index[i].length == 0) {
            // VAR access.
            value = object_dict_entry[i][0];
        } else {
            // ARRAY length is always UNSIGNED16 (UNSIGNED8 in older versions of CANopen).
            value = object_dict_index[i].length;
            type = TYPE_U16;
        }
    } else
    if (object_dict_index[i].length >= subindex) {
        // Array value access.
        value = object_dict_entry[i][subindex - 1];
    } else {
        object_dict_unlock(lock);
        return OBJECT_DICT_INVALID_SUBINDEX;
    }

    object_dict_unlock(lock);

    return copy_i32_into(into, type, value);
}

// Copy a value from local storage to object dict, applying format conversions.
// Returns the number of bytes copied from, or a negative error code.
int  object_dict_set_from(void *from, uint_fast16_t index, uint8_t subindex)
{
    if (index < 1 || index > 0xFFFF)
        return OBJECT_DICT_INVALID_INDEX;

    uint32_t  lock = object_dict_lock();

    int i = index_at(index);
    if (i < 0) {
        object_dict_unlock(lock);
        return OBJECT_DICT_INVALID_INDEX;
    }

    int32_t  value;
    int  result = copy_i32_from(from, object_dict_index[i].type, &value);
    if (result <= 0) {
        object_dict_unlock(lock);
        return result;
    }

    if (subindex == 0) {
        if (object_dict_index[i].length == 0) {
            object_dict_entry[i][0] = value;
            object_dict_unlock(lock);
            return result;
        } else {
            // Setting array length is not supported.
            object_dict_unlock(lock);
            return OBJECT_DICT_INVALID_SUBINDEX;
        }
    }

    if (subindex > object_dict_index[i].length) {
        object_dict_unlock(lock);
        return OBJECT_DICT_INVALID_SUBINDEX;
    }

    object_dict_entry[i][subindex-1] = value;
    object_dict_unlock(lock);
    return result;
}

int object_dict_new(int32_t *mem, uint_fast16_t index, uint8_t length, uint8_t type)
{
    if (index < 1 || index > 65535)
        return OBJECT_DICT_INVALID_INDEX;

    uint32_t  lock = object_dict_lock();
    int       i;

    if (object_dict_entries >= object_dict_entries_max) {
        object_dict_unlock(lock);
        return OBJECT_DICT_FULL;
    }

    if (object_dict_entries < 1) {
        i = 0;
    } else
    if (object_dict_index[0].index == index) {
        object_dict_unlock(lock);
        return OBJECT_DICT_INDEX_IN_USE;
    } else
    if (object_dict_index[0].index < index) {
        i = 0;
    } else
    if (object_dict_index[object_dict_entries - 1].index == index) {
        object_dict_unlock(lock);
        return OBJECT_DICT_INDEX_IN_USE;
    } else
    if (object_dict_index[object_dict_entries - 1].index > index) {
        i = object_dict_entries;
    } else {
        int  imin = 1;                          // inclusive
        int  imax = object_dict_entries - 1;    // exclusive
        while (imin < imax) {
            i = imin + (imax - imin) / 2;
            if (object_dict_index[i].index > index) {
                imin = i + 1;
            } else
            if (object_dict_index[i].index < index) {
                imax = i;
            } else {
                object_dict_unlock(lock);
                return OBJECT_DICT_INDEX_IN_USE;
            }
        }
        if (object_dict_index[imin].index == index) {
            object_dict_unlock(lock);
            return OBJECT_DICT_INDEX_IN_USE;
        } else {
            i = imin;
        }
    }

    if (i < object_dict_entries) {
        for (int k = object_dict_entries; k > i; k--)
            object_dict_index[k] = object_dict_index[k-1];
        for (int k = object_dict_entries; k > i; k--)
            object_dict_entry[k] = object_dict_entry[k-1];
    }

    object_dict_index[i] = (index_type){ .index = index, .type = type, .length = length };
    object_dict_entry[i] = mem;

    object_dict_entries++;

    object_dict_unlock(lock);
    return 0;
}
The idea here is that if you want to map an UNSIGNED16 VAR at index 0x6040 to int32_t  status_word;, you do
    if (object_dict_new(&status_word, 0x6040, 0x00, TYPE_U16)) failed();
Then, to copy the status word into two bytes at ptr, you do
    if (object_dict_get_into(ptr, 0x6040, 0x00) != 2) failed();
Note that the return value, 2, is the number of bytes copied into ptr, and applies to all integer types.  (I've only tested TYPE_I32, though.)
Similarly, to copy the two bytes at ptr to the status word, you do
    if (object_dict_set_from(ptr, 0x6040, 0x00) != 2) failed();
In the SDO receive/transmit functions, the return value will be negative if you reference an invalid index:subindex pair, and the number of bytes used from the buffer otherwise; this should make it rather easy to implement them.

If you want to know the type at index 0x6040, use object_dict_find(0x6040).  It returns an union of index_type; the .index is either 0x6040 or 0 if not found, and .type specifies the CANopen type (TYPE_U16, above), and .length is 0 for VARs, positive for arrays.
(That is, (index_type){ .index > 0, .type != 0, .length = 0 } refers to a VAR, and (index_type){ .index > 0, .type != 0, .length > 0 }refers to an ARRAY.

Because the indexes are sorted in decreasing order, binary search is used, and time complexity for object access is O(N), where N is the number of indexes registered in the object_dict.  (The number of sub-indexes does not affect that at all.)

I did test VARs and ARRAYs (length=2..253 only) of TYPE_I32, i.e. int32_t  some_data[length]; with
    if (object_dict_new(some_data, index, length, TYPE_I32)) failed();
and that object_dict_get_into(&len, index, 0x00) returns the length into an UNSIGNED16, and that object_dict_get_into(&val, index, subindex) and object_dict_set_from(&val, index, subindex) copy the 4-byte values correctly.

The object_dict_set_from() and object_dict_get_into() always use little-endian byte order, as defined in CiA 301.  The code should follow ISO C11 (actually, ISO C99 with the addition of unnamed structs/unions).

If you have more than one thread, or the object_dict is accessed from interrupt context, then you should use uint32_t lock = object_dict_lock(); and object_dict_unlock(lock); around accesses to the variables added to the object dictionary, to ensure the accesses are atomic, except on architectures where (correctly aligned) accesses to int32_t are always atomic.
« Last Edit: February 26, 2024, 10:59:15 pm by Nominal Animal »
 
The following users thanked this post: SiliconWizard

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17857
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Well I did work out a system in the end. I am heavily limited on RAM with just 16kB for the whole program. I created a struct type that has variables in it to store the index, sub index and a variable for the type that includes other markers too. The struct also has an int32 array of the value and an uint32 array to store the timestamps of each variable as it arrives.

So the type variable has as it's least significant bits the number of bytes, bit 7 is the signed marker, any read only value has it's bit 6 set and I just added for more flexibility, setting bit 5 to alter the list of objects that get setup at the start so that I can choose whether to send the object to the slave node or skip it leaving the default.

I did get left with having to setup a set of catch variables of each type so that if the value is say a signed 16 bit value it is copied from a i16 to the i32 rather than have the raw data copied from the can data as this would mean that the i32 value would become a random positive value if a negative i16 value is simply memcpy() in.

At the start of the program I have to fill the array of objects with the data. I have defined the names of the objects to numbers starting at 0.

It seems to be the best compromise. Decoding an SDO message can indeed take several hundred microseconds, usually around 270 µs, this is compared to a message transfer time of no more than 140 µs. The good news of course is that I can transfer most of the data I need with the PDO protocol.

I suppose I could be more memory efficient by having separate arrays of different parts of the object dictionary. The main thing here would be simplification of the array structure but also that those elements that are set up at the start will probably be optimized into constant values in non volatile memory. So the indexes, sub indexes and variable types could each have their own array where the same object number will index to the data about the same object. They would likely end up in NVM and the arrays of values would go into RAM as well as the timestamp records.
« Last Edit: March 31, 2024, 04:29:15 pm by Simon »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf