Programming help - Bitfields in windows (64b) environment with .NET?

[max_torque]

I'm writing a little program grab some data off a CAN bus, using the popular Peak CanUsb dongles, for which they supply a suitable Windows API.  So far so good, no issues, i'm getting data when i should. 

For people who aren't CAN experts (basic) CAN has an 8 byte (64bit) payload, and individual parameters can be encoded by the bus architect into those bytes.  For example, on a car, you might have the engine ecu sending the engine speed to the dash board to display to the driver, and engine speed might be encoded with a scaling of 1, and offset of zero, and a width of 16bits, and those two bytes might say be located in the 1st and 2nd byte of the CAN message.

Ie, the 8 bytes look like this (where X is don't care)

B0,B1,B2,B3,B4,B5,B6,B7
136,019,x,x,x,x,x,x

Here, we take 19, multiply it by 256 (4864) and add 136, for a total of 5000, ie "5000 rpm" for example

So that's an easy one, because to decode the data from the message payload you just grab the first two bytes, multiply the most significant byte by 2^8 and add it to the least significant byte and it's job done (ignoring the issues surrounds motorola and intel encodings for now....).

But, what if i want to decode values that don't fall on nice byte boundaries. Say a parameter was a 3 bit value (0 to 7), encoded into say the 23 to the 25th bit of the message bytes? It would start in the last two bits of the 3rd Byte, and finish on the 1st bit of the 4th byte.

 In an embedded scenario id have no problem in doing some basic bit shifting and some bit masking or use some bitfield unions to pull out just the bits i'm interested in, but how do i do that in a windows environment?

I'm coding in visual studio, with .NET mostly in a visual basic forms style.

Whats the best way of approaching bitwise operations in a 64b native high level environment?  Anyone have experience or ideas?   

[mariush]

You could just use a 8 byte array or 2 32bit variables.
You can use AND and OR bitwise operations to determine the state of a bit inside a byte.

See https://www.tutorialspoint.com/csharp/csharp_bitwise_operators.htm

You can read the individual bits and then add them to a variable by using shift left...
ex.

byte bit_value = [get bit n from byte] 0 or 1
byte result = result <<1 + bit_value    (shift bits in result one bit to the left, then put 0 or 1 on the last bit)

[max_torque]

One thing i need to be a bit (sic) careful with, is that although CAN 2.0b is a maximum of 8 bytes, and hence could be handled directly in a 64b environment, things like CanFD, which i don't (yet) have to accommodate, have up to 64 bytes of payload, so are outside the direct handling / conversion.

Looking at BitArrays ( https://docs.microsoft.com/en-us/dotnet/api/system.collections.bitarray?view=netframework-4.8 ) which turn an array of bytes into an array of bits, stored as booleans, and can handle arbitrary lengths, could be an option.  I could effective convert the whole data byte array into a bit array, then reference each bit as the index of the bitarray, pull out the number of bits i need to look at, and use that boolean info to set the bits in another parameter. That also gives me a way of handling the motorola/intel formatting, because say my data sits between the 23 and 25th bits, i can simply start at the right end and work forwards or backwards through the array?

Should be reasonably fast, as well, as i will have to decode these messages at a reasonable rate (bus baud rate is a max of 1Mb, average data rate could be say 500 700kbs)

[FrankT]

There is the BitConverter class that converts arrays of bytes to built-in types.  Beware of the endian-ness, which is fixed to the local machine architecture.

[Sam Hobbs]

If it were me I would define structures (structs) for each format. You can do bitwise copies of the data from your buffer to the specific structure or perhaps read directly into the relevant structure. I have done that type of thing in C++ but I could figure out how to do it using C# if I needed to. The thing is that you might get resistance from other programmers; your C++ programmers are not familiar with such techniques that were common when C++ (so before C#) was first designed. I think that C# structures are defined (have limitations) to make that type of thing possible.

[pwlps]

If it were me I would define structures (structs) for each format. You can do bitwise copies of the data from your buffer to the specific structure or perhaps read directly into the relevant structure. I have done that type of thing in C++ but I could figure out how to do it using C# if I needed to. The thing is that you might get resistance from other programmers; your C++ programmers are not familiar with such techniques that were common when C++ (so before C#) was first designed. I think that C# structures are defined (have limitations) to make that type of thing possible.

union-type structures like in C can be created in net using the StructLayout attribute, for example :

using System.Runtime.InteropServices;

   [StructLayout(LayoutKind.Explicit)]
   struct Caster64
   {
      [FieldOffset(0)] public ulong uLongValue;

      [FieldOffset(0)] public uint  LowInt;
      [FieldOffset(4)] public uint  HiInt;
      
      [FieldOffset(0)] public ushort Word0;
      [FieldOffset(2)] public ushort Word1;
      [FieldOffset(4)] public ushort Word2;
      [FieldOffset(6)] public ushort Word3;
      
      [FieldOffset(0)] public byte byte0;
      [FieldOffset(1)] public byte byte1;
      [FieldOffset(2)] public byte byte2;
      //... etc.       
   }

in VB the syntax is similar:

<StructLayout(LayoutKind.Explicit)> _
Structure Caster64
   <FieldOffset(0)> Public uLongValue As ULong
   <FieldOffset(0)> Public LowInt As UInteger
   <FieldOffset(4)> Public HiInt As UInteger


   '... etc.       
End Structure

 In an embedded scenario id have no problem in doing some basic bit shifting and some bit masking or use some bitfield unions to pull out just the bits i'm interested in, but how do i do that in a windows environment?

I'm coding in visual studio, with .NET mostly in a visual basic forms style.

In C#/VB.NET you have all the same bit shift and bitwise logical operators as in C, the only differences are: 1) the compiler is more picky about type checking so you need explicit type casts almost everywhere, 2) rules for shifting signed integers are different:
https://stackoverflow.com/questions/1499647/shifting-the-sign-bit-in-net

[elecman14]

You could just store your data in a custom class in a list depending on how much data you plan on collecting. IF you would not run out of memory you could build a class that looks exactly like what you are looking to store. This could be consumed and searched with LINQ