How to send 16 bit data using LwIP?

[rcbuck]

I'm working on a custom board that uses a STM32F407 to send data across the network to another STM32F407 board using LwIP. I created an array of 100 bytes to use as a test program. It works fine as long as the array is declared as uint8_t. I am able to send 100 byres and the other board receives all 100 bytes correctly.

If I declare the array as uint16_t, I only receive half of the bytes. I changed the array size to 50 as a further test. That results in only 25 bytes being received by the remote board.

If I look at the ethernetif.c file in LwIP, I see the following 2 lines that indicate what the problem is.
__ALIGN_BEGIN uint8_t Rx_Buff[ETH_RXBUFNB][ETH_RX_BUF_SIZE] __ALIGN_END; /* Ethernet Receive Buffer */
__ALIGN_BEGIN uint8_t Tx_Buff[ETH_TXBUFNB][ETH_TX_BUF_SIZE] __ALIGN_END; /* Ethernet Transmit Buffer */

If I change the uint8_t to uint16_t and try to compile I get these warnings:
warning: passing argument 3 of 'HAL_ETH_DMARxDescListInit' from incompatible pointer type [-Wincompatible-pointer-types]
warning: passing argument 3 of 'HAL_ETH_DMATxDescListInit' from incompatible pointer type [-Wincompatible-pointer-types]

I have been working on this problem for a few hours but can't find a solution. The only thing I can think of (haven't tried) is to break the 16 bit data into two 8 bit data bytes and re-assemble on the remote end. But that is not really a desirable solution.

Has anyone else encountered this problem and found a work around?

[mark03]

If you want to be really explicit about it, try looking up C unions.  But that shouldn't be necessary.  I've never used LwIP and don't quite follow your reasoning, but at some point you are undoubtedly passing a pointer to the array of data that will be sent.  If this works when the array is an array of uint8_t, all you need to do is make your own array of uint16_t, then cast your pointer to (uint8_t *).  It's all just 32-bit words in the end.

[rcbuck]

Yes, I am pointing to an array of data that needs to be sent. The array is an uint16_t array. The LwIP  is wanting to fill its buffer with uint8_t values. That is what the two __ALIGN_BEGIN statements indicate.

I tried casting the array as (uint8_t *) as you suggested but I still get the same results.

The strange thing is, in the case of the array[100], the first 50 values received by the remote end are correct. The next 50 values all have a value of 128. If I change the array to 50, the first 25 values are correct and the last 25 all have a value of 128.

[rcbuck]

Additional information.

I reduced the size of my array down to 40 bytes to make it easier to work with. When I send the data from the array to the LwIP engine, I have to use memcpy to put the data into a structure that LwIP understands. That is done with this statement:
memcpy(pb->payload, adcbuff, 40);
LwIp then mangles the array values into something strange. That is what is shown in the attachment.

When the data is received by the remote board and I look at the incoming data, I see that pb->payload contains the same strange data that was seen on the transmitting board. I use this statement to retrieve the data:
memcpy(&datain, pb->payload, 40);
The datain array then contains the first 20 bytes that were in the array on the transmit side.

So LwIP has no trouble accepting the 16 bit values but somehow it is only capable of sending half of the values that was given to it to transmit.

The array is loaded with 40 bytes of data starting with a value of 1000. Each additional byte is incremented by 25, so I am sending 1000, 1025, 1050, 1075, 1100, etc.

[greenpossum]

We can't tell unless you show us the code, preferably of a cut-down example.

One thing to note is the the memcpy only copies the data to the buffer. Somewhere else you would be telling LwIP the length of the buffer you want to transmit. Are you setting it to the number of uint_16 elements in the array? It should be the number of bytes, i.e. sizeof(array), or, number of elements * sizeof(unit_16).

Edit: Another thing to pay attention to, if you ever send data between different CPU architectures, is the endianess of the words of those architectures. But for this case, it's not an issue.

[rcbuck]

Cutdown version of the code follows.

Transmit side:

Code: [Select]

uint16_t adcbuff[40]; // global vars
uint16_t count;
uint8_t x = 1000;

int main(void)
{
    for(buffcount=0; buffcount<40; buffcount++) // populate buffer
    {
        adcbuff[buffcount] = x;
        x += 25;
    }

    while(1)
    {
        if(HAL_GPIO_ReadPin(SW1_GPIO_Port, SW1_Pin) == 0) // wait for switch operation
        {       // to send data to remote
            udp_test_send();
        }

        MX_LWIP_Process();
    }
} // end main

void udp_test_send(void)
{
    struct pbuf * pb = pbuf_alloc(PBUF_TRANSPORT, sizeof(adcbuff), (uint16_t)PBUF_POOL);
    if(!pb)
    {
        //error!
        return;
    }

    memcpy(pb->payload, adcbuff, sizeof(adcbuff)); // copy buffer to LwIP pbuf buffer
    asm("NOP");
    udp_sendto(test_pcb, pb, &test_destIp, test_port); // udp_sendto is a LwIP function

    MX_LWIP_Process(); // process transmit data, this is a LwIP funciton

    HAL_GPIO_TogglePin(GRN_GPIO_Port, GRN_Pin); // indicates udp_test_send function
    asm("NOP");                                 // was called and executed
    pbuf_free(pb);
}
That is the complete transmit code. I can attach the receive code if needed, but it basically just receives the incoming data and puts it into a buffer. I have a breakpoint set in the code so it stops when data comes in and I can view the data to see what was received.

If I change uint16_t adcbuff to unit8_t adcbuff[] and populate it with the numbers 1 to 100, I receive 1 to 100 on the remote board. I show sizeof(adcbuff) in this code but the final code that will be replaced with 100 since I will always send 100 bytes. I will always be sending 16 bit data to the remote board.

[greenpossum]

According to https://www.nongnu.org/lwip/2_0_x/group__pbuf.html#ggaee1baa59bb2f85ba575b5a8619ac1ebfa21116654fbab6d5a4dfeb87a1bb8f0ba in the pbuf_alloc call you should not be using PBUF_POOL for TX. Try PBUF_RAM. Also you should not cast the type argument, it already expects an argument of type enum pbuf_type, so the PBUF_* enum constants are already of the right type.

[rcbuck]

greenpossum, I had previously tried using PBUF_RAM but it made no difference. I removed the cast to the type argument. I had accidentally left that in after running another test.

I tested breaking the 40 uint16_t bytes into 80 uint8_t bytes on the transmit board. If I create an array consisting of 80 uint8_t bytes on the receive board, I receive the same 80 uint8_t bytes that were sent. Interestingly though, if I create a 40 uint16_t byte array and perform a memcpy to that array, I get the original 40 uint16_t bytes that were created on the transmit side. There is no need to re-assemble the 8 bit data into 16 bit data. Somehow LwIP does that automatically.

You would think that LwIP would handle the translation between 8 bit and 16 bit data internally. However, it only works on the receive side for some reason.

[greenpossum]

There is no assembly or reassembly of ints and bytes happening. C is very low level. You are just passing a memory address to memcpy. In the case of a pointer to an int location it may be aligned on an even address, or not. The reason memcpy can accept any kind of pointer is because the prototype specifies void * for the source and destination and these are compatible with any pointer so no cast is needed.

What I don't understand is how LWIP knows the length of the data. Certainly memcpy doesn't convey this information to LWIP. Maybe there's a call somewhere where you set the length of the data.

Edit: It seems to me that you should be using the pbuf_take function to copy data from your app to the LwIP buffer, rather than doing it yourself by using the payload pointer member. http://www.nongnu.org/lwip/2_0_x/group__pbuf.html#gad1e31e370271335b197272af2724ca85 When you call a LwIP function to do the copy, it can set the UDP packet length from the pbuf structure appropriately, which memcpy does not.

[HwAoRrDk]

What I don't understand is how LWIP knows the length of the data. Certainly memcpy doesn't convey this information to LWIP. Maybe there's a call somewhere where you set the length of the data.

Edit: It seems to me that you should be using the pbuf_take function to copy data from your app to the LwIP buffer, rather than doing it yourself by using the payload pointer member. http://www.nongnu.org/lwip/2_0_x/group__pbuf.html#gad1e31e370271335b197272af2724ca85 When you call a LwIP function to do the copy, it can set the UDP packet length from the pbuf structure appropriately, which memcpy does not.

Yes, I think you're right. The pbuf structure has a 'len' field that OP's code doesn't set (at least, not as posted).

[cgroen]

I assume you are using the raw API of LwIP ?

I do like this:
(remember pbufs can/will be a chain of buffers!)

//---------------------------------------------------------------------------------------
// Send data to host
// returns 0 if ok
//---------------------------------------------------------------------------------------
int sendDataHostCom(unsigned char *buffer, unsigned int length) {
    struct pbuf  *pBuf;

   // Allocate pbuf (might end up being a chain of buffers!)
   pBuf = pbuf_alloc(PBUF_TRANSPORT, length, PBUF_POOL);
   if (!pBuf) {
      messageDebug(DBG_ERR, __MODULE__, __LINE__, "error allocating buffer");
      return 1;
   }

   // The pBuf we get can be a chain of buffers
   int bytesLeft = length; // Number of bytes we still need to move to buffer(s) 
   struct pbuf *packetTempBuffer; // used to traverse the (possible) list of buffers
   int chunk; // Number of bytes we copy to the current buffer   
   int index = 0;  // Index into the source buffer
   
   packetTempBuffer = pBuf; 
   while ( (bytesLeft) && (packetTempBuffer != NULL) )  { 
      chunk = bytesLeft; 
      if ( chunk > packetTempBuffer->len ) { 
         chunk = packetTempBuffer->len; 
      } 
      // copy one part 
      memcpy(packetTempBuffer->payload, &buffer[index], chunk); 
      // next buffer in chain (if any) 
      packetTempBuffer = packetTempBuffer->next; 
      bytesLeft -= chunk; 
      index += chunk;   
   };        

   // Lock stack
   LOCK_TCPIP_CORE();
      udp_sendto(udphost_raw_pcb, pBuf, (const ip_addr_t *)&ip_host, getSysConfig()->hostPort);
   UNLOCK_TCPIP_CORE();   
   
   pbuf_free(pBuf);
   return 0;
}



//---------------------------------------------------------------------------------------
// Callback function for received data from Host
//---------------------------------------------------------------------------------------
static void rxUDP(void *arg, struct udp_pcb *upcb, struct pbuf *p, const ip_addr_t *addr, u16_t port) {
    struct pbuf  *pBuf;
   int rc, len;
   // if packet is valid
   if (p != NULL) {
      
      // WARNING: p can be a chain of buffers
      len=p->tot_len;
      // Check we have room for all data from all chained pbuf's
      if (len < sizeof(rxBuffer)) {
         // check if data occupies a chain of pbuf's
         struct pbuf *packetTempBuffer; // used to traverse the (possible) list of buffers
         int chunk;       // Number of bytes we copy to the current buffer   
         int index = 0;  // Index into the destination buffer
         
         packetTempBuffer = p; 
         while (packetTempBuffer != NULL) { 
            //messageDebug(DBG_WAR, __MODULE__, __LINE__, "HostCom::pbuf, (pbuf=0x%08X, next=0x%08X, this len=%i, tot_len=%i), port=%i", packetTempBuffer, packetTempBuffer->next, packetTempBuffer->len, p->tot_len, port);
            // Copy the chunk of data from this pbuf
            chunk = packetTempBuffer->len; 
            memcpy(&rxBuffer[index], packetTempBuffer->payload, chunk); 
            // next buffer in chain (if any) 
            packetTempBuffer = packetTempBuffer->next; 
            index += chunk; // Index into destination buffer 
         };              
//         messageDebug(DBG_WAR, __MODULE__, __LINE__, "HostCom::Packet Received, PCB=0x%08X, (pbuf=0x%08X, next=0x%08X, len=%i, tot_len=%i), port=%i", p, p->next, p->len, p->tot_len, port);
         rxLength=p->tot_len;
         os_evt_set(0x0001, tidWaiter);
      }
      pbuf_free(p);
   }
}

[rcbuck]

After a few more hours of testing I have figured out how LwIP processes the transmit and receive data. Working code is attached for both processes.

I think mark03's comments about it just being 32-bit words is the key. The comments greenpossum made about assembly or reassembly is true. I was doing the assembly and reassembly in my user code. LwIp does know about the length of the data from memcpy but I had the memcpy statement wrong. Thanks HwAoRrDk for pointing that out. It should have been memcpy(pb->payload, datain, 40); But 40 is not actually the data length as far as LwIP is concerned. Here is where mark03 is correct about "words".

The adcbuffer[] is declared as uint16_t. But LwIP wants 8 bit data to transmit. What I found was that when I copied data from the adcbuff[] to the pbuff, I had to copy twice as many bytes as what the adcbuff[] actually held. Since I was sending 40 uint16_t bytes, I had to actually copy 80 bytes to the pbuf memory.

The STM32F4xx Family Reference manual states this: "The embedded SRAM can be accessed as bytes, half-words (16 bits) or full words (32 bits)." This probably means uint16_t arrays are stored in contiguous half-word locations. That would explain why copying 80 bytes gives the desired transmit action. I ended up using pbuf_take to copy the data instead of memcpy. I tested both and they both work the same but I believe pbuf_take is probably better.

Looking at the code cgroen posted gave me a couple of hints as to how to approach the problem. I don't have to worry about chained pbuffs. I have 1100 bytes allocated for each pbuff and will only be sending 100 bytes at a time. Therefore I don't have to traverse the pbuf data.

In the receive code you can see I determine the length of the incoming pbuf data. That number tells me how many bytes I have to copy into the datain[] buffer. Again the number here is twice the actual datain[] buffer size since the incoming data is in 8 byte chunks.

Any further comments would be welcome. I have managed to learn more about LwIP than what I intended to know. But it may come in handy in the future. Maybe this chain of posts can help someone else who is having problems with LwIP.

Code: [Select]

// UDP TRANSMIT CODE
/* USER CODE BEGIN PV */
uint16_t adcbuff[40];
uint16_t adcvalue;
uint8_t buffcount;
static struct udp_pcb * test_pcb;
static uint16_t test_port = 3333;
/* USER CODE END PV */

/* USER CODE BEGIN PFP */
void test_recv(void *arg, struct udp_pcb *pcb, struct pbuf *p, const ip_addr_t *srcAddr, uint16_t port);
/* USER CODE END PFP */

int main(void)
{
  /* USER CODE BEGIN 1 */
  uint16_t count, x;
  uint8_t ipaddr[] = {172,16,1,122}, maskaddr[] = {255, 255, 255, 0}, gateaddr[] = {172,16,1,1};
  IP_ADDR4(&test_destIp, 172, 16, 1, 123);
  /* USER CODE END 1 */
 
  for(buffcount=0; buffcount<40; buffcount++)
  {
adcbuff[buffcount] = x; // populate array
x += 25;
  }
 
  while(1)
  {
if(HAL_GPIO_ReadPin(SW1_GPIO_Port, SW1_Pin) == 0)
{
  udp_test_send();
  HAL_Delay(40); // debounce switch
    }

MX_LWIP_Process(); // to check for Ethernet traffic - MX_LWIP_Process function is declared inside lwip.h
  }
 
void udp_test_send(void)
{
  struct pbuf * pb = pbuf_alloc(PBUF_TRANSPORT, 80, PBUF_POOL); // set pbuf size to 80
  if(!pb)
  {
    //error!
    return;
  }

  pbuf_take(pb, adcbuff, 80);
  udp_sendto(test_pcb, pb, &test_destIp, test_port);
  MX_LWIP_Process(); // process send data immediately
 
  HAL_GPIO_TogglePin(GRN_GPIO_Port, GRN_Pin);
  pbuf_free(pb);
} 


// UDP RECEIVE CODE
/* USER CODE BEGIN PV */
uint16_t datain[40];
uint16_t incount = 0, outcount = 0;
static struct udp_pcb * test_pcb;
static uint16_t test_port = 3333;
ip4_addr_t dest_Ip;
ip4_addr_t local_Ip;
/* USER CODE END PV */

/* USER CODE BEGIN PFP */
void test_recv(void *arg, struct udp_pcb *pcb, struct pbuf *pb, const ip_addr_t *srcAddr, uint16_t port);
/* USER CODE END PFP */

int main(void)
{
  uint8_t ipaddr[] = {172,16,1,123}, maskaddr[] = {255, 255, 255, 0}, gateaddr[] = {172,16,1,1};
  IP_ADDR4(&dest_Ip, 172, 16, 1, 122);
  IP_ADDR4(&local_Ip, 172, 16, 1, 123); // set IP of this board for use in bind
                                        // operation in udp_test_int function

  while(1)
  {
    MX_LWIP_Process(); // check for Ethernet traffic - MX_LWIP_Process function declared inside lwip.h
  }
}

// This is a callback function that is called every time ethernet traffic is received
void test_recv(void *arg, struct udp_pcb *pcb, struct pbuf *pb, const ip_addr_t *srcAddr, uint16_t port)
{
  uint8_t len;
 
  if(pb != NULL)
  {
    len = pb->len; // len shows 80 here when running debug session
    memcpy(datain, pb->payload, len); //80);
    HAL_GPIO_WritePin(RED_GPIO_Port, RED_Pin, 1); //DEBUG CODE
    pbuf_free(pb);
  }
}

[ajb]

Like memcpy, pbuf_take receives a void pointer which means that it has no way of knowing the size of the object you want it to copy, hence you must give it a size in bytes.  This will work whether you want it to copy a single 37 byte struct or an array of 32 bit ints, as long as you tell it the correct size. 

HOWEVER, as a general rule, you need to be careful about using memcpy-like functions when moving multi-byte objects, particularly structs and especially arrays of structs, because the layout in memory may not be what you would assume, and may vary from platform to platform.  The first thing you will need to deal with is endianness, which is why we have the hton and ntoh families of utility functions (or macros) as part of most network stacks.  These functions translate multi-byte values from 'host to network' and 'network to host' order, and should be defined appropriately for each compilation target (so if your host layout is the same as the network layout these functions should disappear be optimized out since they don't need to do anything).  As long as you're diligent about using the hton/ntoh you won't have to worry about endianness ever, even when porting to a platform with a different convention.  Of course if you're only communicating between platforms that have the same endianness, you can skip this, but do so under peril of making the next person to deal with your code (which could be you!) rather upset with you.

(Aside, as someone who has had to deal with this problem, please please PLEASE do not mix up endianness within a protocol you expect other people to interact with.  It's super annoying!)

The other thing you need to worry about is struct layout and padding.  The size of a struct in memory is not necessarily equal to the sum of the size of its members, because compilers may insert padding bytes to align members to word sizes.  This is inherently implementation- and optimization-dependent!  Likewise, an array of structs may occupy more memory than the sum of the size of its members because of structs being allocated on word boundaries, leaving de facto padding between structs.  The size of an array of structs in memory may be larger than the size of the struct times the number of elements in the array.  Again, if you are only communicating between platforms where you know that the memory layout will be exactly the same, you can sometimes ignore this, but it can be asking for trouble.  Furthermore, since memcpy has no way of knowing which bytes within the target area are padding and which are real data, you can wind up sending the contents of memory that has not been initialized or has not been re-initialized since its last use--this is an information leak, and can be the basis of a security vulnerability.  Sure, the chances of this becoming a real problem are small, and may be inconsequential on a small embedded project anyway, but this is the sort of thing you as a programmer MUST be cognizant of and should be prepared to program defensively against.

So as a rule, I would absolutely recommend against using memcpy-type functions (including pbuf_take) to marshal data across platform boundaries--and moving data from the host into a network stack always counts as crossing a platform boundary.   Instead, I would always handle the translation from a data object in memory to a protocol buffer in memory explicitly.  If you are dealing with structs, write functions to convert them from their host-dependent memory layout to a byte array in a protocol buffer and vice versa, and always do so by accessing the members of the struct explicitly in the manner appropriate to your platform.  Yes, it's more code to write and maintain, but it makes your intent clear both to the compiler and to the next person to read your code (probably you!), and in a well-architected application you can centralize your data marshaling so you only have to maintain it in one place.  As a bonus, it's also much easier to port an explicit and verbose marshaling function across platforms/languages, because you won't have to worry about things that are implicit in the platform's behavior that may or may not be important to the correct function of the program.  Once you've appropriately marshaled the data into the network layout you can still use memcpy or pbuf_take to move it from eg an intermediate buffer into a pbuf (or maybe you keep a copy of the protocol packet in memory if you're sending out repeated keepalive messages or something).

If you are creating a network protocol of any complexity, don't just start copying memory into a pbuf; start by defining the protocol, including packet structure down the byte level with explicitly-defined endianness, and program to that specification.  Examine the network traffic in Wireshark to make sure the layout in the datagram is correct.  If you're really keen, write a Lua dissector for it--this will especially be appreciated if other people will use your protocol, but it will also help you crosscheck your own work.  This is especially useful if you are using the same (or very similar) target to develop both ends of the protocol, as this can hide endianness or padding issues you may have missed.

[nctnico]

If you are creating a network protocol of any complexity, don't just start copying memory into a pbuf; start by defining the protocol, including packet structure down the byte level with explicitly-defined endianness, and program to that specification.  Examine the network traffic in Wireshark to make sure the layout in the datagram is correct.  If you're really keen, write a Lua dissector for it--this will especially be appreciated if other people will use your protocol, but it will also help you crosscheck your own work.  This is especially useful if you are using the same (or very similar) target to develop both ends of the protocol, as this can hide endianness or padding issues you may have missed.

I second this! Never map structs onto buffers! It will go wrong in one way or another.

If I declare the array as uint16_t, I only receive half of the bytes. I changed the array size to 50 as a further test. That results in only 25 bytes being received by the remote board.

That is because an array of 20 16bit elements is 40 bytes long.

Pseudo code to send the entire array:
int16_t my_array[20];
send_data((int8_t*) my_array, sizeof(my_array));

Using sizeof() makes sure all the data is copied even if it was padded. But read what I've wrote above. Doing it this way is not compatible with other platforms / compilers.

[rcbuck]

Instead, I would always handle the translation from a data object in memory to a protocol buffer in memory explicitly

Is this not what pbuf_take is doing according to the definition? It was written as part of the LwIP stack so I would think it would handle passing the data to the protocol buffer internally. LwIP is far too complex to try to figure out what they are doing with everything internally. It has been around for almost 20 years and everybody uses it so you have to somewhat trust that it works.

I am using identical boards on both ends of the link so endianness is not an issue. The only difference is whether the board is programmed with transmit code or receive code.

That is because an array of 20 16bit elements is 40 bytes long.

If you read my last post you will see that I figured that out. An array of 20 32-bit elements would be 80 bytes long.

[greenpossum]

I had the memcpy statement wrong. Thanks HwAoRrDk for pointing that out. It should have been memcpy(pb->payload, datain, 40); But 40 is not actually the data length as far as LwIP is concerned. Here is where mark03 is correct about "words".

If you look back at my first reply to you, #4, I asked: Are you setting it to the number of uint_16 elements in the array? It should be the number of bytes, i.e. sizeof(array), or, number of elements * sizeof(unit_16). Both memcpy and pbuf_take work in bytes.

There's another thing you should be doing in your code and that is to check the result of pbuf_take. It will return an error if there isn't enough space to copy your data into. memcpy will merrily write outside the buffer if it isn't large enough for your data.

In general if an API provides a way to do what you want without poking into the internals of the API's data structures, you should use it. If pbuf had been written in C++, members like payload and len would probably have been declared private or protected.

[rcbuck]

Yes, for both transmit and receive purposes I am manually setting it to the number of bytes. I will modify my code to use the sizeof operator to get the number of bytes. That way if I change the number of bytes I am working with in the future, I won't have to make changes to the pbuf copy statements.

[ajb]

Instead, I would always handle the translation from a data object in memory to a protocol buffer in memory explicitly

Is this not what pbuf_take is doing according to the definition?

No and no.  The first no is because pbuf_take takes a pointer to void.  Any information about the size or layout in memory of the data structure you pass to it is lost when your pointer-to-whatever is converted to a pointer to void.  This is why you can't dereference a void pointer, because dereferencing requires knowing the size and layout of the item in memory that the pointer addresses.  So all pbuf_take knows is that you want it to copy X bytes starting at memory address Y.  The second no is because even if LWIP had a "pbuf_take_u16_array" function or whatever, it still has no idea how you want it to marshal that data into a datagram.  It doesn't know for example whether those 16-bit values should be transmitted little-endian or big-endian, or if the array should be padded to a 32-bit boundary, or whatever.  The way that data is arranged in a datagram is determined by whatever protocol is operating at the datagram (or TCP stream) level, and LWIP has no knowledge of that.  LWIP just knows which bytes you've told it to transmit.  So it's up to you as the person building that protocol on top of LWIP to either handle all of those details explicitly by writing your own function that translates a data object in (a host- and compiler-dependent layout in) memory into an array of bytes ready for the wire, or to leave them to the vagaries of the compiler and the platform implementation and hope for the best. The more complex your application/communication needs are, the more risky he latter approach is. 

LwIP is far too complex to try to figure out what they are doing with everything internally. It has been around for almost 20 years and everybody uses it so you have to somewhat trust that it works.

Yes, LWIP is complex and hard to trace out its behavior, but you have to understand what it does and does not do for you.  LWIP's responsibility for correct behavior stops abruptly at the datagram or TCP stream level; once you get to that level you are on your own!

I am using identical boards on both ends of the link so endianness is not an issue.

Not an issue....yet    Maybe it will never be an actual problem for your application, but it's always an issue in the sense of something you'll need to be cognizant of.

[nctnico]

I am using identical boards on both ends of the link so endianness is not an issue.

Not an issue....yet    Maybe it will never be an actual problem for your application, but it's always an issue in the sense of something you'll need to be cognizant of.

As a side note: The most trouble-free & easiest to implement and debug protocols are text based. Just look at HTTP, SIP, SCPI / LXI for example.