Author Topic: Lwip send udp packets larger than MTU  (Read 9150 times)

0 Members and 1 Guest are viewing this topic.

Offline tanffnTopic starter

  • Contributor
  • Posts: 17
Lwip send udp packets larger than MTU
« on: May 16, 2015, 01:38:03 pm »
Dear Forum,
I know I am asking quite a bit of questions, the newbe phase will end, eventually :) and I'll be able to give back.

Using ST STM32F427, RTOS, lwip and sending data over UDP, If I try to send data small than the MTU:
   e.g:   netbuf_ref(buf, dataPointer, 1000);
Everything works fine.

If the data is bigger then MTU (e.g 4000):
   1. I don't see any data being sent (using wireshark).
   2. The lwip buffer will ran out (under low_level_output()) after several iterations (ETH_TXBUFNB = 8)
   
In stackoverflow I found this "solution":
   Delete the definition of CHECKSUM_BY_HARDWARE
   http://stackoverflow.com/questions/20851391/lwip-send-udp-packets-larger-than-mtu-but-my-pc-can-not-reassemble-them

Questions:
1. Can I send data bigger than MTU with Hardware checksum? How?
   Should I break my data into segments?
   How can I insert markers into my stream while avoiding unnecessary copy
      (Write header to tempBuff, copy dataBuff to tempBuff, in low_level_output copy tempBuff to Tx_Buff)

2. Is there a zero-copy implementation for Cube?
   Despite reading ST's answer that the copy time is insignificant it is consumes time and unnecessary memory.

3. Any tips for proper system configuration (e.g how much mem to allocate, MEM_SIZE, MEMP_NUM_PBUF)
 

Offline tanffnTopic starter

  • Contributor
  • Posts: 17
Re: Lwip send udp packets larger than MTU
« Reply #1 on: May 20, 2015, 06:49:50 pm »
No lwip large packets tips?  :(
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 4067
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: Lwip send udp packets larger than MTU
« Reply #2 on: May 20, 2015, 06:55:01 pm »
lwip is quite a mess. Several different api's with all their features and drawbacks.
If I recall using it correctly, lwip does have fragmentation options, but it's disabled by default. Receiving fragments of tcp should work though.
Also, when fragmenting UDP, keep in mind that there is no guarantee whatsoever in what order the packets will arrive. If they arrive.
 

Offline hli

  • Frequent Contributor
  • **
  • Posts: 255
  • Country: de
Re: Lwip send udp packets larger than MTU
« Reply #3 on: May 20, 2015, 08:46:02 pm »
UDP, by design, doesn't support fragmentation. Its datagram-orientent, not a stream-oriented protocol: http://en.wikipedia.org/wiki/User_Datagram_Protocol
So you will need to split up your data packets into smaller ones. Remember - UDP is a lossy protocol, so any packet might be lost, and the order is not preserved.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Lwip send udp packets larger than MTU
« Reply #4 on: May 20, 2015, 10:42:27 pm »
Plan 9's rudp might be light enough for your needs:

http://plan9.bell-labs.com/sources/plan9/sys/src/9/ip/rudp.c

Hmm, now that I think of it, what ever is happening with Plan 9? haven't heard of it for a while

http://plan9.bell-labs.com/plan9/

 

Offline helius

  • Super Contributor
  • ***
  • Posts: 3632
  • Country: us
Re: Lwip send udp packets larger than MTU
« Reply #5 on: May 21, 2015, 03:55:29 am »
UDP, by design, doesn't support fragmentation. Its datagram-orientent, not a stream-oriented protocol: http://en.wikipedia.org/wiki/User_Datagram_Protocol
So you will need to split up your data packets into smaller ones. Remember - UDP is a lossy protocol, so any packet might be lost, and the order is not preserved.
On the contrary, fragmentation is a feature of the IP protocol (Layer 3). It's completely independent of transport type (TCP or UDP). Every working Internet stack can transmit and receive UDP packets up to the maximum size, ~64KB.
It's also important to note that a UDP datagram will always have its fragments in order, because fragment reassembly is done before delivery to the application.
« Last Edit: May 21, 2015, 03:58:32 am by helius »
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 4067
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: Lwip send udp packets larger than MTU
« Reply #6 on: May 21, 2015, 05:41:58 am »
Yes, and the fragmentation is something which can be disabled in lwip.
Code: [Select]
/**
 * IP_REASSEMBLY==1: Reassemble incoming fragmented IP packets. Note that
 * this option does not affect outgoing packet sizes, which can be controlled
 * via IP_FRAG.
 */
#ifndef IP_REASSEMBLY
#define IP_REASSEMBLY                   0
#endif
/**
 * IP_FRAG==1: Fragment outgoing IP packets if their size exceeds MTU. Note
 * that this option does not affect incoming packet sizes, which can be
 * controlled via IP_REASSEMBLY.
 */
#ifndef IP_FRAG
#define IP_FRAG                         1
#endif
 

Offline tanffnTopic starter

  • Contributor
  • Posts: 17
Re: Lwip send udp packets larger than MTU
« Reply #7 on: May 21, 2015, 07:02:39 am »
Fantastic, thank you all for the replies!
Jeroen3 I saw that for me the flag was indeed defined as zero, I'll try it later tonight.
 

Offline hamster_nz

  • Super Contributor
  • ***
  • Posts: 2803
  • Country: nz
Re: Lwip send udp packets larger than MTU
« Reply #8 on: May 21, 2015, 08:05:48 am »
Friends don't let friends fragment their UDP packets....

Don't do it. You are opening up a world of hurt, with weird equipment issues and packet loss, strange timeouts and resource starvation. Rework you protocol to.use 576 bytes or less per packet.

Not doing so is like sending a long letter by writing it on multiple postcards to be delivered by random (but generally trustworthy) strangers.

Much better to just send lots of short messages, each on their own postcard....
Gaze not into the abyss, lest you become recognized as an abyss domain expert, and they expect you keep gazing into the damn thing.
 

Offline Scrts

  • Frequent Contributor
  • **
  • Posts: 797
  • Country: lt
Re: Lwip send udp packets larger than MTU
« Reply #9 on: May 21, 2015, 08:53:35 am »
Agreed to guys here regarding not doing any reassembling of UDP packets. It's just wrong :--
I've done an MPEG-TS streamer, which did up to 7 MPEG-TS packets of 188 bytes, so 1316 bytes per UDP packet and it worked perfectly fine. You don't need to do UDP checksum. Write 0x0000 there. It's useless anyway.
 

Offline tanffnTopic starter

  • Contributor
  • Posts: 17
Re: Lwip send udp packets larger than MTU
« Reply #10 on: May 22, 2015, 09:44:22 am »
Thank you Scrts and hamster_nz, I'll probably end up doing what you've suggested.
Breaking the message into several small chunks.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf