LwIP stack - how should it be used? [FreeRTOS + LwIP + STM32F429I] [SOLVED!]

[FlashEF]

Hi,

I am trying to get an Ethernet connection with STM32F429I MCU using DP83848 PHY. So far I've managed to design an Ethernet shield for STM32F429I Discovery board, using full MII + interrupt, and get FreeRTOS up and running. I have utilised STM32CubeMX software to generate the initialisation code. All works fine, got myself a multi-threaded blinky. Physical layer board will arrive next week, so now I am analyzing LwIP. My first goal is to get an TCP/IP echo server running. The problem is - quality of LwIP examples and the fact that each one does things differently.

Here is my list of what I think should happen to get an echo server. If anyone knowledgeable of the topic could confirm that or correct it, it would be great.
1. RTOS starts, initializes hardware.
2. Some threads doing other stuff are initialized (higher priority, now - placeholder blinky).
3. Default thread starts (normal priority), it calls MX_LWIP_Init() func, it:
  3.1 Initializes ethernet hardware (MII pins),
  3.2 Talks to PHY and asks it to generate interrupt on link status change (to be used separately for reconnections on lost link),
  3.3 Starts ETH_MAC thread ethernetif_input() with realtime priority,
  3.4 Runs DHCP IP negotiation.
4. Now in the default thread I can use netconn API to:
  4.1 Create a new connection - netconn_new() - set connection type to TCP
  4.2 Bind my new connection to my IP negotiated by DHCP - netconn_bind()
  4.3 In case of server app use netconn_listen() to wait for connection and then netconn_accept() to accept it
  4.4 Get the data form accepted connection by netconn_recv() in a loop, and echo it back using netconn_write()
5. Do some cleanup in case of lost connection.

Ok, that seems straightforward but there are some confusing things I have found in the examples:
A. I've seen DHCP run in a separate thread with 250 ms thread sleep. I presume this is to be able to re-do the negotiation if link is broken? (cable disconnected?). I run the negotiation once only at least for now, and I plan to re-run it in case of link change interrupt.

B. To what I understand the only thing that needs to be run is the stack thread and the application thread. Netconn API communicates with  the stack from application thread (in this case the default thread) by a mail queue - and that is it. However in the generated init code, just next to  MX_LWIP_Init(), I have found a function MX_LWIP_Process() that should (??) be run in the default/application thread in a loop and by description seems applicable even to RTOS scenario:

Code: [Select]

/**
 * ----------------------------------------------------------------------
 * Function given to help user to continue LwIP Initialization
 * Up to user to complete or change this function ...
 * Up to user to call this function in main.c in while (1) of main(void)
 *-----------------------------------------------------------------------
 * Read a received packet from the Ethernet buffers
 * Send it to the lwIP stack for handling
 * Handle timeouts if NO_SYS_NO_TIMERS not set and without RTOS
 */
void MX_LWIP_Process(void)
{
  ethernetif_input(&gnetif);
       
  /* Handle timeouts */
  #if !NO_SYS_NO_TIMERS && NO_SYS
    sys_check_timeouts();
  #endif
   
}

The same function is called, that is already running in a separate thread, after MX_LWIP_Init() (pt. 3.3)

ethernetif_input()

Code: [Select]

/**
 * This function should be called when a packet is ready to be read
 * from the interface. It uses the function low_level_input() that
 * should handle the actual reception of bytes from the network
 * interface. Then the type of the received packet is determined and
 * the appropriate input function is called.
 *
 * @param netif the lwip network interface structure for this ethernetif
 */
void ethernetif_input( void const * argument )
 
{
 
  struct pbuf *p;
  struct netif *netif = (struct netif *) argument;
 
  for( ;; )
  {
    if (osSemaphoreWait( s_xSemaphore, TIME_WAITING_FOR_INPUT)==osOK)
    {
      do
      {   
        p = low_level_input( netif );
        if   (p != NULL)
        {
          if (netif->input( p, netif) != ERR_OK )
          {
            pbuf_free(p);
          }
        }
      } while(p!=NULL);
    }
 
  }
}

I havn't seen MX_LWIP_Process() function in other examples and I cannot wrap my head around how it all should go together. I've got the receive loop that occupies my application thread, so it cannot run the MX_LWIP_Process there. I'd need yet another thread just for that.

Could anyone clarify what processes should be running alongside each other after the initialization, with what priorities, and how they interact?

[RichardBarry]

Not directly answering your questions - but hopefully useful.....did you see that FreeRTOS's own TCP/IP stack has a pre-configured project for the STM32F4? http://www.freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_TCP/TCP-IP_FAT_Examples_ST_STM32F407.html

[FlashEF]

No, I have used STM32CubeMX software to get the RTOS + LwIP stack:
http://www.st.com/web/catalog/tools/FM147/CL1794/SC961/SS1533/PF259242?sc=stm32cube

I've imported it into Eclipse using this tutorial:
http://www.carminenoviello.com/en/2015/11/02/quickly-import-stm32cubemx-project-eclipse-project/

Examples I used come from different ST docummentation, mainly the one included here (CubeF4 contains LwIP example projects):
http://www.st.com/web/en/catalog/tools/PF259243

To what I can see this FreeRTOS stack is a different thing, and something interesting to check out but has nothing to do directly with LwIP. On the other hand it may be far more useful for others trying to do the same thing, as I bet it is very well integrated with FreeRTOS, while also they claim it is completely thread safe.

Going back to question B:
I am starting to think that  MX_LWIP_Process() is just a function that should be called in the main loop if you are not already running ethernetif_input() in a separate thread. If so, then all my other assumptions would make sense and we would have a clear 2 thread situation.

I'll definitely post back if I get it all up and running and I would be really thankful for any confirmation from others that have similar solutions working to know that I am going the right direction.

[HackedFridgeMagnet]

Im not an expert on networking but I'll give it a go.
Doesn't dhcp use the network layer? ie.calls into it.
Therefore it is fine that DHCP and your application both use the same calls.
If it is threadsafe then it wont matter which threads you are calling from.

The code comment says:

Code: [Select]

* Up to user to call this function in main.c in while (1) of main(void)
The function itself is obviously meant as a "call often and return fast when the flag is not set " type of function.  So just call it often.

[FlashEF]

I've got it up and running and the amount of things I've learned could account for a couple of tutorials.

Some tips & thoughts:
1. First of all do not use STM32F429I Discovery board for Ethernet prototyping like I did. Plan was to design only the physical layer electronics as a shield - just to make things faster. That failed cause MII pinout conflicts the accelerometer, screen and USB peripherals on the board. I had to rip all of it out. Well, I thought that I ripped all of it out only to find that I forgot one capacitor - that was enough to stop one of MII TX channels from woking. 2 days of debugging.
2. Speaking of debugging - probe all MII signals. Do not assume that any single one of them works. When designing your board prepare convenient, labeled test points for all the lines. You'll need a 4-channel scope to see that there actually is data going both ways on 4xTX and 4xRX. If you cannot see outgoing packets on a sniffer (like Wireshark) there has to be an electrical problem.
3. Do not probe the RJ45 side unless You have fully isolated scope (each channel to each channel)! Secondly, you will not find anything there unless you have a scope with proper Ethernet decoder. When debugging TX channels I wanted to see if there is activity on TX wires after something gets send to the PHY chip. That DOES NOT make sense. As long as there is an active link You will see constant activity on the line. Sure you Can decode it by hand, but the signal is extremely hard to capture (trigger) since the line is always doing something.
3. Start debugging with static IP configuration and 1 to 1 connection (direct cable to PC). DHCP on LwIP did not work out of the box. On the other hand DHCP is convenient as a "hello world" packet source. Your device starts up and you shuld see the DHCP packet on Wireshark, comming from 0.0.0.0 IP addres.
4. Use Discovery as a programmer and a platform to test RTOS. Then spin your own board with the same MCU.
5. Use 50 ohm termination on MII lines cause otherwise they ring like crazy. The PHY also injects a lot of noise in to the power supply, so decouple it properly. Read this: http://www.murata.com/~/media/webrenewal/support/library/catalog/products/emc/emifil/c39e.ashx
Pi filters were not enough, in the final version feed-through capacitors will be needed. That is at least if you plan to use the system for analog measurements & interfacing.
6. TCP Builder is a cool bit of software for sending arbitrary TCP and UDP packets.
7. Get yourself some help. This task is so complex and stuff may not work on so many levels that You absolutely need someone to talk it though. I was stuck twice and I made it only because a friend of mine, former Intel employee now running his own company, spared 2 x 20 minutes to consult the problems I was experiencing. Even with help of someone with 10+ years of experience  it took me 3 weeks of 8h+ work to get it up and running. On the other hand - I started with 0 knowledge about networking and RTOS. It is doable but hard.
8. RTOS is the best thing I've ever used on an embedded system! After using it I think that there is no point to program an ARM cortex without it (at least in 90% of cases). Use FreeRTOS + Trace - it will enable You to see what the Kernell is actually doing and get load stats, thread switching... priceless data.
9. Do not bother with socked API. Netconn works and it documented on an acceptable level. Socket API has crap documentation (or in other words almost none).


Answers to my questions:
A. Do not attempt to start with DHCP (see pt. 3). Prepare yourself a connection manager thread that initializes Ethernet peripherals, then inits LwIP and optionally does DHCP or sets static IP config. Then it should signal communication threads that they may bind TCP/UDP connections and start using the LwIP stack & newly established link. This thread should then be able to restart slave connection threads if the link goes down and optionally redo DHCP.
B. There are two LwIP threads - the stack thread and the frame input thread - effectively ethernetif_input() func. running in a loop. The void MX_LWIP_Process(void) should be marked as APPLICABLE ONLY to NO RTOS scenario. Calling it separately in another thread is not only not needed but harmfull as you'll end up with two instances running! Sadly the doccumentation is bad, and there are many such small problems in the code that you'll need to RTFC cause there is no M in the equation

I also found that Ethernet HAL functions use HAL delay. When generating code with CubeMX systick interrupt is used only as a clock source for the RTOS. There was no HAL Systick call. It caused HAL delay functions to wait forever! Since those delays are only used for Ethernet PHY initialization they are acceptable and appropriate. Either try to use RTOS delays (I do not know if that is possible in HAL drivers but should be) or include HAL systick function call in the Systick interrupt routine.

If anyone in the future struggles with LwIP and RTOS feel free to ask questions here

[balas]

Hi FlashEF,

I've seen your post on LWIP + FreeRTOS - and I'd like to ask for some further info if you have the time to answer...

So I'm using a board which is already built with Ethernet PHY and RJ45 (using the DP83848 chip on board Kamami ZL26ARM). So I have the luxury to only deal with software and don't need to troubleshoot the hardware.

I'm using CubeMX to generate the whole pack, just as you did in your example.

I was able to set up a working http server example with dhcp, but without using FreeRTOS. However once I enable FreeRTOS through CubeMX, the MX_LWIP_Process() just hangs, and looks like apart from the first DHCP discovers, nothing is really happening.

I could debug the codes and found that though the package is arriving through Ethernet, the tcpip_input function is called, however when it reads the data from the buffer and tries to send it through an RTOS message into a queue, the queue on some reason is full.

As I can see MX_LWIP_Process() is making sure to read the package from ethernet and send to a queue - however I have no clue what process should I have (another thread?) which reads the queue and finally handles the incoming package.

Ypu mentioned 2 threads in your post (one is the ethernetif_input which is basically wrapped by the MX_LWIP_process) - however you did not provide any specifics on the second thread needed. I suspect that is the missing point I will need to get ethernet packages read from the queue and handled properly.

Can you please elaborate on that? I really appreciate your help .. as I'm running out of ideas.

Many thanks in advance!

Regards,
Balas

[sree]

HEY,
I am planning to help my FRIEND in completing a project ,We have just two weeks to finish off this project as part of our master's internship. I am very much new to embedded field and so lack knowledge about this.I would like to know how and which way should i start working to implement LWiP on STM32F429 MCU and the peripheral .please forgive as i  don't have any idea of how to do it,please help me in f=doing this project.

[picdev]

Hello guys , I have one question about the DP83848IVVX ic and the LINK status, I work with LWIP and stm32f7.
Some times I see thath the LINK status is DOWN even when the cable is on , Is there a way to debug this without special analyzer ?
The error is ON for maybe for 1 or 2 seconds and for this period of time the ethernet doesn't work of coerce.
Where can I found some info about the link status ?

[dgtl]

The link status is read via MDIO interface from the PHY chip. This can be either polled or irq-triggered. Anyway, look for the MDIO intrerface code outside LwIP (this is customization for each platform, so platform-specific code, not core LwIP). Add some debug printf-s or debug via breakpoints and try to figure out, why the updated status is not read. Is it polled too infrequently? Is the interrupt not set up correctly or triggered not correctly?

[spacejunkie]

I've got it up and running and the amount of things I've learned could account for a couple of tutorials.

Some tips & thoughts:
1. First of all do not use STM32F429I Discovery board for Ethernet prototyping like I did. Plan was to design only the physical layer electronics as a shield - just to make things faster. That failed cause MII pinout conflicts the accelerometer, screen and USB peripherals on the board. I had to rip all of it out. Well, I thought that I ripped all of it out only to find that I forgot one capacitor - that was enough to stop one of MII TX channels from woking. 2 days of debugging.
2. Speaking of debugging - probe all MII signals. Do not assume that any single one of them works. When designing your board prepare convenient, labeled test points for all the lines. You'll need a 4-channel scope to see that there actually is data going both ways on 4xTX and 4xRX. If you cannot see outgoing packets on a sniffer (like Wireshark) there has to be an electrical problem.
3. Do not probe the RJ45 side unless You have fully isolated scope (each channel to each channel)! Secondly, you will not find anything there unless you have a scope with proper Ethernet decoder. When debugging TX channels I wanted to see if there is activity on TX wires after something gets send to the PHY chip. That DOES NOT make sense. As long as there is an active link You will see constant activity on the line. Sure you Can decode it by hand, but the signal is extremely hard to capture (trigger) since the line is always doing something.
3. Start debugging with static IP configuration and 1 to 1 connection (direct cable to PC). DHCP on LwIP did not work out of the box. On the other hand DHCP is convenient as a "hello world" packet source. Your device starts up and you shuld see the DHCP packet on Wireshark, comming from 0.0.0.0 IP addres.
4. Use Discovery as a programmer and a platform to test RTOS. Then spin your own board with the same MCU.
5. Use 50 ohm termination on MII lines cause otherwise they ring like crazy. The PHY also injects a lot of noise in to the power supply, so decouple it properly. Read this: [url]http://www.murata.com/~/media/webrenewal/support/library/catalog/products/emc/emifil/c39e.ashx[/url]
Pi filters were not enough, in the final version feed-through capacitors will be needed. That is at least if you plan to use the system for analog measurements & interfacing.
6. TCP Builder is a cool bit of software for sending arbitrary TCP and UDP packets.
7. Get yourself some help. This task is so complex and stuff may not work on so many levels that You absolutely need someone to talk it though. I was stuck twice and I made it only because a friend of mine, former Intel employee now running his own company, spared 2 x 20 minutes to consult the problems I was experiencing. Even with help of someone with 10+ years of experience  it took me 3 weeks of 8h+ work to get it up and running. On the other hand - I started with 0 knowledge about networking and RTOS. It is doable but hard.
8. RTOS is the best thing I've ever used on an embedded system! After using it I think that there is no point to program an ARM cortex without it (at least in 90% of cases). Use FreeRTOS + Trace - it will enable You to see what the Kernell is actually doing and get load stats, thread switching... priceless data.
9. Do not bother with socked API. Netconn works and it documented on an acceptable level. Socket API has crap documentation (or in other words almost none).


Answers to my questions:
A. Do not attempt to start with DHCP (see pt. 3). Prepare yourself a connection manager thread that initializes Ethernet peripherals, then inits LwIP and optionally does DHCP or sets static IP config. Then it should signal communication threads that they may bind TCP/UDP connections and start using the LwIP stack & newly established link. This thread should then be able to restart slave connection threads if the link goes down and optionally redo DHCP.
B. There are two LwIP threads - the stack thread and the frame input thread - effectively ethernetif_input() func. running in a loop. The void MX_LWIP_Process(void) should be marked as APPLICABLE ONLY to NO RTOS scenario. Calling it separately in another thread is not only not needed but harmfull as you'll end up with two instances running! Sadly the doccumentation is bad, and there are many such small problems in the code that you'll need to RTFC cause there is no M in the equation

I also found that Ethernet HAL functions use HAL delay. When generating code with CubeMX systick interrupt is used only as a clock source for the RTOS. There was no HAL Systick call. It caused HAL delay functions to wait forever! Since those delays are only used for Ethernet PHY initialization they are acceptable and appropriate. Either try to use RTOS delays (I do not know if that is possible in HAL drivers but should be) or include HAL systick function call in the Systick interrupt routine.

If anyone in the future struggles with LwIP and RTOS feel free to ask questions here

That was quite helpful. Thanks. I was researching for a project involving network connectivity. I have some questions:

1. I have heard about RMII stability issues. The posts I read it from may be old. Have you tried RMII recently, have the issues gone or become bearable? I would like to avoid building my own board at this stage if possible. I was planing to order a Nucleo-H743ZI board, but it has a RMII connection (CubeMX auto populates RMII in Ethernet options when I choose this board). 

2. Is it possible to have two Ethernet connections? I am designing a system that's going to be an EtherCAT master. That would take one Ethernet port. And I need another port for network connectivity.
EtherCAT does not use TCP/IP, it uses raw Ethernet frames. So I would not need an IP stack for this connection. Only the second connection requires TCP/IP.
If having two Ethernet connections is not possible, can I write and read raw Ethernet Frames into a connection managed by LwIP or whatever stack CubeMX provides. The frames have appropriate flags so a well written stack would know that they are not carrying IP payload. That should prevent incoming EtherCAT frames from confusing the stack.