Anyone used the Wiznet ethernet chips?

[Siwastaja]

In my design I have used an external oscillator. Or to be more specific: a clock output from the microcontroller.

I considered that but the 1.2V level requirement makes an oscillator an unobtanium, too, although one could use a higher voltage oscillator and level-shift the output.

This can't be really this difficult, I contacted Wiznet, hope they come back. They don't participate on the forum where many many have asked about the crystal specs (and reported weird crystal issues).

[mikeselectricstuff]

In my design I have used an external oscillator. Or to be more specific: a clock output from the microcontroller.

Yes, use an oscillator, let someone else worry about drive etc. 25 cents at 100x on LCSC
25MHz is a bit on the high side for a simple inverter based oscillator.

As I mentioned before DO NOT use a MEMS oscillator if you want all your packets to arrive!

[nctnico]

In my design I have used an external oscillator. Or to be more specific: a clock output from the microcontroller.

I considered that but the 1.2V level requirement makes an oscillator an unobtanium, too, although one could use a higher voltage oscillator and level-shift the output.

This can't be really this difficult, I contacted Wiznet, hope they come back. They don't participate on the forum where many many have asked about the crystal specs (and reported weird crystal issues).

The W5500 datasheet I have says that you can use a 3.3V square wave as the clock input.

[Siwastaja]

In my design I have used an external oscillator. Or to be more specific: a clock output from the microcontroller.

I considered that but the 1.2V level requirement makes an oscillator an unobtanium, too, although one could use a higher voltage oscillator and level-shift the output.

This can't be really this difficult, I contacted Wiznet, hope they come back. They don't participate on the forum where many many have asked about the crystal specs (and reported weird crystal issues).

The W5500 datasheet I have says that you can use a 3.3V square wave as the clock input.

Sorry, I'm talking about W6100, which seems to have the same requirements as W5500S.

W5500 seems to have better availability at distributors, too, so considering skipping IPv6 support just to get a product with sane clock input.

W6100 would seem to require a 3V3 oscillator + a level shifter, in which case I need to be quite careful not to distort the duty cycle or introduce jitter.

[nctnico]

Level shifter shouldn't be hard. A resistive divider should do it; the clock input should have a high impedance anyway. Another option is to AC couple the clock into the W6100 (optionally using a capacitive divider). The internal bias circuit should take care of centering the signal where the W6100 likes it best.

[Siwastaja]

Yeah with just a few pF of input capacitance I'm probably going with 3V3 oscillator and resistor divider but I'm just getting a bit paranoid for reading so many posts about issues caused by clocking of this chip. This isn't the first Ethernet device I have designed but to be frank the earlier ones I didn't care much and they just Did Work with whatever 25MHz +/-50ppm crystal I got. Now I want to avoid problems when production is possibly ramped up to tens of thousands, recalls is last thing I need.

[peter-h]

One gets the same forum debates around the 32F417 + LAN8742 PHY chip and the famous 50MHz clock feeding the 32F4's ETH subsystem... Almost nobody seemed to actually know something. All I can say is that I followed the data sheets and it is rock solid, across numerous boards, but I did keep the PCB track length to about 2cm, and put resistors in series as recommended, to reduce ringing.

A resistive divider won't work at 25MHz. Look at the DC loading of such a divider, versus the required Zout to drive even a few pF. You will probably need caps in parallel, so e.g. 2 x 1k to set the DC level at 50%, and 2x 10pF to produce an "AC" divider.

[Siwastaja]

A resistive divider won't work at 25MHz. Look at the DC loading of such a divider, versus the required Zout to drive even a few pF. You will probably need caps in parallel, so e.g. 2 x 1k to set the DC level at 50%, and 2x 10pF to produce an "AC" divider.

Say 5mA loading on 3V3 oscillator R = 3.3V/0.005A = 660 ohm,
Assuming pin capacitance of 5pF + layout capacitance 3pF = 8pF
t_10-90% = 2.2*R*C = 2.2*660 * 8e-12 = 11.6ns
Rise/fall time for clock input of W6100 per datasheet max 8ns

You are right, it might marginally work with lower value resistors, otherwise capacitors AC bypassing the resistors would be needed.

[peter-h]

Indeed; although the 5mA is probably just a waste. Depends on the input circuit it is driving. If there is a pullup or pulldown, that changes things, plus the value of such internal Rs tends to have a huge tolerance on it.

I also doubt one needs that much xtal drive, because all that matters is the amplitude on the clock input. The crystal drive relates to the amplitude at the other end of the xtal. But I don't know enough about this. The thing which might drive (no pun intended) the high xtal drive power would be if the clock input was low-Z and you needed to achieve a given amplitude at it. For example to drive a 50 ohm input straight off a xtal circuit like this



with say 1.5V P-P, you would need to drive the xtal really hard.

[nctnico]

There is likely some internal biasing going on but the input should be able to deal with a sine wave as well. After all, when used as a crystal oscillator the input receives a sine wave. Due to the biasing this sine wave likely has the zero crossing just at the right level for the input. Might be worth investigating if you really want to flesh things out. It kind of sucks the clock input needs such a low level for an external clock input.

[Siwastaja]

And finally I found a suitable crystal, I don't know how I managed to overlook this quite inexpensive part:
https://www.jauch.com/downloadfile/5d5283eb8ea55300421fd84937fcba163/jauch_datasheet_j49smh.pdf
Cshunt = 5pF (better than the usual 7pF of large crystals)
ESR = 30ohm (better than the usual 40ohm)

Gain margin calculation
> 8.43/(4*30*(2*3.14*25e6)^2*(5e-12+12e-12)^2*1000)
ans =  9.8616 > 6.9897

Drive level max 500µW. I still don't feel good about running something at absolute maximum rating and 5x over recommended, OTOH I don't know what the "recommended 500µW" in the W6100 datasheet actually means, maybe it means "pick a crystal with maximum rating of 500µW and W6100 will drive it at lesser power".

Going in circles with this crystal drive level thing, it seems some kind of tradition for manufacturers to pull ridiculous drive level numbers out of thin air, e.g. see https://community.infineon.com/t5/PSoC-5-3-1/External-crystal-drive-level-calculation/td-p/242159 and the company response apologizing the wrong numbers.

W6100 datasheet does not specify the crystal amplifier voltage swing, but assuming W6100 oscillator circuit runs from 1.2V, possible maximum drive level for a typical CL=12pF, CO=7pF, ESR=40ohm crystal would be, from first formula of above link, 2*40*(3.14*25e6*1.2*(12e-12+7e-12))^2 * 1e6 = 256uW

So maybe the 500uW advice is more like "better pick a crystal with 500uW absolute maximum rating" and not "the oscillator circuit will drive the crystal at 500uW and any attempt to reduce drive by increasing Rext results in instability due to too little amplitude".

I'm probably just going with a crystal after all, but have been enjoying documenting this weird trip, I think I have never spent this much time choosing a crystal just for such ubiquitous use case as 100M Ethernet.

[peter-h]

The xtal oscillator should drive the xtal with a rail to rail swing. It won't drive it at any particular "power" - no way to do that.

But if there was a real requirement for a high power xtal, it would suggest that the clock input is low impedance of some sort.

[Siwastaja]

The xtal oscillator should drive the xtal with a rail to rail swing. It won't drive it at any particular "power" - no way to do that.

Not true at all, configurable crystal drive strength is often available, even in many cheap microcontrollers, you can easily verify this by measurement, swing is very rarely "rail to rail".  For example, see random Google result: https://community.silabs.com/s/article/choosing-c8051-crystal-oscillator-drive-level-oscxcn-xfcn-value-x?language=en_US

How they internally exactly achieve this, I don't know, I never looked into it.

W6100 does not have configurable drive strength, but that does not mean it must be a full swing between GND and Vcc (they don't even explicitly tell which Vcc domain, but I assume it's the 1.2V domain) without any internal current limitation.

So yes, it will drive at some particular power, but of course the crystal ESR, CL and CO are all part of the equation, it doesn't measure and regulate the power.

In absence of software-configurable drive strength, one adds external series resistance.

[hans]

Rail-to-rail doesn't make sense. If a crystal has an ESR of 100Ohm, with a 3.3Vpp square-wave drive, then that is 3.3^2/100 /2 = 54.5mW. Most crystals are much lower amplitude.. e.g. a 100uW sine-wave drive with 100Ohm ESR is 0.242Vpp.

 The ESR is the only resistive element of the crystal (e.g. it dissipates power), whereas the capacitors and inductors are only used for oscillation or tuning characteristics. A pierce oscillator circuit may have a series  and shunt resistor into/across the inverter to set the gm of that amplifier.

I remember that PICs have fuses to set the frequency range of the oscillator. E.g. for a PIC16F1509 it says these bits will change the shunt resistor between 2M and 10M. Lower gm = less gain = lower power. The external series resistor may be needed to limit power into a low-power crystal.
I don't remember seeing this tunable shunt resistor on e.g. a STM32. The series resistor may still be needed. I presume they have set the crystal driver power reasonably high to drive a wide range of oscillators (and possibly lower frequency ones need some scaling down)
 

[peter-h]

The 0.242V is across the xtal itself. This is not related to the swing of the amplifier output driving it.

[hans]

I happened to work on a board bring up today with a W5500 and a STM32H725 chip. Having worked with ethernet phy's before and TCP/IP stacks.. pretty amazed how simple it is. Just write a few IP registers and it's ready to open a socket.

The internal engine of the chip seems to max out at around 92Mbit/s while transmitting the (same) 1K of data with a TCP socket. When the program also writes the Tx buffer and then waits for packet completion, it maxes out around 23 18Mbit/s with a SPI speed 34.375MHz, and 27 22Mbit/s with a SPI speed of 43.75MHz. So could be potentially a little bit faster if new data is written while the old packet is transmitted.

Unfortunately this SPI bus breaks down at 68MHz, as I had to patch it with a few thin coil wires.

This is still far higher than the figures listed on their official page: https://docs.wiznet.io/Product/iEthernet/W5500/Application/spi-performance
(Honestly 3Mbit/s sounds quite atrocious.)

For some reason, I get worse performance with UDP (max 80Mbit/s). But still very decent and much faster than a "regular" microcontroller TCP/IP stack can hold up (I remember researching some TCP/IP stacks for PIC32MX, most topped out at max 1-2.5MiB/s in TCP).

I'm posting this because the "raw" performance (note: my code is polling the status registers, etc.) of this chipset was a bit hard to find for me.

[SiliconWizard]

Well the W5500 supports SPI @up to 80MHz from the datasheet. Haven't tried it yet so I don't know what the problems could be. Not sure what you mean by "this SPI bus breaks down at 68MHz, as I had to patch it with a few thin coil wires." Could you detail?

[hans]

I've a stack of 2 boards (see: https://twitter.com/BitsOfHans/status/1571145207177158657/photo/1), but the board-to-board connector pinout was not correct so I had to jump the lines between boards with ~10cm wires (I use very thin coil wire for that, ideal to solder onto small pads). I do have some series damping resistors in the PCB to damp reflections, but wires in open air don't have a proper 50Ohm impedance.

My current code either transmits data into the buffer, or waits for the packet to be transmitted. For my code, the measured throughputs match pretty well using the harmonic mean of the SPI speed and "44Mbit/s" (estimated limitation of statemachine and firmware etc.):

1/ (1/34.375 + 1/44) = 19.3Mbit/s
1/ (1/43.75 + 1/44) = 21.9Mbit/s
Actually these calculations are a bit on the high side, so my prediction for SPI @ 80MHz is: 1/ (1/80 + 1/44) = 28.4Mbit/s.

If I want faster speeds, I would need to write into the bufferspace while another packet is being sent. I'm not sure if the buffer RAM is dual port, or if it needs arbitration, but I assume that at some point either the packet engine (statemachines, firmware polling etc.) or the SPI bus (80MHz) will be the limiting factor.

It's pretty promising, because other ethernet mac controllers (like the classic ENC28J60 or ENC424J600) have pretty slow SPI buses. Some multiple of tens of Mbit/s is pretty good.

[coppice]

Weird that such a common interface/protocol is still not mainstream good quality open source available, esp. with the huge trend towards networking small embedded devices 

Some of the very best implementations of TCP/IP are open source, but they are also full featured, large, and in big solutions like BSD and Linux. When people are trying to squeeze into something small they have so many incompatible notions of what limited resources means that generic solutions don't work well.

[hans]

Small update..

Read the docs, which says not to write to TxBuffer space while a TCP packet is transmitted. However, I'm not sure why. I think they don't want you to change the Tx WR pointers while a packet is in transmission. But why should the buffer space be inaccessible? I suppose they would use dual-port RAM to have the Ethernet packet engine and SPI engine operate independently.. So I went ahead and implemented this dual process anyway.

The old code was:

Code: [Select]

        while(1) {
            // check for available space
            do {
                fsr = le2be(SpiW5500_Read<uint16_t>(s0, 0x20));
            } while (fsr < sizeof(myPacket));

            // write packet
            myPacket.data[0]++;
            SpiW5500_Write<Packet1K>(s0tx, wp, myPacket);
            wp += sizeof(myPacket);
            SpiW5500_Write<uint16_t>(s0, 0x24, le2be(wp)); // Tx WR pointer

            // transmit packet
            SpiW5500_Write<uint8_t>(s0, 1, 0x20); // Command: Send

            // wait till transmit is done
            do {
                ir = SpiW5500_Read<uint8_t>(s0, 2); // Rd IR (SendOK = 0x10)
            } while ((ir&0x10) == 0);
            SpiW5500_Write<uint8_t>(s0, 2, 0x1F); // Clr IR (SendOK = 0x10)

This first checks for free space, then writes new 1K packet, and transmits it, which is sequential and slow.

New code does not:

Code: [Select]

        while (1) {
            fsr = le2be(SpiW5500_Read<uint16_t>(s0, 0x20)); // Rd Tx freespace
            ir  = SpiW5500_Read<uint8_t>(s0, 2); // Rd IR
            if (!hasWritten && fsr > sizeof(myPacket)) { // write packet when space is available and it wasn't done so
                myPacket.data[0]++;
                SpiW5500_Write<Packet1K>(s0tx, wp, myPacket);
                wp += sizeof(myPacket);
                hasWritten = true;
            }
            if (ir & 0x10) { // Check if triggered: SendOK = 0x10, which arms the send command
                SpiW5500_Write<uint8_t>(s0, 2, 0x10); // Clr IR (SendOK = 0x10)
                canSend = true;
            }
            if (hasWritten && canSend) { // new data written & send armed => set new pointer and transmit
                SpiW5500_Write<uint16_t>(s0, 0x24, le2be(wp));
                SpiW5500_Write<uint8_t>(s0, 1, 0x20); // Command: Send
                hasWritten = false;
                canSend = false;
            }
        }
This code works fine. No TCP retransmits or weird issues to be seen in Wireshark, and the packet counter is working normally.

For tests, socket 0 has maximum buffer space (16KiB). Throughput went up from ~27.8Mbit/s (old code, SPI clock of 43.75MHz) to 37Mbit/s (new code).

When I change packet size to MTU (1472bytes), the throughput peaks at 28.1Mbit/s (old) and 38.4Mbit/s (new).
Improved SPI driver (without STM32 HAL overhead): max 28.75 / 39.3Mbit/s respectively.

All throughputs were measured in user space with a small python socket script.

That last test translates 90% throughput of the SPI bus clock to application throughput. That's a nice promise for designs that can get up to 70 or 80MHz SPI clocks. (which hopefully I can when I fix the board-to-board connectors).

Not sure why I see so many benchmark figures of this chip with lower numbers. Obviously SPI bus is a bottleneck (this STM32H7 has a FIFO, so with CPU cycles it doesn't need DMA to saturate the SPI bus), but if you DMA that it combined with this 'trick' it should be a matter of maximizing the SPI clock.

[Siwastaja]

High-speed SPI needs some thought, like only route it on a PCB (no wires/connectors), route it as kinda-impedance controlled over a contiguous ground plane, and series terminate the signals at source, matching the impedance. For example, if you assume the CMOS Rds_on to be 20 ohms, and add 33 ohms in series, then calculate the trace width for 53 ohms impedance, given you know your stackup (distance to the ground plane). Remember vias from the IC ground leads to the ground plane near the signal traces on both ends. And if you absolutely need to go through a connector, use ground-signal-ground-signal scheme, and try to work out the characteristic impedance of the wire by googling or from the geometry, so you can match the PCB tracks and series termination resistors to the wire impedance.

Of course, just minimizing the bus length to a few cm is easier and then all of this does not matter.

[SiliconWizard]

Read the docs, which says not to write to TxBuffer space while a TCP packet is transmitted. However, I'm not sure why. I think they don't want you to change the Tx WR pointers while a packet is in transmission. But why should the buffer space be inaccessible? I suppose they would use dual-port RAM to have the Ethernet packet engine and SPI engine operate independently.. So I went ahead and implemented this dual process anyway.

The old code was:

Code: [Select]

        while(1) {
            // check for available space
            do {
                fsr = le2be(SpiW5500_Read<uint16_t>(s0, 0x20));
            } while (fsr < sizeof(myPacket));

            // write packet
            myPacket.data[0]++;
            SpiW5500_Write<Packet1K>(s0tx, wp, myPacket);
            wp += sizeof(myPacket);
            SpiW5500_Write<uint16_t>(s0, 0x24, le2be(wp)); // Tx WR pointer

            // transmit packet
            SpiW5500_Write<uint8_t>(s0, 1, 0x20); // Command: Send

            // wait till transmit is done
            do {
                ir = SpiW5500_Read<uint8_t>(s0, 2); // Rd IR (SendOK = 0x10)
            } while ((ir&0x10) == 0);
            SpiW5500_Write<uint8_t>(s0, 2, 0x1F); // Clr IR (SendOK = 0x10)

This first checks for free space, then writes new 1K packet, and transmits it, which is sequential and slow.

New code does not:

Code: [Select]

        while (1) {
            fsr = le2be(SpiW5500_Read<uint16_t>(s0, 0x20)); // Rd Tx freespace
            ir  = SpiW5500_Read<uint8_t>(s0, 2); // Rd IR
            if (!hasWritten && fsr > sizeof(myPacket)) { // write packet when space is available and it wasn't done so
                myPacket.data[0]++;
                SpiW5500_Write<Packet1K>(s0tx, wp, myPacket);
                wp += sizeof(myPacket);
                hasWritten = true;
            }
            if (ir & 0x10) { // Check if triggered: SendOK = 0x10, which arms the send command
                SpiW5500_Write<uint8_t>(s0, 2, 0x10); // Clr IR (SendOK = 0x10)
                canSend = true;
            }
            if (hasWritten && canSend) { // new data written & send armed => set new pointer and transmit
                SpiW5500_Write<uint16_t>(s0, 0x24, le2be(wp));
                SpiW5500_Write<uint8_t>(s0, 1, 0x20); // Command: Send
                hasWritten = false;
                canSend = false;
            }
        }
This code works fine. No TCP retransmits or weird issues to be seen in Wireshark, and the packet counter is working normally.

For tests, socket 0 has maximum buffer space (16KiB). Throughput went up from ~27.8Mbit/s (old code, SPI clock of 43.75MHz) to 37Mbit/s (new code).

When I change packet size to MTU (1472bytes), the throughput peaks at 28.1Mbit/s (old) and 38.4Mbit/s (new).
Improved SPI driver (without STM32 HAL overhead): max 28.75 / 39.3Mbit/s respectively.

All throughputs were measured in user space with a small python socket script.

That last test translates 90% throughput of the SPI bus clock to application throughput. That's a nice promise for designs that can get up to 70 or 80MHz SPI clocks. (which hopefully I can when I fix the board-to-board connectors).

Not sure why I see so many benchmark figures of this chip with lower numbers. Obviously SPI bus is a bottleneck (this STM32H7 has a FIFO, so with CPU cycles it doesn't need DMA to saturate the SPI bus), but if you DMA that it combined with this 'trick' it should be a matter of maximizing the SPI clock.

This is because indeed most of them are made with rather low SPI clock and no double buffering at all. As you mentioned, the docs say not to do double buffering actually, so probably people just stick to that advice. Or maybe they just don't know any better anyway. But of course maximizing SPI clock and filling the TX FIFO while the chip is transmitting rather than wait for it to complete are the only ways of maximizing throughput. No way around it.

The more concerning point IMO is not so much that simple users do not manage to maximize throughput but that the vendor itself does not either apparently.