looking for a simple linux uart application that handles /dev/ttyUSB0 things

[legacy]

this project has been developed by different guys
from the hardware guy i got a board which looks like the Yun
so there is a SoC (Atheros MIPS @400Mhz with built-in USB controller (1))
interfaced to an ATmega328 (datasheet) @ 16Mhz

SoC ---- built in USB (OHCI->USB1) -------- FTDI232 -------- serial 5V -------- ATmega328 @ 16Mhz

(1) the first problem is: why the firmware (linux side) guy has not used EHCI (USB2) instead of OHCI (USB1) ?
i have to understand it


btw, the Arduino side looks like a clone of the board "Arduino 2009"
wandering if i could get the same SoC around and the same board, in order to get more investigations

p.s.
the YUN seems to have higher uart speed

p.s.2
from my point of view, the "teensy-v2" hardware is better
i have tested it on my laptop  (laptop x86 ---USB2---teensy2 with built in USB device)
up to 900Kbyte/sec over the usb-serial with the same software i am talking about
no issues, no data corrupted, no timeout, everything works great

[legacy]

You may have to consider 115200 cannot work. Be sure to check the USB speed with the output from dmesg.

dmesg says the USB is handled by OHCI (wandering why not EHCI instead? .. i will try to recompile the kernel and see what happens)
and the device3 (arduino) is USB1 full speed which should be 12M bit/sec
in theory it should be fine enough to handle 115K bit/sec

the first problem is: what happens if linux gets a long burst stream (128x128+2=16386byte)
in where every single URB/bulk has 1 byte instead of 64 ?

supposing the baud rate drift is not relevant
for 115200bit/sec  it means something like 10Kbyte/sec -> 10K URB / sec
i trust the kernel is perfectly able to handle it, even if …
… it's not an ideal situation and probably it is not the best optimized idea

teensy-v2 has no baud generator, so no drift issues
and the linux kernel side uses serial-class instead of a serial-driver, that permits to send 64byte per packet
it means that you know exactly what happens in the chain, so you could organize your firmware that way
and in the linux application side you could also use read(fd,buffer,64) instead of 64 times for {read(fb,&ch,1)}

just considerations … unfortunately i can ask the team to change the hardware
btw i am suspecting the problem is related to the % error due to the baud rate (frequency drift)
like you say, guys (thank your for your considerations)

or something in the hardware layer 

[Monkeh]

(1) the first problem is: why the firmware (linux side) guy has not used EHCI (USB2) instead of OHCI (USB1) ?
i have to understand it

Because EHCI is used for high speed devices, and virtually every USB connected UART is a low speed or full speed device? This is not a choice made by your firmware guy.

[legacy]

Because EHCI is used for high speed devices, and virtually every USB connected UART is a low speed or full speed device?
This is not a choice made by your firmware guy.

you are absolutely right !

my point is: while i am waiting for the kernel sources (someone should send me sooner) i have looked a the Carambola's kernel because it looks very similar to the SoC i am talking about: the kernel support for v3.3.8 i see these choices

Code: [Select]

CONFIG_USB_EHCI_HCD:                                                                                                                                               ?
  ?                                                                                                                                                                    ?
  ? The Enhanced Host Controller Interface (EHCI) is standard for USB 2.0                                                                                              ?
  ? "high speed" (480 Mbit/sec, 60 Mbyte/sec) host controller hardware.                                                                                                ?
  ? If your USB host controller supports USB 2.0, you will likely want to                                                                                              ?
  ? configure this Host Controller Driver.                                                                                                                             ?
  ?                                                                                                                                                                    ?
  ? EHCI controllers are packaged with "companion" host controllers (OHCI                                                                                              ?
  ? or UHCI) to handle USB 1.1 devices connected to root hub ports.  Ports                                                                                             ?
  ? will connect to EHCI if the device is high speed, otherwise they                                                                                                   ?
  ? connect to a companion controller.  If you configure EHCI, you should                                                                                              ?
  ? probably configure the OHCI (for NEC and some other vendors) USB Host                                                                                              ?
  ? Controller Driver or UHCI (for Via motherboards) Host Controller                                                                                                   ?
  ? Driver too.               

which has a combo EHCI/OHCI support and needs "EHCI support for AR7XXX/AR9XXX SoCs"

so i should see EHCI and OHCI in the dmesg, initialized during the boot, while i am seeing only OHCI … and i wander why 

(and unfortunately the USB has no connector, everything is soldered on the PCB, so i can't plug an USB-hard drive to see what happens with a USB2 device, perhaps not without an hw hack  )

btw, these are the dmesg's line when the SoC sees Arduino

Code: [Select]

usb 4-1: new full speed USB device number 3 using ohci
usb 4-1: New USB device found, idVendor=0403, idProduct=6001
usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-1: Product: FT232R USB UART
usb 4-1: Manufacturer: FTDI
usb 4-1: SerialNumber: A700emu6
usb 4-1: usb_probe_device
usb 4-1: configuration #1 chosen from 1 choice
usb 4-1: adding 4-1:1.0 (config #1, interface 0)
usbserial_generic 4-1:1.0: usb_probe_interface
usbserial_generic 4-1:1.0: usb_probe_interface - got id
ftdi_sio 4-1:1.0: usb_probe_interface
ftdi_sio 4-1:1.0: usb_probe_interface - got id
ftdi_sio 4-1:1.0: FTDI USB Serial Device converter detected
usb 4-1: Detected FT232RL
usb 4-1: Number of endpoints 2
usb 4-1: Endpoint 1 MaxPacketSize 64
usb 4-1: Endpoint 2 MaxPacketSize 64
usb 4-1: Setting MaxPacketSize 64
usb 4-1: FTDI USB Serial Device converter now attached to ttyUSB0

it's OHCI, USB1, full speed

i'd like to test both endpoints with a maximal usage equal to MaxPacketSize, i mean 64byte per packet
in the afternoon i will try to change the Arduino firmware in order to test this feature
the current firmware has the endpoints maximal usage equal to 1 byte per packet

[legacy]

about Arduino 2009, there 2 versions (due to the possibility to replace the DIP package MPU), the first with AtMega328, and the second with AtMega168, which looks quite similar (even if not exactly equals)

i am reading the AtMega168 data sheet: on page 197 it would seem to suggest that with a 16MHz clock the serial port can handle 1Mbps, in case … 1Mbps >> 115200bps, so it "should" be fine in my case, or not 

which is the maximum baud rate you have successfully experimented with Arduino 328/168 and a linux SoC ?

 

[janoc]

Um, silly question - since it fails only at higher baudrates, are you sure that the Arduino is actually using a crystal and not a (cheap) ceramic resonator? Some clones have these!

[nctnico]

Code: [Select]

usb 4-1: new full speed USB device number 3 using ohci
usb 4-1: New USB device found, idVendor=0403, idProduct=6001
usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-1: Product: FT232R USB UART
usb 4-1: Manufacturer: FTDI
usb 4-1: SerialNumber: A700emu6
usb 4-1: usb_probe_device
usb 4-1: configuration #1 chosen from 1 choice
usb 4-1: adding 4-1:1.0 (config #1, interface 0)
usbserial_generic 4-1:1.0: usb_probe_interface
usbserial_generic 4-1:1.0: usb_probe_interface - got id
ftdi_sio 4-1:1.0: usb_probe_interface
ftdi_sio 4-1:1.0: usb_probe_interface - got id
ftdi_sio 4-1:1.0: FTDI USB Serial Device converter detected
usb 4-1: Detected FT232RL
usb 4-1: Number of endpoints 2
usb 4-1: Endpoint 1 MaxPacketSize 64
usb 4-1: Endpoint 2 MaxPacketSize 64
usb 4-1: Setting MaxPacketSize 64
usb 4-1: FTDI USB Serial Device converter now attached to ttyUSB0

it's OHCI, USB1, full speed

i'd like to test both endpoints with a maximal usage equal to MaxPacketSize, i mean 64byte per packet
in the afternoon i will try to change the Arduino firmware in order to test this feature
the current firmware has the endpoints maximal usage equal to 1 byte per packet

Then there is your problem. You really need to fill USB packets to the max because in USB there is a roundtrip time for each packet. If you fill a packet with only one byte then you limit the USB bandwidth way below the maximum bandwidth.

[Monkeh]

so i should see EHCI and OHCI in the dmesg, initialized during the boot, while i am seeing only OHCI … and i wander why 

Who cares? You don't need EHCI.

[legacy]

right, too
just curious 

[legacy]

Um, silly question - since it fails only at higher baudrates, are you sure that the Arduino is actually using a crystal and not a (cheap) ceramic resonator? Some clones have these!

Good question!

The hardware may be described as "custom clone", practically speaking the Arduino side is like the "Arduino 2009 board" but developed by our hardware guys and integrated in the same PCB with an Atheros SoC: it looks like the Arduino Yun but it has a DIP-package MPU (AtMega 328)

ummm you are right about a thing i have not considered yet: the board has cheap ceramic resonator 

[nctnico]

The baudrate doesn't have to be very accurate. There is a tolerance of 1% IIRC.

[legacy]

@nctnico
you get the issue fixed, definitively !

i have changed both the arduino firmware and the linux application sides in order to packet things in 64byte (according to the size of the bulk endpoints)

results

• 9600 still Ok
• 19200 still Ok
• 38400 still Ok
• 115200bps stable, no errors, no data lost, no corruption, good checksum
• 230400bps stable, no errors, no data lost, no corruption, good checksum
• 460800bps and upper bauds rates are untested

it works better and faster, thank your for your help 


now i have to contact the Arduino firmware guy, and make him to commit my patch
let me say, i am not the arduino firmware guy, about this project … in an ideal world my task should be related ONLY to the linux application, an more specifically to the algorithm model (using matlab to experiment and tune up ideas into code) to process all the stream from the arduino, which is connected to an optical sensor.

in the real world, as usual, i have to become the hardware/firmware/everythings guy 

no problem with this, except the fact that my knowledges are limited, so very very thanks for your help, really, guys 

[janoc]

The baudrate doesn't have to be very accurate. There is a tolerance of 1% IIRC.

If the board has only a resonator, then it is going to be marginal at 115200bps - those things have a tolerance +-0.5% (crystals have ~0.001% typical).

I would still change it for a crystal, just to be safe.

[nfmax]

'Asynchronous' serial communications actually synchronises each time a new start bit arrives, and most UARTs sample data during the middle third of each bit time. The bit times are based on the baud rate as set by the local UART clock; there is also a sampling uncertainty relative to this clock in knowing when the start bit arrived. Including parity and stop bits, a character is usually 10 bits long. For the sampling instant on the 10th bit not to have drifted out ofthe middle third, i.e. not by more than 1/6 bit time, the maximum clock error between sending & receiving clocks must be less than 1.667%. Allowing a margin for the uncertainty in the start bit detection time gives around 1.5% as the maximum tolerable error between the send & receive clocks. In practice, you can go a bit worse than this (in the absence of noise) as again most UARTs actually make multiple samples during each bit time & decide between 1 & 0 by the majority of the sample values.

[westfw]

If the board has only a resonator, then it is going to be marginal at 115200bps

I don't know why people keep repeating this.  The ability of a uart to accurately receive data when the clock is "not quite right" is NOT dependent on the speed.  Only on the percentage of error.

That said, a 16MHz (exactly) AVR as used on most arduinos is "marginal" at 115200bps  (2.1% off) because of the limitations of the bitrate generator.  It works fine on most modern Arduino boards, because the USB/Serial converter is ALSO an AVR with the same logic, and the code is very careful to make sure the error goes in the same direction.  If you happen to use some other USB/Serial logic that introduces error in the OTHER direction, you can easily have failures.
(My FTDI-based older arduinos and derivatives seem to work fine at 115200, but ... I'm continually nervous, and I'm not sure they'd pass a long-term data test.)

Also, 115200bps is one character every 86 microseconds (~1200 instruction times.)  It's pretty easy to write code  that doesn't service the uart often enough to avoid overruns.

[legacy]

so, 115200bps is the upper limits ?

[legacy]

new tests applying the data-packet size patch (64 byte per packet over usb/BULK)

• setting 460800 on arduino side, 460800 on linux side not correctly recognized by linux, so not working, time out, 100% data lost
• setting 460800 on arduino side, 500000 on linux side, is working at 500Kbps, no corruption, no data lost, everything is OK from a long run test (10Mbyte sent)

wandering why 460800 on arduino side, and 500000 on linux
minicom is confirming that (not working if linux set to 460800bps, perfectly working if set to 500000)   


btw, i am going faster (500000bps) than expected (115200bps), excellent guys 

[westfw]

115200bps is the upper limits?

No.  People have reported success with 1Mbps; in that case the 16MHz yields an exactly correct bit rate.
200k, 250k, and 500k should come out exactly work as well.
(Of course, you have to be even more careful that your code is fast enough to handle the bitrate.  The 1Mbps example I know if is the Arduino Optiboot Bootloader, which does ~256 byte flash sectors that it receives all at once (and nothing else at the same time, and not using interrupts), with "got it" and "send me the next" on either side.)

[janoc]

If the board has only a resonator, then it is going to be marginal at 115200bps

I don't know why people keep repeating this.  The ability of a uart to accurately receive data when the clock is "not quite right" is NOT dependent on the speed.  Only on the percentage of error.

That said, a 16MHz (exactly) AVR as used on most arduinos is "marginal" at 115200bps  (2.1% off) because of the limitations of the bitrate generator.

Which is not in contradiction to what I have said. What you are saying is obviously true, but there is another thing to consider. If you have a resonator and its frequency is "wandering" with temperature, your bitrate is going to (obviously) wander as well.

It works fine on most modern Arduino boards, because the USB/Serial converter is ALSO an AVR with the same logic, and the code is very careful to make sure the error goes in the same direction.  If you happen to use some other USB/Serial logic that introduces error in the OTHER direction, you can easily have failures.
(My FTDI-based older arduinos and derivatives seem to work fine at 115200, but ... I'm continually nervous, and I'm not sure they'd pass a long-term data test.)

I am not so sure about the "error going in the same direction" - it is not only about the clockspeed being a multiple of the bitrate so that you minimize the error but also frequency stability. The USB AVR uses a regular crystal on the new Arduino Uno. The main AVR is using a resonator. So if their frequences drift, it is going to be tougher and tougher to keep things in sync and error-free over long transmissions at high bitrates.

[legacy]

Arduino vs Teensy

The atmega328p in the Arduino/Uno serie does serial transmit and receive, and a separate chip converts this into serial-over-USB. The Teensy uses a different AVR chip that has built-in USB support, so there is no need to go via serial. The manufacturer of the Teensy claims that the Teensy has direct 12 MBit/sec USB access and also claims for the RAWHID library functions:  You can send up to 1000 packets per second in each direction, where each packet is 64 bytes

my protocol is able to work at 500Kbps over usb-serial just because it has acknowledges (send data packet and wait, then go on), but it's a dirty trick

[legacy]

The reason people have trouble with none-standard baudrates on Linux is because the kernel developers created 2 different interfaces to set none-standard baudrates and not all drivers/kernel version support both, and not all serial libraries support both. In this case, PySerial does not support Termios2.

from here, also … it seems the problem is kernel side and libc side, also i am not using the the termio2