Anyone know how to write a basic USB stack?

[technix]

The light switch boards arrived and I have assembled one. Now I need to get the USB of the STM32F042F4P6 up. Almost all material I read online points me at ST's firmware library. That library depends on either Cube or Std_Periph_Library, neither of which I feel comfortable touching with a ten-foot pole due to the overly complicated structure and the associated code bloat. This is a microcontroller with 16kB Flash and 6kB SRAM - bloat is the last thing I want to introduce.

So is there any guide on writing a simple USB stack from the ground up, directly operating on the registers? I am okay with being limited to one endpoint as that is all needed to implement a vendor-class device with WinUSB response headers, which would work anywhere with libusb.

This is also relevant when writing code for those 8-bit USB microcotnrollers, like ATmega16U2 or PIC16F1455.

[xani]

https://www.eevblog.com/forum/microcontrollers/how-do-you-write-a-usb-stack/

[Howardlong]

You could take a look at how V-USB does it, it doesn't even need a USB peripheral, just GPIOs and some careful timing. It's limited to low speed devices though, which might or might not be a problem depending on your application.

[mikeselectricstuff]

You could take a look at how V-USB does it, it doesn't even need a USB peripheral, just GPIOs and some careful timing. It's limited to low speed devices though, which might or might not be a problem depending on your application.

If it has USB hardware, it make sense to use it. The problem is where the documentation of the product says little more than "Use our driver library" , or is inaccurate & issues only know internally by the people who did the library.

[technix]

You could take a look at how V-USB does it, it doesn't even need a USB peripheral, just GPIOs and some careful timing. It's limited to low speed devices though, which might or might not be a problem depending on your application.

If it has USB hardware, it make sense to use it. The problem is where the documentation of the product says little more than "Use our driver library" , or is inaccurate & issues only know internally by the people who did the library.

Even the library does NOT work: I decided to take a look at Cube's emitted code. The compiled library boilerplate (ST's USB library + Cube, its hard dependency) took 16,144 out of 16,384 available bytes of Flash memory with -Og optimization, 13,764 if optimized with -Os (breaking debugging.) So 240 bytes for the whole application?!? I am seriously wondering how did ST fit their DFU bootloader into the 13k mask ROM on the chip...

Currently my hand written code used 2,020 bytes. (No USB yet, not going to touch the bloatware known as Cube or ST's libraries any time soon.)

[tszaboo]

Even the library does NOT work: I decided to take a look at Cube's emitted code. The compiled library boilerplate (ST's USB library + Cube, its hard dependency) took 16,144 out of 16,384 available bytes of Flash memory with -Og optimization, 13,764 if optimized with -Os (breaking debugging.) So 240 bytes for the whole application?!? I am seriously wondering how did ST fit their DFU bootloader into the 13k mask ROM on the chip...

Currently my hand written code used 2,020 bytes. (No USB yet, not going to touch the bloatware known as Cube or ST's libraries any time soon.)

Wow, that's gotta be a new awesome checkbox functionality. Yes it has USB. Yes the code uses 99% of the flash. So what? We said it has USB 
It also has a bunch of other peripherals. All of them on the same pins. Checkbox checked!

[__daz__]

I found this blog entry to be very useful for similar purposes I had in the past  http://andybrown.me.uk/2016/01/09/f042usbhid

[JPortici]

(^ Link is broken.. http://andybrown.me.uk/2016/01/09/f042usbhid/ )

I have the same complaint for microchip.
USB stack for PIC32MK is bigger than the boot flash area, how the * i'm supposed to write an usb bootloader using your tools? Of course i can't use your ready-made usb BL because there isn't one for MK in harmony 2.04 so i have to wait and hope for the new harmony release.. which will be when? who knows!

[ataradov]

An improved version of my USB stack is available here https://github.com/ataradov/vcp . It takes < 4K, and works perfectly on a 16 K device (SAM D11).

And ig you want USB bootloader, then I've got one as well https://github.com/ataradov/bootloaders .

[technix]

Even the library does NOT work: I decided to take a look at Cube's emitted code. The compiled library boilerplate (ST's USB library + Cube, its hard dependency) took 16,144 out of 16,384 available bytes of Flash memory with -Og optimization, 13,764 if optimized with -Os (breaking debugging.) So 240 bytes for the whole application?!? I am seriously wondering how did ST fit their DFU bootloader into the 13k mask ROM on the chip...

Currently my hand written code used 2,020 bytes. (No USB yet, not going to touch the bloatware known as Cube or ST's libraries any time soon.)

Wow, that's gotta be a new awesome checkbox functionality. Yes it has USB. Yes the code uses 99% of the flash. So what? We said it has USB 
It also has a bunch of other peripherals. All of them on the same pins. Checkbox checked!

Umm what are you talking about? 

I have enabled the following peripherals in Cube, all of which needed for my application:

* USB (duh)
* External crystal (dependency of USB)
* 2 PWM on TIM3 (to dim the light and the status LED)
* The USB HID Class driver
* Serial port (for debug messages)

When I tried removing the serial port driver (as if in the final product, where debug messages are no longer needed) and -Os optimization there is still 11,992 bytes of boilerplate.

Just for fun, enabling just the same set of peripherals as my own startup code, Cube emitted 3,596 bytes of code under -Os. While my own code emitted 2,020 bytes with -Og (a lot less optimization than -Os.) Speaking of bloat, that is 1.5k pure bloat introduced by the underwhelming architecture and WET code.

[technix]

An improved version of my USB stack is available here https://github.com/ataradov/vcp . It takes < 4K, and works perfectly on a 16 K device (SAM D11).

And ig you want USB bootloader, then I've got one as well https://github.com/ataradov/bootloaders .

What is the licensing on your code? Can I take some of your code and put it in my project, which is under a 3-clause BSD license?

[ataradov]

What is the licensing on your code? Can I take some of your code and put it in my project, which is under a 3-clause BSD license?

It is all 3-BSD, feel free to do whatever you want to do with it. No attribution or acknowledgment is required.

[technix]

What is the licensing on your code? Can I take some of your code and put it in my project, which is under a 3-clause BSD license?

It is all 3-BSD, feel free to do whatever you want to do with it. No attribution or acknowledgment is required.

Thanks. I will try to fit your code (maybe not the VCP part - just the enumeration part as I intend to end up using libusb for my project) onto STM32F042.

[IconicPCB]

Technix,

chek out http://www.fourwalledcubicle.com/LUFA.php

[funkathustra]

Even the library does NOT work: I decided to take a look at Cube's emitted code. The compiled library boilerplate (ST's USB library + Cube, its hard dependency) took 16,144 out of 16,384 available bytes of Flash memory with -Og optimization, 13,764 if optimized with -Os (breaking debugging.) [...]
I have enabled the following peripherals in Cube, all of which needed for my application:

* USB (duh)
* External crystal (dependency of USB)
* 2 PWM on TIM3 (to dim the light and the status LED)
* The USB HID Class driver
* Serial port (for debug messages)

For what it's worth, when I enable LTO, I get a slightly-lower 15116-byte flash file with these peripherals enabled. 1300 bytes of flash is plenty of space to implement an RGB LED thing. You can do it with, like, five lines of code.

Removing the serial port gets you down to 13944. -Os gets you wayyy down to 7880 bytes. That's *plenty* of flash for any USB FS HID application I've ever heard of.

I think you're missing the point. The 16 KB (F4) version of the '042 is what you deploy in production (-Os) to save a few cents --- but you'd never develop on them. You develop on the F6 version, which has a much-roomier 32 KB of flash.

Why on earth would you write a USB stack from scratch? If your time is really that worthless, please let me know and I'll send you some contract work I need done 

I think you need to adjust your code size expectations to the platform you're on. If you were on an 8-bit part, 16 KB of flash is plenty for USB (even 8 KB works fine). But on a 32-bit ARM part — even in Thumb mode — that flash gets eaten up pretty quickly (thanks, RISC... ugh).

An improved version of my USB stack is available here https://github.com/ataradov/vcp . It takes < 4K, and works perfectly on a 16 K device (SAM D11).

technix is using HID, not VCP (who uses VCP for anything other than legacy products?)

[ataradov]

Why on earth would you write a USB stack from scratch? If your time is really that worthless, please let me know and I'll send you some contract work I need done 

It is not that hard. It actually saves time in a long run.

technix is using HID, not VCP (who uses VCP for anything other than legacy products?)

That's why I also gave a link to HID bootloader. HID is just raw endpoints, it is way easier and simpler than VCP.

[funkathustra]

Why on earth would you write a USB stack from scratch?

It is not that hard. It actually saves time in a long run.

I'm not saying it's hard — just that it takes time, and seems redundant. And, if you look through the ST USB stack, I don't see a lot of room for shrinking it in size. It's only got three source files, none of them more than 500 lines long (and 50% of that is Doxygen).

And: how does it save time in the long run?

I've regularly done USB projects for at least 8 years — with Microchip, Freescale, Silicon Labs, and ST stacks — and I've never had any issues nor reasons to build a stack from scratch. I've definitely had to get in there and modify the vendor stack to do different things, but that's a far cry from reinventing the wheel.

I found a minor bug once in the SiLabs stack, sent them a message with the fix, and they patched it in the next release — along with patches for all sorts of other bugs that cover edge cases that I could never thoroughly test myself.

That's why I also gave a link to HID bootloader. HID is just raw endpoints, it is way easier and simpler than VCP.

Didn't notice anything about HID. Comment retracted! And yes, HID and Vendor-class devices are super lightweight.

[NorthGuy]

I'm not saying it's hard — just that it takes time, and seems redundant.

IMHO, it's the best way to learn how USB works.

[technix]

Why on earth would you write a USB stack from scratch?

It is not that hard. It actually saves time in a long run.

I'm not saying it's hard — just that it takes time, and seems redundant. And, if you look through the ST USB stack, I don't see a lot of room for shrinking it in size. It's only got three source files, none of them more than 500 lines long (and 50% of that is Doxygen).

And: how does it save time in the long run?

I've regularly done USB projects for at least 8 years — with Microchip, Freescale, Silicon Labs, and ST stacks — and I've never had any issues nor reasons to build a stack from scratch. I've definitely had to get in there and modify the vendor stack to do different things, but that's a far cry from reinventing the wheel.

I found a minor bug once in the SiLabs stack, sent them a message with the fix, and they patched it in the next release — along with patches for all sorts of other bugs that cover edge cases that I could never thoroughly test myself.

That's why I also gave a link to HID bootloader. HID is just raw endpoints, it is way easier and simpler than VCP.

Didn't notice anything about HID. Comment retracted! And yes, HID and Vendor-class devices are super lightweight.

ST's USB stack itself may be small, its dependency sure isn't. The serial port driver should not take a lot of kB. The blank startup code should be a kB or two smaller. Cube is the center of bloat here, and if I am using ST's libraries I cannot get rid of it.

[krho]

@technix:

I don't know if there is low-level driver for f0, but if it is, try checking that option in cubeMX, this should shrink things.
The other thing as mentioned in this thread use LTO, it should throw out a lot of code even with Og.

[mikeselectricstuff]

Why on earth would you write a USB stack from scratch?

It is not that hard. It actually saves time in a long run.

I'm not saying it's hard — just that it takes time, and seems redundant. And, if you look through the ST USB stack, I don't see a lot of room for shrinking it in size. It's only got three source files, none of them more than 500 lines long (and 50% of that is Doxygen).

And: how does it save time in the long run?

I've regularly done USB projects for at least 8 years — with Microchip, Freescale, Silicon Labs, and ST stacks — and I've never had any issues nor reasons to build a stack from scratch. I've definitely had to get in there and modify the vendor stack to do different things, but that's a far cry from reinventing the wheel.

I found a minor bug once in the SiLabs stack, sent them a message with the fix, and they patched it in the next release — along with patches for all sorts of other bugs that cover edge cases that I could never thoroughly test myself.

That's why I also gave a link to HID bootloader. HID is just raw endpoints, it is way easier and simpler than VCP.

Didn't notice anything about HID. Comment retracted! And yes, HID and Vendor-class devices are super lightweight.

ST's USB stack itself may be small, its dependency sure isn't. The serial port driver should not take a lot of kB. The blank startup code should be a kB or two smaller. Cube is the center of bloat here, and if I am using ST's libraries I cannot get rid of it.

If any serial port driver takes more than 1K it needs to be taken out the back and shot. Ditto startup code.

[technix]

@technix:

I don't know if there is low-level driver for f0, but if it is, try checking that option in cubeMX, this should shrink things.
The other thing as mentioned in this thread use LTO, it should throw out a lot of code even with Og.

There isn't a LL driver for F0. As of LTO it somehow does not play nice with the copy of newlib ARM shipped. Speaking of, my code contains a good portion of linker script magic, which is not compatible with LTO.

[technix]

Why on earth would you write a USB stack from scratch?

It is not that hard. It actually saves time in a long run.

I'm not saying it's hard — just that it takes time, and seems redundant. And, if you look through the ST USB stack, I don't see a lot of room for shrinking it in size. It's only got three source files, none of them more than 500 lines long (and 50% of that is Doxygen).

And: how does it save time in the long run?

I've regularly done USB projects for at least 8 years — with Microchip, Freescale, Silicon Labs, and ST stacks — and I've never had any issues nor reasons to build a stack from scratch. I've definitely had to get in there and modify the vendor stack to do different things, but that's a far cry from reinventing the wheel.

I found a minor bug once in the SiLabs stack, sent them a message with the fix, and they patched it in the next release — along with patches for all sorts of other bugs that cover edge cases that I could never thoroughly test myself.

That's why I also gave a link to HID bootloader. HID is just raw endpoints, it is way easier and simpler than VCP.

Didn't notice anything about HID. Comment retracted! And yes, HID and Vendor-class devices are super lightweight.

ST's USB stack itself may be small, its dependency sure isn't. The serial port driver should not take a lot of kB. The blank startup code should be a kB or two smaller. Cube is the center of bloat here, and if I am using ST's libraries I cannot get rid of it.

If any serial port driver takes more than 1K it needs to be taken out the back and shot. Ditto startup code.

My own startup code takes about 500 bytes if bypassing newlib's _start. Calling _start (to properly initialize the C and C++ runtime) the code grew to 1.5k though.

[technix]

If your time is really that worthless, please let me know and I'll send you some contract work I need done 

Please let me know if you have any. Be aware that my time will be costly.

I am forced to write my own USB stack because of the sheer bloat of ST's libraries. 99% of the flash for just the boilerplate... You got to be kidding me.

[mark03]

I don't know if there is low-level driver for f0, but if it is, try checking that option in cubeMX, this should shrink things.

At the last place I worked, we were implementing a suite of STM32 drivers for an IoT reference design.  The client insisted that we build it on top of the ST LL drivers.  I could not see the point of this.  As far as I could tell, the LL "drivers" are nothing more than a thin obfuscation layer on top of the register interface.  Instead of writing a register directly, you call an LL function which...  writes a register.  Adding code layers is no benefit unless you are abstracting away some details, or encapsulating complexity somehow.  AFAICT the LL drivers do neither.

I haven't tried it out yet, but here's an independent USB driver which looks interesting:

Lightweight USB Device Stack

[donotdespisethesnake]

If your time is really that worthless, please let me know and I'll send you some contract work I need done 

Please let me know if you have any. Be aware that my time will be costly.

I am forced to write my own USB stack because of the sheer bloat of ST's libraries. 99% of the flash for just the boilerplate... You got to be kidding me.

That's really a surprise, because companies like ST make all their money giving away high-quality source code, and not from selling bigger chips. As you know, hiring people to write good software is incredibly cheap, because it's so easy anyone can do it.

 

[funkathustra]

If your time is really that worthless, please let me know and I'll send you some contract work I need done 

Please let me know if you have any. Be aware that my time will be costly.

I am forced to write my own USB stack because of the sheer bloat of ST's libraries. 99% of the flash for just the boilerplate... You got to be kidding me.

Who cares? Use the 32 KB chip like I mentioned, and then swap out to the 16 KB part for production?

Or, just... don't swap it out. The 32K part is seven cents more expensive in singles than the 16K part on DigiKey. Give me your address and I'll personally cut you a check for the price difference.

My own startup code takes about 500 bytes if bypassing newlib's _start. Calling _start (to properly initialize the C and C++ runtime) the code grew to 1.5k though.

Most ARM Cortex-M0 MCUs have a vector table that takes 1K alone. That's not even counting code.

If any serial port driver takes more than 1K it needs to be taken out the back and shot. Ditto startup code.

This isn't an 8051-type CISC-style processor, this is a 32-bit ARM microcontroller. A UART library that weighs in at 1K on an ARM chip is equivalent to 500 RISC instructions. You think you could write a software-buffered, interrupt-based UART library in less than 500 RISC instructions? Something that actually handles and reports all the error conditions? I'm not talking about a three-function Init()/Read()/Write() blocking driver.

Unless you're stuck on an island with the 16 KB chips, I really see no point to getting pedantic over this stuff.

But I get the feeling that I'm in the minority in this thread. Still, I wish someone could articulate specific reasons why anyone should care about spending time trimming down this stuff.

Different strokes, I guess....

[mikeselectricstuff]

I don't know if there is low-level driver for f0, but if it is, try checking that option in cubeMX, this should shrink things.

At the last place I worked, we were implementing a suite of STM32 drivers for an IoT reference design.  The client insisted that we build it on top of the ST LL drivers.  I could not see the point of this.  As far as I could tell, the LL "drivers" are nothing more than a thin obfuscation layer on top of the register interface.  Instead of writing a register directly, you call an LL function which...  writes a register.  Adding code layers is no benefit unless you are abstracting away some details, or encapsulating complexity somehow.  AFAICT the LL drivers do neither.

I haven't tried it out yet, but here's an independent USB driver which looks interesting:

Lightweight USB Device Stack

Apart from complex stuff like USB, TCP etc. it rarely makes any sense to use manufacturer libraries - things like serial are trivial to write for a specific application and pretty much impossible to write a general-purpose driver that suits everything  it's a waste of time trying.
A developer's time is much better spent learning how the hardware works than learning how a device library works, and it will be way easier to debug your own code that something that comes from elsewhere.
There seems to be a stupid attitude across many manufacturers that as they need to supply drivers for complex stuff, the also need to do it for simpler stuff as well, and it's generally a total waste of time for everyone. I'm in the process of extracting the useful parts from Microchip's USB Host driver from the car-crash of ridiculousness created by their Harmony framework - 10 levels of directories ( most contain only one object), multiple levels of abstraction for the simplest I/O port access - it's just ridiculous.
 

[technix]

If your time is really that worthless, please let me know and I'll send you some contract work I need done 

Please let me know if you have any. Be aware that my time will be costly.

I am forced to write my own USB stack because of the sheer bloat of ST's libraries. 99% of the flash for just the boilerplate... You got to be kidding me.

Who cares? Use the 32 KB chip like I mentioned, and then swap out to the 16 KB part for production?

Or, just... don't swap it out. The 32K part is seven cents more expensive in singles than the 16K part on DigiKey. Give me your address and I'll personally cut you a check for the price difference.

I'd like to, but sourcing them in China is another story.

My own startup code takes about 500 bytes if bypassing newlib's _start. Calling _start (to properly initialize the C and C++ runtime) the code grew to 1.5k though.

Most ARM Cortex-M0 MCUs have a vector table that takes 1K alone. That's not even counting code.

For Cortex-M0 the maximum possible vector table size is 192 bytes (16 core vectors and up to 32 NVIC vectors. You may want to take a good read at Cortex-M0 TRM.) The remaining 300 bytes is more than enough for setting up clocks and copying data sections.

If any serial port driver takes more than 1K it needs to be taken out the back and shot. Ditto startup code.

This isn't an 8051-type CISC-style processor, this is a 32-bit ARM microcontroller. A UART library that weighs in at 1K on an ARM chip is equivalent to 500 RISC instructions. You think you could write a software-buffered, interrupt-based UART library in less than 500 RISC instructions? Something that actually handles and reports all the error conditions? I'm not talking about a three-function Init()/Read()/Write() blocking driver.

Unless you're stuck on an island with the 16 KB chips, I really see no point to getting pedantic over this stuff.

But I get the feeling that I'm in the minority in this thread. Still, I wish someone could articulate specific reasons why anyone should care about spending time trimming down this stuff.

Different strokes, I guess....

At least for me my driver implemented almost non-blocking (block only when buffer is full) interrupt driven PIO within that limit, if optimized with -Os.

[lucazader]

Just throwing my 2c in here, feel free to ignore this..

I am currently developing a product with the F042K4/6 processor.

In regards to generated code size:
I have full USB-CDC and UART comms, custom serial protocol handling using the crc  and some leds and GPIOs.
As of this afternoon that all fits in less than 12k of flash, leaving a nice 4k buffer for other things. This is all based on an stm cube generated project using -Os and -flto.
One of the main things I have found in the stm cube that can help in reducing code size is to disable interupts that are not being used. Eg with the F042, if you dont need to wake up off uart1, or you dont need the interrupt handler for it, then disabling that save a few hundred kb.
Yes the generated code may appear large at first. However when I got stuck into the development I realised that adding extra features and implementing protocol decoding didnt actually increase the binary size significantly.

In regards to the chip pricing:
While the 32kb chip is only 7c more on digikey they are about 20-30c more (than the 16k variant) when sourced in china (currently). Which on a chip that only costs $1.2 ish is quite a large percentage.

On the argument of creating a USB stack:
I can see the cost-benefit of this if you are producing 10's of thousands of units. However with the current project i started coding on Tuesday (3 days of work so far). I am currently going through final functionality testing right now. I doubt developing, or even porting a usb stack of any sort would enable a quicker development cycle like this.

[BradC]

I'm not saying it's hard — just that it takes time, and seems redundant.

IMHO, it's the best way to learn how USB works.

I did that on the Parallax Propeller. It was a great way to learn about the low level details of USB. It also taught me a crap load about buggy host implementations and how to diagnose and work around them. The authors of the VIA stack back around the Windows 98 era needed taking out and shooting.

[andyturk]

As far as I could tell, the LL "drivers" are nothing more than a thin obfuscation layer on top of the register interface.  Instead of writing a register directly, you call an LL function which...  writes a register.  Adding code layers is no benefit unless you are abstracting away some details, or encapsulating complexity somehow.  AFAICT the LL drivers do neither.

I actually like using the LL "drivers". They are rather thin, but they do abstract away some of the STM32 minutiae that vary between families. Basically, what you get is a big collection of macros with a standard naming convention and InitStruct routines to configure the hardware. In contrast to the Cube's HAL, the LL stuff stays away from timing issues, buffering, callbacks, etc.

[technix]

I ended up using this: https://github.com/dmitrystu/libusb_stm32

It enumerated. I kind of have to use HID, but Linux grabbing the device isn't that much of a hassle after all thanks to libhidapi, which is cross platform like libusb, and does not have problems with Linux kernel grabbing the device.

[mikeselectricstuff]

From what I've seen in various places, once you have a working stack that's often only half the battle.
The rest is knowing all the quirks of incorrectly implemented features on the things you want it to connect to.
There are some comments that refer to this in the Keysight interview, where he remarks that Microsoft have the best knowledge of USB and networking from all the zillions of PCs out there.

[ataradov]

From what I've seen in various places, once you have a working stack that's often only half the battle.
The rest is knowing all the quirks of incorrectly implemented features on the things you want it to connect to.

This may be true as a host, but as a device, I don't see an issue. There are differences between the OSes when it comes to marginal stuff, but if you follow the spec on your end, things will just work. At least in my experience.

It is interesting that is pretty easy to crash both Windows and Linux though a bad behaving device, which happens sometimes during stack development.

[technix]

I took a third party stack actually, and thankfully it is entirely interrupt driven so I just initialized it in a constructor and set up the USB interrupt for it. No direct reference in main. Yay? And it took me a little over 5k Flash. Speaking of bloat in Cube.

Now I need to get the USB HID Vendor Class Device up... Do you know how to get Windows enumerate it properly? It enumerates for now, but it says "requested operation failed."

[ataradov]

Now I need to get the USB HID Vendor Class Device up... Do you know how to get Windows enumerate it properly? It enumerates for now, but it says "requested operation failed."

Show your descriptors. It usually just works out of the box.

[technix]

Now I need to get the USB HID Vendor Class Device up... Do you know how to get Windows enumerate it properly? It enumerates for now, but it says "requested operation failed."

Show your descriptors. It usually just works out of the box.

Code: [Select]

static const struct usb_device_descriptor device_descriptor =
{
    .bLength            = sizeof(struct usb_device_descriptor),
    .bDescriptorType    = USB_DTYPE_DEVICE,
    .bcdUSB             = VERSION_BCD(2,0,0),
    .bDeviceClass       = USB_CLASS_PER_INTERFACE,
    .bDeviceSubClass    = USB_SUBCLASS_NONE,
    .bDeviceProtocol    = USB_PROTO_NONE,
    .bMaxPacketSize0    = USB_EP0_SIZE,
    .idVendor           = USB_VID,
    .idProduct          = USB_PID,
    .bcdDevice          = VERSION_BCD(0,1,0),
    .iManufacturer      = 1,
    .iProduct           = 2,
    .iSerialNumber      = INTSERIALNO_DESCRIPTOR,
    .bNumConfigurations = 1
};

static const struct config_descriptor config_descriptor =
{
.config =
{
        .bLength                = sizeof(struct usb_config_descriptor),
        .bDescriptorType        = USB_DTYPE_CONFIGURATION,
        .wTotalLength           = sizeof(config_descriptor),
        .bNumInterfaces         = 1,
        .bConfigurationValue    = 1,
        .iConfiguration         = NO_DESCRIPTOR,
        .bmAttributes           = USB_CFG_ATTR_RESERVED,
        .bMaxPower              = USB_CFG_POWER_MA(300)
},
.interface =
{
        .bLength                = sizeof(struct usb_interface_descriptor),
        .bDescriptorType        = USB_DTYPE_INTERFACE,
        .bInterfaceNumber       = 0,
        .bAlternateSetting      = 0,
        .bNumEndpoints          = 2,
        .bInterfaceClass        = USB_CLASS_HID,
        .bInterfaceSubClass     = USB_SUBCLASS_NONE,
        .bInterfaceProtocol     = USB_PROTO_NONE,
        .iInterface             = NO_DESCRIPTOR,
},
.endpoints =
{
{
        .bLength                = sizeof(struct usb_endpoint_descriptor),
        .bDescriptorType        = USB_DTYPE_ENDPOINT,
        .bEndpointAddress       = USB_RXD_EP,
        .bmAttributes           = USB_EPTYPE_INTERRUPT,
        .wMaxPacketSize         = USB_PKT_SZ,
        .bInterval              = 100,
},
{
        .bLength                = sizeof(struct usb_endpoint_descriptor),
        .bDescriptorType        = USB_DTYPE_ENDPOINT,
        .bEndpointAddress       = USB_TXD_EP,
        .bmAttributes           = USB_EPTYPE_INTERRUPT,
        .wMaxPacketSize         = USB_PKT_SZ,
        .bInterval              = 100,
}
}
};

static const uint8_t usb_hid_report[] =
{
    0x06, 0x00, 0xff,              // USAGE_PAGE (Vendor Defined Page 1)
    0x09, 0x01,                    // USAGE (Vendor Usage 1)
    0xa1, 0x01,                    // COLLECTION (Application)
    0x85, 0x01,                    //   REPORT_ID (1)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x27, 0xff, 0xff, 0x00, 0x00,  //   LOGICAL_MAXIMUM (65535)
    0x75, 0x10,                    //   REPORT_SIZE (16)
    0x95, 0x01,                    //   REPORT_COUNT (1)
    0x81, 0x82,                    //   INPUT (Data,Var,Abs,Vol)
    0x85, 0x02,                    //   REPORT_ID (2)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x27, 0xff, 0xff, 0x00, 0x00,  //   LOGICAL_MAXIMUM (65535)
    0x75, 0x10,                    //   REPORT_SIZE (16)
    0x95, 0x01,                    //   REPORT_COUNT (1)
    0x91, 0x82,                    //   OUTPUT (Data,Var,Abs,Vol)
    0xc0                           // END_COLLECTION
};

static const struct usb_hid_descriptor hid_descriptor =
{
.bLength                = sizeof(struct usb_hid_descriptor),
.bDescriptorType        = USB_DTYPE_HID,
.bcdHID                 = VERSION_BCD(1,1,1),
.bCountryCode           = USB_HID_COUNTRY_US,
.bNumDescriptors        = 1,
.bDescriptorType0       = USB_DTYPE_HID_REPORT,
.wDescriptorLength0     = sizeof(usb_hid_report)
};
Should I make the TX and RX buffers larger? Currently they are 2-byte buffers matching the uint16_t from the PWM side of things.

[ataradov]

Your report descriptor looks unusual. It may work, but typically what you see for HID used as a general purpose data transfer thing you want to specify reports to be exact size of the endpoint.

Copy the descriptor from here https://github.com/ataradov/bootloaders/blob/master/firmware/usb_hid_samd11/usb_descriptors.c , it is known to work without problems on all OSes.

[technix]

Your report descriptor looks unusual. It may work, but typically what you see for HID used as a general purpose data transfer thing you want to specify reports to be exact size of the endpoint.

Copy the descriptor from here https://github.com/ataradov/bootloaders/blob/master/firmware/usb_hid_samd11/usb_descriptors.c , it is known to work without problems on all OSes.

I am using 2-byte transfers... Maybe I have other implementation problems...

[janoc]

If your time is really that worthless, please let me know and I'll send you some contract work I need done 

Please let me know if you have any. Be aware that my time will be costly.

I am forced to write my own USB stack because of the sheer bloat of ST's libraries. 99% of the flash for just the boilerplate... You got to be kidding me.

If you can deal with the license (or are willing to purchase a commercial one), then libopencm3 and ChibiOS contain both very good USB stacks for various ST chips, with lots of examples. Getting a HID device or serial port over USB running with these takes no time at all.

Both also contain decent HAL, so you can skip the entire Cube mess.

[NorthGuy]

Now I need to get the USB HID Vendor Class Device up... Do you know how to get Windows enumerate it properly? It enumerates for now, but it says "requested operation failed."

What do you mean "It enumerates for now". Do you see the HID device in the device manager?

Who says "requested operation failed"?

[technix]

Now I need to get the USB HID Vendor Class Device up... Do you know how to get Windows enumerate it properly? It enumerates for now, but it says "requested operation failed."

What do you mean "It enumerates for now". Do you see the HID device in the device manager?

Who says "requested operation failed"?

It says HID, but there is a yellow exclamation mark on it, and Windows is the one claiming "requested operation failed."

[technix]

Well I have managed to get it to enumerate under macOS and Windows. Now Linux is spitting out EPIPE like this:

Code: [Select]

[71430.459031] usb 3-3.1: new full-speed USB device number 21 using xhci_hcd
[71430.568139] usb 3-3.1: New USB device found, idVendor=0002, idProduct=c001
[71430.568141] usb 3-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=254
[71430.568142] usb 3-3.1: Product: USB Light Switch
[71430.568143] usb 3-3.1: Manufacturer: SushiBits by Max Chan <xcvista@me.com>
[71430.568143] usb 3-3.1: SerialNumber: CD40BBB5
[71430.582643] usbhid 3-3.1:1.0: can't add hid device: -32
[71430.582653] usbhid: probe of 3-3.1:1.0 failed with error -32
[71434.831073] usb 3-3.1: USB disconnect, device number 21

[ataradov]

Try my HID report descriptor. I don't know what different OSes actually do with them, but I see no other reason for this behavior.

[technix]

Try my HID report descriptor. I don't know what different OSes actually do with them, but I see no other reason for this behavior.

It is not working under Linux. I think my heavily reduced descriptor is perfectly fine:

Code: [Select]

static const uint8_t usb_hid_report[] =
{
    0x06, 0x00, 0xff,              // USAGE_PAGE (Vendor Defined Page 1)
    0x09, 0x01,                    // USAGE (Vendor Usage 1)
    0xa1, 0x01,                    // COLLECTION (Application)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x27, 0xff, 0xff, 0x00, 0x00,  //   LOGICAL_MAXIMUM (65535)
    0x75, 0x10,                    //   REPORT_SIZE (16)
    0x95, 0x01,                    //   REPORT_COUNT (1)
    0x09, 0x01,                    //   USAGE (Vendor Usage 1)
    0x81, 0x82,                    //   INPUT (Data,Var,Abs,Vol)
    0x09, 0x01,                    //   USAGE (Vendor Usage 1)
    0x91, 0x82,                    //   OUTPUT (Data,Var,Abs,Vol)
    0xc0                           // END_COLLECTION
};

[NorthGuy]

Well I have managed to get it to enumerate under macOS and Windows. Now Linux is spitting out EPIPE like this:

Code: [Select]

[71430.459031] usb 3-3.1: new full-speed USB device number 21 using xhci_hcd
[71430.568139] usb 3-3.1: New USB device found, idVendor=0002, idProduct=c001
[71430.568141] usb 3-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=254
[71430.568142] usb 3-3.1: Product: USB Light Switch
[71430.568143] usb 3-3.1: Manufacturer: SushiBits by Max Chan <xcvista@me.com>
[71430.568143] usb 3-3.1: SerialNumber: CD40BBB5
[71430.582643] usbhid 3-3.1:1.0: can't add hid device: -32
[71430.582653] usbhid: probe of 3-3.1:1.0 failed with error -32
[71434.831073] usb 3-3.1: USB disconnect, device number 21

I would guess something is wrong with the HID descriptor. It gets the device, but cannot decode report descriptiors or doesn't receive the report descriptors at all.

Your descriptor is similar to my descriptors which are working except that the sizes/counts are different. Except that mine don't have report ids (which you don't need anyway since you only have one report of every kind) and specify usage for every report.

It is very easy with Linux. You just look at the source code and figure out in what case it returns the above error -32. You can even re-compile it and output your own log messages to see what's wrong.

[ataradov]

My descriptor absolutely works on all platforms. It was created a long time ago and travels from project to project.

There must be something  wrong with the way you send it.

[NorthGuy]

My descriptor absolutely works on all platforms. It was created a long time ago and travels from project to project.

Yes, mine too. It is practically identical to the last one technix has posted except for sizes, counts and presence of the feature report.

[technix]

My descriptor absolutely works on all platforms. It was created a long time ago and travels from project to project.

There must be something  wrong with the way you send it.

I get the same error with your descriptors. Maybe Linux need some other method to feed the report descriptor?

[NorthGuy]

I get the same error with your descriptors. Maybe Linux need some other method to feed the report descriptor?

It is all standard on all platform. HID descriptor is passed two ways - in between interface descriptors and EP descriptors within device descriptor and also in response to a separate request. Report descriptor is passed in response to a special  request. Read the specs if in doubt.

You have the source code of your lib and the Linux source code. Have you looked at these? Is Linux requesting something your lib doesn't give? Very easy to figure out what's going on if you just look. No reason to guess.

[technix]

I get the same error with your descriptors. Maybe Linux need some other method to feed the report descriptor?

It is all standard on all platform. HID descriptor is passed two ways - in between interface descriptors and EP descriptors within device descriptor and also in response to a separate request. Report descriptor is passed in response to a special  request. Read the specs if in doubt.

You have the source code of your lib and the Linux source code. Have you looked at these? Is Linux requesting something your lib doesn't give? Very easy to figure out what's going on if you just look. No reason to guess.

I am also poking at it on Windows (Thanks god there is VMware and the super versatile and tolerating USB stack of macOS) and it seem to me that the USB stack is not handling reenumeration properly. Is there any pointer on what is happening when a USB device is being reenumerated?

[ataradov]

Your USB device seems to be enumerating correctly  according to the Linux output. HID part is having problems.

There is no fixed enumeration process, host will request descriptors in any order it feels like (starting with device descriptor). Just log all requests and see if you handle all of them.

Specifically HID descriptor is requested with a Get Descriptor command as a standard interface descriptor. And your stack should send the response over the control endpoint.

[NorthGuy]

Is there any pointer on what is happening when a USB device is being reenumerated?

http://www.usb.org/developers/docs/usb20_docs/usb_20_100617.zip

You need a file called usb_20.pdf

or in popular form:

http://www.usbmadesimple.co.uk/ums_4.htm

[technix]

I get HID Vendor Class Device enumeration. Source code here: https://github.com/SushiBits/LightSwitchUSB-Firmware

Now I need to fill in the actual code to turn HID reports on the interrupt endpoints into actions. Since the USB stack is entirely interrupt driven I think I have the main() loop all to this task. Or maybe I should run a timer to perform timed data synchronizes and leave main() be the loop of low power mode?

[ataradov]

From the quick look at the code, you just call usbd_ep_write() and the rest will happen automatically. Host will send an IN request (when software asks for it), and USB controller will send the data.

[technix]

It worked with macOS now. The host app is built with libhidapi, and somehow it is likely I can submit it to the Mac App Store after a makeover and some polishing. I am going to take a nap and start writing code for Linux and Windows. (My VB.net is rusty now, not having touched it for years...)

[funkathustra]

It worked with macOS now. The host app is built with libhidapi, and somehow it is likely I can submit it to the Mac App Store after a makeover and some polishing.

It's nice that HID API actually calls native IOKit stuff, instead of relying on LibUSB.

I am going to take a nap and start writing code for Linux and Windows. (My VB.net is rusty now, not having touched it for years...)

Out of curiosity, why not C#? You're obviously a capable C/C++ programmer.

[technix]

It worked with macOS now. The host app is built with libhidapi, and somehow it is likely I can submit it to the Mac App Store after a makeover and some polishing.

It's nice that HID API actually calls native IOKit stuff, instead of relying on LibUSB.

libhidapi have four backends now: native Windows, native macOS (IOKit), libusb on Linux, and hidraw on Linux.

I am going to take a nap and start writing code for Linux and Windows. (My VB.net is rusty now, not having touched it for years...)

Out of curiosity, why not C#? You're obviously a capable C/C++ programmer.

I had a history with VB.net: back when I was mainly developing for Windows I was using that language over C#.

[josip]

IMHO, it's the best way to learn how USB works.

Exactly.

I had a plenty of free time, and didn't know anything about USB. Tried with some books, but gave up (what the hell is endpoint  ).

Started assembler coding, by analyzing TI MSP430 open source USB stack (written in C), and soon all become clear. Found (I guess) all undocumented staff (related to USB module), and at the end, it (CDC) was fast (1MByte/s, unlimited buffer) small (almost 1 KByte) and lite for CPU. Later, I also made CDC bootloader, with (software) AES, and interface commands that fit in 2 Kbytes.

Now, there is no more plenty of free time, but there is no way that I will use any prepared USB stack for different MCU family.

[westfw]

and at the end, it was fast small, and lite for CPU

"and now I know enough that vendors' USB stacks all look even worse than they used to!"

Serious question: have you written another USB stack for a different chip?   If not, how much do you think the understanding you've gained would help, writing for the 2nd chip?

[ataradov]

Serious question: have you written another USB stack for a different chip?   If not, how much do you think the understanding you've gained would help, writing for the 2nd chip?

I did port mine across multiple different Atmel MCUs (with different USB controllers). I can say, it helped a lot.

Now instead of asking a question "where do I get a sample code for this controller, because USB is hard", I ask questions like "how do I do X on this controller", and I know exact set of Xs that are required to make USB work. It is much easier this way.

And my simple "stack" is structured in a way that you only need to re-implement one file to do the porting, so you pretty much have a minimal list of functions you need to make a USB stack work.

[josip]

"and now I know enough that vendors' USB stacks all look even worse than they used to!"

Vendors just give you example, how things can be done, and that's it. There are 3 options for user. If there is no performance / size / bug issue, original stack can be used as it is. Original stack can be optimized (MSP430, for example, http://www.indigresso.com/_blog/?p=33), or written from zero (as I did).
 

Serious question: have you written another USB stack for a different chip?   If not, how much do you think the understanding you've gained would help, writing for the 2nd chip?

I agree with Alex. All MSP430 devices have identical USB module, and my USB stack will work with all. Till today I didn't write USB stack for another family, but I have plan for the next year, to start with Cortex-M0+ (again, from zero, in assembler). And I don't expect significant problems. From my point of view, it is little more complicated, than writing code for UART on different families. If I be able to find some (undocumented) tricks (related to USB hardware module) to boost it up, like with MSP430, is another topic.

[NorthGuy]

Serious question: have you written another USB stack for a different chip?   If not, how much do you think the understanding you've gained would help, writing for the 2nd chip?

It took me about 2 weeks to get working USB on PIC16F1454. Most of that time was studying not-so-clear USB docs. Fortunately, the PIC's module was documented very well, so there was no problem there. If I needed to do it for other chip (and most likely I will have to do it soon for PIC32MZ, which is generally undocumented), I'd estimate 2, may be 3 days.

[westfw]

Quote

is there any guide on writing a simple USB stack from the ground up

Getting away from particular implementations, my understanding of USB Device in General goes something like this:

• Data is exchanged in "messages."  In general, messages are queued by the device and sent when polled by the hosts (at about 1kHz, for low and full speed.)
• Most USB device will have hardware that handles messages "per endpoint", and support multiple endpoints (number of endpoints varies.  Amount of buffering (# messages) per endpoint varies.  Max size of messages varies.)
• Endpoint 0 is the "control" endpoint and is magic and required.  When a device comes up, it communicates on EP0 about what it is, what other endpoints exist, and so on ("enumeration.")  EP0 is bidirectional (all other endpoints are unidirectional.)
• A lot of what passes for a "USB driver" is setup of the initial enumeration, going from some human-readable description of the device to the rather verbose message formats and initial communications.  Once everything is set up, a USB device is pretty similar to any other time-sliced message-based  flow-controlled communications.
• Which means that things like performance and latency depend pretty heavily on buffering strategies, particularly when going to/from byte-stream sources (CDC.)  You get so many messages-per-second, and high throughput requires putting as much data as possible in each message, as well as making sure that data that's been sent from user APIs is ready to be sent as a USB message when the host requests it.
• (this essentially skips whatever you'd need for really high-speed and superspeed comm.  Yeah, usb1.x!)

An corrections on the basic outline would be appreciated...

[ataradov]

Your biggest problem would be figuring when to send STALL, when to send Zero Length Packets, when to send nothing, and all that higher level stuff.

I just recently updated my VCP implementation, since it had very incorrect implementation for the control transfers (they use one more level of abstraction). I never needed them before, so it was not a huge deal, but I needed them for another projects, and discovered a problem. This may be an argument for using "professional" stacks, but it took a couple hours to fix, and I learned more about USB in the process, so time well spend, really.

With USB everything related to control endpoint is very messed up. Regular endpoints are very easy.

Supporting specific classes adds a whole new level of problems, especially given pretty poor USB documentation. But it is solvable as well.