32F417: How does USB CDC OUT (PC -> target) flow control work?

[peter-h]

I have my board running well, and flow control works in the "IN" direction (board -> PC).

But there are strange issues. I am testing with a loopback program which has variable size packets and at a point which is very close to 1024 bytes I see a lot of errors.

I do have a 1024 byte buffer in that data flow but changing that to 2048 doesn't change anything, and I suspect the flow control isn't working.

As with everything-USB it is damn complicated but AFAICT the data path is:

1) USB (FS mode) does an interrupt on each RX packet, and it has 1-64 bytes of data.
2) An ISR copies this into a 64 byte buffer
3) The ISR calls this

Code: [Select]


/**
  * @brief  Data received over USB OUT endpoint are sent over CDC interface
  *         through this function.
  *
  * We get here when there is some OUT data (OUT = PC to target). That data is
  * loaded into cdc_receive_temp_buffer[64], supplied to us here as "buf".
  * Data is then copied into our circular buffer cdc_receive_buffer[1024].
  * No more data can be sent by PC until USBD_CDC_ReceivePacket gets called.
  * This function is called by an ISR so no need to disable interrupts.
  *
  * @param  Buf: Buffer
  * @param  Len: Number of data received (in bytes)
  *
  */

static int8_t CDC_Receive_FS(uint8_t* buf, uint32_t *len)
{

if (*len > 0)
{

// copy over the data from buf, length len, into our circular buffer
for (uint16_t i = 0; i < *len; i++)
{
cdc_receive_buffer[cdc_receive_buffer_rx_index++] = buf[i];
cdc_receive_buffer_rx_index %= CDC_RX_BUFFER_SIZE;
if (cdc_receive_buffer_rx_index == cdc_receive_buffer_get_index)
{
// Buffer is full - discard a byte from it. This is necessary to resolve the ambiguity of
// the two pointers being equal if FULL or EMPTY. We make sure they are never equal after an input.
// Increment the get index to ensure that only the last BUFFER-1 bytes can be accessed
cdc_receive_buffer_get_index++;
cdc_receive_buffer_get_index %= CDC_RX_BUFFER_SIZE;
}
}

}

// Enable reception of next packet from Host
USBD_CDC_ReceivePacket(&hUsbDeviceFS);

return (USBD_OK);
}

4) The target application extracts data from the circular buffer

The last line above, USBD_CDC_ReceivePacket, is supposed to enable the next 1-64 byte packet to arrive from the PC. This is executed regardless of whether that circular buffer overflowed, which is not ideal because there is no flow control, but if I don't call USBD_CDC_ReceivePacket I need to call it later when the target application has extracted some data, but this is a difficult problem because with no data arriving that interrupt won't be happening. So I would need to call it from some foreground code, but then I have reentrancy issues because USBD_CDC_ReceivePacket calls a whole load of stuff which might get called by USB interrupts also. The obvious way is to skip the USBD_CDC_ReceivePacket if that buffer is full and call it from the code which is extracting data from that circular buffer, whenever >=64 bytes of room is in there.

As usual there are loads of people posting all over the place trying to solve this.

However I have established that when there are errors, the above buffer overflow condition is not happening. So it is something else.

AIUI, when a PC application sends data to USB VCP, it opens a file called say COM3 and sends data to it, in packets of any length. USB then packages this into 1-64 byte packets and sends it down the wire. There is no 1024 or whatever packet limit involved.

For an idea of how many layers there are in the data flow, this is the USB OUT (PC to target) interrupt:

[peter-h]

In case someone finds this in years to come, I have got this running, I think, and posted it here
https://community.st.com/s/question/0D53W00001qnO9ESAU/32f417-how-does-usb-cdc-out-pc-target-flow-control-work?t=1664975189841

[pcprogrammer]

USB is indeed complicated. There are many aspects in the whole control, but most of it is done by the USB controller. The number of bytes transferred from the host (PC) to the device (board) depends on the number given in the descriptor for the device. When set to 64 the host will send blocks of 64 bytes and the device has to handle these blocks.

I have played with this on the STM32F103 and STM32F303 devices on a bare metal basis. All my tries with the code generated with the ST tools failed for what ever reason. This is what I made for the F103 mimicking a CH340 on the USB side. https://github.com/pecostm32/STM32F103C8_USB_CH340

I found this site very helpful: https://www.beyondlogic.org/usbnutshell/usb1.shtml

[peter-h]

I think the USB code generated by Cube MX does not implement any of the "practical" stuff like flow control. It just does the data moving.

So for example, with MSC, there is no way to control data flow PC -> board when writing a serial FLASH filesystem (sector write ~15ms). The only way I found which works was to just delay the ACK to the PC by the sector write. USB does not seem to care and somebody said that FLASH sticks do the same (I posted a thread or two already on here on the MSC stuff).

And CDC didn't do it either.

I didn't realise the 64 bytes was configurable. I thought it was inherent in USB FS and CDC, thus limiting CDC to 64kbytes/sec (USB packet rate is 1kHz).

[bson]

Don't you just do an IN transfer indicating RX data available on the CDC interrupt endpoint?  The host polls the interrupt endpoint and when it sees the RX flag the driver will proceed to do a regular IN data transfer.  But I'm not that familiar with CDC, having never implemented it or even looked at an implementation.

[wek]

I didn't realise the 64 bytes was configurable. I thought it was inherent in USB FS and CDC, thus limiting CDC to 64kbytes/sec (USB packet rate is 1kHz).

Sometimes it's worth to actually read the normative material.

The USB defines the allowable maximum bulk data payload sizes to be only 8, 16, 32, or 64 bytes for full-speed endpoints and 512 bytes for high-speed endpoints.  A low-speed device must not have bulk endpoints.

AFAIK some hosts ignore this restriction, so you can try to increase packet size, if you are fan of the "it works for me" principle.

OTOH, as long as there's available bandwidth (as bulk transfers have the lowest priority of all), there is no restriction on number of scheduled bulk transfers to the same endpoint per frame - see USB2.0 5.8.4; this again is then up to the host's implementation.

JW

[pcprogrammer]

I didn't realise the 64 bytes was configurable. I thought it was inherent in USB FS and CDC, thus limiting CDC to 64kbytes/sec (USB packet rate is 1kHz).

The packet rate is not set at 1KHz. There is a 1ms interval as a kind of alive polling from the host. As long as there is available bandwidth the host can send more packets to the device and the device can setup more packets to be read by the host.

With a full speed device it is possible to use isochronous transfers and fill up the available bandwidth to the max, but it needs a dedicated host driver to do this. I have done this for a 16bit 100KSa/s system with a STM32F103 and a driver for linux. Problem with the driver for linux was I had to enable it every time I started the computer because it was my own development and not part of the kernel.

For the FNIRSI-1013D I made a bare metal mass storage class driver by dissecting code I found on the net. It lacks some bits, but it works very well with the default linux drivers for it.

USB development is a bitch though. The host needs things to happen within time limits, so no debugging with breakpoints. Wireshark is my friend during such developments.

[peter-h]

I am more than happy with 64kbytes/sec on CDC. It's way more than this product will never need.

The reason one needs flow control to work is because a 168MHz 32F4 can generate byte-serial data quite a bit faster than the other end on USB can absorb it. I spent quite a bit of time trying to characterise this, before I discovered that flow control is possible. The limits were ~10k packets/sec (packets of any size), and some others. Obviously this was PC dependent so the wrong way to go about it.

And same the other way round; one can have code running on the target which can't consume data fast enough (obviously).

if you are fan of the "it works for me" principle.

I am not, which is why I ask here

[pcprogrammer]

The thing to consider here is that there are two levels of operation.

1) The USB layer where the controller handles most of the communication, and the device can hint the host to wait or resend by controlling ACK.
2) The protocol layer on top of the USB hardware, which in your case is CDC, and this is software. I have not looked into this specific one.

For the CH340 device I found that it uses an extra endpoint to communicate the state of flow control lines to the host. So it might be that CDC has a similar system to communicate flow control on an application level.

There is this old thread about hardware flow control in CDC that might be helpful: https://www.eevblog.com/forum/microcontrollers/hardware-flow-control-in-usb-cdc/

[peter-h]

That's an interesting thread- thank you. Yes; he has to implement his hardware handshake in the UART, not further back. Then he will get a proper per-byte flow control. Lots of UARTs do this in hardware, including the 32F4 ones. You can literally stop "on the byte".

I am not trying to do that. I have this

target -> PC:
embedded code -> 1k circular buffer -> 256 byte linear buffer -> mystery ST USB code -> PC

where the move between the two buffers is done by an RTOS task running every 5ms i.e. a 50kbyte/sec max data rate.

PC -> target:
USB interrupt (mystery ST USB code) -> 64 byte linear buffer -> 1k circular buffer -> embedded code

where the USB interrupt does the circular buffer loading.

It seems to be working OK now, mostly, and there is definitely flow control working as it should be, but I can still create a data loss somewhere and it's not clear where, by making that 256 byte buffer much smaller, so throttling the speed there.

The testing is being done with the embedded code implementing a loopback

Code: [Select]

// ========= VCP echo test ==========
// Note the max data rate is 64kbytes/sec for CDC.
// This copy code limits the speed to buffer size x period i.e. 1000 every 10ms
static uint8_t usb_rx_buffer[1000];

// Test USB serial via KDE_serial interface
// Get data from CDC into temp buffer and output it to CDC

uint16_t bytes1 = KDE_serial_get_ipqcount(0);
uint16_t bytes2 = KDE_serial_get_opqspace(0);

// Run this only if the data can be copied without blocking anywhere

if ( bytes2 > bytes1 )
{

uint16_t tomove = MIN(bytes1,sizeof(usb_rx_buffer));

KDE_serial_receive(0, usb_rx_buffer, tomove);
KDE_serial_transmit(0, usb_rx_buffer, tomove);

osDelay(10); // give time for USB RTOS task etc to run

}
Port 0 is the VCP. I have four real serial ports also.

And throttling the speed there (with say osDelay(100)) slows it down but does not lose data at all, so the PC -> target flow control is definitely working.

And the target -> PC flow control has been tested separately; it is easy to test by sending a large block of text to Teraterm, and without flow control TT will lose some of it. This is how flow control is done to the PC:

Code: [Select]


/**
  * @brief  CDC_Transmit_FS
  *         Data to send over USB IN endpoint are sent over CDC interface
  *         through this function. Note that "IN" in USB terminology means
  *         KDE -> PC; the opposite of what KDE docs use.
   *
  * @param  Buf: Buffer of data to be sent
  * @param  Len: Number of data to be sent (in bytes)
  *  *
  * This function had a call rate limit of about 10kHz, and had a packet length limit
  * of about 800 bytes - until the flow control mod below. These limits still apply if
  * flow_control=false.
  *
  * This function is BLOCKING if flow_control=true. That means that if there is no Host
  * application receiving the data (e.g. Teraterm) it will block output.
  * Flow control is disabled for the debugging output functions.
  *
  * If called before USB thread starts, all output is dumped - for obvious reasons!
  *
  */

uint8_t CDC_Transmit_FS(uint8_t* Buf, uint16_t Len, bool flow_control)
{

if ( g_USB_started && (Len>0) )
{

osMutexAcquire(g_CDC_transmit_mutex, osWaitForever);

USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef*)hUsbDeviceFS.pClassDataCDC;

if (flow_control)
{
// Loop around until USB Host has picked up the data
while (hcdc->TxState != 0) // This gets changed by USB thread, hence the g_USB_started test
{
osDelay(1); // This actually needs to be just ~100-200us
}
}

USBD_CDC_SetTxBuffer(&hUsbDeviceFS, Buf, Len);
USBD_CDC_TransmitPacket(&hUsbDeviceFS);

osMutexRelease(g_CDC_transmit_mutex);

}

return USBD_OK;
}

The mutex is there because this function may also get called from debug "printfs", plus it ensures that debug messages are atomic (if done with just one call to the above) which is obviously desirable in an RTOS environment.

The USB code, running all under interrupts (not RTOS) toggles the hcdc->TxState flag so you just check it before transmission. Looks simple!

[peter-h]

I am revisiting this topic because somebody on the ST forum alleged that the function below has a bug (see comments) but then disappeared.

If it is indeed a bug, I am not sure how it could be fixed. You can't put that TxState test inside that ints-disabled region because the USB int won't get in either to toggle the flag

Code: [Select]


/**
  * @brief  CDC_Transmit_FS
  *         Data to send over USB IN endpoint are sent over CDC interface
  *         through this function. Note that "IN" in USB terminology means
  *         box -> PC.
   *
  * @param  Buf: Buffer of data to be sent
  * @param  Len: Number of data to be sent (in bytes)
  *  *
  * This function is BLOCKING if flow_control=true. That means that if there is no Host
  * application receiving the data (e.g. Teraterm) it will block output.
  * The problem is that if no USB device is connected from startup, the while() loop
  * would block for ever, but there is no simple fix for that.
  *
  * Flow control is normally disabled for the debugging output functions.
  *
  * If called before USB RTOS thread starts, all output is dumped - for obvious reasons!
  * Same if PA9=0 (VBUS_SENSE) i.e. no USB cable connected.
  * Same if CDC device inactive.
  *
  * Mutexed to enable multiple RTOS tasks to output to port 0.
  *
  * It has been alleged that the test
  * while (hcdc->TxState != 0)
  * should be inside the non-interruptible region
  * __disable_irq();
USBD_CDC_SetTxBuffer(&hUsbDeviceFS, Buf, Len);
USBD_CDC_TransmitPacket(&hUsbDeviceFS);
__enable_irq();
  * but that would introduce an indeterminate hold-up of interrupts. Also it would not work
  * as suggested because the USB interrupt would also be blocked from changing the TxState flag!
  * Also it seems unlikely that TxState could go Busy after it has been found in the Ready state.
  * That hcdc structure is only for CDC, not for MSC, so what else could be toggling TxState?
  *
  *
  */

uint8_t CDC_Transmit_FS(uint8_t* Buf, uint16_t Len, bool flow_control)
{

if ( g_USB_started && (Len>0) && (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_9)!=0) && g_CDC_FS_active )
{

osMutexAcquire(g_CDC_transmit_mutex, osWaitForever);

USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef*)hUsbDeviceFS.pClassDataCDC;
uint32_t timeout=0;
bool error=false;

// Wait on USB Host accepting the data.
// A simple 1 sec timeout prevents a hang.
if ( flow_control )
{
// Loop around until USB Host has picked up the previous data
while (hcdc->TxState != 0) // This gets changed by USB interrupt
{
osDelay(2);
timeout++;
if (timeout>500)
{
hcdc->TxState=0;
error=true;
break;
}
}
}

if (!error)
{
// Output the data to USB.
// USB interrupts must be disabled around this.
__disable_irq();
USBD_CDC_SetTxBuffer(&hUsbDeviceFS, Buf, Len);
USBD_CDC_TransmitPacket(&hUsbDeviceFS);
__enable_irq();
}

osMutexRelease(g_CDC_transmit_mutex);

g_comms_act[0]=LED_COM_TC; // indicate data flow on LED 0

// If flow control is selected OFF, we do a crude wait to make CDC output work
// even with a slow Host.
// In the usage context (outputting debugs to Teraterm) this actually needs to be only
// a ~100-200us wait, but is host dependent. This gives us a 1-2ms delay between
// *blocks* (typically whole lines).
if ( !flow_control )
{
asm("nop"); // for breakpoints
osDelay(2);
}

}

return USBD_OK;
}

Thank you for any comments. This code is ex ST USB FS and after various patches it runs well. So the issue is theoretical, but due to USB being pretty fast, it is hard to test.

[Sauli]

A few thoughts.

The TxState variable must be set to a nonzero value before the mutex is released. So it must happen in USBD_CDC_SetTxBuffer() or USBD_CDC_TransmitPacket().

With more than one task calling the transmit function, the 'no flow control' option does not work at all:
Suppose a task has just set up a transmit packet and immediately thereafter another task disregards TxState and sets up a new packet. Unless the USBD_CDC_SetTxBuffer() / USBD_CDC_TransmitPacket() function pair somehow queues packets, the new packet will trash the previous one. The previous packet is not gone until the host reads it.

[peter-h]

The TxState variable must be set to a nonzero value before the mutex is released. So it must happen in USBD_CDC_SetTxBuffer() or USBD_CDC_TransmitPacket().

Yes that is what happens:

Code: [Select]


/**
  * @brief  USBD_CDC_DataOut
  *         Data received on non-control Out endpoint
  * @param  pdev: device instance
  * @param  epnum: endpoint number
  * @retval status
  */
uint8_t  USBD_CDC_TransmitPacket(USBD_HandleTypeDef *pdev)
{     
  USBD_CDC_HandleTypeDef   *hcdc = (USBD_CDC_HandleTypeDef*) pdev->pClassDataCDC;
 
  if(pdev->pClassDataCDC != NULL)
  {
    if(hcdc->TxState == 0)
    {
      /* Tx Transfer in progress */
      hcdc->TxState = 1;
     
      /* Transmit next packet */
      USBD_LL_Transmit(pdev,
                       CDC_IN_EP,
                       hcdc->TxBuffer,
                       hcdc->TxLength);
     
      return USBD_OK;
    }
    else
    {
      return USBD_BUSY;
    }
  }
  else
  {
    return USBD_FAIL;
  }
}


With more than one task calling the transmit function, the 'no flow control' option does not work at all:

Yes. That's fine. The "no flow control" option is purely for debugs, and I don't care if there is data loss.

The basic problem is that with USB you cannot concurrently achieve

- flow control on transmit data (data Box -> Host)
- the Box not getting hung up if Host is missing / failed / no CDC profile supported / whatever

About all one can do is have some sort of a timeout that if output data is not disappearing (the Host CDC device profile is not retrieving it) for say 1 second, a flag gets set which discards the output.

That is AIUI and I may well be wrong! My tests below suggest I am wrong, at least in some cases, but why?

That is why, before CDC outputting, I implement the tests for e.g. VBUS=1 to eliminate the most obvious cases. But as has been pointed out to me before, there is no perfect solution. There was some disagreement on the detail e.g. one person said that if you want to detect whether the Host is running a CDC device profile on that "endpoint" (I think that is the rioght name) you can't do that unless you have a custom driver, which is not viable because this needs to work on any Windows etc PC.

However I tested this by forcing flow control ON and seeing if the whole box hangs when Teraterm is not running on the PC... It does not hang.  The data still disappears somewhere. Is there a defined action for Windows (to collect all CDC data and if there is no app running, just dump it) or is this just "luck".

It also does not hang when the USB cable is unplugged! But I think that is a different thing: the ST USB code detects the link state and calls usb_deinit. Then the ST code just dumps the data.

I think the scenario where I saw the target hanging must have been a subtle failure at the Host end.

This whole thing is really complicated.

The funny thing is that in the context of driving say Teraterm with debugs you don't need flow control. You just need about a 200 us delay between lines of text Or limit the packet rate to < 5000/sec. I determined these experimentally, and these numbers are obviously Host dependent. But I found that if you do push out a few k of data flat out, the Host (Teraterm) gets locked up. The data still gets "consumed" but Teraterm needs a restart.

A real need for flow control comes into use only if you are using CDC for transferring data to some more normal application.

[nctnico]

Flow control for USB serial ports is done in software, not at the USB level. So when the buffer of the UART starts to get full, you set the flags (which mimic the hardware flow control lines) in the CDC messages to indicate to the host to stop sending data. At the host end, the application talking to the VCOM port must have flow control enabled ofcourse.

[tru]

Are using an older ST CubeF4 library?
Perhaps, a bug was fixed already in newer libraries.

See latest middle class CDC source file (https://github.com/STMicroelectronics/STM32CubeF4/blob/master/Middlewares/ST/STM32_USB_Device_Library/Class/CDC/Src/usbd_cdc.c):

Code: [Select]

**
  * @brief  USBD_CDC_DataIn
  *         Data sent on non-control IN endpoint
  * @param  pdev: device instance
  * @param  epnum: endpoint number
  * @retval status
  */
static uint8_t USBD_CDC_DataIn(USBD_HandleTypeDef *pdev, uint8_t epnum)
{
  USBD_CDC_HandleTypeDef *hcdc;
  PCD_HandleTypeDef *hpcd = (PCD_HandleTypeDef *)pdev->pData;

  if (pdev->pClassDataCmsit[pdev->classId] == NULL)
  {
    return (uint8_t)USBD_FAIL;
  }

  hcdc = (USBD_CDC_HandleTypeDef *)pdev->pClassDataCmsit[pdev->classId];

  if ((pdev->ep_in[epnum & 0xFU].total_length > 0U) &&
      ((pdev->ep_in[epnum & 0xFU].total_length % hpcd->IN_ep[epnum & 0xFU].maxpacket) == 0U))
  {
    /* Update the packet total length */
    pdev->ep_in[epnum & 0xFU].total_length = 0U;

    /* Send ZLP */
    (void)USBD_LL_Transmit(pdev, epnum, NULL, 0U);
  }
  else
  {
    hcdc->TxState = 0U;

    if (((USBD_CDC_ItfTypeDef *)pdev->pUserData[pdev->classId])->TransmitCplt != NULL)
    {
      ((USBD_CDC_ItfTypeDef *)pdev->pUserData[pdev->classId])->TransmitCplt(hcdc->TxBuffer, &hcdc->TxLength, epnum);
    }
  }

  return (uint8_t)USBD_OK;
}Look at:

Code: [Select]

if ((pdev->ep_in[epnum & 0xFU].total_length > 0U) &&
      ((pdev->ep_in[epnum & 0xFU].total_length % hpcd->IN_ep[epnum & 0xFU].maxpacket) == 0U))Notice a variable called total_length is used to determine whether to send a ZLP or not, and this is necessary to indicate end of a transfer for USB BULK data flow under those conditions.
Is this code or similar in the older library you're using?

Just some nitpicking, with the last code you've posted, in the error timeout handling, I think should also call the function (stm32f4xx_hal_pcd.h):
HAL_PCD_EP_Abort(PCD_HandleTypeDef *hpcd, uint8_t ep_addr)
So atleast it puts the endpoint in not transmit mode, and also sends NAK immediately.

[peter-h]

Thank you for this.

It's funny. My existing file says

Code: [Select]

  * @file    usbd_cdc.c
  * @author  MCD Application Team
  * @version V2.4.2
  * @date    11-December-2015
and both say Revision 1.2 November 16, 2007" but doing a comparison there is a vast amount of differences, understanding which is way above my pay grade. It is possible that all that has changed is that my version has all the HS stuff removed (I am supporting FS only) and that was done by the original implementer (the Monday guy who spent ~ 2 years patching ST code with various patches.

One thing we did was to replace the heap usage in USBD_CDC_Init() and de-init, which is dumb and crashes the target (due to fragmentation) after x days or weeks, because Windows periodically goes to sleep on USB, with a static buffer.

Looking at USBD_CDC_DataIn(), yes, I have an older version

Code: [Select]

/**
  * @brief  USBD_CDC_DataIn
  *         Data sent on non-control IN endpoint
  * @param  pdev: device instance
  * @param  epnum: endpoint number
  * @retval status
  */
uint8_t  USBD_CDC_DataIn (USBD_HandleTypeDef *pdev, uint8_t epnum)
{
  USBD_CDC_HandleTypeDef   *hcdc = (USBD_CDC_HandleTypeDef*) pdev->pClassDataCDC;
 

  if(pdev->pClassData != NULL)

  {
   
    hcdc->TxState = 0;
 
    return USBD_OK;
  }
  else
  {
    return USBD_FAIL;

  }
 
}
I went to patch in the newer one but it generated lots of errors in missing structure members and then I found a lot of other stuff differed, so I gave up.

It's a pity that I ask for help and when I get it, it turns out to be too complicated for me to implement, but here a) I have something which works and b) the ST changes are spread across several files and there is no way for me to debug this stuff and c) how come the old code was working at all?

Just some nitpicking, with the last code you've posted, in the error timeout handling, I think should also call the function (stm32f4xx_hal_pcd.h):
HAL_PCD_EP_Abort(PCD_HandleTypeDef *hpcd, uint8_t ep_addr)
So atleast it puts the endpoint in not transmit mode, and also sends NAK immediately.

The function HAL_PCD_EP_Abort() does not exist in my ST code. I found one here but it has dependencies on other stuff...
https://github.com/STMicroelectronics/STM32CubeF4/blob/master/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_pcd.c

So I have to leave that one too. However, I see no evidence that that timeout ever occurs if Teraterm is running.  I cannot get a breakpoint to occur on it (and I am using the method using "asm nop" to make sure there really is code at the breakpoint). It was just an attempt to prevent that RTOS task getting hung if the USB code never cleared that flag.

However if Teraterm is not running, I can make it occur, though extremely rarely. So something is going on with the host collecting the data. I reckon me clearing the TxState flag to 0 here

Code: [Select]

{
// Loop around until USB Host has picked up the previous data
while (hcdc->TxState != 0) // This gets changed by USB interrupt
{
osDelay(2);
timeout++;
if (timeout>500)
{
hcdc->TxState=0;
error=true;
break;
}
}
}
is wrong. And that is probably the same thing which you suggested with calling HAL_PCD_EP_Abort().

The specific situation was if the system was running and outputting debugs, some of which were with flow control ON (for complicated reasons), and Teraterm was not running, then sometimes when Teraterm was started then, only a few bytes came out of it and after that it did nothing. Commenting out the hcdc->TxState=0 has fixed that issue. And it seems safe because I am now not touching any ST USB code if the timeout occurs.

[peter-h]

Commenting out the hcdc->TxState=0 has dramatically improved things. I can see why: we are clearing a flag which the interrupt-driven USB code is toggling.

The curious thing is that that timeout does occur (with flow control enabled, as per the code) but does so very rarely, if there is no VCP application running on the PC, on that COM port. This looks like whatever process in Windoze is dumping this "unwanted" incoming data, it sometimes goes to sleep for more than 1 second. It seems to happen of the order of once every few minutes. It looks like the data ends up in some buffer, say 64k, and when this fills up, there is some interrupt, and windoze dumps that buffer, but it takes a while.

With Teraterm running, the timeout is never seen, which is not surprising since I had already experimentally determined the limits of a 3.5GHz PC running Teraterm and these limits are very high, with Windows (or perhaps Teraterm) needing just a 200us gap here and there.

Flow control for USB serial ports is done in software, not at the USB level

We may have wires crossed, because the TxState flag does tell you if the Host has collected the data. Obviously this is specific to the ST USB code being discussed here.