USB data transfer protocol for rpi scanner

[amaschas]

I've been working on a project to re-create the old commercial film scanners of a few decades ago with updated technology and added flexibility. In essence, rather than using a line scan sensor, I'm using a rpi hi quality camera, and instead of being confined (mostly) to a single axis of movement, my scanner will move the sensor on three axes when completed (currently 2). I'll also be taking advantage of OpenCV for the post-processing.

My initial approach involved doing all of the OpenCV work on the rpi, for which purpose I luckily had a 8GB rpi 4 lying around. It has occurred to me, however, that this is not the best way of going about the sort of resource intensive processing, and I might have better luck transferring the image data to a host machine and just relying on the rpi to interface with the camera and capture via lib-camera.

I figure I could rig something up a socket over some form of network, but it occurred to me that most scanners—even modern ones—do essentially the same thing. They send the image data progressively to the host machine and let the scanner software do the hard work of assembling the image. This got me to wondering how exactly this worked on something like one of those cheap USB document scanners that canon and epson sell. Is it one of the USB gadget protocols, and if so, which one? I've been able to find surprisingly little information about how this works, but since an rpi w can operate as a gadget I'm wondering if I can emulate that approach.

Attached an image of the current state of my scanner for the curious.

[Nominal Animal]

I recommend you check the SANE project, which provides scanner support for Linux for a quarter of a century now.

Applications that support scanning via SANE are called frontends, and the "drivers" which support families of scanners are called backends, so you'll want to go see about the backends for the device(s) you intend to use.  Note that the supported scanners list contains a lot of old scanners, because the project is so long-lived; I recommend you try and find a scanner whose support is marked complete.

[amaschas]

I recommend you check the SANE project, which provides scanner support for Linux for a quarter of a century now.

Applications that support scanning via SANE are called frontends, and the "drivers" which support families of scanners are called backends, so you'll want to go see about the backends for the device(s) you intend to use.  Note that the supported scanners list contains a lot of old scanners, because the project is so long-lived; I recommend you try and find a scanner whose support is marked complete.

Just to note, I'm not trying to support and existing scanner, but rather build a new one. I'm trying to find a good way to transfer the image data from the capturing rpi to another machine for post-processing. Optimally I'd love to transfer the raw frame buffer from the python script capturing the data from the camera to another python script that stitches the images together.

[Nominal Animal]

Ok, let me retry.

This got me to wondering how exactly this worked on something like one of those cheap USB document scanners that canon and epson sell.

See SANE.

Is it one of the USB gadget protocols, and if so, which one?

No.  There is USB Still Image Capture Device Definition, and PTP/MTP, but they're mostly used with cameras.  Scanner families have tended to have their own communications protocols and drivers, with scanner-specific variations.

I've been able to find surprisingly little information about how this works, but since an rpi w can operate as a gadget I'm wondering if I can emulate that approach.

You can.  Simply use USB bulk transfers and your own protocol.

I would not do it that way, personally; I'd use a TCP/IP connection over Ethernet or WiFi.  This is simpler, because the host OS does not matter, and you get galvanic isolation for free (because the Ethernet interface includes magnetic isolation for all signals).  (No ground loop risk, if you power the 'Pi using a grounded or safety-grounded wall wart, i.e. not a class II isolated wall wart with no connection to ground.)

As to the protocol, I would definitely design it as an interface boundary: as a control and data interface to an appliance.  That is, each image (uncompressed RGB) would be preceded by a header, which identifies all the settings and properties of the image (and unless you standardize on all multi-byte values being in little-endian order, even the byte order), and perhaps a simple checksum of the entire frame.  In addition to coordinates (x, y, z?), I'd also include a serial number based on the currently active document.

An important facet of this is the asynchronous nature of the beast.  If you design your control protocol as a synchronous one, you'll waste time waiting for stuff to complete everywhere.  Instead, you must design each command (sent via the TCP/IP connection) so that it is immediately acknowledged/rejected, and if acknowledged/accepted, responded with a completed/failed status.  This means that each command you send should have a nonce, for example a monotonic counter, that is included in each immediate response and status message.  For example, to move the camera to a specific location, you can have separate commands for each axis, that are sent concurrently; the device will only send the status message when each axis has reached the desired position.
This also means you can add new features, like say zoom capability, light/backlight intensity controls, etc., by adding new commands to the protocol.

[amaschas]

As to the protocol, I would definitely design it as an interface boundary: as a control and data interface to an appliance.  That is, each image (uncompressed RGB) would be preceded by a header, which identifies all the settings and properties of the image (and unless you standardize on all multi-byte values being in little-endian order, even the byte order), and perhaps a simple checksum of the entire frame.  In addition to coordinates (x, y, z?), I'd also include a serial number based on the currently active document.

An important facet of this is the asynchronous nature of the beast.  If you design your control protocol as a synchronous one, you'll waste time waiting for stuff to complete everywhere.  Instead, you must design each command (sent via the TCP/IP connection) so that it is immediately acknowledged/rejected, and if acknowledged/accepted, responded with a completed/failed status.  This means that each command you send should have a nonce, for example a monotonic counter, that is included in each immediate response and status message.  For example, to move the camera to a specific location, you can have separate commands for each axis, that are sent concurrently; the device will only send the status message when each axis has reached the desired position.
This also means you can add new features, like say zoom capability, light/backlight intensity controls, etc., by adding new commands to the protocol.

This makes a lot of sense to me, thanks! I was initially thinking of getting this working via a TCP/IP socket, but using high level commands to do stuff like kick off the entire scan process and have all of the movement and capturing handled on the appliance end. But the more I think about it the more it makes sense to create an API with discrete commands.

I'm thinking of approaching it similarly to the way communication with peripherals on an ESP32 works, with threaded event loops that are publishing and subscribing to event queues. This should provide the degree of asynchronous control, while still allowing me to execute synchronous behaviors (aka move and then take an exposure).

[Nominal Animal]

This makes a lot of sense to me, thanks! I was initially thinking of getting this working via a TCP/IP socket, but using high level commands to do stuff like kick off the entire scan process and have all of the movement and capturing handled on the appliance end. But the more I think about it the more it makes sense to create an API with discrete commands.

I agree.

It is typical for me to design or implement a prototype before I actually create the real thing, because I'm not smart enough nor prescient to see the actual bugs and issues that crop up.  What I do do, is create each part in a modular fashion, so I can test (unit testing) and combine them easily.  That way, when there is a problem, I can usually immediately limit it to a specific part.  (The exception is things like the sync/async approach issue, which usually require a redesign.  In GUI applications I use a worker thread to compute slow stuff, and even in HTML/JS I use a timeout timer to trigger the slow work on the background, so the UI stays responsive all the time.  So, while asynchronous is a little bit more work to design, it will pay itself back in the end.)

I have thought about a somewhat similar device, but with stereoscopic images and a separately triggered lights (to combine into one optimal image), for example for easily recording and annotating coin collections with professional quality images (and perhaps even coin face height maps).

I'm thinking of approaching it similarly to the way communication with peripherals on an ESP32 works, with threaded event loops that are publishing and subscribing to event queues. This should provide the degree of asynchronous control, while still allowing me to execute synchronous behaviors (aka move and then take an exposure).

Sounds a good starting point to me.

Do consider reporting on your progress on the project in the Projects, Designs, and Technical Stuff -subforum; it sounds very interesting.

If you have any Linux socket programming questions, the Programming -subforum is a bit more frequently used than this Embedded Computing forum is.  For that kind of low-level systems programming (services etc.), I personally use C (Gnu99 variant currently for Linux), and am quite familiar with the networking and pthreads stuff in Linux.  There are other even more experienced members there, too; but not all check the forum daily.

[Berni]

TCP/IP over Ethernet is the way to go.

On a Rpi 4 you get gigabit Ethernet so that is one of the fastest interfaces you can easily use. No fancy driver shenanigans needed to talk to it either, just open a TCP connection in user space and away you go.

[amaschas]

I have thought about a somewhat similar device, but with stereoscopic images and a separately triggered lights (to combine into one optimal image), for example for easily recording and annotating coin collections with professional quality images (and perhaps even coin face height maps).

This should actually work for reflective targets as well. I'm thinking of it as a film scanner because that's my primary use case, but really it's a configurable, motorized xyz movement stage for imaging that I'm designing to be as cheap and modular as possible.

Do consider reporting on your progress on the project in the Projects, Designs, and Technical Stuff -subforum; it sounds very interesting.

If you have any Linux socket programming questions, the Programming -subforum is a bit more frequently used than this Embedded Computing forum is.  For that kind of low-level systems programming (services etc.), I personally use C (Gnu99 variant currently for Linux), and am quite familiar with the networking and pthreads stuff in Linux.  There are other even more experienced members there, too; but not all check the forum daily.

Those are great suggestions, I was actually wondering which forum would be the best place to ask about this sort of thing, and just landed here because I was using an rpi. I'm doing all of my prototyping in Python currently because I can iterate faster, but at some point I will probably rewrite the more performance-sensitive parts in C. I'll definitely solicit some feedback in the projects subforum.

TCP/IP over Ethernet is the way to go.

On a Rpi 4 you get gigabit Ethernet so that is one of the fastest interfaces you can easily use. No fancy driver shenanigans needed to talk to it either, just open a TCP connection in user space and away you go.

Yeah, that's a good point, can't get any faster and simpler than ethernet!