Use Hti-301 ht on Linux (both x86_64, aarch64 and armhf)

[ericb]

Hello,

I'm the happy owner of a brand new Hti-301  ht, and I'd like to use it on the desktop (only). I'm prof in an french engineers school, and my goal is to study the temperature precision, mainly around the temperature measuring. Got one student working with me, and doing the same with the Seek Compact pro.

FYI, we can already do a lot of things with the Seek camera (using the great libseek-thermal) and we started the - difficult- study of temperature value measurement-accuracy and computation. BTW, a big thank you to Fraser for his topic (https://www.eevblog.com/forum/thermal-imaging/seek-thermal-core-measurement-accuracy-whats-the-story/)

On the Hti-301, I got it since one day only, and I only found some information. The camera seems to work well (I got no cell phone btw :-) ), but I wasn't ableto see anything yet, but sure I'll progress fastly.

For example :

Code: [Select]

~ $ uvcdynctrl -f
Listing available frame formats for device video0:
Pixel format: YUYV (YUYV 4:2:2; MIME type: video/x-raw-yuv)
  Frame size: 384x292
    Frame rates: 25


v4l2-ctl returns :

Code: [Select]

~ $ v4l2-ctl --list-formats --list-formats-ext -d0
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'YUYV' (YUYV 4:2:2)
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 384x292
Interval: Discrete 0.040s (25.000 fps)


Other lsusb command :

Code: [Select]

~ $ lsusb -t
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 10000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
    |__ Port 2: Dev 2, If 1, Class=CDC Data, Driver=cdc_mbim, 480M
    |__ Port 2: Dev 2, If 2, Class=Vendor Specific Class, Driver=usbserial_generic, 480M
    |__ Port 2: Dev 2, If 0, Class=Communications, Driver=cdc_mbim, 480M
    |__ Port 3: Dev 42, If 0, Class=Video, Driver=uvcvideo, 480M
    |__ Port 3: Dev 42, If 1, Class=Video, Driver=uvcvideo, 480M
    |__ Port 10: Dev 3, If 0, Class=Wireless, Driver=btusb, 12M
    |__ Port 10: Dev 3, If 1, Class=Wireless, Driver=btusb, 12M




Code: [Select]


~ $ v4l2-ctl -d0 -D
Driver Info:
Driver name      : uvcvideo
Card type        : T3-317-13
Bus info         : usb-0000:00:14.0-3
Driver version   : 4.15.23
Capabilities     : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps      : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name      : uvcvideo
Model            : T3-317-13
Serial           : (hidden)
Bus info         : 3
Media version    : 4.15.23
Hardware revision: 0x00000200 (512)
Driver version   : 4.15.23
Interface Info:
ID               : 0x03000002
Type             : V4L Video
Entity Info:
ID               : 0x00000001 (1)
Name             : T3-317-13
Function         : V4L2 I/O
Pad 0x01000004   : Sink




Other (partial) information returned by lsusb -v

Code: [Select]

Bus 001 Device 042: ID 1514:0001 Actel
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x1514 Actel
  idProduct          0x0001
  bcdDevice            2.00
  iManufacturer           1
  iProduct                2
  iSerial                 3
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          162
    bNumInterfaces          2
    bConfigurationValue     1
    iConfiguration          4
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass         14 Video
      bFunctionSubClass       3 Video Interface Collection
      bFunctionProtocol       0
      iFunction               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass        14 Video
      bInterfaceSubClass      1 Video Control
      bInterfaceProtocol      0
      iInterface              0
      VideoControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdUVC               1.00
        wTotalLength           40
        dwClockFrequency        6.000000MHz
        bInCollection           1
        baInterfaceNr( 0)       1
      VideoControl Interface Descriptor:
        bLength                18
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Camera Sensor
        bAssocTerminal          2
        iTerminal               6
        wObjectiveFocalLengthMin      0
        wObjectiveFocalLengthMax      0
        wOcularFocalLength            0
        bControlSize                  3
        bmControls           0x00000220
          Focus (Absolute)
          Zoom (Absolute)
      VideoControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             2
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          1
        bSourceID               1
        iTerminal               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              9
      VideoStreaming Interface Descriptor:
        bLength                            14
        bDescriptorType                    36
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bNumFormats                         1
        wTotalLength                       71
        bEndPointAddress                  129
        bmInfo                              0
        bTerminalLink                       2
        bStillCaptureMethod                 0
        bTriggerSupport                     0
        bTriggerUsage                       0
        bControlSize                        1
        bmaControls( 0)                    27
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        1
        bNumFrameDescriptors                1
        guidFormat                            {xxxxxxxxxxxxxxxxxxxxxxxxx privacy ? xxxxxxxxxx}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
          bCopyProtect                      0
      VideoStreaming Interface Descriptor:
        bLength                            30
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x00
          Still image unsupported
        wWidth                            384
        wHeight                           292
        dwMinBitRate                  5606400
        dwMaxBitRate                  5606400
        dwMaxVideoFrameBufferSize      224256
        dwDefaultFrameInterval         400000
        bFrameIntervalType                  1
        dwFrameInterval( 0)            400000
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x030c  1x 780 bytes
        bInterval               1


As you can see, there is something missing to initialize the camera, and I got some questions :

-  I got it half-working using OpenCV 3.4.0. Mainly the zoom access is broken in this release. Does someone suggest a better OpenCV version to use, to be sure I can correcly initialize the camera ?

I got it working with SeekGui (a software I wrote), but the only frame type I got is a green one, probably because using BGRA by defaut.  What is the right pixel format ? YUYV ?

Other possible issue: the current Linux kernel version is 4.15.23, maybe obsolete (investigations in progress).

Last but not least, any help is welcome (including in private) !!


To be continued :-)


[EDIT : fixed typos, wording, and some other little glitches]

[therwp]

Code: [Select]

~ $ uvcdynctrl -f
Listing available frame formats for device video0:
Pixel format: YUYV (YUYV 4:2:2; MIME type: video/x-raw-yuv)
  Frame size: 384x292
    Frame rates: 25


Code: [Select]


~ $ v4l2-ctl -d0 -D
Driver Info:
Driver name      : uvcvideo
Card type        : T3-317-13
Bus info         : usb-0000:00:14.0-3
Driver version   : 4.15.23
Capabilities     : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities

I got it working with SeekGui (a software I wrote), but the only frame type I got is a green one, probably because using BGRA by defaut.  What is the right pixel format ? YUYV ?

Very cool! I don't have the knowledge to be of assistance, but I appreciate your moving everyone forward.

-Regarding "I got it working with...," that is referring to the Seek camera only, right?

-Interesting that the HT-301 reports a resolution of 384x292. As you know, it's sold as a 384 x 288. Where do the extra four pixels come from?

[tonykids]

Very cool! I don't have the knowledge to be of assistance, but I appreciate your moving everyone forward.

-Regarding "I got it working with...," that is referring to the Seek camera only, right?

-Interesting that the HT-301 reports a resolution of 384x292. As you know, it's sold as a 384 x 288. Where do the extra four pixels come from?

They are some addtion info of this frame,like max&min temp and their x,y position,centre temp,some user defined positions' temp etc.

[ericb]

Hello,

@therwp : I'm using libseek-thermal (you can find on github) adapted to Seek Compact pro. It works very well, and is really reliable.

Back to the Hti 301 : to answer you about the 384 x 288 pixels, I think there is some extra lnformation, like done with e.g. TV line / TV frame for TV signals, but I'm currently discovering all of that, and I take care to experiment without brake anything. 

Current status : on Linux x86_64 + OpenCV 3.4.11 I got frames 388x292@25fps, and I only found that some values for zoom and focus cause something like new something,  maybe shutter reset or something.  The big issue is that I got NO image yet, and I'm very disappointed

If it doesn'twork, I'll return it soon (sort of big 800 euros deception in this case  :-/ )

@tonykids

Thanks for the information, I'll investigate this path

[Cat]

The big issue is that I got NO image yet, and I'm very disappointed

No image with OpenCV or no image at all?
According to your snippets the Hti-301 is an UVC (USB Video Class) device which can be accessed with an mediaplayer like VLC and the thermal image should be seen.

[ericb]

@Cat:

q4l2  /dev/video0  => start the capture gives  grey vertical lines ~ 3 s  then uniform grey  ?

$ v4l2-ctl -L
                 focus_absolute 0x009a090a (int)    : min=0 max=65535 step=0 default=0 value=0
                  zoom_absolute 0x009a090d (int)    : min=0 max=65535 step=0 default=0 value=0

Playing with  focus_absolute and zoom absolute, gives a green fixed frame, and nothing else.

Playing with any player , no image at all : means when moving the camera, I see no change.

I forgot : I hear no shutter (like with the Seek Compact pro)  I can hear it again :-)


Can the camera be broken ? probably not, but sure it is not correctly initialized.

[Edit] : removed useless information, update : it works !



Ok, it starts to work. See attached screenshot, taken with miniDart. As expected, it works directly with my software ! UFF !  Last, SeekGui will be adapted to both Seek Compact pro (I'll integrate libseek-thermal) and Hti-301 + I'll cross compile everything to work on Windows too.

[ericb]

Hello,

Got all colour pallettes workings. See the attached screenshot (add the selector using ImGui was easy)

+ works on x86_64 (obviously), plus  RPi 4  (8GB but should work on 4GB, aarch64, GPU @650 MHz / MPU @2 GHz)) and works somewhat with armhf on RPi 3b+ 2GB (32 bits)

Important: the code has not been commited, for good reasons.

Next steps :
- provide a simple way to init/calibrate.
- resize to image only (remove the 4 extra lines)
- provide a Windows port (cross compiled binary + installer)
- add cross + temperature information (works already for Seek Compact pro on my machine, but needs some love)

Note:  select an area + zoom and draw over the image are already included

To be continued

[riccardo.pittini]

Amazing job!!!
Do you think that changing the resolution will make it work also for the HTI-201?

[ericb]

@riccardo.pittini

Hello,

To be honest, I don't know the Hti 201 camera well, but it should, since this camera is very probably seen as  (USB 2.x) USB device providing any of RGBA or YUYV or YUV420 or even RGB24 pixel format for frames. e.g. if you are using Linux, qv4l2 should work out of the box, including controls. 

+ IMHO this should work with ffmpeg too (all OS ! )

For the radiometric data, and any control operation, I don't know (but v4l2 will provide you tracks to explore).

@Fraser or some expert : can you please confirm the  201 is seen as an USB 2 device ?

About resolution, just take care to dissociate the sensor resolution (pixels width by pixels height) with image resolution obtained using cv::resize(). For the hti 301, got 388 x 292 resized into 720p (aka 1280 x 720 @25 FPS) or any of the 640 x480, 800 x 600 ... 2k ...etc. Or whatever other image size I propose.


If you want to test on Windows, I'd suggest you to give it a try with miniDart (I provide Windows versions here : https://framagit.org/ericb/miniDart/-/tree/master/testing_version (please be gentle, this is the Handball version and I'll adapt to thermogaphy with SeekGui in a while).

BE CAREFULL : if you want to test miniDart, or SeekGui later, install OpenGL drivers on your Windows machine IS MANDATORY. Else miniDart won't even launch !

On Linux, you'll have to build it, but this is very complicated. The problem is I customized Freetype inside OpenCV (there in a minimal canvas included in miniDart, and you can draw over the images - WIP-). I'll have more time in next summer to simplify. Sorry, this is  the only moment in the year I have less work :-/

HTH

[Odiug]

@ericb

Hi

How did you get the palettes to work. Is it also a setting inside UVC or are they applied afterwards in the application?

BTW, my HT-301 worked out of the box as a UVC device under Linux with the app "Cheese", but also even under Win10 with Camera and Teams. It seems to come up with reasonable default settings as a working UVC device.

Regards
Guido

[ericb]

@Odiug

Of course it will work for you on Linux (even VLC should work).

Though, everything is not yet perfect yet : for example, I can record video (thermal view + min max + radiometric data) using miniDart + Seek Compact pro, using v4l2loopback + the viewver, but the hti-301 ht recording is broken. Maybe my fault. If I'm not wrong, I still have to add the right change to v4l2loopback with the hti-301 ht too. The current track is, currently, the software refresh rate is not enough to record the video, or it could be the framesize (720p). Investigating ...

Indeed paletts will miss you, since I'm using a not documented way to select one of them. Tell me if you want to try, I'lll start to build a windows version soon (should work directly, since it does compile straight away since a while).

As short anwser, I found the trick randomly (confirmed by someone who reversed some parts of the sdk). If you can read "between the lines" in my previous answers, you'll understand what happens.

To tell you more, I can change the pallet in runtime, and it rocks really, and more, but this is undocumented. I'm still waiting the official SDK from the reseller .... got no news since a while.

pm me if you want to know more.

[jonnys]

Hi i see that no news from a few month ago so i refresh this topic,
and ask are is any new information about this case ?
kind regards

[ericb]

Hello,

Well, I don't have much of feedback (nor help in any way) ... no idea why : it works well and is promising.

I'm sorry I was crazy busy with my -new- job, and didn't find the time to improve, but I'll have more time very soon (bettwen Christmas and New Year day).

Other important point : I worked a lot to improve miniDart (record and SYNC correctly video and audio)

Please stay tuned, big changes are scheduled during 2022 Q1 (or maybe Q2)

My plans are:
- change a lot of code (and rebase with loudly improved miniDart), including video recording
- add min max + some radiometic data
- improve palettes (currently not working well)
- provide a Windows version

 [edit] : fixed typos

[ericb]

The code is noW adapted to OpenCV 3.4.17, and I have added the color palettes to miniDart too.  Tests and WIP  with OpenCV 4.5.5

For further details, see : https://framagit.org/ericb/ir_thermography  (Linux only Windows asap)

(or https://framagit.org/ericb/miniDart/-/commit/deadd5a95f9fdc097fd6cb23658789ff939f96b2 )

I think I'll have a look at radiometric values during the coming days. Feel free to help me if you are interested.

[ericb]

In fact, I found 10 color palettes + other features (including internal chessboard for calibration, maybe other things hidden)

Works perfectly using both OpenCV 3.4.17 AND OpenCV 4.5.5.

Tested on Linux only (x86_64 and aarch 64). Since the code is very portable, it should work on Windows.  For the new palettes, just have a look at the headers in seekGui code (I'll update miniDart source code soon, because the images can be recorded in a video -.mp4, .mkv or .avi @25 fps- very easely).

I'll have to stop for some days. To be continued.