TFT displays, play videos with microcontroller (SPI, Parallel, MIPI DSI)

[luiHS]

Hi.

I am looking for a solution to make a video player with small screens, some of rectangular shape from 3 to 5 inches and a round one from 2 to 3 inches. I checked the ports of the available TFT screens and also the possible microcontrollers.

I found these screen ports available, I do not know if there are more (that can be used with a microcontroller):

1.- Parallel port
2.- SPI, ILI9341
3.- MIPI DSI

On the microcontroller, I checked the new NXP RT1020, but unfortunately this does not have any port dedicated to displays, Parallel or MIPI DSI, only the standard SPI.

For MIPI DSI, I see that some STM32F7 or H7 are good options, also for parallel or even SPI. I recently purchased two evaluation boards STM32F7, STM32F769I-DISCO and STM32F746G-DISCO. The 769 with MIPI DSI and the 746 with parallel TFT port, both with demos of applications that play videos. I also bought all these displays from the attached pictures, to test, there are parallel port, SPI and MIPI DSI.

I do not want to use a Raspberry, because I need the product to be very compact, economical and I do not like to depend on a RPY Zero or similar.

Some suggestions, experiences, TFT screen technology / Microcontroller ?.

[jeremy]

The problem you have not considered which is critical is the format of video. If you are expecting to play back h264 or similar, none of the micros will be able to handle it. RPi has built in hardware video decoders, which is why lots of people use it for media applications.

[funkathustra]

First thing's first: generally, for displays in this size category, there are two common interfaces that may look similar on the surface, but are totally different:

• Parallel video interface. You get 16, 18, or 24 data pins that carry your pixel color (RGB565, RGB666, or RGB888), plus a pixel clock, H/V sync, and sometimes EN. This is much more common than MIPI for smaller (and lower-resolution) displays. Typically tops out at around 1024x768 resolution. In this case, the framebuffer lives on your MCU/MPU, and it has an LCD controller that can generate these signals for you. You can find this on only a few high-end Cortex-M4/M7s, plus basically every Cortex-A (except the very basic ones that lack LCD support). Keep in mind none of these parts have enough RAM to hold a reasonably-sized framebuffer, so all will require external RAM.
• 8080 memory interface. You get 8 or 16 data pins that carry commands or data, plus standard 8080-style control signals (RD, WR, RS, E). These are "smart" displays that have integrated framebuffers and fancy controllers. The ILI9341 you mentioned is an example. Many of these displays also have SPI mode (which you mentioned), but on MCUs with an external memory controller, these displays can be accessed much faster using their parallel interface. Basically, all high-end Cortex-M4s/M7s have this peripheral — and it's even found on some Cortex-M3 parts. Since the framebuffer lives on the display, you don't need RAM to hold it.

MIPI is usually only found on higher-resolution displays that go into portable devices. You'll have a hard time sourcing these more generally, and few MCUs support this interface (even many Cortex-A parts lack MIPI output support).

SPI is not really suitable for anything bigger than 320x240-resolution screens, and even then, don't expect more than 10-15 fps.

Onto the real problem: video decoding.

For any sort of reasonable framerate, you'll need an MCU with a hardware JPEG decoder, and you'll be limited to playing back MJPEG movies. I've seen decent playback at 320x240 resolution on high-end Cortex-M4/M7 parts, but nothing higher-resolution than that.

Even Cortex-As without hardware video codecs struggle to decode modern video formats in software — so be sure you know what you're getting into before diving in too much!

[Rasz]

You said product, what kind of MOQ are you able to swallow? Bom cost?
My suggestion would be a trip to China.

[luiHS]

 
There is no problem with MOQ, the Chinese offer has very cheap prices, I do not need to go to China, I have bought all the displays that are seen in the pictures attached in my first publication, to try here, with the three technologies I know, MIPI DSI , SPI and Parallel. And some more round displays of 2.4 inches, all MIPI DSI. I ordered everything on Aliexpress and Alibaba from Chinese suppliers.

On the parallel bus, I mean RGB data, Clock and Vertical-Horizontal Synchronisms, no commands are sent. About MIPI DSI I have several models of displays with sizes between 3 and 5 inches, and there are cheap microcontrollers STM32F7 with MIPI DSI port, for example the F769. It is not complicated, nor expensive, to have a solution based on MIPI DSI

About video decoding, demo applications with F746 and F769 work with MF files (up to 640x480), I do not know what this format is. I've tried the 480x272 and it works fine. With SPI ILI9341 I do not remember now what format of video files they use, this with an STM32F4xx microcontroller, although I prefer to use an STM32F7 or H7.

My idea, if there is no better one, is to use the STM32F769 microcontroller that has MIPI DSI, rectangular MIPI DSI displays between 3 and 5 inches, and round MIPI DSI 2.2 inches.

[vaualbus]

if you plain to build the app from scratch be sure to read the stm32 tft manual and read the section about what you need to do for the F/H 7 series.
Both the two processo have two caches inside that can cause the screen to blink while drawin. I spend a lot of time find and resolve this issue in my codebase.
Also if you do double buffering you need to place the two frame buffers in the low sdram memory address.

[westfw]

Keep in mind that to drive, say, a 240x320x12 display at 15fps requires IO bandwidth of ~15Mbps.  Smart displays don't buy you anything if you're talking about arbitrary video.  That is a lot of bits to be pushing out of any sort of IO port that isn't purpose-built for video, even without the need to computer or read it from somewhere...

[funkathustra]

[...] a 240x320x12 display at 15fps requires IO bandwidth of ~15Mbps.  That is a lot of bits to be pushing out of any sort of IO port that isn't purpose-built for video

What the heck are you talking about? I don't know what weirdo 12-bit color depth you're using, but a 240x320 display operating at 16-bit (RGB565) can be refreshed by writing 76800 words to it. At 15 fps, you'd need to write 1.152 Mwords/second. Even if you're bit-banging it with a GPIO port, you'd only need the data/clock signals to work at 2.3 MHz (allowing one period for the data, and another for the WR/RD signals). I don't know of any MCU the OP is considering that would struggle at all to even bit bang that.

And all the MCUs we're considering in this thread have external memory controllers, parallel RGB interfaces, and/or MIPI support, so this is either a direct memory operation, or a direct memory operation 99.99% of the time, and a software-defined control sequence for the remaining 0.001%. Not hard.

Contrary to your opinion, pushing the bits out of the bus to the display is exactly the least problematic part of this project. The hard part is decoding the video. But the MCUs the OP is considering have hardware JPEG decoding, which means MJPEG video files should be relatively easy to decode.

[viperidae]

320*240*16bits is 1,228,800 bits. To get 15fps you need about 20Mbps. The ili9341 controller only supports 10MHz spi, so you're limited to 8fps doing full screen refreshes. To go faster you need to use the parallel interface

[luiHS]

320*240*16bits is 1,228,800 bits. To get 15fps you need about 20Mbps. The ili9341 controller only supports 10MHz spi, so you're limited to 8fps doing full screen refreshes. To go faster you need to use the parallel interface

Teeny 3.6 (Kinetis MK66, 180Mhz) + ILI9341, plays SD video.
This is for you 8fps ??, I do not think so.

[NW27]

Have a look at the FTDI FT800 series of IC's and perhaps add in the FT90x to feed it.

Sent from my SM-N920I using Tapatalk

[westfw]

IL9341 also has a parallel interface, right?  So if you've got enough pins that'll cut it down to ~1M tansfers/s, which should be doable with DMA (like a teensy3.6.)  But I doubt that it's a "walk in the park", either.
I thought the FTDI parts were specifically aimed at GUI-like (NOT VIDEO) applications?

[luiHS]

 
It is possible that the examples of Youtube with ILI9341 screen that plays videos from SD card, connect the MK66 to display with a parallel port. Checking the data sheet of ILI9341, I see that can work with several formats of display data:

3-line Serial Interface
4-line Serial Interface

8-bit Parallel MCU Interface
9-bit Parallel MCU Interface
16-bit Parallel MCU Interface
18-bit Parallel MCU Interface

6-bit Parallel RGB Interface
16-bit Parallel RGB Interface
18-bit Parallel RGB Interface

For me there is no problem, connect a Kinetis MK66 180Mhz (using DMA) with a parallel port to ILI9341 screens, to play videos from the SD card.

I see in Youtube examples, videos with MK66 and ILI9341 screen, they play very well, without flickering or jams, the playback is smooth and at a very good fps speed, I would say that at least it plays at 24pfs or more. It's perfect for what I need.

I do not know yet, what exactly is the difference between the Parallel MCU interface and the Parallel RGB interface, I suspect that it is related to using an external or internal buffer to store data.