Numato Opsis: FPGA-based open video platform

[mithro]

An open platform for recording, routing, and manipulating HDMI and DisplayPort video signals.

https://www.crowdsupply.com/numato-lab/opsis

The Numato Opsis is a powerful new FPGA-based open source video platform for videographers and visual artists. The Opsis board was designed to give the user complete control over high-speed video, enabling everything from real-time conference capturing solutions, to experimental visual art and even general FPGA-based video research.

• A: HDMI Out 1
• B: HDMI Out 2
• C: HDMI In 1
• D: HDMI In 2
• E: DisplayPort Out
• F: DisplayPort In
• G: USB 2.0 Device / JTAG Programmer / UART Adapter
• H: Expansion Port (TOFE, not PCI-Express compatible)
• I: microSD (underside of board)
• J: USB 2.0 OTG
• K: Gigabit Ethernet

As a collaboration between the TimVideos.us live event streaming project and FPGA development board manufacturer Numato Lab, the Opsis is the ideal device for the HDMI2USB.tv video capturing firmware, which allows for “fool-proof” video recording from any computer. Enabling direct acquisition of video streams in realtime to a computer through the USB 2.0 interface or sent over the network through the Gigabit Ethernet interface, the Opsis is a vital component within the complete conference recording solution envisioned and pioneered by the TimVideos.us project.

Complete Hardware Design - https://github.com/timvideos/HDMI2USB-numato-opsis-hardware

--

Lots more information on the website. Hackaday also did a post on the board at http://hackaday.com/2015/10/02/numato-opsis-fpga-based-open-video-platform

As the designer of the board, I would love feedback and your thoughts on it! I'd really love us to hit our 100 board stretch goal so we can give everyone the expansion board included and ultimately the reason I spend all my spare time on doing things like this is to get people the access to the board.

[marshallh]

Why is there so much empty space on the board?

- waste of money for PCB
- unnecessary pickup of RFI/EMI
- routing shouldn't need that much for only 6 video IOs
- looks kind of ugly

Silkscreen is also unappealing.

Moving parts/rerouting with Kicad is worse than a root canal at 30 thousand feet.
Possibly sized to fit an industrial AV case.

[Richard Crowley]

It looks like an exciting project. Considering backing it.   

@brobbuilder, do you have examples of similar projects with better execution?

[mithro]

Hi brobbuilder,

The board is designed to meet the PC Mini-ITX standard to allow using any case designed for those little multimedia PCs (which then dictates it size). With the cost of the FPGA part, the actually size of the PCB makes very little difference to the cost when making 50+ of them.

Aesthetics are totally subjective, so I can't really comment there . In my opinion the most important aspect of the silkscreen is to relay information about the board which I believe it does much better then many other boards.

Thanks for the feedback!

Tim 'mithro' Ansell

[miguelvp]

Looks awesome.

But with 4K maturing USB 2.0 doesn't make much sense for video capture, specially since it seems you support ultra HD (consumer 4K) via the displayport at 30Hz and true 4K digital cinema at 24Hz wouldn't it make more sense to support USB 3.0 and type c connectors as well?

The DisplayPort connector supports resolutions up to a bandwidth of ~12Gbit/s which include extremely high resolutions around 4096 × 2160 @ 24Hz.
The DisplayPort connector is dual-mode enabled and thus can operate as high speed HDMI ports using cheap adapters. When operating in HDMI modes, many high resolutions up to 3840 x 2160 @ 30Hz should work.

Then again USB 3.0 still falls short, even 3.1 is still a bit shy on the bandwidth.

[mithro]

We would have loved to have used USB 3.0 but need to manage risk for our first board. Adding the high speed DisplayPort connectors was the big change we wanted to do in this revision.

The board (http://hdmi2usb.tv/digilent-atlys/) we where using to develop the HDMI2USB.tv gateware had the FX2 chip, so we decided to use the same chip as we knew it worked.

To make matters harder, the good options that have recently become available for USB3.0 development (like the FT600 board you suggested on IRC) were not easily sourcable when we were making the decisions.

The current solution for conference video capture is VGA at NSTC resolution, so the step up to 720p30 is a *huge* improvement.

A couple of other useful things to note;

• There is a Gigabit Ethernet which doubles the bandwidth available of the USB2.0
• The HDMI2USB.tv firmware currently use a JPEG encoder core on the FPGA to compress video input down to a bandwidth. JPEG still isn't enough to get 4k down to USB bandwidth.
• Begin an FPGA better encoding options are definitely possible.
• The FPGA can downscale the video for capture while still passing through the higher resolution video.
• The DisplayPort connectors are connected to the GTP transceivers and thus can instead be used for PCI-Express interface with an adapter board. Still experimenting with making that work.

If the Opsis is successful, we are definitely looking at doing another board with USB3.1 (probably on a Type-C connector) and either an Artix-7 or Kintex-7 FPGA on it. It will definitely won't be until middle or late next year before we get close to finishing that design though.

Hope that explains the decision process! Actually delivering on creating hardware is all about the compromises, hopefully we made the right decision.

Tim 'mithro' Ansell

[Bud]

Regardless of how good the product might be, don't you think it is inappropriate to give yourself a thumbs up in the original post

[miguelvp]

(like the FT600 board you suggested on IRC)

I have not been on IRC for decades

About the JPEG encoder, maybe JPEG 2000 will compress better (wavelet based) but not sure if that's an option, although that might be feasible with an FPGA. It will lose some high frequency information but for video your eye will interpolate the result so high compression rates 1:100 might be possible but might require a beefier FPGA as the ones you have on your roadmap.

As for the decisions made, they seem very reasonable 

[mithro]

(like the FT600 board you suggested on IRC)

I have not been on IRC for decades

Sorry about that! It appears that I confused you with a person asking exactly the same question on #eevblog IRC channel who also has a nick starting with m.

About the JPEG encoder, maybe JPEG 2000 will compress better (wavelet based) but not sure if that's an option, although that might be feasible with an FPGA. It will lose some high frequency information but for video your eye will interpolate the result so high compression rates 1:100 might be possible but might require a beefier FPGA as the ones you have on your roadmap.

Commercial JPEG2000 cores exist but I couldn't find one which would fit in the Spartan-6 45T on the Opsis. Down scaling is probably the only real option for capture of 4k on the Opsis board at the moment. The 4k video does allow you to put the HDMI inputs side-by-side at full resolution or doing some type of overlay / up scaling too.

[mithro]

Regardless of how good the product might be, don't you think it is inappropriate to give yourself a thumbs up in the original post

I'm not a regular user of these forums so I apologies if I've made a social faux pas (some friends suggested I post here). I just selected a random "good" emoticon from the message icon list - if you don't think it is appropriate, I can change it (assuming that is possible)?

[Richard Crowley]

Don't worry about it.  Sniping at newbie's online behavior is rather beneath the general quality of discourse on this website.
Stick to discussions of the technology and let the socialites talk amongst themselves.

[mithro]

If people are confused about the video stuff on the board, I wrote up a longish FAQ (http://j.mp/opsis-video-faq) about all the video formats and what is/isn't supported. Sadly with the area being confusing itself and the Opsis being an FPGA board (so everything is just "software") things get complicated quickly. There is even a section on the "4k" video too -> http://j.mp/opsis-video-faq#heading=h.f52wag5243kp

[hamster_nz]

I'm tainted, as Mithro offered me a board to "have a play with, no strings attached", and I've been hacking away on it for a while. Apparently it is the 3rd board ever made . So here's my biased review.

Value for money
Pricing is really good, esp given that it doesn't have any of the discounts that 'official' dev board manufactures get. It must be priced pretty close to cost. You can pay $800 for the same FPGA on a module (eg https://www.opalkelly.com/products/xem6310mt/)

Video Interfaces
Most video oriented FPGA devboards have only one or two of each connector - e.g. the US$500 Digilent Nexys Video (HDMI-in, HDMI-out, MiniDisplayPort) or US$75 Scarab miniSpartan6+ (HDMI in, HDMI out, with only 1/5th the FPGA logic), and they also have a lot of cruft that you don't really need for pure digital video work - switches. LEDs, small expansion ports. This is the only board I've seen that has two HDMI inputs and two HDMI outputs, and DisplayPort in and out.

For all the Spartan-6 based boards HDMI tops out a little over 720p due to the clocking limits of 1,050Mb/s on standard I/O pins, and even for the newer Artix-7 boards 1080p is a little out of the FPGA's spec (but it does work). As the DisplayPort is on the high speed transceivers, it can move 2.7Gb/s per channel. and as it has all four channels it can REALLY can do 4k resolutions (either 2160p60 in YCC 422 or 2160p30 in RGB/YCC 422).

Other interfaces
As mentioned it is light on more generic I/O and expansion, but then it is a purpose-designed board.

Memory
I haven't looked into it, but Video memory bandwidth is always going to be a problem. If you are doing UHD video you won't be able to squirt 10Gb/s of video into RAM and suck it back out again. 2.4GB/s of sustained bandwidth is too much to ask for. The board has got DDR-1600 on it, so it should be fine for processing 1080p, streams, but not 2160p. So it's not a platform for 4K development unless the video stream stays within the FPGA.

It is plenty of memory if you want to run an embedded soft-CPU design acting as command and control of the video streams.


FPGA type and size
It is quite rare for a Spartan 6 devboard to have the 'T' model (the ones with high-speed transceivers) on it. I found the transceivers to be a real pain to get to grips with - lots of exceptions and oddness to deal with, especially with the clocking options. I've got them working but I'm still getting to grips with them after two weeks. Everything seems to be planed out correctly (e.g. reference clocks on the correct pins), with just a few gotchas on the prototype boards - but that is what prototypes are for!

The FPGA size is my main concern - video processing uses lots of logic - relatively fast wide buses (e.g. using internal RGB 12:12:12 video data paths at 150MHz), and 58 DSP blocks. A matrix operation (e..g colour space conversion) might require 9 DSP blocks, so by the time you allow mixed RGB/YCC inputs over half your DSP blocks have been used.

However, this system is designed for when you have full control over the input and output formats (e.g using EDID to advertise only RGB support), so it is the right-sized FPGA for the system's target use.

Debug issues
Debug - Without some generic pins to hook up to my logic analyser, debug is a bit more painful than I would like. I'm currently using a passive microSD->JTAG adapter to extract a couple of bits of debug. If somebody could implement a multi-channel logic analyser with output to a HDMI screen that would be really cool. There is the option to get data off the board over the gigabit ethernet too, but I don't have any experience of that.

Who is this for?
It isn't for the first time FPGA user, you don't have much fun things to experiment with. Maybe look at the miniSpartan6+ or a Digilent board.

If you are looking for an FPGA board to play with switching and mixing and light processing of digital video signals (e.g. brightness, colour processing, audio processing) it is going to be perfect.

If you want to play with or prototype DisplayPort it is going to be perfect too - it costs maybe half as much as an DisplayPort FMC card.

If you want to perform simple processing of video from two cameras (e.g. machine vision) it might be a good fit, but you may have problems with offloading the processed data to your host.

If you want to do extensive video processing you most likely will find the FPGA too small, but you will be spending a lot more for a less 'open' board.

[ehughes]

https://digilentinc.com/Products/Detail.cfm?NavPath=2,719,1476&Prod=NEXYS-VIDEO


This is a much better deal.   Much better FPGA that is better suited to video work. 

That and you get a copy of the vivado tools...

Every 'maker' FPGA board is seems like a bad deal.  Digilent gets the FPGAs at very special pricing.....    It is tough to beat them for xilinx products when they getting the chips for next to free.

[mithro]

https://digilentinc.com/Products/Detail.cfm?NavPath=2,719,1476&Prod=NEXYS-VIDEO

This is a much better deal.

The Nexys Video is a great board if you can get the academic pricing or Digilent sends you one for free.

We actually have a comparison table on the crowd funding page at https://www.crowdsupply.com/numato-lab/opsis#comparison -- In my (very biased) opinion think the Opsis comes out very strong against the Nexys Video. The biggest advantage of the Opsis is the large number of video inputs and outputs (which Mike also highlights in his review).

Much better FPGA that is better suited to video work. 

I can't argue with the Artix-7 being a better FPGA then the Spartan-6 (any "version 2" of the Opsis would probably be using an Artix-7 FPGA). However, I wouldn't say that the Spartan-6 is a bad FPGA for video work -- for a long time it was the FPGA for low-cost video processing. The Spartan-6 also has the advantage of being extremely popular for a long time meaning the amount of open sample/example code available is pretty huge.

That and you get a copy of the vivado tools...

From what I understand, the Vivado WebPack is available for free and the only advantage that the license gets you is the "debug IP" and "Accelerating Integration" features (unless I'm missing something?). You can get a lot of the same features as the debug IP by using misoc (http://m-labs.hk/gateware.html) and liteScope (http://enjoy-digital.fr/litescope/docs/intro/about.html).

Every 'maker' FPGA board is seems like a bad deal.  Digilent gets the FPGAs at very special pricing.....    It is tough to beat them for xilinx products when they getting the chips for next to free.

Can't argue that it is hard to beat Digilent but I do think we have at-least come pretty close.

Thanks for the feedback ehughes!

[ehughes]

The advantage of full Vivado is the high level synthesis tools (which you won't get with Spartan 6) and better verification tools.

I agree that the extra Video I/O is nice but that S6 will have a tough time doing complicated video processing before running out of resources for all those channels.  At that point you are playing with Plan-ahead trying to meet timing closure..

I do both low and  high end FPGA for my day job (and teach a course from time to time).    My observation is that the FPGA dev tool market is a  tough but to crack.  I had a board I wanted to release but realized the business case wasn't there.

1.)   The Maker crowd will never tolerate the 30-minute Synthesis + Place&Route times for non-trivial designs. 
2.)  People from #1 don't want to pay much. 
3.)   The people from #1 tend not to see the value of verification and simulation tools.    Any non trivial design spends 90% in the verification/simulation stage.  This takes a ton of work.
4.)   People who do this for a living understand the importance of the verification tools and want the best.   FPGAs are unforgiving.    They will get access to vendor tools/boards which are often heavily subsidized (i.e. Digilent)
5.)  People who who know what they want will be able to instantly know if an FPGA board is suitable.   
6.)  As a result of 1-5,   you are going after a very small slice of what is already a small slice.      The margins are very thin and supporting new users is tough without the margins

I hope this venture is profitable for you!   Hopefully you find a way to make money where I couldnt.   If not,  it sounds like you like it enjoy it!   Good luck to your venture.    I'll keep my eye out for a Artix (or Kintex) version in the future