LUT-heavy morphological image processing architecture on FPGA – does this approa

[newdawn15]

Hi everyone,

I've been working on a side project to learn FPGA architecture design by building a small image-processing accelerator entirely in Verilog.

The idea is a Local Interaction Processor (LIP), where every pixel is assigned its own processing element and operations are performed using only local neighbourhood interactions.

Current architecture:

8×8 image

↓

8×8 processing element array (64 PEs)

↓

Each PE receives a 3×3 neighbourhood

↓

Min/Max engine + Threshold engine

↓

Output image

Implemented operations:

- Pass-through
- Erosion
- Dilation
- Opening
- Closing
- Uniformity
- Roughness

The processing fabric is parameterized and supports programmable 3×3 structuring element masks.

I also wrote several testbenches to visualize the behavior of the various operations.

I ran Yosys synthesis and then ECP5 FPGA synthesis.

One thing that surprised me was the resource profile.

Resource utilization:

- Total LUT4: 54298 / 83640 (64%)
- Logic LUT: 35034 / 83640 (41%)
- Carry LUT: 19264 / 83640 (23%)
- DFF: 2048 / 83640 (2%)
- BRAM: 0%

This architecture seems extremely LUT-heavy and almost memory-free, which I suppose is expected since it is basically a giant neighbourhood-comparison engine.

I eventually discovered that directly targeting nextpnr also exposed another issue: my current top-level exports the entire image bus, resulting in more than 2000 external IO pins, which obviously isn't realistic for an actual board deployment.

I'm now trying to decide what direction to take next.

Some possibilities:

- Add BRAM and streaming interfaces
- AXI integration
- Cloud FPGA deployment
- OpenLane ASIC flow
- Larger image fabrics

I'd appreciate feedback from people with more FPGA experience.

Questions:

1) Is this a sensible architecture for morphological image processing?

2) Is LUT-heavy / memory-light expected for this kind of accelerator?

3) If this were to evolve into a practical accelerator, what would you change first?

GitHub:

https://github.com/abhi15-bose-max/LIP

also my zenodo link( IDK if its working , first time posting on zenodo) :https://zenodo.org/records/20822134

[booscrawl]

Interesting project. Given that this is basically a single convolutional layer it is surprising that you are using 65% of an 80K lut array. 8x8 x 3x3 is only 576. I guess the reason is that you used 16 bit depth for every pixel value.

Your repository is also missing a license.

[newdawn15]

The 16 bit pixel width is likely the cause , Each PE is performing 16-bit neighbourhood comparisons (min/max), thresholding, and operation selection, so the comparison and mux logic blows up. My next goal is to explore pipelining and more realistic memory/streaming interfaces to see how the architecture changes. And thanks for pointing out the missing license—that was an oversight on my part. I'll add one to the repository.

[booscrawl]

It might also be worth noting what LLM you used to draft it 

[asmi]

You might want to try porting this to a AMD/Xilinx 7 series (or higher) FPGA - they have LUT6 architecture as opposed to LUT4, and so logic-intensive designs tend to have lower logic depth and thus higher frequency, also since you can do more with LUT6 than with LUT4, the same design usually consumes less LUTs than on LUT4 devices.
As for ASIC, there is a saying that in FPGA registers are basically free (since there are always a lot of them and they are already there, you might as well use them), while in ASICs logic is free because you are not limited by the max number of inputs your logic gates might have, consequentially combinatorial logic which requires multiple levels of gates in FPGA due to there not being enough inputs, in ASICs can be synthesized with much less logic levels consisting of gates with larger number of inputs. Because of that designs optimized for FPGA tend to not do well when synthesized for ASIC, and vice versa. But that is only a problem if you try to extract every last drop of performance out of your design target, for most "normal" designs which have good timing margins, this is not really a problem.

[newdawn15]

Hmm , at the moment I am using open source tooling and github actions , so my FPGA experiments have been on ECP5 flow. I don't currently have access to Vivado or a Xilinx board to compare against a LUT6 architecture, but doing an Artix-7 synthesis comparison is definitely something I'd like to explore in the future. On the ASIC side I'll run the design through openLANE within a few days, hopefully the results will be better .

And thank you for your time.

[laugensalm]

Interesting one, I've attempted to do similar a few years back, running under the internal name "Cottonpicken engine" that is using fixed PE and microcode to switch/route data inside the pipeline. It ended up turning into a hardwired codec (mostly JPEG DCT style transforms), eventually.
The problem with the OpenSource tooling (I've also used ECP5) is that the HDL description does rarely turn out into an optimum during the yosys based mapping. So I ended up doing a lot of direct procedural inferencing into RTL to avoid the painful subject of teaching synthesis to spit out something sensible. This abstraction (in order to still keep portable HDL) took quite some research into a new HDL concept.

Then, 6-LUT versus 4-LUT, there's some marketing relevant controversy, but in reality, I've never come across any bottlenecks (except silly ones introduced by synthesis) that would justify selection of one over the other. So, the manufacturer-supported synthesis tools would handle that quite ok.

Anyway, if I had to do this again, I would invest into a proper inference framework earlier and start with a microcoded approach from the beginning, as it's also quite easy to incrementally verify it.
Moreover, during development, you can iron out effects from re-pipelining simply in the microcode that is typically loaded by a small front end processor into a dual port EBR. Saves you from re-synthesizing and hoping it does things right.

[newdawn15]

So if I am not wrong , your kept the neighbourhood logic but kept the PE behavior and the routing logic microcode - controlled. That's really interesting , also are there any particular FPGA families you would suggest looking into for these type of processors ?

[laugensalm]

Yep, correct. You seem to already have the first steps thought through by using opcodes (as suggested by your README). Then it will be about figuring out an effective structure that runs pipelined at the pixel clock of the image, plus you'll need line buffers for incoming pixel streams from a camera. You might try a direct Bayer pattern to YUV conversion as proof of concept (this is already non-trivial).
The line buffers will have to map into EBR, by having a short glance at your code my quick guess would be that your array is consuming all the LUTs. So that part is not really viable in the engineering domain, say, as part of a camera IP core.
Other than that, yosys and ECP5 is a good setup for learning and focusing on synthesis, you will need to dive into inference rules (what description maps to what logic). This trial & error game will be the same again with vendor tools, but at a way lower effort, typically.
The other option is to focus on the "processor design" entirely and run everything as a dry dock simulation using CXXRTL (I'm doing so with the mentioned JPEG encoder). That would allow you to push a real image through your RTL design, providing you with a working reference in the first place, then you can tackle the logic inference optimizations on the desired target (the EVDK from Lattice Semi is a nice stereo cam for such development). A long way to go though, and AI can't help you there.