VUnit, UVVM, OSVVM: What are similarities and differences?

[matrixofdynamism]

The VUnit, UVVM and OSVVM are all aimed to make testbench design easier for VHDL projects by introducing HVL like features into the verification process. They all target SystemVerilog UVM and other language's verification methodologies.

I can see that there is some overlap between VUnit, UVVM and OSVVM but don't understand what each of these are really good for and not good for. Also, I do not understand the features that are present on one but not the other. Ultimately, each of these will excel in something so it would be worth using something from all of them to create the best verification environment.

Has anyone used all VUnit, UVVM and OSVVM?

[laugensalm]

I've had to use some OSVVM and had a look at VUnit and its cousin cocotb,
OSVVM is sure an approach with a lot of motivation behind it, but after moving all HDL development to the Python world, I wouldn't want to mess with such a massive overhead any longer. This is mostly due to VHDL being very difficult when it comes to separating pure hardware elements from simulation constructs and limited language functionality WRT to verify constructs 'inline'. This is something where Python as an interpreted language with its meta programming caps has way more potential.

I got stuck with MyHDL co-simulation for a long time for complex mixed language projects where algorithms sketched in Python run alongside their hardware-inferred counterparts. Made VUnit void in my case, but if you have to test your setup with various simulators, it might still be the tool with best interfacing support, but I'm not up to date.
The only problem with MyHDL is that one has to live with the status quo of its architectural issues.

It all depends a bit on what kind of hardware you're developing and what the customers request. With self-verifying approaches from the Python world you can skip a lot of traditional pitfalls that actually require the overhead of *VM libraries due to the V* language deficiencies as design languages for authoring hardware logic.

If you're going 'traditional' with the standard money tools, VUnit via its *VM APIs is probably a safe way to develop proper CI/CD tests. However it may still lack some features for proper Co-Simulation with open source simulators like GHDL and ICARUS, so it might not support sticking complex designs into the model in the loop.

[matrixofdynamism]

If Python is adept at hardware design and verification, I am interested to know why it has not taken over the industry yet. I am not at all well informed on the subject.

By co-simulation you mean processor running software connected to peripherals that are HDL designed via some sort of standard bus like AMBA AXI?

Jim Lewis of OSVVM (also leading VHDL language development) has done something for cosimulation and here is article. But, I am not sure if this is actually any good since it is quite new.
https://osvvm.org/archives/2159

I am actually kind of fed up with the hardware domain lagging SOOOOOOO much behind the Software domain. VHDL-2019 was released a few years ago but most tools still do not even support all of VHDL-2008. Then there is this divide between VHDL and SystemVerilog. Then we have SytemVerilog UVM which the VHDL world is trying to catch up with by developing UVVM, OSVVM, VUnit. We have different vendors like Intel, Xilinx, Microsemi, LatticeSemiconductor and who knows what else that might have decent silicone but the software tools are just terrible or almost unuseable sometimes. I mean, why not all work together and make a lot of open source tools as well? All we need is decent simulators with decent wave viewers and BFMs for complex things like full AXI4. Is that really asking for a lot? But that is a different topic altogether.

[laugensalm]

I second your pain WRT V *HDL. Have to be careful not to stir up another language war, but one aspect from software development is that a proper VHDL parser and AST representation is pretty darn hard to fully implement (I've tried some static analysis and moved on to Python, eventually). So, we got a massively over engineered VHDL and...an easy to parse hack called Verilog tricking developers with its implicit behaviour.
Then classic hardware developers at least in the german speaking domain where I'm from are extremely unwilling to learn new methodologies, thus 80's HDL material is taught on technical universities, mostly. So there's quite a lag.
Other than that, I'm aware of quite a bunch of developers who don't only *test* their HDL using Python, but also develop in it (including me).
Big vendors are also good at telling you your new methodology is crap, until they suddenly happened to have come up with it by themselves (I could tell stories..). So, I can even see OSVVM lagging behind 10 years...I believe it was 2013 when I published virtual debugging methods via co-simulation using GHDL/ghdlex on the embedded world show.

Co-Simulation would be anything that runs in both worlds, not necessarily a processor. It's basically just looping in your own simulation driver into the HDL simulator's main loop in an asynchronous way. A kind of a 'standard' to do that is the VPI interface of most simulators, then there's VHPI on the VHDL side. And there are various code-emitting backends (MLIR, CXXRTL) that can be driven alongside a python simulation.
Example: A hashing algorithm is to be put into hardware. Algo is sketched in Python, provided with a unit test, someone has implemented a core, now your task is to verify correct behaviour for an incoming data stream. So you'll run the software and the hardware model in a virtual 'lock step' mode driven by a Python main loop.

There's a lot in the OpenSource that is really usable, sometimes also a lot more performant than some of the official money tools. I've got a few examples, if you're interested.

So, you've basically got the choice to go traditional or be stubborn about it and find a method that fits you best, despite what 'industry' says. Eventually, you've got to do the interfacing and testing right so that you can for example emit your testbench to a classic HDL for 'traditional' engineering so that they can verify the design the good old way.

[SiliconWizard]

I've looked an toyed with OSVVM, it's a well thought out framework, but ultimately I've failed to see any benefit compared to writing testbenches with just VHDL. I can't say for the 2 others.

But it's like with all frameworks: they may give you a real benefit in some contexts, or just give you the illusion of a benefit, or may just please upper management, or you may like that for personal preferences.
I personally have a general rule of limiting tool "bloat" and dependencies on third-party tools, especially those that are too big and complex to be fully inspected before using them.

[matrixofdynamism]

After using the VUnit front end Python capability, JSON-for-VHDL, VUnit message passing mechanism to create BFMs and the VUnit logging library; I don't think I can go back to just being plain VHDL.

The reason SystemVerilog UVM is taking over for large designs is it provides capabilities that are otherwise absent from plain HDL being used for verification.

SystemVerilog UVM was preceded by OVM and other HVLs like Vera. VUnit, OSVVM and UVVM are trying to add those high level HVL features into VHDL based testbenches. But, we will only understand the benefits of these features once we have used them properly in a project. For many of us, the cocktails that we have created to ease the verification process ourself means that we are already using many features found in VUnit, OSVVM and UVVM without even realizing it.

Note: SystemVerilog UVM and VHDL UVVM are completely different things.

[lasplund]

While VUnit is an alternative to UVM, it was never created with the intention to be "a UVM for VHDL". A major reason for starting the project was to provide what was missing in UVM and other frameworks focused on system-level verification. Most important is support for short code/test iterations to get quick feedback on bad design decisions/bugs/defects that cause a lot of harm if they live for a long time. This has many implications:

* The designer must be an integral part of the overall verification effort. With a separate verification team, the feedback loop is easily weeks in length. This doesn't mean that separate verification teams aren't needed, it just means that the overall development cycle becomes much more efficient if very few mistakes are discovered that way. To engage designers in verification, we need a framework which is HDL-centric. Switching to a brand new language to get started is not a pragmatic approach and not really needed. Use the HDL for what it is good for and add Python (or something else) when the HDL runs out of steam. VUnit has supported co-simulation for many years but lately we're starting to enable HDL-embedded Python to avoid all the complexities around managing co-simulation.

* Short code/test iterations means running many tests continuously while working with the RTL. That will only happen if everything is fully automated and tests are simple to add/remove/update and fast to run. Minimal boilerplate code and automatic code discovery, compilation, execution, and reporting running in multiple simulator threads (afforded by support for free simulators) are important things we missed when we started.

* Reporting must also be reliable and report on any type of error detected, not only the errors detected by the framework's own assertion functions. If not, verification isn't fully automated. VUnit tests will fail on errors regardless if you use the provided checks, HDL-native asserts, PSL, if the simulator fails on an out of range conditions, division by zero etc. As a consequence, VUnit must be very non-intrusive. VUnit will detect errors from UVM, OSVVM, UVVM, proprietary solutions and others if you mix them. If you look around it's easy to find examples where VUnit is used with one of the others.

* Enabling integration with other frameworks is also part of our core philosophy. As open-source advocates we would like to see a well-functioning ecosystem of tools. There is more to open-source than openly available source code. Open collaboration, open design discussions, etc.

* Since coding with HDL is different than programming software, our industry has been a bit isolated from the much larger "computer science universe" and we felt that we were a bit underdeveloped, missing out on ideas and design patterns that are also applicable to what we do. VUnit has been a platform for reusing such ideas.

[laugensalm]

I've looked an toyed with OSVVM, it's a well thought out framework, but ultimately I've failed to see any benefit compared to writing testbenches with just VHDL. I can't say for the 2 others.

When you don't need to automate, and when you trust full coverage of your TB, ok.
The non-intrusiveness argument from lasplund is a strong point: You'll want to keep your original code 'as is' and bombard it with test vectors from another, possibly completely asynchronous domain (key word glitch tests). That's best done by a cosimulation front end that can loop in real life data.

Then there's the issue with portability where VHDL is just terrible at, or rather, many implementations in commercial tools (take xsim and '2008), let aside compliexities like a full SoC verification, including coverage (like the classic omitted 'else' clause in a conditional 'if' chain).
Even though being turing complete, VHDL lacks some procedural features to keep complex simulations maintainable.

The master pain with all these kludgy methodologies is indeed that most developers still clearly distinguish between hardware and software and are not taught the bridging concepts that already exist (as various LLVM'ish approaches). Same goes for verification, if modern tools such as Python can create self-verifying testbenches, the distinction between developer and verification team is kinda void. (ok, kinda not, in the ASIC domain). Not everyone likes  management to hear about that :-)

[tggzzz]

The master pain with all these kludgy methodologies is indeed that most developers still clearly distinguish between hardware and software and are not taught the bridging concepts that already exist

I wish more people recognised that. It does enable you to fun at their expense: get them to provide a definition that will allow you to distinguish hardware from software. Then point out an example that causes their definition to fail. Rinse and repeat.

But there's worse: too many softies just pile one abstraction on top of another without understanding the underlying layers. Start by considering the sync/async comms mechanisms in a networking stack starting at the PHY level and progressing to all the application and web-level protocols.

Same goes for verification, if modern tools such as Python can create self-verifying testbenches, the distinction between developer and verification team is kinda void. (ok, kinda not, in the ASIC domain). Not everyone likes  management to hear about that :-)

Yes and no.

It is Sunday, so I will agree with you.
On Monday I will point out that domain-experts and software experts have different languages and concepts, and that bridging that impedance mismatch is non-trivial.
On Tuesday I will point out that coders are notoriously bad at creating tests. Doubly so if when they believe "the unit test light is green, therefore it works".

[lasplund]

Even though being turing complete, VHDL lacks some procedural features to keep complex simulations maintainable.

Could you give some examples of the procedural features that you had in mind?

[SiliconWizard]

Even though being turing complete, VHDL lacks some procedural features to keep complex simulations maintainable.

Could you give some examples of the procedural features that you had in mind?

I'm also curious. There is actually more in VHDL than 99% of VHDL users know in my experience, so that statement must certainly be backed up with some examples and proof that the author of this statement knows VHDL in its entirety, which if true, will be a relatively rare case.

I for one have never missed a single feature for writing testbenches. That was definitely not the case with Verilog, although SystemVerilog fixes most of that.

Ultimately, it's exactly like in the software world. Using big "frameworks" or your own set of tools is a matter of context, company politics and usually only 10-20% technical. The fact they can save you time is also rarely a given. Analyze that in your own context, and make decisions based on that. Avoid cargo cult engineering.

[laugensalm]

Even though being turing complete, VHDL lacks some procedural features to keep complex simulations maintainable.

Could you give some examples of the procedural features that you had in mind?

Ok, here's a few. I admit that I'm spoiled by Python, and you might argue this is not needed/can be done differently:
* Lack of yield` alike mechanisms to create well readable co-routines
* `generate` limitations to architecture level
* Lack of built-in language constructs to separate synthesis from simulation specific statements (I'm aware of vendor specific pragmas). This is also related to the above generator concept of the pythonic `yield` combined with decorators or factory classes.
* No possibility to procedurally create interfaces (inline)

The last one is a corner case for SoC systems with plenty of configureable I/Os that need to be portable among various architectures. Not directly related to TB creation, but having to test the entire MxN matrix requires many ugly hacks using the good old cpp.

Then there's a number of cases which are not really of the procedural sort or can be implemented in a complicated way (like configurations), but are just not understood by some tools.
This all requires kludgy workarounds or worst case a complete refactoring of VHDL projects when not 'done right'.

I really don't want to ignite any fires, but my personal dogma is just: minimal coding and keep things in places where they belong. Plus even though if you're the best VHDL crack who avoids all language obstacles, and magically create your code with emacs to avoid verbose VHDL typing, you'll still have more code to process and mistakes are natural. This is where  tgzzzz's tuesday developer comes in, and yes, peer reviews are mandatory and the main purpose of co-simulation is to make SW and HW developers meet.

And this is all based on the thumb rule, that code should be maximally portable, i.e. *one description in one place*  translates to several targets (be that some LLVM IR, yosys RTL, V*) where one counts as the golden reference to co-simulate against.

As for my knowledge of VHDL, well, I've worked myself through the VHDL grammar (keyword static analysis), some remains are found here (buried in XSL files): https://hackfin.gitlab.io/xhdl
I did stop at VHDL2008 though and gave up on the original goal. Nowadays I rather focus on outputting correct V* HDL.

[tggzzz]

Even though being turing complete, VHDL lacks some procedural features to keep complex simulations maintainable.

Could you give some examples of the procedural features that you had in mind?

Ok, here's a few. I admit that I'm spoiled by Python, and you might argue this is not needed/can be done differently:
* Lack of yield` alike mechanisms to create well readable co-routines
* `generate` limitations to architecture level
* Lack of built-in language constructs to separate synthesis from simulation specific statements (I'm aware of vendor specific pragmas). This is also related to the above generator concept of the pythonic `yield` combined with decorators or factory classes.
* No possibility to procedurally create interfaces (inline)

Isn't there are risk that some of those are the moral equivalent of using SQL+database where synchronous RPC is what's needed. (Yes, I've seen something very close to that, remarkably. Clearly that developer only had a hammer in his toolbox!)

I really don't want to ignite any fires, but my personal dogma is just: minimal coding and keep things in places where they belong.

My dogma is to make the code easy to understand and maintain/extend. Often that is aligned with your dogma.

And this is all based on the thumb rule, that code should be maximally portable, i.e. *one description in one place*  translates to several targets (be that some LLVM IR, yosys RTL, V*) where one counts as the golden reference to co-simulate against.

I've seen that taken stupidly far in a pure software product... One file for all variants. Variant selected by if X then else, where X is a variable held in a configuration database (because changing code is hard but changing config is simple!). Consequence: FSMs containing if-the-else nested 10 deep, so it took 6 months to get a change to a customer. Quite remarkable, really. Even heroic, if it wasn't so dimwitted.

A lot of that was human-language variants. My solution was to create a separate file for each language. Yes, there was a lot of duplication, but if a customer wanted a change to French, you could see the line and make the change in the knowledge that it wouldn't affect a customer using Swiss-French.

Summary: Good Taste and Consistency and Conceptual Integrity are key. No surprises there!

N.B. please don't start discussing the details of that analogy, since analogies often end up generating more heat than illumination. Let's keep this discussion to the well-named thread title: VUnit, UVVM, OSVVM: What are similarities and differences?

[laugensalm]

Isn't there are risk that some of those are the moral equivalent of using SQL+database where synchronous RPC is what's needed. (Yes, I've seen something very close to that, remarkably. Clearly that developer only had a hammer in his toolbox!)

That analogy is a bit far fetched, honestly. And doesn't apply, because those mechanisms are something that makes code (c)leaner, not introduce overhead - once you've wrapped your mind around it. But yes, one has to get familiar with a few modern language paradigms that are more common in the SW world, but can be applied to HW generation as well. I guess I'd have to elaborate on those, but I don't want to hijack the thread any longer with too much pythonism, there's a load of examples up already (https://github.com/hackfin/cyrite.howto) for those not hating Jupyter notebooks.

[tggzzz]

Isn't there are risk that some of those are the moral equivalent of using SQL+database where synchronous RPC is what's needed. (Yes, I've seen something very close to that, remarkably. Clearly that developer only had a hammer in his toolbox!)

That analogy is a bit far fetched, honestly. And doesn't apply, because those mechanisms are something that makes code (c)leaner, not introduce overhead - once you've wrapped your mind around it. But yes, one has to get familiar with a few modern language paradigms that are more common in the SW world, but can be applied to HW generation as well. I guess I'd have to elaborate on those, but I don't want to hijack the thread any longer with too much pythonism, there's a load of examples up already (https://github.com/hackfin/cyrite.howto) for those not hating Jupyter notebooks.

We are all well aware of the "writing Fortran in any language" phenomenon.

The SQL/RPC analogy is stretching it, but isn't that also true of your "* Lack of yield alike mechanisms to create well readable co-routines"?

Hardware is inherently parallel, so a major impedance mismatch for software developers switching to hardware is that Verilog/VHDL looks like C/Ada - but the semantics are necessarily very different. Until I understand otherwise, in this context I regard Python as being the same as C/Ada.

Overall, appropriate architecture/design/test strategies for bridging the hardware-software impedance mismatch is an interesting topic. I have only seen project-specific "point solutions", but that is a weak statement.

[lasplund]

Ok, here's a few. I admit that I'm spoiled by Python, and you might argue this is not needed/can be done differently:
* Lack of yield` alike mechanisms to create well readable co-routines
* `generate` limitations to architecture level
* Lack of built-in language constructs to separate synthesis from simulation specific statements (I'm aware of vendor specific pragmas). This is also related to the above generator concept of the pythonic `yield` combined with decorators or factory classes.
* No possibility to procedurally create interfaces (inline)

Agree, these are areas where pure VHDL is lacking support but there are solutions, at least to some extent. These are also areas where the frameworks in the OP differ.

* Rather than co-routines VUnit implements the actor model (message passing) to provide a high-level and powerful approach to asynchronous programming. This is a well-understood and supported construct in the SW world, Python included, and that's why it was brought to VUnit.
* Agree, it would be nice with a more dynamic way of generating logic.
* IEEE actually standardized a pragma: -- rtl_synthesis on/off. However, the vendors had already their pragmas in place so the standardized version wasn't generally adopted. It is also possible to define an is_simulation function that can be used to selectively execute code.
* When it comes to variants of DUTs, whether it's interfaces or something else, test(bench) parameterization is the way to go. VUnit advocates that generics/parameters are passed to the top-level testbench such that relevant combinations can be generated and applied by the Python test runner. Every such parameterization of a test(bench) is called a VUnit configuration. This is not to be confused with VHDL configurations. They can also be used but they don't scale well as the number of combinations grows (see https://vunit.github.io/blog/2023_08_26_vhdl_configurations.html).

[laugensalm]

The SQL/RPC analogy is stretching it, but isn't that also true of your "* Lack of yield alike mechanisms to create well readable co-routines"?

I don't think so. It's a rather fundamental language construct. In fact, it's the key to a number of elegant solutions with the advantage of not having to invent (yet) another verification language or extension.

For example, it enables one to build thin co-simulation and verification layers on various levels, be it as a co-sim that speaks VPI to a state of the art simulator to do lock-step verification or a high level pipeline inference that makes sure that the inferred logic  calculates the same as the Python 'native' model written by a pure SW mind. This is heading towards 'inline verifikation', but without the '#ifdef' alike messups. Last, but not least, you can get your own code coverage-proven, i.e. show that there is no unvisited code left in your design.  You won't get there that easily with translaction layer models or HDL enhanced by RPC mechanisms.

Just my opinion: If VHDL had integrated such (standardized!) fundamentals, plenty of kludgy ballast could be thrown over board and vendors wouldn't have to be doing their proprietary VPI/VHPI implementations (let aside vendor lock in purposes) - something causing quite some maintenance overhead for generic cosim tools.
To make VHDL not look bad: It's still the SOTA-language to unroll the above constructs into - to avoid garbage-in-garbage-out scenarios.

* When it comes to variants of DUTs, whether it's interfaces or something else, test(bench) parameterization is the way to go. VUnit advocates that generics/parameters are passed to the top-level testbench such that relevant combinations can be generated and applied by the Python test runner. Every such parameterization of a test(bench) is called a VUnit configuration. This is not to be confused with VHDL configurations. They can also be used but they don't scale well as the number of combinations grows (see https://vunit.github.io/blog/2023_08_26_vhdl_configurations.html).

Parametrization is mostly unproblematic with variable signal lengths and simple flags, but when it comes to conditional compilation, it can get hairy and in some cases lead to the mentioned cpp hacks or code duplication, this due to some further unnamed vendor painware.
With 'interface' I've meant  port and generics, in fact. In VHDL there's obviously no such thing as variable length parameter list. To some extent you can cover some details using records, but this is again not well thought through until '2019...and, well, you know the rest of the story.

[matrixofdynamism]

There one specific things about VHDL that bother be the most since I have used C and SV. It is that it does not contain #if which can be used to compile code based on conditions. In my workplace they have ASIC design written VHDL and parts of it are being converted to SV gradually. The VHDL design is stored in .vhdm files containing lot of #if and #endif. The build process first preprocesses these files and removes code inside #if that is not required by current configuration and generated .vhd files from it for simulation and synthesis. But with SV, the #if is part of the language. The VHDL generate could be so much better.

Anyway, I really wish that some academic minded person would compare the VUnit, OSVVM and UVVM. The UVVM forum does not seem to be very active. I posted two questions and never got a reply at all even after many days.

I really wish that instead of VUnit, UVVM, OSVVM, Cocotb e.t.c. there would be one great solution. But such solution would come from industry which it hasn't so we are all left to create open source solutions.

To each his/her own.

[tggzzz]

The SQL/RPC analogy is stretching it, but isn't that also true of your "* Lack of yield alike mechanisms to create well readable co-routines"?

I don't think so. It's a rather fundamental language construct. In fact, it's the key to a number of elegant solutions with the advantage of not having to invent (yet) another verification language or extension.

That sounds close to wanting to write Fortran in an HDL. Too close for my comfort.

Parallelism is inherent in any HDL, and can be used when writing tests. Use what the language provides, to its fullest extent. Only when it can be demonstrated that the language is insufficient should such things be considered.

Thus I don't see the need to introduce another mechanism from a very different domain.

For example, it enables one to build thin co-simulation and verification layers on various levels, be it as a co-sim that speaks VPI to a state of the art simulator

Can you describe the salient features of such a "state of the art simulator", and why the HDL is insufficient.

"For my convenience because I think that way" is an insufficient justification

to do lock-step verification or a high level pipeline inference that makes sure that the inferred logic  calculates the same as the Python 'native' model written by a pure SW mind. This is heading towards 'inline verifikation', but without the '#ifdef' alike messups. Last, but not least, you can get your own code coverage-proven, i.e. show that there is no unvisited code left in your design.  You won't get there that easily with translaction layer models or HDL enhanced by RPC mechanisms.

A "pure SW mind" is a bad thing to have anywhere near hardware.

C style #ifdefs are a revolting concept that had some value in 1980, when compilers and memory were very limited. No modern language has them, for many very sound reasons.

[tggzzz]

There one specific things about VHDL that bother be the most since I have used C and SV. It is that it does not contain #if which can be used to compile code based on conditions.

Good

You will note that #ifdef was invented when memory was very tight, and compilers were much cruder. No modern language has #ifdef, for many solid reasons. There are better ways of achieving the same ends.

It is a long time since I used VHDL, so I've probably got the terminology wrong, but ISTR VHDL had specific language features allowing different implementations of the external interface. Thus you could have a behavioural implementation, a structural implementation, and the relevant one is defined/selected during compilation.  That mechanism should be sufficient.

[laugensalm]

..
That sounds close to wanting to write Fortran in an HDL. Too close for my comfort.

Parallelism is inherent in any HDL, and can be used when writing tests. Use what the language provides, to its fullest extent. Only when it can be demonstrated that the language is insufficient should such things be considered.

Thus I don't see the need to introduce another mechanism from a very different domain.

Again, no.
Fortran has absolutely no relation to this. You might want to check up on the yield concept of modern languages (Python, C++). Then you will understand that this is the way for building simulators. It's not a different domain either, because the yield mechanisms exactly allow you to model parallelism or clearly denote sequential processes, depending on context.

Can you describe the salient features of such a "state of the art simulator", and why the HDL is insufficient.

"For my convenience because I think that way" is an insufficient justification

You were already given the arguments above. A SOTA simulator is a priori a simulator you trust, I won't go into the details of explaining simulator differences, as this gets too OT.

A "pure SW mind" is a bad thing to have anywhere near hardware.

C style #ifdefs are a revolting concept that had some value in 1980, when compilers and memory were very limited. No modern language has them, for many very sound reasons.

You're off the track again, I'm afraid. It is a very much valid desire to not emit code to a compiler in a certain context (see portable kernels, like Linux), and I've already mentioned the scenarios where VHDL can't cope.
Yes, vendors do provide code generators to generate SoC peripherals, and badly nested #ifdefs can create maintenance issues.

Don't believe the fairy tales about SW developers getting into HDL. As I said, this distinction is somewhat void for the folks who understand LLVM concepts and modern SW methods.

To at least address some of the original questions:

Anyway, I really wish that some academic minded person would compare the VUnit, OSVVM and UVVM. The UVVM forum does not seem to be very active. I posted two questions and never got a reply at all even after many days.

I really wish that instead of VUnit, UVVM, OSVVM, Cocotb e.t.c. there would be one great solution. But such solution would come from industry which it hasn't so we are all left to create open source solutions.

I'm not aware of any up to date papers. There was some discussion on the GHDL chats a while ago, i believe it was user 'umarcor' who exercised through some scenarios. The problem is the complexity, it's like basically outlining language differences, which will lead nowhere as we can see. So you might have to be more specific in what you're looking for.

The info I can provide is about Co-Simulation with VPI/VHPI and how to loop in stimuli from various language fronts, like MyHDL/Python or C via RPC calls or qemu setups speaking to virtual peripherals. For creating robust tests and CI/CD setups, Python and modules as pytest or VUnit are state of the art software methods you won't want to miss in a dev team. For functional coverage you can get some stuff crossed with OSVVM/UVVM, for formal verification again or code coverage you'll have to use something else (tool specific).

So these are boiler plate methods where you typically weave some code around it and go.
If you wish to simulate the entire system past the technology mapping, you'll use another method again where all the above is very tool specific. Making this all a lot easier using Python or open circuit languages from the CIRCT is considered advanced, there it's more like your private choice if you like/trust it or not. Seldom, management has the insight on this topic.

As long as the 'industry' tends to mutilate standards and exercise vendor locking on their software as well (I just want their silicon, darn), we have to forget about a uniform industry solution where pragmatic decisions are seldom made (why did VHS even become a standard?)

[tggzzz]

..
That sounds close to wanting to write Fortran in an HDL. Too close for my comfort.

Parallelism is inherent in any HDL, and can be used when writing tests. Use what the language provides, to its fullest extent. Only when it can be demonstrated that the language is insufficient should such things be considered.

Thus I don't see the need to introduce another mechanism from a very different domain.

Again, no.
Fortran has absolutely no relation to this. You might want to check up on the yield concept of modern languages (Python, C++). Then you will understand that this is the way for building simulators. It's not a different domain either, because the yield mechanisms exactly allow you to model parallelism or clearly denote sequential processes, depending on context.

What is insufficient about the concepts and constructs that already exist in every HDL for parallelism?

Wanting to add "alien" constructs to any language just because they are "neat" is not a valid reason. That way leads to C++.

[laugensalm]

What is insufficient about the concepts and constructs that already exist in every HDL for parallelism?

Wanting to add "alien" constructs to any language just because they are "neat" is not a valid reason. That way leads to C++.

I am not saying that there are issues WRT parallelism in a HDL hardware *description*. You just can't  write a VHDL simulator (and or any sort of procedural inline-verification) in VHDL itself. That's why all these different external frameworks are needed.
Please, again, read up on the 'yield' mechanisms before you call them 'alien'. They're as basic as a function return, and no, I have absolutely no desire that functional iterator concepts are ever added to a VHDL-Draft at all.
But again, if you'd want to discuss the advantages of Python as HDL and verification setup, I'd suggest to open up another thread.

[tggzzz]

What is insufficient about the concepts and constructs that already exist in every HDL for parallelism?

Wanting to add "alien" constructs to any language just because they are "neat" is not a valid reason. That way leads to C++.

I am not saying that there are issues WRT parallelism in a HDL hardware *description*. You just can't  write a VHDL simulator (and or any sort of procedural inline-verification) in VHDL itself. That's why all these different external frameworks are needed.
Please, again, read up on the 'yield' mechanisms before you call them 'alien'. They're as basic as a function return, and no, I have absolutely no desire that functional iterator concepts are ever added to a VHDL-Draft at all.
But again, if you'd want to discuss the advantages of Python as HDL and verification setup, I'd suggest to open up another thread.

I have been writing applications using a yield() function since 1982, i.e. over 40 years. They are not new to me!

Your statements are confusing. Sometimes it appears you are referring to augmenting the VHDL language, sometimes it appears you are referring to augmenting tools that compile/execute VHDL, sometimes it appears that you are referring to tools that are added to tools that compile/execute VHDL.

[lasplund]

* When it comes to variants of DUTs, whether it's interfaces or something else, test(bench) parameterization is the way to go. VUnit advocates that generics/parameters are passed to the top-level testbench such that relevant combinations can be generated and applied by the Python test runner. Every such parameterization of a test(bench) is called a VUnit configuration. This is not to be confused with VHDL configurations. They can also be used but they don't scale well as the number of combinations grows (see https://vunit.github.io/blog/2023_08_26_vhdl_configurations.html).

Parametrization is mostly unproblematic with variable signal lengths and simple flags, but when it comes to conditional compilation, it can get hairy and in some cases lead to the mentioned cpp hacks or code duplication, this due to some further unnamed vendor painware.
With 'interface' I've meant  port and generics, in fact. In VHDL there's obviously no such thing as variable length parameter list. To some extent you can cover some details using records, but this is again not well thought through until '2019...and, well, you know the rest of the story.

Depending on what you are using interfaces for, there are solutions. The message passing-based interfaces used for VUnit verification components are inherently variable length as messages are variable length. Within a message there can also be variable length objects such as `integer_vector_ptr` which can contain an arbitrary number of integers. In the realm of VUnit verification components, we consider a verification component interface (VCI) to be a set of supported messages. By that definition, a VC can support any number of interfaces and interfaces can be reused between VCs. Most VCs would have a fix number of VCIs that they support but nothing prevents you from making that a configuration of a VC instance or even something that is decided at runtime. Any message can be sent to anyone, it's whether or not the VC accepts the message that decides if it's supported, i.e. if the interface is present.

Variable length port/generic lists is another thing and depending on the use case there might be solutions. Example?