Embedded programming and hardware design with VHDL are worlds apart. In fact, they have essentially nothing in common.
For embedded programming, I would start simple with the Arduino. Everything that can be built with an Arduino has been built and the projects are all over the internet. But Arduino programming is a long way from bare iron embedded programming. You can skip over all the Arduino libraries and get right down in the dirt. The only thing you want to save is the bootloader. You will be using the same IDE/toolchain, just omitting the library stuff. You will need to understand avr-gcc and avr-libc, both of which the Arduino system is already using.
The STM boards are a big step up but there is a tendency to use the factory provided bloatware libraries. I haven't tried to get around this on STM boards. Then there is the original mbed LPC1768 board which is my personal favorite. You can get right down in the weeds with interrupt handlers and all the hardware gadgets. Getting the NVIC (interrupt controller) set up is an interesting learning experience. Stop by mbed.org. BTW, their toolchain works for many 'mbed compatible' STM boards. This allows use of the online toolchain (work on your project from any browser in the world) plus drag and drop device programming. There are mixed opinions on using this toolchain. I like it! If you're going to buy an STM board, it might as wel be 'mbed compatible' whether you intend to use those tools or not.
OR...
There's a project out in the Google-verse called LC3B (or the earlier version LC3) which is a hardware design project. The given state diagram can be coded up in VHDL (or whatever) to form a functional CPU with nothing more than an interrupt driven serial port. It has separate system and user privileges and while the instruction set is clearly RISC, there is a C compiler for the LC3 version. I don't know if it has been upgraded for the byte oriented LC3B. There is also an assembler for the LC3 version.
This is a pretty nice project because first you build the CPU, then you add peripherals (PWM, more serial, <whatever>) then you write bare metal code in assembly language. In the end, you will have a pretty good handle on both hardware and embedded programming.
There's a book: "Introduction to Computing Systems" by by Patt and Patel.
I am having fun playing with the PSoC 6 and the video series with the end result of creating a robotic arm controlled by a cell phone. This is a most interesting way to create projects but 90% of the code is toolchain created. The entire infrastructure is provided, all you do is add a few lines of specific code. You'll see what I'm talking about. PSoC Creator is the toolchain.
OR, if you like command line work, consider installing the AVR-GCC toolchain under Linux and using a simple editor (I use gedit) and connecting to a simple chip like the ATmega128. Now you're as close to the iron as you can get. I did quite a bit of work with this setup a long time back and I really enjoyed it. The '128 is a great chip with lots of peripherals and pins.
http://maxembedded.com/2015/06/setting-up-avr-gcc-toolchain-on-linux-and-mac-os-x/https://www.mouser.com/ProductDetail/Olimex-Ltd/AVR-H128-Chttps://www.digikey.com/product-detail/en/olimex-ltd/AVR-ISP-MK2/1188-1008-ND/3471448As to the ICSP programmer, expect to have to play with udev rules to get the USB working under Linux. According to Google, it works just fine under Windows.
What you are looking for is an ICSP programmer that will work with avr-dude.
It's a pretty large sandbox, you need to decide where you want to play.