[Solved] Cyclone V (de10-nano) SDRAM controller - how to reserve memory for FPGA

[plastygrove]

I've been able  to successfully write to SDRAM from the FPGA using a custom Avalon MM Host/Master component. This is pretty barebones and all I'm doing is writing one byte at memory location 0x20000000 (512M).

When I boot up the de10-nano in Linux, I'm able to access the byte that I'm writing using:

Code: [Select]

busybox devmem 0x20000000
But to make sure that I'm not overwriting anything that is already using that memory space, I added the kernel parameter in the boot options to reserve 10K.

Code: [Select]

memmap=10K$512M
Is this the right way to do it?

[plastygrove]

Looks like the kernel parameter memmap isn't working for me. When I check in /proc/iomem, it doesn't show any reserved memory space:

Code: [Select]

root@de10-15aug21:~# cat /proc/iomem
00000000-3fffffff : System RAM
  00008000-00cfffff : Kernel code
  00e00000-00eff01b : Kernel data

However, I modified the device tree as shown in this post and I can now see the reserved memory in /proc/iomem. I reserved about 50MB (0x3200000) starting at 512MB (0x20000000)

Code: [Select]

root@de10-15aug21:~# cat /proc/iomem
00000000-1fffffff : System RAM
  00008000-00cfffff : Kernel code
  00e00000-00eff01b : Kernel data
23200000-3fffffff : System RAM

The modified device tree code I used is:

Code: [Select]

#include "socfpga_cyclone5.dtsi"

/ {
  model = "Terasic DE-0(Atlas)";
  compatible = "terasic,de0-atlas", "altr,socfpga-cyclone5", "altr,socfpga";

  memory@0 {
    name = "memory";
    device_type = "memory";
    reg = <0x0 0x40000000>; /* 1GB */
  };

  reserved-memory {
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;

    some_reserved: buffer@20000000 {
      compatible = "my-dev";
      no-map;
      reg = <0x20000000 0x3200000>; /* 50 MB */
    }; 
  }; 

  myfpgawrite {
    memory-region = <&some_reserved>;
    compatible = "foo";
  }; 
};


Minirant: Strangely, this is not the first time that Xilinx forum posts have helped me with an Intel/Altera device. I wonder why the Intel communities at rocketboards.org and community.intel.com are dead.

[Daixiwen]

Minirant: Strangely, this is not the first time that Xilinx forum posts have helped me with an Intel/Altera device. I wonder why the Intel communities at rocketboards.org and community.intel.com are dead.

I've never been at rocketboards but for the other one, alteraforums was very active and helpful but Intel killed it when they moved it to their crappy platform.

[Daixiwen]

Don't forget about the data cache. When you write directly to memory with a master interface, you also need to invalidate the CPU's data cache for the area before reading it. I don't know how you do this in Linux but I assume there is an API or macros for that.

[BrianHG]

Don't forget about the data cache. When you write directly to memory with a master interface, you also need to invalidate the CPU's data cache for the area before reading it.

That's cheaply designed...  Though I guess if that cache is part of the CPU core and not part of the ram controller or it's cache doesn't consider write hits within it's cache on the bus when there is an alternate master, I can see how this may be an extra step which would complicate the design.

I remember when Motorola came out with the 68040 and it's internal cache killed Amiga OS 1.3.  The only needed trick was adding 2 strategically placed 'NOP' instructions in the Amiga 1.3 ROM since with Motorola 68K line of CPUs, 'NOP' was actually not a 'NOP', but a flush cache instruction.

[Daixiwen]

It actually depends on where the master is connected. If using the f2h master interface, AFAIK it will go through the cache and be updated automatically. I'm not 100% sure on this though.
But the HPS DDR SDRAM controller also has the possibility of having several masters connected directly to it, and in that case you completely bypass the cache and need to manually invalidate it.

[mfro]

The only needed trick was adding 2 strategically placed 'NOP' instructions in the Amiga 1.3 ROM since with Motorola 68K line of CPUs, 'NOP' was actually not a 'NOP', but a flush cache instruction.

NOPe. "NOP" doesn't flush caches on m68k, it flushes the instruction pipeline (and does so on ColdFire still).

(apologize, I'm in nitpicking mood today)

[BrianHG]

The only needed trick was adding 2 strategically placed 'NOP' instructions in the Amiga 1.3 ROM since with Motorola 68K line of CPUs, 'NOP' was actually not a 'NOP', but a flush cache instruction.

NOPe. "NOP" doesn't flush caches on m68k, it flushes the instruction pipeline (and does so on ColdFire still).

(apologize, I'm in nitpicking mood today)

Is that not the level 0 instruction cache?

[mfro]

The only needed trick was adding 2 strategically placed 'NOP' instructions in the Amiga 1.3 ROM since with Motorola 68K line of CPUs, 'NOP' was actually not a 'NOP', but a flush cache instruction.

NOPe. "NOP" doesn't flush caches on m68k, it flushes the instruction pipeline (and does so on ColdFire still).

(apologize, I'm in nitpicking mood today)

Is that not the level 0 instruction cache?

Good opportunity to continue nitpicking 

While I never heard of level 0 caches before, you managed to get me googling . And indeed, it appears such concept does exist - https://www.ics.uci.edu/~nlduong/pubs/cases2012.pdf - learned something new. Thanks!

This appears to be a data cache only, however, while the instruction pipeline surely doesn't qualify as such as it contains just that - instructions

[BrianHG]

The only needed trick was adding 2 strategically placed 'NOP' instructions in the Amiga 1.3 ROM since with Motorola 68K line of CPUs, 'NOP' was actually not a 'NOP', but a flush cache instruction.

NOPe. "NOP" doesn't flush caches on m68k, it flushes the instruction pipeline (and does so on ColdFire still).

(apologize, I'm in nitpicking mood today)

Is that not the level 0 instruction cache?

Good opportunity to continue nitpicking 

While I never heard of level 0 caches before, you managed to get me googling . And indeed, it appears such concept does exist - https://www.ics.uci.edu/~nlduong/pubs/cases2012.pdf - learned something new. Thanks!

This appears to be a data cache only, however, while the instruction pipeline surely doesn't qualify as such as it contains just that - instructions

If I remember correctly, the 040 had software controls enable/disable each data caching / instruction caching / branch prediction for the instruction cache & writeback mode.  I remember seeing the switch options on one of my Amiga 040 utilities.

[mfro]

If I remember correctly, the 040 had software controls enable/disable each data caching / instruction caching / branch prediction for the instruction cache & writeback mode.  I remember seeing the switch options on one of my Amiga 040 utilities.

That's correct. Most of the m68k machines (including Amiga & Atari) that originally started 68000 based had something similar to maintain compatibility with older (68000-only) software that didn't know how to deal with caches.

... and yes, I certainly did recognise you're trying to distract from the "NOP" topic 

[BrianHG]

... and yes, I certainly did recognise you're trying to distract from the "NOP" topic 

No need to continue on a settled matter.  NOP flushes the instruction cache pipeline...

[mfro]

Apologize for nitpicking, anyway.

I'm right now debugging an 68030 core that is supposed to later be married to your DDR3 controller (thank you again), so I'm not keen to get in trouble with you

[BrianHG]

Apologize for nitpicking, anyway.

I'm right now debugging an 68030 core that is supposed to later be married to your DDR3 controller (thank you again), so I'm not keen to get in trouble with you

V1.5 is has just been completed and in testing right now.  Huge performance boost with the multiport and ease of achieving all in black timing @ 400MHz, or even 450MHz on Cyclone IV.
The only change is each multiport # is a read/write port instead of separate read and write ports.
Even CycloneV can now do 375MHz in quarter-rate mode with my ellipse demo from it used to be at 300MHz.
Though, Cyclone IV/MAX10 can now do 375MHz in half-rate mode doubling the speed of the multiport and even 400MHz in half rate mode if you code your access carefully and want to wait for a compile with all the effort & enhancement options turned up to the max while it can even now function overclocked to 500MHz making the multiport 250MHz, 16 access ports.

Technically speaking, with enough IOs, you can run 2-4 68030 simultaneously with a 32bit high res display, or 4 hires 8bit displays all in 1 fpga with 2x DDR3x16 ram chips.

[Jiale]

Hi:

I am also recently working on the project requiring the FPGA to write to HPS SDRAM. However, I cannot find explicit or useful documentation. Would you please give me some insights about how you achieve to make FPGA write to HPS SDRAM>

Thanks!

[Daixiwen]

In the HPS subsystem, you can enable the F2H (Fpga to HPS) interface. It will create for you an Avalon Memory Mapped slave interface, available in QSys. Your component inside the FPGA that needs to access the memory must export an Avalon Memory Mapped Master interface, and you can connect the two in QSys.