Block copy instructions

[SiliconWizard]

Just having this thought lately, about potential benefits (or lack thereof) of block copy instructions in a CPU.

Memory block copy is a *very* common operation in typical programs. Efficient block copy seems critical. (We can also refer to the other thread dealing with strcpy() and memcpy() for instance.)
Some CPUs implement block copy instructions (such as Intel x86 processors), others do not.

Random thoughts about that:
* As a "complex" kind of instructions, they may be more a CISC thing than a RISC thing.
* On RISC processors, it may make more sense to "fuse" simpler instructions than implement such "complex" instructions. Fusing instructions, super-scalar execution, and such, is not trivial though.
* Block copy instructions would typically be non-interruptible, possibly taking MANY cycles and thus "blocking" the CPU, which could cause issues with interrupt latency and just thread scheduling.
* They could be made interruptible, but it would make the CPU design more complex.
* A DMA controller could be used instead, but it fits different needs IMO, would have extra setup overhead...

An example piece of RISC-V (RV32I) assembly code for a block copy of N 'words' looks like (do not hesitate to provide more optimized code if there is):
[Assume: a0 contains N, a1 the source start address and a2 the destination start address]

Code: [Select]

BlockCopy:
    beqz a0, BlockCopyEnd
    slli a0, a0, 2
    add t1, a1, a0
   
BlockCopyLoop:
    addi a1, a1, 4
    lw t2, -4(a1)
    addi a2, a2, 4
    sw t2, -4(a2)
    bgtu t1, a1, BlockCopyLoop

BlockCopyEnd:

That's 8 instructions, and 5 in the copy loop.

We could imagine a series of block copy instructions doing just that, with varying width (per byte, per half word, per word copy), and 'N' in a register or immediate.
For instance, the block copy instruction dealing with words and 'N' in a register could look like:

Code: [Select]

memcpy a2, a1, a0

Drawback for the "'N' in a register" versions: they would require 3 source registers. The immediate version would require only 2.
Such instructions don't seem very complicated to implement at all.

Your thoughts?

[Cerebus]

I suspect that a few minutes spent with an appropriate search engine will turn you up more papers and PhD theses than you'll care to read on evaluating block copy instructions versus macro-coding block copies.

Without empirical data any opinions I'd have would largely amount to 'hand waving' and the last thing we need on here is more of that kind of stuff. I will however say that my instinct is that in a well designed RISC CPU a block copy instruction won't be worth the complexity it adds.

[T3sl4co1l]

I mean, that's all the 8086 is doing, via microcode.  In essence, you've shifted some user-space software into the "BIOS", except not even that, it's a deeper level, inside the CPU itself.  It's like having a very brief library built into the hardware.

RISC tends to be a bit verbose on instruction length (preferring fixed length encodings at full bus width, and obviously there are many exceptions, on the ISA level (e.g. ARM Thumb), or within an ISA (an instruction with a full-bus-width parameter e.g. jump-immediate)), so the software version does tend to take up a fair amount of space.  But that space all fits in the instruction cache, so it's no execution penalty.  And such systems usually have more than enough program memory or cache to handle it (as long as you don't mind the cost of that memory, which was a bigger deal say in the 80s and 90s).

As I understand it, the point of RISC is to avoid, as much as possible, anything that can't be done in a single cycle.  So you very commonly get load-store architectures, even very ornate ones that can do quite a lot in that cycle (complex ALU ops on multiple arguments -- gobs and gobs of hardware, deep pipelines).  But never a humble little ADD [mem], imm that requires a read-modify-write cycle.

And because RISC is speed focused, and microcode is subject to the same restrictions, it's not like one can be faster (given the same depth of optimization, obviously).  Your only cost seems to be program space.  And the jump-return overhead, if using it as a library function rather than inlined (which is just a simple time-space tradeoff).

The natural exceptions then being, anywhere caches or multiple buses can fill in for those hazards.  A memory bus could in principle be designed for certain elementary modify operations.  I don't know the implementation details of course, but some AVR instructions or peripheral registers manifest in that way: certain IO locations can be modified at no cycle penalty; or some peripherals present a set of registers that perform the respective operations, and presumably the peripheral is resolving those internally.  Or DSPs for instance, where multiple buses are the norm, so massive throughputs can be had even on very dense operations like MAC [dest], [src1], [src2] (where all three operands are memory).  Anyway, these are architecture specific and I have very little practical knowledge about the architectures that use these methods (caches and types, multiple buses).

Put another way, the reason I think CISC tend to implement block instructions, is because, due to their complex instruction decoding and multiple housekeeping cycles per instruction, you are at a significant disadvantage doing certain very common -- namely, block or string -- operations on them.  So it is advantageous to provide that shortcut.

(With peculiar side cases being understood, like, wasn't it on several Pentiums that the written-out loop can perform better than REP STOS?  A side effect of the early transition to micro-op architectures, I think?)

Finally, the differences disappear in older and simpler architectures; I've seen 68k called RISCy before, though maybe that's not a good description of it after all.  Its instructions tend to be faster than 8086's?  Though that's also mainly due to 8086's stupid EA modes.  Architectures lacking pipelines tend to perform about the same.  Even AVR (firmly RISC?) has comparable speed to those classic chips, despite being only 8 bits, and a much newer process (though hardly cutting edge <10nm or anything).

Tim

[DiTBho]

[..] because RISC is speed focused, and microcode is subject to the same restrictions, it's not like one can be faster (given the same depth of optimization, obviously).  Your only cost seems to be program space.  And the jump-return overhead, if using it as a library function rather than inlined (which is just a simple time-space tradeoff).

sums up perfectly what I think 

[brucehoult]

An example piece of RISC-V (RV32I) assembly code for a block copy of N 'words' looks like (do not hesitate to provide more optimized code if there is):
[Assume: a0 contains N, a1 the source start address and a2 the destination start address]

Code: [Select]

BlockCopy:
    beqz a0, BlockCopyEnd
    slli a0, a0, 2
    add t1, a1, a0
   
BlockCopyLoop:
    addi a1, a1, 4
    lw t2, -4(a1)
    addi a2, a2, 4
    sw t2, -4(a2)
    bgtu t1, a1, BlockCopyLoop

BlockCopyEnd:

That's 8 instructions, and 5 in the copy loop.

And 5 clock cycles per word copied, on a typical single-issue RISC pipeline (assuming the branches are correctly predicted).

You can easily drop that as close as you want to 2 instructions (cycles) per 4 bytes as you want by unrolling a few times -- four times will give you 11 instructions for 4 words copied (2.75 cycles/word), 16 times will give you 35 instructions for 16 words copied (2.19 cycles/word)

That's assuming you're copying in SRAM or L1 cache. If you're copying in DRAM then you're going to quickly hit the memory bandwidth limit anyway.

You need a bit of overhead to deal with left-over bytes, misalignment etc.

Newlib unrolls four times. Newlib Nano does a byte loop.

Code: [Select]

/* Nonzero if either X or Y is not aligned on a "long" boundary.  */
#define UNALIGNED(X, Y) \
  (((long)X & (sizeof (long) - 1)) | ((long)Y & (sizeof (long) - 1)))

/* How many bytes are copied each iteration of the 4X unrolled loop.  */
#define BIGBLOCKSIZE    (sizeof (long) << 2)

/* How many bytes are copied each iteration of the word copy loop.  */
#define LITTLEBLOCKSIZE (sizeof (long))

/* Threshhold for punting to the byte copier.  */
#define TOO_SMALL(LEN)  ((LEN) < BIGBLOCKSIZE)

void *
__inhibit_loop_to_libcall
memcpy (void *__restrict dst0,
const void *__restrict src0,
size_t len0)
{
#if defined(PREFER_SIZE_OVER_SPEED) || defined(__OPTIMIZE_SIZE__)
  char *dst = (char *) dst0;
  char *src = (char *) src0;

  void *save = dst0;

  while (len0--)
    {
      *dst++ = *src++;
    }

  return save;
#else
  char *dst = dst0;
  const char *src = src0;
  long *aligned_dst;
  const long *aligned_src;

  /* If the size is small, or either SRC or DST is unaligned,
     then punt into the byte copy loop.  This should be rare.  */
  if (!TOO_SMALL(len0) && !UNALIGNED (src, dst))
    {
      aligned_dst = (long*)dst;
      aligned_src = (long*)src;

      /* Copy 4X long words at a time if possible.  */
      while (len0 >= BIGBLOCKSIZE)
        {
          *aligned_dst++ = *aligned_src++;
          *aligned_dst++ = *aligned_src++;
          *aligned_dst++ = *aligned_src++;
          *aligned_dst++ = *aligned_src++;
          len0 -= BIGBLOCKSIZE;
        }

      /* Copy one long word at a time if possible.  */
      while (len0 >= LITTLEBLOCKSIZE)
        {
          *aligned_dst++ = *aligned_src++;
          len0 -= LITTLEBLOCKSIZE;
        }

       /* Pick up any residual with a byte copier.  */
      dst = (char*)aligned_dst;
      src = (char*)aligned_src;
    }

  while (len0--)
    *dst++ = *src++;

  return dst0;
#endif /* not PREFER_SIZE_OVER_SPEED */
}

I actually have a criticism of this code. I'd remove the TOO_SMALL() test as being both pointless (the "while (len0 >= BIGBLOCKSIZE)" will do the same job) but also harmful because even going around a word copy loop once (let alone three times) is going to be better than copying a byte at a time.

Through the maqic of modern compilers, all those *foo++ get converted to optimal code with offsets from a pointer that is updated once per loop.

[fanOfeeDIY]

The are probably many reasons of not having special block copy instruction on RISC-V but easiest one I could imagine is when the pipeline of the RISC-V implementation gets deeper, then it will have more read after write registrar stall and becomes slower.

Code: [Select]

    lw t2, -4(a1)
    addi a2, a2, 4
    sw t2, -4(a2) <- will stall reading from register t2 for Read after write (RAW), if the implementation have more than 6 or 7 stages.

Even the modern compiler for x86 do not generate string copy instruction even the instruction exists.
The RISC-V is open specification so it could have highly pipelined implementation in the future by someone and probably RISC-V would like to maintain natural from the implementations.

I wrote similar topic here at RISC-V study meeting.
This is the link to the slide, page 7.
https://1drv.ms/b/s!AnLZDi2WpU1Ogx8bmfm5HvzBGOm-?e=kZ0f8S

I found nice chart on the pipile hazard here.
https://stackoverflow.com/questions/37396533/pipeline-stall-after-a-load-instruction-but-not-after-an-add-instruction

This was the link to the event.
https://risc-v.connpass.com/event/194872/

[westfw]

Hmm.  Does any implemented block copy instruction copy the data out-of-order to match up with the way memory and/or cache is actually implemented?

[DiTBho]

Hmm.  Does any implemented block copy instruction copy the data out-of-order to match up with the way memory and/or cache is actually implemented?

What is the problem you are thinking about?

[andersm]

Drawback for the "'N' in a register" versions: they would require 3 source registers.

If you use the third register as a counter that is updated for each iteration, that would make the instruction interruptible without too much complication.

Edit: That kind of resembles the 68010 loop mode.

[DiTBho]

That kind of resembles the 68010 loop mode.

m68k does it this way:

Code: [Select]

// Compare two blocks of memory for equality

     lea #Source,A0              // A0 points to source block
     lea #Destination,A1         // A1 points to destination block
     move.w #Count-1,D0          // Compare Count words
rpt:
     cmpm.w (A0)+,(A1)+          // Compare pair of words
     dbne D0,rpt                 // Repeat until all done

dbne: decrement D0 and  branch until D0 is not equal to zero

In a RISC design like PowerPC, D0 is not a common register file but rather a dedicated register.
So the pipeline knows exactly what's happening without the need of looking at the the branch prediction.

cmpm compares two words in memory for equality, the m68k ISA also has a similar complex move opcode 

Code: [Select]

     move.w (A0)+,(A1)+          // copy mem(p_src) into mem(p_trg)  and auto increment pointers

It was good for CISCs. It's terrible and doesn't add any gain when implemented in RISCs.
Except in the PowerPC case, which then adds extra instructions and extra complexity to manage the hardware register loop counters.

There is a reason if MIPS doesn't have anything similar 

[tggzzz]

Just having this thought lately, about potential benefits (or lack thereof) of block copy instructions in a CPU.

Memory block copy is a *very* common operation in typical programs. Efficient block copy seems critical.

Instruction execution time is no longer the determinant of program execution time. Main memory latency and bandwidth and cache coherence protocols are the killer.

The best way to speed up memory copying was very well illustrated a couple of decades ago when people were optimising LAN protocol stacks up to the application program. The successful technique was, unsurprisingly, don't copy buffers - just pass the relevant pointers up the protocol stack.

That isn't too difficult in special cases such as network stacks, but correctly sharing buffers/references in complex large applications is inherently complex. The best starting point is a language that takes care of it for you, i.e. not C/C++.

The dynamics of such applications is one reason that the microbenchmarks beloved of the C/C++ community can be very misleading with large complex applications.

[DiTBho]

The best starting point is a language that takes care of it for you, i.e. not C/C++

Probably here Rust gains some points

[techman-001]

The Z80 had special commands just for this purpose:

Block Transfer and Search

      Table 9 lists the extremely powerful block transfer instructions. These instructions operate
      with three registers.

      · HL points to the source location
      · DE points to the destination location
      · BC is a byte counter


After the programmer initializes these three registers, any of these four instructions can be
used. The Load and Increment (LDI) instruction moves one byte from the location pointed
to by HL to the location pointed to by DE. Register pairs HL and DE are then automati-
cally incremented and are ready to point to the following locations. The byte counter (i.e.,
register pair BC) is also decremented at this time. This instruction is valuable when the
blocks of data must be moved but other types of processing are required between each
move. The Load, Increment and Repeat (LDIR) instruction is an extension of the LDI
instruction. The same load and increment operation is repeated until the byte counter
reaches the count of zero. As a result, this single instruction can move any block of data
from one location to any other.

Because 16-bit registers are used, the size of the block can be up to 64KB long
(1KB = 1024 bits) and can be moved from any location in memory to any other location.
Furthermore, the blocks can be overlapping because there are no constraints on the data
used in the three register pairs.

The LDD and LDDR instructions are similar to LDI and LDIR. The only difference is that
register pairs HL and DE are decremented after every move so that a block transfer starts
from the highest address of the designated block rather than the lowest.

Table 10 specifies the op codes for the four block search instructions. The first, CPI (Com-
pare and Increment) compares the data in the Accumulator with the contents of the mem-
ory location pointed to by Register HL. The result of the compare is stored in one of the
flag bits and the HL register pair is then incremented and the byte counter (register pair
BC) is decremented.

http://www.zilog.com/docs/z80/um0080.pdf

[Nominal Animal]

Just having this thought lately, about potential benefits (or lack thereof) of block copy instructions in a CPU.

I, too, have wondered and examined these.  I came to the conclusion that auto-increment loads, stores, and comparisons (if using a separate comparison status register, unlike e.g. RISC-V where the comparison operation is inherent in the conditional branch instruction) would make the biggest difference.  It does not even matter if the increments occur before or after the access, nor even if it differs between loads and stores.  If you explore the assembly needed to implement the most commonly used operations – memset(); memcpy_inc() and memcpy_dec() (implements memcpy(), memmove(), memrepeat()); strncmp(), strnlen(), memcmp(), etc.; whichever are most common for your workloads – you'll see that the address increment/decrement is the only part common to all of these, and the one that saves the most cycles per iteration if folded to the load/store/comparison operations.

Note that the autoincrement/autodecrement matching the access size is perfectly acceptable, and that larger than byte/char accesses may have to be aligned.  I personally would like to see both 8-bit and 32/64-bit (native register size) versions; everything else can be done in software.

bzero()/memset(,0,) is only worth accelerating for full aligned cachelines or pages, and then basically only if there is some trick you can do with the memory management to do the clearing efficiently.

There is a rare case of copying unaligned memory to aligned memory, or vice versa, in e.g. the network stack and binary file format handling.  If the architecture does not support unaligned memory accesses, having a special load operation that shifts the target register up by a byte as it loads the byte from the address specified by another register (and the inverse for a store), preferably with optional autoincrement/autodecrement, would help a lot.  That way, on a 32-bit architectures, such copies would take just 4 loads, 1 store, 1 conditional branch per 32-bit word (or 1 load, 4 stores, 1 conditional branch); but might be useful for other scatter/gather byte composition/extraction too.  Note that little-endian byte order uses autoincrement, but big-endian autodecrement.

I am sure you can see the difference these make for the most common cases – str*() and mem*() functions, but also for endian-specific unaligned accessor functions (that I intend my base library to provide by default as static inline functions) – but if not, I'd be happy to post some particular examples in pseudo-assembly.

[DiTBho]

auto-increment loads, stores, and comparisons [..] would make the biggest difference

So basically you want these two, right?

Code: [Select]

load_postinc.size D_reg, (A_reg)   // load D_reg from mem[A_reg], and then increment A_reg+=size
store_postinc.size D_reg, (A_reg)  // store D_reg into mem[A_reg], and then increment A_reg+=size
size={ sizeof(u8), sizeof(u16), sizeof(u32)} = { 1, 2, 4} bytes
D_reg is the data register.
A_reg is the address register.

In a RISC design both D_reg and A_reg belong to the register file set.

MIPS doesn't have this kind of special load/store instructions with post-increment, but I think that's possible job and makes sense even for a pure RISC design.

But I wouldn't add anything more than this, otherwise the "RISC" will become a "CRISC"

[SiliconWizard]

Judging from the sheer amount of benchmarks, studies and available pieces of code for optimizing block copy, this is certainly NOT a solved issue IMHO, at least not in the general case.

And I actually found *very few* articles about block copy instructions. Very few. Most articles you can find are about optimizing block copy in assembly with instructions sets that don't have dedicated instructions. Exceptions, as said earlier, is mostly with x86 processors, on which you can find various comparisons using either the block copy instructions (that are x86 legacy and are usually not particularly well optimized on modern x86 processors), or vectored operations, with or without loop unrolling, etc. And now even when optimizations are possible with specific instruction scheduling, vector operations, etc, it requires specific CPU features and is almost an art in itself.

@bruce: taking the RISC-V example, loop unrolling (along with using the offset for load and store operations) will indeed make it almost as efficient as a dedicated controller. Drawback is: it's not a general solution, loop unrolling entirely depends on the code, and another one: it makes code size quickly grow, a lot. It's hard to beat 1 instruction when the equivalent would be anything from 8 to possibly tens of instructions (with loop unrolling). Code size is not just a size issue, it also fills up instruction caches unnecessarily.

Of course if code size is no issue, and/or you have a super-scalar CPU, and/or with vector operations, and/or huge caches, dedicated instructions/a dedicated controller are probably not useful. In other cases, I'm convinced at this point that it can be.

And implementing such instructions would definitely take much less ressources than implementing a super-scalar processor, and/or vector operations.

@andersm: a number of ways to work around the interruptibility issue and the 3 sources issue indeed. Not a big problem. As for interruptibility specifically, I'm thinking it could be interesting to have instructions that would optionally make the block copy operation either interruptible or not: for implementing exclusive access to memory at some point, a non-interruptible version would certainly be efficient IMO, compared to mutexes or the like.

For people who still think that's a useless feature, especially on a RISC CPU, I found this report:
"Hardware Acceleration for Memory to Memory Copies" - UCB/EECS-2017-2 - supervised by Randy Katz and Krste Asanovic themselves. An interesting reading.

Another read, less interesting but still dealing with the issue: AN12628 - "Optimizing Memory Copy Routines", NXP.

Finally, as to how frequent "block copy" operations are, I think many do not realize how much.
They are certainly not just coming from explicit memcpy()-like calls. Anytime data is moved around (variables, objects, ...), a "block copy" is actually issued. I have no "benchmarks" to give figures (and I'd be interested in seeing some), but I'm certain moving memory around is a very significant part of most programs. And even if your particular microarch *and* compiler allow copies as efficient as dedicated instructions/dedicated controllers, that would potentially save significant code memory. Anyway, I recommend reading the above UCB report.

[SiliconWizard]

(...)
But I wouldn't add anything more than this, otherwise the "RISC" will become a "CRISC"

This is one of the points I mentioned in my first post. But thinking about it, I'm not sure it's really a valid point.

Instead of calling such instructions "complex instructions", just call the block copy feature an "accelerator" - that happens to be invocable through single instructions, part of some IS extension - and suddenly that all becomes trendy, and not many will object.

[Nominal Animal]

auto-increment loads, stores, and comparisons [..] would make the biggest difference

So basically you want these two, right?

Code: [Select]

load_postinc.size D_reg, A_reg   // load D_reg from mem[A_reg], and then increment A_reg+=size
store_postinc.size D_reg, A_reg  // store D_reg into mem[A_reg], and then increment A_reg+=size
size={1,2,4} as for {u8, u16, u32} I/O, D_reg, data register; A_reg, address register; in a RISC design Both D_reg and A_reg belong to the register file set.

I'd trade 16-bit for autodecrement, and I don't care if the increment is preincrement or postincrement, but yes.

Autodecrement is necessary for memmove() when dst > src within the same array, and very commonly used when inserting data into a sorted array.

single instructions, part of some IS extension

That's what I'd prefer, too.  Essentially, there are ten instruction forms that I'd like to see:

• load byte from memory[reg2] to least significant byte of reg1, incrementing reg2 by one
• load byte from memory[reg2] to least significant byte of reg1, decrementing reg2 by one
• load byte from memory[reg2] to least significant byte of reg1, clearing the other bits of reg1, incrementing reg2 by one
• load byte from memory[reg2] to least significant byte of reg1, clearing the other bits of reg1, decrementing reg2 by one
• load memory[reg2] to register reg1, incrementing reg2 by register size
• load memory[reg2] to register reg1, decrementing reg2 by register size
• store the least significant byte of reg1 to memory[reg2], incrementing reg2 by one
• store the least significant byte of reg1 to memory[reg2], decrementing reg2 by one
• store reg1 to memory[reg2], incrementing reg2 by register size
• store reg1 to memory[reg2], decrementing reg2 by register size

From the first four, either pair is acceptable, both forms have their uses.  I mean, for block memory functions, either keep-upper-bits-unchanged or zero-upper-bits is fine; just do not do sign extension.

I've always had a slight dislike for sign extending a byte to a full word, because it is so rarely needed.  In most code I've seen, it is actually a bug, where a single character from a string is cast to int, and the underlying char type happens to be signed.  It is certainly not needed in block memory functions.

For example, in C, <ctype.h> functions must not be used on char; an explicit cast to (unsigned char) is required, but most programmers forget/don't know/don't care.  That is, the following code is buggy:

Code: [Select]

static inline const char *skipspaces(const char *src) {
    if (src)
        while (isspace(*src))
            src++;
    return src;
}
whereas the correct version is

Code: [Select]

static inline const char *skipspaces(const char *src) {
    if (src)
        while (isspace((unsigned char)(*src)))
            src++;
    return src;
}
The only actual case for sign extension from byte to full register is when you have a variable of type signed char or int8_t, and you load it for integer manipulation.  So I do see a need for sign extension for a single byte load to a register; but not as part of memory block functions.

[tggzzz]

Finally, as to how frequent "block copy" operations are, I think many do not realize how much.
They are certainly not just coming from explicit memcpy()-like calls. Anytime data is moved around (variables, objects, ...), a "block copy" is actually issued. I have no "benchmarks" to give figures (and I'd be interested in seeing some), but I'm certain moving memory around is a very significant part of most programs. And even if your particular microarch *and* compiler allow copies as efficient as dedicated instructions/dedicated controllers, that would potentially save significant code memory. Anyway, I recommend reading the above UCB report.

Language semantics have a significant effect. Some languages mandate copying, others don't. T

To some extent optimisation can reduce such overhead, if and only if there can be no aliasing (to the specific data) anywhere in the program.

[SiliconWizard]

Finally, as to how frequent "block copy" operations are, I think many do not realize how much.
They are certainly not just coming from explicit memcpy()-like calls. Anytime data is moved around (variables, objects, ...), a "block copy" is actually issued. I have no "benchmarks" to give figures (and I'd be interested in seeing some), but I'm certain moving memory around is a very significant part of most programs. And even if your particular microarch *and* compiler allow copies as efficient as dedicated instructions/dedicated controllers, that would potentially save significant code memory. Anyway, I recommend reading the above UCB report.

Language semantics have a significant effect. Some languages mandate copying, others don't. T

To some extent optimisation can reduce such overhead, if and only if there can be no aliasing (to the specific data) anywhere in the program.

True, although even so, data copy is IMO still significant. What language do you have in mind?

I'd like to find or write a tool that allows identifying data copy at runtime and yield statistics such as min, max, average block size and total amount of copied data, so we get real figures instead of just guessing.

[tggzzz]

Finally, as to how frequent "block copy" operations are, I think many do not realize how much.
They are certainly not just coming from explicit memcpy()-like calls. Anytime data is moved around (variables, objects, ...), a "block copy" is actually issued. I have no "benchmarks" to give figures (and I'd be interested in seeing some), but I'm certain moving memory around is a very significant part of most programs. And even if your particular microarch *and* compiler allow copies as efficient as dedicated instructions/dedicated controllers, that would potentially save significant code memory. Anyway, I recommend reading the above UCB report.

Language semantics have a significant effect. Some languages mandate copying, others don't. T

To some extent optimisation can reduce such overhead, if and only if there can be no aliasing (to the specific data) anywhere in the program.

True, although even so, data copy is IMO still significant. What language do you have in mind?

C/C++ mandate copying of structs/objects as parameters to function calls. Many other languages copy a reference to the struct/object,but that can have it's own disadvantages.

Rust has an intriguing approach to ameliorating the problems, but I haven't "kicked the tyres".

I'd like to find or write a tool that allows identifying data copy at runtime and yield statistics such as min, max, average block size and total amount of copied data, so we get real figures instead of just guessing.

Don't forget to also measure the size of the working set vs cache size,and cache line alignment.

[brucehoult]

C/C++ mandate copying of structs/objects as parameters to function calls. Many other languages copy a reference to the struct/object,but that can have it's own disadvantages.

Yes, but programmers who care about performance never do that for large objects/structs.

[RenThraysk]

m68k has movem, for block copies above a certain size. Could copy something like 48 bytes in 2 instructions.

The general solution if copying is the performance bottleneck, is to completely eliminate the need to copy. AKA Zero-copy.

Most modern OS's have zero copy APIs for things like files and networking.

[brucehoult]

m68k has movem, for block copies above a certain size. Could copy something like 48 bytes in 2 instructions.

On the actual M68000 that is 18 cycles per 4 bytes copied, plus 20 cycles overhead. You can't use all 16 registers (64 bytes), obviously, because you need some for the src and dst pointers, minimum. And whatever registers you do use, you probably have to save them somewhere at the start, and reload them later.

OK, let's say 12 registers. So 12*18+20 = 236 cycles to save them at the start and restore them after. And then 236 cycles plus some loop overhead (dbnz, say, 10 cycles) for each 48 bytes copied.

A simple MOVE.L (An)+,(An)+ is 20 cycles and doesn't use any registers except the src and dst pointers so MOVEM saves very little if you don't mind unrolling the MOVE.L loop a little.

[tggzzz]

C/C++ mandate copying of structs/objects as parameters to function calls. Many other languages copy a reference to the struct/object,but that can have it's own disadvantages.

Yes, but programmers who care about performance never do that for large objects/structs.

Yes, they accept the difficulties and problems (in C/C++) with the alternatives.

But that becomes much more problematic where code from multiple people, projects, organisations and companies has to be combined.

In such cases language semantics can and does make a big difference, especially with the vast majority of programmers.