any possibilty to check the correct behaviour of SoftCore CPU ?

[legacy]

hi guys
i have an uncommon question, i friend of mine has developed a MIPS-1 (pipelined) SoftCore in VHDL
and he has found there are issues (1) between what the binary code does and what it is expiated to be done

he says that, in same cases, the same binary code has a different behavior

so my question is: do you see any possibilty to check the correct behaviour of SoftCore CPU?
 
I mean, a sort of automatic Hardware in the loop like:
- For each existing opcode
- Check against all rules
- Is this a correct opcode? If yes, compare Hardware with a simulator or real mips-CPU.

what do you think about ?
apologies for the dummy question, i am new about this world


(1) i have personally seen that compiling with gcc-mips has different behavior
for example i got a piece of code that is working if compiled with -O0, -O1
but that it is not working (and shows unexpected behavior) with -O2, -Os

for example, in this pseudo code i have observed

Code: [Select]

i=0;
base=2;
do
     an=an/base;
     i++;         <---------- if observed, it grows with unexpected step
                              it is expected to grow like i=i+1
                              it grows like i=i+1 if compiled with -O0, -O1
                              but if compiled with -O2 the local variable "i" is filled with garbage

loop

as far as i know gcc-mips uses the -Ox, x={0,1,2,s} flag in order to do optimization
i don't exactly know what breaks things from -O1 to -O2
but it may be something like
- Hazards
- Alee

or what else  ?

[Jeroen3]

between that the binary code does and what it is expiated to be done

Expected by the binary code, or expected by the programming language?

The C(++) compiler might optimize some things away. Read the documentation for your compiler to find out what.
After that the softcore should do as the binary says, if that is not the case then your vhdl synthesizer might have been wrong somewhere.
Is the softcore shipped with a testbench?

[legacy]

between that the binary code does and what it is expiated to be done

Expected by the binary code, or expected by the programming language?

we are talking about MIPS-1 target (with the SoftCore is compliant), so with the same binary code the SoftCore should have the same behavior of the ASIC (MIPS R3000), but in same cases it doesn't ant it shows unexpected behavior, i suspect due to a bug inside the HDL code.

Is the softcore shipped with a testbench?

yes it has, but it is a hard job to manually use the tb in order to check everything, and that is the question: is there a smarter way to do such a job ?

[bwat]

hi guys
i have an uncommon question, i friend of mine has developed a MIPS-1 (pipelined) SoftCore in VHDL
and he has found there are issues (1) between what the binary code does and what it is expiated to be done

he says that, in same cases, the same binary code has a different behavior

This isn't uncommon at all with cheap and large FPGAs and free development tools making CPU design and implementation easier than ever. If I understand you correctly this is simply a case of checking that a CPU implementation is correct with respect to the specification (manufacturer's ISA description, definitional implementation, etc.). To me it sounds like your friend just hasn't done enough testing.

1) Use the "-S" flag to look at the instruction stream. Compare and contrast the "-O1" and "-O2" options.
2) Simulate the execution of the fault inducing code to see how being executed. Do the instructions flow through the pipeline correctly? Is the ALU being given the right operands and function selection code? Is the ALU result being fed into the register file? Is the result being stored in the right register? ......

I'm guessing your branching is wrong. I have very little to back that up to be honest other than ALUs tend to fail in obvious ways that are caught easily.

Good luck.

Edit: Just saw this:

but it is a hard job to manually use the tb in order to check everything,

So you have a test bench but you don't want to check.... hmmm.... I can't see any other way to be honest.

[legacy]

If I understand you correctly this is simply a case of checking that a CPU implementation is correct with respect to the specification (manufacturer's ISA description, definitional implementation, etc.).

yes, correct =)

To me it sounds like your friend just hasn't done enough testing.

exactly, he said that he has manually tested the minimal part of the whole.

1) Use the "-S" flag to look at the instruction stream. Compare and contrast the "-O1" and "-O2" options.

i did, but i can't understand exactly where/why -O1 has good results, while -O2 and -Os has not

2) Simulate the execution of the fault inducing code to see how being executed. Do the instructions flow through the pipeline correctly? Is the ALU being given the right operands and function selection code? Is the ALU result being fed into the register file? Is the result being stored in the right register? ...…

i think i am too newly to afford this part, but my friend has much more skills in HDL

I'm guessing your branching is wrong. I have very little to back that up to be honest other than ALUs tend to fail in obvious ways that are caught easily

ok, i will consider this, thank your for your advices =)

[legacy]

what do you use to simulate ? and any tutorial about that ?

[Jeroen3]

i did, but i can't understand exactly where/why -O1 has good results, while -O2 and -Os has not

http://en.wikipedia.org/wiki/Optimizing_compiler

[legacy]

http://en.wikipedia.org/wiki/Optimizing_compiler

i know the theory, but i can't understand WHERE in the optimized code (exactly what/where/why) the optimization causes un expected behavior due to a problem in the pipeline: would you like to see the assembly code for -O0, -O1, -O2 and try to compare them ?

[bwat]

Yes, show us the assembly output. 

Edit: Make your test routine as small as possible and give us the output of "ggc -S -O1" and gcc -S -O2" to be more specific. It's been over 20 years since I worked with MIPS assembly but it's not that hard to get up and running with.

[legacy]

this it the piece of C that has the issue

Code: [Select]

basen_result_t basen_value_to_ascii
(
    uint32_t value,
    uint8_t  ascii_value[],
    uint8_t  mybase,
    uint32_t format_len
)
{
    basen_result_t ans;
    uint32_t       evaluated_digits_len;
    uint32_t       required_format_len;
    uint32_t       digits_offset;
    uint32_t       An;
    uint32_t       i;
    uint32_t       j;
    uint32_t       k;
    uint8_t        ch;

    ans.is_valid = False;
    ans.value    = value;
    ans.status   = basen_result_ok;

    if (mybase <= 16)
    {
        /* early evaluate of required format len */
       required_format_len = format_len;
       if (required_format_len>0)
          {
          required_format_len--;
          }

        /* evaluate of digit len */
        An = value;
        i  = 0;
        while ((An != 0)) // logicalAnd (i < basen_digit_n))
        {
            i++;
            An = An / mybase;
        }
        if ((required_format_len < string_safe_len)logicalAnd(i < string_safe_len))
        {
            evaluated_digits_len = i;

            /* check required_format_len */
            if (evaluated_digits_len >= required_format_len)
            {
                /*
                 * fix1
                 */
                required_format_len = evaluated_digits_len;
            }

            /* evaluate format offset */
            if (required_format_len > 0)
            {
                /*
                 * offset case A
                 *   do format digits
                 */
                digits_offset = required_format_len - evaluated_digits_len;
            }
            else
            {
                /*
                 * offset case B
                 *   do not format digits
                 */
                /* do not format digits */
                digits_offset = 0;
            }

            /* zero stuffing */
            for (i = 0; i < required_format_len; i++)
            {
                ascii_value[i] = '0';
            }
            ascii_value[i] = '\0';

            /* evaluate  */
            An = value;
            i  = 0;
            do
            {
                /*
                 * is the mips1 bug here ???
                 */
                j = (evaluated_digits_len - i) + digits_offset;
                k = (An % mybase);
                ch=basen_digit[k];
                ascii_value[j] = ch;
               
                An = An / mybase;
                i++; // <-------------------- unexpected behavior happens here, i++ is not i incremented correctly

            } while (An != 0);

            j = (evaluated_digits_len + digits_offset + 1);

            ascii_value[j] = '\0';
            ans.is_valid   = True;
        }
        else
        {
            /*
             * failure1
             */
            ascii_value[0] = '\0';
            ans.status     = basen_result_not_defined;
        }
    }
    else
    {
        /*
         * failure2
         */
        ascii_value[0] = '\0';
        ans.status     = basen_result_unsupported_base;
    }

    return ans;
}

this is the gcc-mips ENV used in the build up

Code: [Select]

MY_CC=mips-elf-gcc -G0 -fPIE -O0 -mips1 -N -mno-check-zero-division -mdivide-breaks -mno-gpopt


Code: [Select]

MY_CC=mips-elf-gcc -G0 -fPIE -O1 -mips1 -N -mno-check-zero-division -mdivide-breaks -mno-gpopt


Code: [Select]

MY_CC=mips-elf-gcc -G0 -fPIE -O2 -mips1 -N -mno-check-zero-division -mdivide-breaks -mno-gpopt


this is the assembly with -O0, no unexpected behavior

Code: [Select]

basen_value_to_ascii:
.frame $fp,56,$31 # vars= 48, regs= 1/0, args= 0, gp= 0
.mask 0x40000000,-4
.fmask 0x00000000,0
.set noreorder
.set nomacro

addiu $sp,$sp,-56
sw $fp,52($sp)
move $fp,$sp
move $2,$4
sw $5,60($fp)
sw $6,64($fp)
move $3,$7
sb $3,68($fp)
sw $0,32($fp)
lw $3,60($fp)
nop
sw $3,36($fp)
li $3,1 # 0x1
sw $3,40($fp)
lbu $3,68($fp)
nop
sltu $3,$3,17
beq $3,$0,$L40
nop

lw $3,72($fp)
nop
sw $3,24($fp)
lw $3,24($fp)
nop
beq $3,$0,$L41
nop

lw $3,24($fp)
nop
addiu $3,$3,-1
sw $3,24($fp)
$L41:
lw $3,60($fp)
nop
sw $3,16($fp)
sw $0,12($fp)
b $L42
nop

$L43:
lw $3,12($fp)
nop
addiu $3,$3,1
sw $3,12($fp)
lbu $3,68($fp)
lw $4,16($fp)
nop
divu $0,$4,$3
mfhi $4
mflo $3
sw $3,16($fp)
$L42:
lw $3,16($fp)
nop
bne $3,$0,$L43
nop

lw $3,24($fp)
nop
sltu $3,$3,40
beq $3,$0,$L44
nop

lw $3,12($fp)
nop
sltu $3,$3,40
beq $3,$0,$L44
nop

lw $3,12($fp)
nop
sw $3,28($fp)
lw $4,28($fp)
lw $3,24($fp)
nop
sltu $3,$4,$3
bne $3,$0,$L45
nop

lw $3,28($fp)
nop
sw $3,24($fp)
$L45:
lw $3,24($fp)
nop
beq $3,$0,$L46
nop

lw $4,24($fp)
lw $3,28($fp)
nop
subu $3,$4,$3
sw $3,20($fp)
b $L47
nop

$L46:
sw $0,20($fp)
$L47:
sw $0,12($fp)
b $L48
nop

$L49:
lw $4,64($fp)
lw $3,12($fp)
nop
addu $3,$4,$3
li $4,48 # 0x30
sb $4,0($3)
lw $3,12($fp)
nop
addiu $3,$3,1
sw $3,12($fp)
$L48:
lw $4,12($fp)
lw $3,24($fp)
nop
sltu $3,$4,$3
bne $3,$0,$L49
nop

lw $4,64($fp)
lw $3,12($fp)
nop
addu $3,$4,$3
sb $0,0($3)
lw $3,60($fp)
nop
sw $3,16($fp)
sw $0,12($fp)
$L50:
lw $4,28($fp)
lw $3,12($fp)
nop
subu $4,$4,$3
lw $3,20($fp)
nop
addu $3,$4,$3
sw $3,8($fp)
lbu $3,68($fp)
lw $4,16($fp)
nop
divu $0,$4,$3
mfhi $3
sw $3,4($fp)
lw $4,4($fp)
lui $3,%hi(basen_digit)
addiu $3,$3,%lo(basen_digit)
addu $3,$4,$3
lb $3,0($3)
nop
sb $3,0($fp)
lw $4,64($fp)
lw $3,8($fp)
nop
addu $3,$4,$3
lbu $4,0($fp)
nop
sb $4,0($3)
lbu $3,68($fp)
lw $4,16($fp)
nop
divu $0,$4,$3
mfhi $4
mflo $3
sw $3,16($fp)
lw $3,12($fp)
nop
addiu $3,$3,1
sw $3,12($fp)
lw $3,16($fp)
nop
bne $3,$0,$L50
nop

lw $4,28($fp)
lw $3,20($fp)
nop
addu $3,$4,$3
addiu $3,$3,1
sw $3,8($fp)
lw $4,64($fp)
lw $3,8($fp)
nop
addu $3,$4,$3
sb $0,0($3)
li $3,1 # 0x1
sw $3,32($fp)
nop
b $L52
nop

$L44:
lw $3,64($fp)
nop
sb $0,0($3)
sw $0,40($fp)
b $L52
nop

$L40:
lw $3,64($fp)
nop
sb $0,0($3)
li $3,32 # 0x20
sw $3,40($fp)
$L52:
lw $5,32($fp)
lw $4,36($fp)
lw $3,40($fp)
sw $5,0($2)
sw $4,4($2)
sw $3,8($2)
move $sp,$fp
lw $fp,52($sp)
addiu $sp,$sp,56
j $31
nop

this is the assembly with -O1, no unexpected behavior

Code: [Select]

basen_value_to_ascii:
.frame $sp,0,$31 # vars= 0, regs= 0/0, args= 0, gp= 0
.mask 0x00000000,0
.fmask 0x00000000,0
.set noreorder
.set nomacro

move $2,$4
lw $3,16($sp)
andi $7,$7,0x00ff
sltu $4,$7,17
beq $4,$0,$L43
nop

beq $3,$0,$L45
move $8,$3

addiu $8,$3,-1
$L45:
beq $5,$0,$L47
move $3,$0

move $9,$5
$L48:
divu $0,$9,$7
mflo $9
bne $9,$0,$L48
addiu $3,$3,1

$L47:
sltu $4,$8,40
beq $4,$0,$L49
nop

sltu $4,$3,40
beq $4,$0,$L49
nop

sltu $4,$3,$8
beq $4,$0,$L50
move $9,$3

move $9,$8
$L50:
beq $9,$0,$L52
move $10,$0

move $4,$0
li $10,48 # 0x30
addu $8,$6,$4
$L60:
sb $10,0($8)
addiu $4,$4,1
sltu $8,$4,$9
bne $8,$0,$L60
addu $8,$6,$4

subu $10,$9,$3
$L52:
addu $9,$6,$9
sb $0,0($9)
addu $4,$10,$3
addu $4,$6,$4
move $8,$5
lui $9,%hi(basen_digit)
addiu $9,$9,%lo(basen_digit)
$L54:
divu $0,$8,$7
mfhi $8
addu $8,$8,$9
lbu $8,0($8)
nop
sb $8,0($4)
mflo $8
bne $8,$0,$L54
addiu $4,$4,-1

addu $6,$6,$3
addu $10,$6,$10
sb $0,1($10)
li $3,1 # 0x1
b $L55
li $4,1 # 0x1

$L49:
sb $0,0($6)
move $3,$0
b $L55
move $4,$0

$L43:
sb $0,0($6)
move $3,$0
li $4,32 # 0x20
$L55:
sw $4,8($2)
sw $5,4($2)
j $31
sw $3,0($2)

this is the assembly with -O2, unexpected behavior, the local variable is out of range

Code: [Select]

basen_value_to_ascii:
.frame $sp,0,$31 # vars= 0, regs= 0/0, args= 0, gp= 0
.mask 0x00000000,0
.fmask 0x00000000,0
.set noreorder
.set nomacro

andi $7,$7,0x00ff
sltu $3,$7,17
lw $8,16($sp)
beq $3,$0,$L47
move $2,$4

beq $8,$0,$L49
nop

addiu $8,$8,-1
$L49:
beq $5,$0,$L51
move $3,$0

move $9,$5
$L52:
divu $0,$9,$7
mflo $9
bne $9,$0,$L52
addiu $3,$3,1

$L51:
sltu $4,$8,40
beq $4,$0,$L53
sltu $4,$3,40

beq $4,$0,$L53
sltu $4,$3,$8

beq $4,$0,$L54
subu $10,$8,$3

$L55:
move $4,$0
li $12,48 # 0x30
$L58:
addu $11,$6,$4
addiu $4,$4,1
sltu $9,$4,$8
bne $9,$0,$L58
sb $12,0($11)

$L57:
addu $8,$6,$8
addu $4,$10,$3
lui $9,%hi(basen_digit)
sb $0,0($8)
addu $4,$6,$4
addiu $9,$9,%lo(basen_digit)
move $8,$5
$L59:
divu $0,$8,$7
mfhi $8
addu $8,$9,$8
lbu $11,0($8)
mflo $8
sb $11,0($4)
bne $8,$0,$L59
addiu $4,$4,-1

addu $3,$6,$3
addu $10,$3,$10
li $4,1 # 0x1
li $3,1 # 0x1
sb $0,1($10)
sw $4,8($2)
sw $5,4($2)
j $31
sw $3,0($2)

$L47:
move $3,$0
li $4,32 # 0x20
sb $0,0($6)
sw $4,8($2)
sw $5,4($2)
j $31
sw $3,0($2)

$L53:
move $3,$0
move $4,$0
sb $0,0($6)
sw $4,8($2)
sw $5,4($2)
j $31
sw $3,0($2)

$L54:
bne $3,$0,$L56
nop

move $8,$0
b $L57
move $10,$0

$L56:
move $8,$3
b $L55
move $10,$0


[nctnico]

I'd try to reduce the piece of code to a smaller one with a loop. A weird quirk of the MIPS architecture is that it also executes the instruction after a branch or jumps. That is why you see so many nops after branches/jumps in the unoptimised code. Maybe that is what is going wrong.

If you could somehow single step through the code it would show where it goes wrong.

[legacy]

About the unexpected behavior, as the fact that basen_digit is an array of char defined this way

Code: [Select]

basen_digit:
.ascii "0123456789ABCDEF????\000"

if the local variable "i" is not incremented in the right way (i=i+1) it causes the variable "j" to be wrong too

Code: [Select]

j = (evaluated_digits_len - i) + digits_offset;

In particular it seems that j becomes  negative (which is impossible for its range check, see the code, it could only happens if "i" is extremely out of range, which is impossible because of the increment inside the loop)

btw, if it happens then "j" will not point to the right char

Code: [Select]

ascii_value[j] = ch

So i observe a completely wrong ascii_value in the uart terminal

Code: [Select]

            do
            {
                /*
                 * is the mips1 bug here ???
                 */
                j = (evaluated_digits_len - i) + digits_offset;
                k = (An % mybase);
                ch=basen_digit[k];
                ascii_value[j] = ch;
               
                An = An / mybase;
                i++; // <-------------------- unexpected behavior happens here, i++ is not i incremented correctly

            } while (An != 0);


Wandering why, what is causing it

[legacy]

If you could somehow single step through the code it would show where it goes wrong.

i do not have such tool, gdb-stub (which is a part of gnu debugger) is not working for this SoC (i have to fix a lot of thing, also the "break" instruction is used by gdb-stub but the SoftCore has no "break" implementation, so i can't use it): in other words i can't debug on this target step by step, i have no jtag tap machine in the HDL code … so i can do step by step on VHDL simulator (e.g. gtkWave + gHDL), or with an other MIPS machine

p.s.
i put all these assembly code (-O0, -O1, -O2) on a real ASIC MIPS, such is my Atlas-MIPS32-R2 board, and i have no issue, ok it is not MIPS1 as the Softcore, but … it may be i could exclude
- a bug in the C source code
- a bug in the toolchain
- a bug in the Env

[legacy]

it also executes the instruction after a branch or jumps

is there a flag (or a pretty/ugly trick workaround) to force gcc-mips to put a "nop" after each conditional/unconditional branch ?

 

[andersm]

Here's the inner loop, -O1 version, annotated:

Code: [Select]

# $9 = &(basen_digit[0])
# $8 = An
# $7 = mybase
# $4 = &(ascii_value[evaluated_digits_len+digits_offset+max_i])
$L54:
    divu    $0,$8,$7    # divmod(An, mybase)
    mfhi    $8          # k = remainder
    addu    $8,$8,$9    # $8 = &(basen_digit[k])
    lbu     $8,0($8)    # ch = basen_digit[k]
    nop
    sb      $8,0($4)    # ascii_value[j] = ch
    mflo    $8          # $8 = quotient, An = An / mybase
    bne     $8,$0,$L54  # if An != 0, jump to L54
    addiu   $4,$4,-1    # decrease ascii_value pointer, executed in delay slot
Here is the -O2 version:

Code: [Select]

$L59:
    divu    $0,$8,$7
    mfhi    $8
    addu    $8,$9,$8
    lbu     $11,0($8)
    mflo    $8
    sb      $11,0($4)
    bne     $8,$0,$L59
    addiu   $4,$4,-1
The only significant difference is that register $8 is changed by the mflo instruction immediately after being used as the address for a memory load. It should be trivial to construct a test case to show if there is a pipeline error here.

[legacy]

may be also "lbu" has an hazard ?

[andersm]

There shouldn't be, but if the pipeline is buggy who knows? There is a hazard between mflo and the divu of the next loop iteration, but there are more than the two required instructions between them.

[legacy]

i have reported everything to the HDL guy, the one who has put this project on.

what are your advices in order to help me (and him) to identify exactly what is buggy ? let me know if i could arrange something by firmware, he will try to simulate everything on HDL simulator (and, may be i will learn how to do it, too)

[legacy]

also, why if i wrote this assembly code (base.s)

Code: [Select]

   .text
   .align 2
   .global entry
entry:
   .set noreorder

StartTest:
   li    $9, 0x4
   li    $8,0x2000
   li    $7,0x7
$L54:
   divu  $8,$7
   mfhi  $8
   addu  $8,$9,$8
   lbu   $11,0($8)
   mflo  $8
   bne $8,$0,$L54
   addiu  $4,$4,-1

and i invoke

Code: [Select]

mips-elf-as base.s -o base.o
mips-elf-ld base.o -Ttext 0 -eentry -o base.elf
mips-elf-objdump -D --headers -S base.elf

i got this dissembled code ?

Code: [Select]

Disassembly of section .text:

00400000 <entry>:
  400000: 24090004 li t1,4
  400004: 24082000 li t0,8192
  400008: 24070007 li a3,7
  40000c: 14e00002 bnez a3,400018 <entry+0x18> <-----------???
  400010: 0107001b divu zero,t0,a3
  400014: 0007000d break 0x7 <----------------???
  400018: 00004012 mflo t0
  40001c: 00004010 mfhi t0
  400020: 01284021 addu t0,t1,t0
  400024: 910b0000 lbu t3,0(t0)
  400028: 00004012 mflo t0
  40002c: 1500fff7 bnez t0,40000c <entry+0xc>
  400030: 2484ffff addiu a0,a0,-1


[andersm]

It's a generated division by zero check:

--trap
--no-break
    as automatically macro expands certain division and multiplication instructions to check for overflow and division by zero. This option causes as to generate code to take a trap exception rather than a break exception when an error is detected. The trap instructions are only supported at Instruction Set Architecture level 2 and higher.
--break
--no-trap
    Generate code to take a break exception rather than a trap exception when an error is detected. This is the default.

[legacy]

is it possible to remove it ? i need the test as i have written without any add on

[andersm]

You have to write the instruction in the three-operand format with zero as the destination register.

Code: [Select]

    divu  $0,$8,$7

[legacy]

thanks, it compiles in the right way 

[legacy]

just to understand things, what is the different between

divu zero,Rx,Ry

and 

divu Rx,Ry ?

in other words why mips-as is taking different action if i use zero as the destination register ?

[bwat]

I haven't touched MIPS assembly since 1991/1992 but isn't $0 always zero so writes to that register are ignored? Hopefully a MIPS expert will give you a proper answer.


What I wanted to say was you're probably going to have to build a test suite. Relying on GCC generated code to exercise your CPU probably wont exercise your datapath and control unit completely. It's also much easier to test each component individually in the HDL of your choice that trying to write target programs to do the same thing. When you're satisfied the separate modules of your CPU work then you might want to run some compiled code. Just a suggestion.