HP 3478A: How to read/write cal SRAM

[lmester]

Would you or fenugrec be able to help me get the gain data properly re-packed? Also, let me know if I need to make changes in the unpack algorithm.

Sure, but as I mentioned we may be slightly wrong (or at least uncertain) about the decoding; it would be really good to do some kind of test as I described, before spending too much time on a possibly-wrong encoding implementation.

That sounds good. Apply a fixed input to the meter and then enter various gain values. Is this correct? The most stable reference for me would be a resistor. I assume it won't matter which function or range is used. You only want the meter response to changes in the gain value.

[fenugrec]

[...] I assume it won't matter which function or range is used. You only want the meter response to changes in the gain value.

Correct. And to make the math easier, set the Offset for that range to 0 too. Your resistor can be fine as long as you can ensure a tight connection for the duration of the tests; in 5-digit mode the last digit(s) should be pretty stable and repeatable. If not, @Miti can probably help (again !)

Gain constants: I think these should cover all edge cases:

Code: [Select]

00000
00700
00800
00900
00a00
00f00
90700
77777
88888
99999


[Miti]

And how do I send these raw gains? I would have to read the calibration constants, modify the gain for the 3V range, calculate the checksum, upload the new constants?

[lmester]

And how do I send these raw gains? I would have to read the calibration constants, modify the gain for the 3V range, calculate the checksum, upload the new constants?

That's what I'm testing now. I modified my cal edit program to accept a raw gain value. I enter 0 for the offset and 5 hex digits for the gain. The program calculates the checksum and sends the new cal entry to the meter. I then log the new reading from the meter.

If you're interested I can put the modified program up on my web site.

I may need to hold off on this and just let the meter run overnight. It's been on for several hours. The reading started at .68324 K Ohms it's slooowly decreased to .68311. I'm assuming that the meter and the resistor that I'm measuring have not yet reached a stable temperature.

Just looked over at the meter reading. It's been stable during the time I was typing this.  I'll try to get some more data.

[Miti]

That's what I'm testing now. I modified my cal edit program to accept a raw gain value. I enter 0 for the offset and 5 hex digits for the gain. The program calculates the checksum and sends the new cal entry to the meter. I then log the new reading from the meter.

If you're interested I can put the modified program up on my web site.

Sure I'm interested. Even if you do this experiment, I still intend to do it in the weekend.

[lmester]

Sure I'm interested. Even if you do this experiment, I still intend to do it in the weekend.

Download the file HP3478AGainCheck.exe from https://mesterhome.com/gpibsw/hp3478a/Software/

Replace your existing Hp3478a.exe

Go to edit configuration. Select the desired range. Enter 0 for the offset. Enter exactly 5 hex digits for the raw gain. Click update cal. Exit edit calibration data and click start measurement from the main window. The meter reading should change to reflect the new gain and offset settings.

Warning! Range and error checking are disabled. Make sure to enter exactly 5 hex digits for the gain.

Below is the data from three tests. The second test was the most stable. The starting and ending meter reading was the same.  I'm measuring  resistance. It'll be interesting to see if you get better stability when measuring voltage.

Code: [Select]

initial reading .68309
3K range default calibration: offset -1 gain 1.0036  (04C00)

Start: and End: are the calibrated resistance readings from the beginning and end of each test


Start: .68309
RawG  Meter reading
00000 68064
00700 68112
00800 68009
00900 68016
00a00 68022
00f00 68056
90700 63346
77777 73356
88888 62013
99999 62769
End: .68307


Start: .68307
00000 68062
00700 68110
00800 68008
00900 68014
00a00 68021
00f00 68055
90700 63345
77777 73356
88888 62012
99999 62769
End: . 68307


Start: .68306
00000 68061
00700 68109
00800 68006
00900 68013
00a00 68020
00f00 68053
90700 63343
77777 73353
88888 62010
99999 62766
End: . 68304

[fenugrec]

Below is the data from three tests. The second test was the most stable.

Very nice, thanks for those rigorous tests. Plugging the numbers :

Code: [Select]

gain const reading 'x' decode(x) Un-apply gain
(= reading / decode(x))
TEST SERIES 1
00000 68064 1.000000 68064.0
00700 68112 1.000700 68064.4
00800 68009 0.999200 68063.5
00900 68016 0.999300 68063.6
00a00 68022 0.999400 68062.8
00f00 68056 0.999900 68062.8
90700 63346 0.930700 68062.7
77777 73356 1.077777 68062.3
88888 62013 0.911112 68063.0
99999 62769 0.922223 68062.7

TEST SERIES 2
00000 68062 1.000000 68062.0
00700 68110 1.000700 68062.4
00800 68008 0.999200 68062.4
00900 68014 0.999300 68061.6
00a00 68021 0.999400 68061.8
00f00 68055 0.999900 68061.8
90700 63345 0.930700 68061.7
77777 73356 1.077777 68062.3
88888 62012 0.911112 68061.9
99999 62769 0.922223 68062.7

As you can see, the results fit nearly perfectly - as good as I could expect, down to the last digit, which definitely proves the quality of your experimental methodology.

The code I used in that spreadsheet is unfortunately an awful BASIC macro,

Code: [Select]

gain = 1

for cur=1 to 5
temp = clng("&h" & mid(gainstring, cur, 1))    ### this converts a single digit from the "raw gain constant" text string to a numerical value.
if (temp >= 8) then
temp = temp - 16
endif
rem digit 1 (cur = 1) is 1/100)
gain = gain + (temp / (10^(cur + 1)))
next cur

but that code, to the best of my knowledge, is equivalent to my original C code and Steve's, assuming he checks for "digit >= 8" too.

Now, for the reverse function, encode(gain), I'd have to think about this before I could say anything smart.

Again, thanks for that data. Always nice to validate theory.

[Miti]

Here are my readings. First I tried on the 3V range, it goes out of range with raw gain 77777. Then I tried on 30V range.
Some misalignment in the data but you get the idea.

Code: [Select]

Initial reading 3.00048V 3V DC Range 30V DC Range Gain reported Correct gain
Initial offset -2
Initial gain 1.027762

00000 2.91943 2.9198 1.00000 1.00000 2.91943 2.9198
00700 2.92148 2.9219 1.0007 1.0007 2.919436395 2.919856101
00800 2.9171 2.9175 1.0008 0.9992 2.919435548 2.919835869
00900 2.91739 2.9178 0.9993 0.9993 2.919433604 2.919843891
00a00 2.91768 2.9181 0.9994 0.9994 2.919431659 2.919851911
00f00 2.91914 2.9195 0.9999 0.9999 2.919431943 2.919791979
90700 2.71712 2.7175 0.9307 0.9307 2.919436983 2.919845278
77777 Over 3.1469 1.077777 1.077777 0.00000 2.919806231
88888 2.65993 2.6603 1.088888 0.911112 2.919432518 2.919838615
99999 2.69237 2.6927 0.922223 0.922223 2.919434887 2.919792718



[lmester]

Miti,

The gain unpack is also working with your data. Your calculated gain values are very close to the 2.91943 (gain = 1) value.

Code: [Select]


GainC is the compressed gain value used for the test
MeterDisp is the meter reading for the current GainC
GainNum is the uncompressed gain value
Calc-Gain is MeterDisp/GainNum. Should be 2.91943

GainC MeterDisp GainNum Calc-Gain
00000 2.91943 1.000000 2.91943
00700 2.92148 1.000700 2.919436
00800 2.9171 0.999200 2.919435
00900 2.91739 0.999300 2.919433
00a00 2.91768 0.999400 2.919431
00f00 2.91914 0.999900 2.919431
90700 2.71712 0.930700 2.919436
77777 OverRng 1.077777 ?
88888 2.65993 0.911112 2.919432
99999 2.69237 0.922223 2.919434



Thanks to steve1515 and fenugrec I'm getting a better understanding of what's going on here.

That's the problem with using code that I don't fully understand. Debugging can be really hard!

My gain unpack code is currently using "digit>8" not "digit>=8". I assume that I should switch this to "digit>=8"? With my meter cal data there are no compressed cal digits equal to 8. The change makes no difference using my data.

Also, I'd still like to know what range check to use for the gain. What limits should I set for user entered gain values?

[fenugrec]

My gain unpack code is currently using "digit>8" not "digit>=8". I assume that I should switch this to "digit>=8"?

yes, otherwise the raw constant "88888" wouldn't decode/unpack properly. As you saw, it corresponds to a gain of 0.911112 which is validated by many tests now.

Maybe it would help to think about how the gain is applied by the stock firmware: it doesn't convert the constant to a fixed-point decimal number like what we're doing. That would be incredibly wasteful on that mcs-48 architecture with no hardware multiply. Here's roughly what it does, starting with a raw reading 'R' , with for example a gain constant K=90700. I'll call each digit Kn such that K1 = 9, K3 = 7, ...

Code: [Select]

C = R;   //C will be the "almost calibrated" reading, just before the offset is applied.

K1 = 9;
// At the "1" digit, we're working at the 1/100 decimal position, so R is "decimal-shifted" 2 digits right.
// Since K1 >= 8, we need to subtract (16-9)=7 times the shifted raw reading.
C = C - ((16-9) * R/100);

K2 = 0;
//do nothing

K3 = 7:
// digit 3 : means "add 7 times", but now R is shifted 4 positions
C = C + (7 * R/10000);

//and the rest of the digits are 0, so do nothing.

//The equivalent operation is C = (1 * R) - (7 * 0.01 * R) + (7 * 0.00001 * R)
//can also be written as C = R * (1 - 0.07 + 0.00007) or
//of course, C = R * 0.9307


The reverse process is sortof like a long division I guess ? I think it goes something like this. Assume all offsets are 0 for simplicity. The meter has a raw ADC reading 'R', and it knows the expected calibrated value 'C'.

• calculate error 'e = c - r' . Can be negative of course
• shift raw value right, i.e. divide by 10. First iteration needs to start at a 1/100 division. let's call it 'sr = r * (1/100)'
• how many times does 'sr' fit in 'e' ?
• if it fits 0 to 7 times, then that encodes directly to digit 1.
• if it fits -1 to -8 times, then "encode" that digit as (16 - <value>), so 0x0F for -1 to 0x08 for -8.
• if it fits 8 or 9 times, problem : those digits indicated negative values, so we need to cheat:
   carry 10 to the digit to the left (by incrementing by 1 - careful if the digit was already 7, you'll need to do the same manoeuvre recursively), and set the current digit to (value - 10) i.e. -2 or -1.
   In other words, use the fact that 8 * sr = (10 * sr) + (-2 * sr)
• if it fits -9 times, same idea : borrow 10 from digit on the left, adjust, and continue.
• calculate new 'e' value for next loop, i.e. remainder. Right shift sr again and continue

I think that should work, conceptually at least. Interesting consequence of the +7/-8 limit on digits : some gain values can be encoded in more than one way, for example 0001C and 00006 both correspond to a gain of 1.000006 .

Also, I'd still like to know what range check to use for the gain. What limits should I set for user entered gain values?

That would be the two extremes we've tested,

Code: [Select]

77777 1.077777
88888 0.911112

[lmester]

The reverse process is sortof like a long division I guess ? I think it goes something like this. Assume all offsets are 0 for simplicity. The meter has a raw ADC reading 'R', and it knows the expected calibrated value 'C'.

• calculate error 'e = c - r' . Can be negative of course
• shift raw value right, i.e. divide by 10. First iteration needs to start at a 1/100 division. let's call it 'sr = r * (1/100)'
• how many times does 'sr' fit in 'e' ?
• if it fits 0 to 7 times, then that encodes directly to digit 1.
• if it fits -1 to -8 times, then "encode" that digit as (16 - <value>), so 0x0F for -1 to 0x08 for -8.
• if it fits 8 or 9 times, problem : those digits indicated negative values, so we need to cheat:
   carry 10 to the digit to the left (by incrementing by 1 - careful if the digit was already 7, you'll need to do the same manoeuvre recursively), and set the current digit to (value - 10) i.e. -2 or -1.
   In other words, use the fact that 8 * sr = (10 * sr) + (-2 * sr)
• if it fits -9 times, same idea : borrow 10 from digit on the left, adjust, and continue.
• calculate new 'e' value for next loop, i.e. remainder. Right shift sr again and continue

I think that should work, conceptually at least. Interesting consequence of the +7/-8 limit on digits : some gain values can be encoded in more than one way, for example 0001C and 00006 both correspond to a gain of 1.000006 .

Thank you for the info.

I'm sorry, it seems that I still don't understand the procedure. I've ben trying to calculate the packed gain for .911112  packed =0x88888. I'm not getting correct results.

I'm assuming that  e  is the gain value? e=.911112

 Could you please go through the procedure using .911112? This would give me a better understanding of the process.

[steve1515]

I've updated my release to correct for the decode error. If anyone wants to check it out and verify it would be appreciated.
https://github.com/steve1515/hp3478a-calibration/releases
https://github.com/steve1515/hp3478a-calibration

I've also been thinking about how to convert from floating point back to a raw gain, and I'm not sure there is a good way. I see what fenugrec is saying... The meter just keeps the raw nibbles and uses them as is. No floating point calculations are done in the meter. Maybe it would be better to have the interface allow the user to adjust the raw gain values as 5 sliders or 5 numeric up-down boxes where the floating point value is displayed for reference as the value is changed. 

[fenugrec]

I've also been thinking about how to convert from floating point back to a raw gain, and I'm not sure there is a good way.

Let me try again. Say the gain constant K is  "0001C". The digits are :

Code: [Select]

k1 = 0
k2 = 0
k3 = 0
k4 = 1
k5 = 'C' = -4
As we know, 0-7 mean 0-7, but 8 to F are negatives, so 0x0C corresponds to -4.

What is the meter calculating ? please read through this. scroll !

Code: [Select]

R = raw ADC reading,
delta = correction to apply
C = corrected value :

C = R + delta
delta = R * ( (K1 * 0.01) + (K2 * 0.001) + (K3 * 0.0001) .... + (K5 * 0.000 001))

######
all good ? in this case K1 to K3 are 0, so we can rewrite as

delta = R * ((1 * 0.00001) + (-4 * 0.000 001))
and then of course,
delta = R * (0.000006)

######
easy enough, I hope. This last value, 0.000006, I'll call it "k_factor" to distinguish it from the gain. So,

delta = R * k_factor

Remember from the beginning,
C = R + delta
C = R + R*k_factor
C = R * (1 + k_factor)
but we like to think about gain. What's gain ?
C = R * gain
so,
gain = 1 + k_factor


therefore "gain" in our example is 1.000006.

############
what's the opposite process ? Let's say for gain=1.049000 ? We want to determine the digits K1 to K5, such that

k_factor = (K1 * 0.01) + (K2 * 0.001) + (K3 * 0.0001) + (K4 * 0.00001) + (K5 * 0.000 001)

1) finding k_factor : super easy. Of course this can give a negative number, since "gain" can be <1.
k_factor = gain - 1 = 0.049000


2) how do you "solve" the equation to find K1 to K5 ?

2A) determine the desired k_factor  =  gain - 1 = 0.049000

2B) for digit K1, which corresponds to the position "0.04", can we solve (0.04 = K1 * 0.01) ? Of course. K1 = 4.

2C) for digit K2, corresponding to the position "0.009". Can we solve (0.009) = (K2 * 0.001) ?
  yes, but we have a problem :  we would like K2 = 9, but we can only have K values between -8 and +7.
What can we do ?

2D) well, 9 = (10 - 1). Serious math. what does that mean ?

k_factor = (K1 * 0.01) + (K2 * 0.001) + ...
k_factor = (4 * 0.01) + ((10 * 0.001) + (-1 * 0.001)) + ...
k_factor = (4 * 0.01) + (0.01 + (-1 * 0.001)) + ...
k_factor = (5 * 0.01) + (-1 * 0.001) + ...
see what happened ? I incremented K1, and set K2 = -1. All's good, we've made the digits fall in the valid range -8.... +7

the digits are now
K1 = 5
K2 = -1

2A3) : next digits are 0, thankfully. But it would be the same process.

#####  note: if a carry/borrow operation causes a digit to fall outside the valid range, then another carry/borrow would have to be repeated with the next digit to the left.

2A5) we end up with a  gain constant "5F000".




[fenugrec]

I forgot to push to GH last time I worked on this. I added an "encode_gain" function and selftest (verifies that encode(gain) == expected_gain).
The encode function isn't accessible from the CLI though; ran out of time to add that. Should be easy enough : sscanf(%d, &gain), bounds check, encode(gain).

https://github.com/fenugrec/hp3478a_utils/commit/04584cd4eb7f9f741913e01591b73cf6db307348

[Miti]

Kirill added Prologix support to his adapter and it works with Lmester's program.

https://www.eevblog.com/forum/projects/project-extending-hp3478a-functionality/new/#new

A big thanks to both, Kirill and Lmester!

[lmester]

fenugrec,

I think I have the gain encode function working properly. I still might have a problem.

You'd mentioned that a gain value of 0001C is equivalent to 00006 and both correspond to a gain of 1.000006. I think I see these differences in my gain encoding. It looks like my implementation of your algorithm prefers "06" encoding instead of  "1C" encoding.

I've looked at several different instrument cal backup data files. It looks like the instrument always encodes gain as "1C" instead of "06".

I've done some testing. The instrument does seem to handle the compressed gain in either format. My testing shows that both encoded gain values give the same instrument reading.

I'm just not comfortable about writing different compressed gain data into the instrument for the same gain setting.

Below are some tests.
Inst:    The value read from the cal RAM.
Re-enc:    The re-encoded value written back into the cal RAM.
Gain:    The same gain is reported for both compressed gain values.

Inst.   Re-enc.   Gain.
04C00   03600   1.0036
22C01   21601   1.021601
1CFE3   06FE3   1.005883

It'd be interesting to process some of the above gains with your code.
Do you generate 04C00 or 03600 for a gain of 1.0036?

Finally, I did some reality checks for gain updates entered using my software. The offset is set to 0. Various gain multipliers were applied and compared with the instrument reading.
This looks very good.


Readings with offset set to 0. and various gain settings.
Calc gain = .68085 * Inst. gain

Inst. gain   Inst. reading   Calc. gain
1.000000   .68085         0.68085   
1.010000   .68766         0.6876585
1.077777   .73380         0.73380447045   
 .911112   .62033         0.6203306052
 .990000   .67404         0.6740415

[fenugrec]

Hi,

I've done some testing. The instrument does seem to handle the compressed gain in either format. My testing shows that both encoded gain values give the same instrument reading.

Excellent, glad to hear.

I'm just not comfortable about writing different compressed gain data into the instrument for the same gain setting.

Fair point. The only theory I have is that the original algo tries to clamp each digit value to the range [-4...+5] i.e. allowing only  C,D,E,F,0,1,...,5 . I checked a few cals (non exhaustive) and indeed the digits 6,7,8,9,A,B seem to be always absent. One reason to do this might be to limit rounding error propagation by keeping multiplications small (<= 5) ?

You're welcome to check against the annotated disassembly listing I have in my repo, but I find that reading hand-optimized BCD code is super tedious. Or you could emulate that portion of the code under MAME, but tracking 6-digit packed-BCD operations is also headache-inducing...

[lmester]

Fair point. The only theory I have is that the original algo tries to clamp each digit value to the range [-4...+5] i.e. allowing only  C,D,E,F,0,1,...,5 . I checked a few cals (non exhaustive) and indeed the digits 6,7,8,9,A,B seem to be always absent. One reason to do this might be to limit rounding error propagation by keeping multiplications small (<= 5) ?

I have six cal files that i've been using to test my program. Two from my instruments and the rest from posts on EEVBLOG. All are missing 6,7,8,9,A,B in the gain digits. It does look like these digits are not used.

You're welcome to check against the annotated disassembly listing I have in my repo, but I find that reading hand-optimized BCD code is super tedious. Or you could emulate that portion of the code under MAME, but tracking 6-digit packed-BCD operations is also headache-inducing...

I must give you a big thank you for your work disassembling the code.

Many years ago I got stuck with a project to recover 6502 code from a disassembly listing. The programmer got into an argument with the company and took the source code when he left. All that I had was an EPROM with the object code. That was a really nasty job.

Working with disassembled code and no comments is certainly headache-inducing!!

For now I'm just going to use the existing gain compress code. The instrument does seem to handle the different encoding. Also, I expect that the gain adjust feature won't be used very often.

If I have any problem reports I may then adjust the algorithm to avoid the unused digits.

[Mp3]

Just wanted to thank everyone involved in this effort, I am about to build one of the atmega328 adapters to back up my own calibration data, and i will share it here.

I have one unit that hasn't been calibrated since 2004 and I will soon be buying a friend's 3478A which was calibrated only six months ago. I'll be sure to share data backed up from both units.

Has anyone tested writing the backed up data to a 3478A after a battery swap? I'm wondering if this utility replaces the need to provide auxiliary power to the SRAM while replacing the internal battery.

[Miti]

Yes, indeed, you don’t need to worry about hot swapping the battery anymore. And you also don’t need to design your own adapter. It’s all done and tested.

Adapter:
https://www.eevblog.com/forum/projects/ar488-arduino-based-gpib-adapter/

Adapter with bonuses :
https://www.eevblog.com/forum/projects/project-extending-hp3478a-functionality/

Free software :
https://www.eevblog.com/forum/testgear/free-hp3478a-multimeter-control-program/

You can also use the SW to “tune” the calibration if you need  and you have precision reference gear.

Bonus - backlight :
https://www.eevblog.com/forum/testgear/hp3478a-backlight-my-version/msg1506655/#msg1506655

[cbueche]

Hello all,

a quick thank you to all who contributed to decoding the intricacies of the HP 3478a. For the people coming late to the party, here is a full log of my actions. It would be cool to check the data I got from my procedure - I know your values would diverge but you do have a sense of what they should look like.

• get gpib to work in Linux : https://linux-gpib.sourceforge.io/ (somewhat outdated but worked for me on Kubuntu 18.04)
• create a virtual env for Python script
• dump the data using below's Python script : python ./getcal.py > hp3478_cal_dump.bin
• compile hp3478util from https://github.com/fenugrec/hp3478a_utils (see issue 1 if you can't create an executable)
• hp3478util -a hp3478_cal_dump.bin -d

virtualenv

Code: [Select]

python3.8 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install pyusb
pip install pyvisa
pip install pyvisa-py
pip install gpib-ctypes

getcal.py

You need to modify the hardcoded address (16 below).

Code: [Select]

#!/bin/env python3

import sys
import pyvisa

rm = pyvisa.ResourceManager()
dvm = rm.open_resource('GPIB::16::INSTR')

vals_per_line = 16
for addr in range(256):
    dvm.write_raw(bytes([ord('W'), addr]))
    val = dvm.read()
    sys.stdout.write(val)

    if ((addr + 1) % vals_per_line == 0):
        sys.stdout.write('\n')

sys.stdout.write('\n')
dvm.close()
rm.close()

dump of my data from python script


@@@@@FGAM@NOLG@@
@@@GAMMONL@@@@@@
AAMMMNLHIIIIIDAL
DBBKIIIIIIIALDOC
JF@@@@@@@@@@@@@I
IH@CHAAABCMBIIIH
HH@EALAKIIIIIHH@
EMEDK@IIIIII@EMC
MJGIIIIIH@EOAOJF
IIIIII@EOB@KCIII
IIH@ENCAKCIIIIIH
ALOCLIOIIIHDEBML
AAKFIIIIHCBLDBCK
I@@@@@@@@@@@@@II
GICBB@C@BMA@@@@@
@@@@@@@@@@@@@@@@


output of hp3478util -a hp3478_cal_dump.bin -t


************ hp3478util, (c) 2018-2020 fenugrec ************
entry 0x00: OK (30 mV DC)
entry 0x01: OK (300 mV DC)
entry 0x02: OK (3 V DC)
entry 0x03: OK (30 V DC)
entry 0x04: OK (300 V DC)
entry 0x05 ((Not used)): bad cks (0x00) (unused entry)
entry 0x06: OK (ACV)
entry 0x07: OK (30 Ohm 2W/4W)
entry 0x08: OK (300 Ohm 2W/4W)
entry 0x09: OK (3 KOhm 2W/4W)
entry 0x0A: OK (30 KOhm 2W/4W)
entry 0x0B: OK (300 KOhm 2W/4W)
entry 0x0C: OK (3 MOhm 2W/4W)
entry 0x0D: OK (30 MOhm 2W/4W)
entry 0x0E: OK (300 mA DC)
entry 0x0F: OK (3A DC)
entry 0x10 ((Not used)): bad cks (0x00) (unused entry)
entry 0x11: OK (300 mA/3A AC)
entry 0x12 ((Not used)): bad cks (0x00) (unused entry)


output of hp3478util -a hp3478_cal_dump.bin -d


************ hp3478util, (c) 2018-2020 fenugrec ************
entry # offset  (rawgain)       gain    range
00      000067  1D0EF   1.006979        30 mV DC
01      00A000  71DDF   1.070669        300 mV DC
02      000000  A11DD   0.941067        3 V DC
03      C89999  941CA   0.934054        30 V DC
04      2B9999  9991C   0.922306        300 V DC
05      3AA600  00000   1.000000        (Not used)
06      00009A  98038   0.922022        ACV
07      123D29  998A8   0.922132        30 Ohm 2W/4W
08      51C1B9  99998   0.922222        300 Ohm 2W/4W
09      A5D54B  09999   0.992223        3 KOhm 2W/4W
0A      05D3AD  A7999   0.946223        30 KOhm 2W/4W
0B      805F1F  A6A99   0.945323        300 KOhm 2W/4W
0C      9905F2  0B399   0.995223        3 MOhm 2W/4W
0D      99805E  31B39   1.030523        30 MOhm 2W/4W
0E      998A1C  F3C9F   0.992529        300 mA DC
0F      98452D  CA11B   0.954105        3A DC
10      999832  C423B   0.964225        (Not used)
11      000000  00000   1.000000        300 mA/3A AC
12      99A793  22030   1.022030        (Not used)

[pqass]

I came here looking for the formula for processing the raw gain value, however, I've discovered a problem with the hp3478util -a hp3478_cal_dump.bin -d output as produced with cbueche's calibration data in the previous post.

Specifically, the -d switch forgets to remove the end-of-line line-feed character (0x0A) before processing the line.  If it were the same for the -t switch, then the program wouldn't produce "OK" crc's.

If I run the linux hex-dump command cat cbueche_lf.cal | od -A x -t x1z -w13 --skip=1 on a cal file where there is a trailing line-feed every 16 bytes, then it will produce the following output which is consistent with the offset and raw gain columns shown in cbueche's post:
(After the ASCII column, I've included processed offset and gain which should just be the lower nibble of each byte except the last two bytes.)

Code: [Select]

000001 40 40 40 40 46 47 41 4d 40 4e 4f 4c 47  >@@@@FGAM@NOLG< 00: offset=000067 raw_gain=1d0ef
00000e 40 40 0a 40 40 40 47 41 4d 4d 4f 4e 4c  >@@.@@@GAMMONL< 01: offset=00a000 raw_gain=71ddf
00001b 40 40 40 40 40 40 0a 41 41 4d 4d 4d 4e  >@@@@@@.AAMMMN< 02: offset=000000 raw_gain=a11dd
000028 4c 48 49 49 49 49 49 44 41 4c 0a 44 42  >LHIIIIIDAL.DB< 03: offset=c89999 raw_gain=941ca
000035 42 4b 49 49 49 49 49 49 49 41 4c 44 4f  >BKIIIIIIIALDO< 04: offset=2b9999 raw_gain=9991c
000042 43 0a 4a 46 40 40 40 40 40 40 40 40 40  >C.JF@@@@@@@@@< 05: offset=3aa600 raw_gain=00000
00004f 40 40 40 40 49 0a 49 48 40 43 48 41 41  >@@@@I.IH@CHAA< 06: offset=00009a raw_gain=98038
00005c 41 42 43 4d 42 49 49 49 48 0a 48 48 40  >ABCMBIIIH.HH@< 07: offset=123d29 raw_gain=998a8
000069 45 41 4c 41 4b 49 49 49 49 49 48 48 40  >EALAKIIIIIHH@< 08: offset=51c1b9 raw_gain=99998
000076 0a 45 4d 45 44 4b 40 49 49 49 49 49 49  >.EMEDK@IIIIII< 09: offset=a5d54b raw_gain=09999
000083 40 45 4d 43 0a 4d 4a 47 49 49 49 49 49  >@EMC.MJGIIIII< 10: offset=05d3ad raw_gain=a7999
000090 48 40 45 4f 41 4f 4a 46 0a 49 49 49 49  >H@EOAOJF.IIII< 11: offset=805f1f raw_gain=a6a99
00009d 49 49 40 45 4f 42 40 4b 43 49 49 49 0a  >II@EOB@KCIII.< 12: offset=9905f2 raw_gain=0b399
0000aa 49 49 48 40 45 4e 43 41 4b 43 49 49 49  >IIH@ENCAKCIII< 13: offset=99805e raw_gain=31b39
0000b7 49 49 48 0a 41 4c 4f 43 4c 49 4f 49 49  >IIH.ALOCLIOII< 14: offset=998a1c raw_gain=f3c9f
0000c4 49 48 44 45 42 4d 4c 0a 41 41 4b 46 49  >IHDEBML.AAKFI< 15: offset=98452d raw_gain=ca11b
0000d1 49 49 49 48 43 42 4c 44 42 43 4b 0a 49  >IIIHCBLDBCK.I< 16: offset=999832 raw_gain=c423b
0000de 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 17: offset=000000 raw_gain=00000
0000eb 49 49 0a 47 49 43 42 42 40 43 40 42 4d  >II.GICBB@C@BM< 18: offset=99a793 raw_gain=22030
0000f8 41 40 40 40 40 40 0a 40 40 40 40 40 40  >A@@@@@.@@@@@@< 19: offset=100000 raw_gain=a0000
000105 40 40 40 40 40 40 40 40 40 40 0a        >@@@@@@@@@@.< 20: offset=000000 raw_gain=0000a

I believe the correct output should be (without any added line-feed characters):
(Note that my gain column may be wrong although I've taken it from hp3478util.c getgain(). Please confirm otherwise.  "crc=ff" means valid crc.)

Code: [Select]

000001 40 40 40 40 46 47 41 4d 40 4e 4f 4c 47  >@@@@FGAM@NOLG< 00: offset=000067 raw_gain=1d0ef gain=1.006979 crc=ff  30 mV DC
00000e 40 40 40 40 40 47 41 4d 4d 4f 4e 4c 40  >@@@@@GAMMONL@< 01: offset=000007 raw_gain=1ddfe gain=1.006688 crc=ff  300 mV DC
00001b 40 40 40 40 40 41 41 4d 4d 4d 4e 4c 48  >@@@@@AAMMMNLH< 02: offset=000001 raw_gain=1ddde gain=1.006668 crc=ff  3 V DC
000028 49 49 49 49 49 44 41 4c 44 42 42 4b 49  >IIIIIDALDBBKI< 03: offset=999994 raw_gain=1c422 gain=1.006422 crc=ff  30 V DC
000035 49 49 49 49 49 49 41 4c 44 4f 43 4a 46  >IIIIIIALDOCJF< 04: offset=999999 raw_gain=1c4f3 gain=1.006393 crc=ff  300 V DC
000042 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 05: offset=000000 raw_gain=00000 gain=1.000000 crc=00  <not used>
00004f 49 49 48 40 43 48 41 41 41 42 43 4d 42  >IIH@CHAAABCMB< 06: offset=998038 raw_gain=11123 gain=1.011123 crc=ff  AC V
00005c 49 49 49 48 48 48 40 45 41 4c 41 4b 49  >IIIHHH@EALAKI< 07: offset=999888 raw_gain=051c1 gain=1.005061 crc=ff  30 Ohm 2W/4W
000069 49 49 49 49 48 48 40 45 4d 45 44 4b 40  >IIIIHH@EMEDK@< 08: offset=999988 raw_gain=05d54 gain=1.004754 crc=ff  300 Ohm 2W/4W
000076 49 49 49 49 49 49 40 45 4d 43 4d 4a 47  >IIIIII@EMCMJG< 09: offset=999999 raw_gain=05d3d gain=1.004727 crc=ff  3 kOhm 2W/4W
000083 49 49 49 49 49 48 40 45 4f 41 4f 4a 46  >IIIIIH@EOAOJF< 10: offset=999998 raw_gain=05f1f gain=1.004909 crc=ff  30 kOhm 2W/4W
000090 49 49 49 49 49 49 40 45 4f 42 40 4b 43  >IIIIII@EOB@KC< 11: offset=999999 raw_gain=05f20 gain=1.004920 crc=ff  300 kOhm 2W/4W
00009d 49 49 49 49 49 48 40 45 4e 43 41 4b 43  >IIIIIH@ENCAKC< 12: offset=999998 raw_gain=05e31 gain=1.004831 crc=ff  3 MOhm 2W/4W
0000aa 49 49 49 49 49 48 41 4c 4f 43 4c 49 4f  >IIIIIHALOCLIO< 13: offset=999998 raw_gain=1cf3c gain=1.005926 crc=ff  30 MOhm 2W/4W
0000b7 49 49 49 48 44 45 42 4d 4c 41 41 4b 46  >IIIHDEBMLAAKF< 14: offset=999845 raw_gain=2dc11 gain=1.016611 crc=ff  300 mA DC
0000c4 49 49 49 49 48 43 42 4c 44 42 43 4b 49  >IIIIHCBLDBCKI< 15: offset=999983 raw_gain=2c423 gain=1.016423 crc=ff  3A DC
0000d1 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 16: offset=000000 raw_gain=00000 gain=1.000000 crc=00  <not used>
0000de 49 49 47 49 43 42 42 40 43 40 42 4d 41  >IIGICBB@C@BMA< 17: offset=997932 raw_gain=20302 gain=1.020302 crc=ff  300 mA/3A AC
0000eb 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 18: offset=000000 raw_gain=00000 gain=1.000000 crc=00  <not used>
0000f8 40 40 40 40 40 40 40 40                 >@@@@@@@@< 19: <padding>


[cbueche]

Hi,

after a laptop upgrade (now running Kubuntu 20.04) and recompiling all tools and GPIB setup is done, I have a new GPIB adapter as well (bought used on Ebay, BEIMING F82357), I read the calibration data again and got the same as the last time :

Code: [Select]

@@@@@FGAM@NOLG@@
@@@GAMMONL@@@@@@
AAMMMNLHIIIIIDAL
DBBKIIIIIIIALDOC
JF@@@@@@@@@@@@@I
IH@CHAAABCMBIIIH
HH@EALAKIIIIIHH@
EMEDK@IIIIII@EMC
MJGIIIIIH@EOAOJF
IIIIII@EOB@KCIII
IIH@ENCAKCIIIIIH
ALOCLIOIIIHDEBML
AAKFIIIIHCBLDBCK
I@@@@@@@@@@@@@II
GICBB@C@BMA@@@@@
@@@@@@@@@@@@@@@@

Then running through hp3478util :

Code: [Select]

hp3478a_utils/hp3478util -a calibration_data/hp3478_cal_dump_1.bin -t

************ hp3478util, (c) 2018-2020 fenugrec ************
entry 0x00: OK (30 mV DC)
entry 0x01: OK (300 mV DC)
entry 0x02: OK (3 V DC)
entry 0x03: OK (30 V DC)
entry 0x04: OK (300 V DC)
entry 0x05 ((Not used)): bad cks (0x00) (unused entry)
entry 0x06: OK (ACV)
entry 0x07: OK (30 Ohm 2W/4W)
entry 0x08: OK (300 Ohm 2W/4W)
entry 0x09: OK (3 KOhm 2W/4W)
entry 0x0A: OK (30 KOhm 2W/4W)
entry 0x0B: OK (300 KOhm 2W/4W)
entry 0x0C: OK (3 MOhm 2W/4W)
entry 0x0D: OK (30 MOhm 2W/4W)
entry 0x0E: OK (300 mA DC)
entry 0x0F: OK (3A DC)
entry 0x10 ((Not used)): bad cks (0x00) (unused entry)
entry 0x11: OK (300 mA/3A AC)
entry 0x12 ((Not used)): bad cks (0x00) (unused entry)

so far so good, now look at this dump :

Code: [Select]

hp3478a_utils/hp3478util -a calibration_data/hp3478_cal_dump_1.bin -d
************ hp3478util, (c) 2018-2020 fenugrec ************
entry # offset  (rawgain)       gain    range
00      000067  1D0EF   1.006979        30 mV DC
01      000007  1DDFE   1.006688        300 mV DC
02      000001  1DDDE   1.006668        3 V DC
03      999994  1C422   1.006422        30 V DC
04      999999  1C4F3   1.006393        300 V DC
05      000000  00000   1.000000        (Not used)
06      998038  11123   1.011123        ACV
07      999888  051C1   1.005061        30 Ohm 2W/4W
08      999988  05D54   1.004754        300 Ohm 2W/4W
09      999999  05D3D   1.004727        3 KOhm 2W/4W
0A      999998  05F1F   1.004909        30 KOhm 2W/4W
0B      999999  05F20   1.004920        300 KOhm 2W/4W
0C      999998  05E31   1.004831        3 MOhm 2W/4W
0D      999998  1CF3C   1.005926        30 MOhm 2W/4W
0E      999845  2DC11   1.016611        300 mA DC
0F      999983  2C423   1.016423        3A DC
10      000000  00000   1.000000        (Not used)
11      997932  20302   1.020302        300 mA/3A AC
12      000000  00000   1.000000        (Not used)

I now get the gain values you have in your post. I'm not sure why my initial data extracted with "-d" was wrong. I wrote the getcal.py in steps and maybe I had some CR/LF in it while debugging, causing the errors.

[pqass]

Hi,

...snip...

I now get the gain values you have in your post. I'm not sure why my initial data extracted with "-d" was wrong. I wrote the getcal.py in steps and maybe I had some CR/LF in it while debugging, causing the errors.

Good to hear that it was just an input file mixup between the -t and -d outputs.  The raw offset column is a binary coded decimal (see fenugrec earlier in this thread) and we shouldn't see any alpha characters in it.
Thanks for confirming my gain output.

For future readers, this is the Linux shell script that I use to dump raw calibration data (don't feed it new-lines):

Code: [Select]

#!/bin/sh

if [ -f "$1" ]; then
cat "$1" | od -A x -t x1z -w13 --skip=1 | awk 'BEGIN{
SEP="|";
rectypestr="30 mV DC" SEP "300 mV DC" SEP "3 V DC" SEP "30 V DC" SEP "300 V DC" SEP "<not used>" SEP "AC V" SEP "30 Ohm 2W/4W" SEP "300 Ohm 2W/4W" SEP "3 kOhm 2W/4W" SEP "30 kOhm 2W/4W" SEP "300 kOhm 2W/4W" SEP "3 MOhm 2W/4W" SEP "30 MOhm 2W/4W" SEP "300 mA DC" SEP "3A DC" SEP "<not used>" SEP "300 mA/3A AC" SEP "<not used>" SEP "<padding>";
rectype[0] = "";
split(rectypestr, rectype, SEP);
}{
if(NF==15) {
crc=0; oset = ""; gain = ""; cgain = 1.0; multi = 0.01;
for(i=1; i<NF; i++) {
if(length($i)==2) {
crc += strtonum("0x"(substr($i,2))""((i==13)?"0":""));
if (i < 8) { /* first 6 bytes */
oset = oset "" substr($i,2);
} else if (i < 13) { /* next 5 bytes */
gain = gain "" substr($i,2);
digit = strtonum("0x"(substr($i,2)));
cgain += (digit - ((digit > 7) ? 16 : 0)) * multi;
multi = multi / 10;
}
}
}
coset = strtonum(oset);                      /* raw offset is in bcd; not hex */
if (strtonum(oset)>499999) coset -= 1000000; /* if raw offset is greater than 499999 then its a negative number */
printf "%s\t%.2d: raw_offset=%s raw_gain=%s offset=%+07d gain=%1.6f crc=%.2x  %s\n", $0, (NR-1), oset, gain, coset, cgain, crc, rectype[NR];
} else if(NF>1) {
printf "%s\t%.2d: %s\n", $0, (NR-1), rectype[NR];
}
}'
else
echo "$0 <cal file>"
fi

And this is the output using cbueche's calibration data as input (I now process the raw_offset to get +/- values):
(Use tr -d '\n' <cbueche2_lf.cal >cbueche2.cal to remove any new-line characters before feeding the script.)

Code: [Select]

$ ./verify.sh cbueche2.cal
000001 40 40 40 40 46 47 41 4d 40 4e 4f 4c 47  >@@@@FGAM@NOLG< 00: raw_offset=000067 raw_gain=1d0ef offset=+000055 gain=1.006979 crc=ff  30 mV DC
00000e 40 40 40 40 40 47 41 4d 4d 4f 4e 4c 40  >@@@@@GAMMONL@< 01: raw_offset=000007 raw_gain=1ddfe offset=+000007 gain=1.006688 crc=ff  300 mV DC
00001b 40 40 40 40 40 41 41 4d 4d 4d 4e 4c 48  >@@@@@AAMMMNLH< 02: raw_offset=000001 raw_gain=1ddde offset=+000001 gain=1.006668 crc=ff  3 V DC
000028 49 49 49 49 49 44 41 4c 44 42 42 4b 49  >IIIIIDALDBBKI< 03: raw_offset=999994 raw_gain=1c422 offset=-000006 gain=1.006422 crc=ff  30 V DC
000035 49 49 49 49 49 49 41 4c 44 4f 43 4a 46  >IIIIIIALDOCJF< 04: raw_offset=999999 raw_gain=1c4f3 offset=-000001 gain=1.006393 crc=ff  300 V DC
000042 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 05: raw_offset=000000 raw_gain=00000 offset=+000000 gain=1.000000 crc=00  <not used>
00004f 49 49 48 40 43 48 41 41 41 42 43 4d 42  >IIH@CHAAABCMB< 06: raw_offset=998038 raw_gain=11123 offset=-001962 gain=1.011123 crc=ff  AC V
00005c 49 49 49 48 48 48 40 45 41 4c 41 4b 49  >IIIHHH@EALAKI< 07: raw_offset=999888 raw_gain=051c1 offset=-000112 gain=1.005061 crc=ff  30 Ohm 2W/4W
000069 49 49 49 49 48 48 40 45 4d 45 44 4b 40  >IIIIHH@EMEDK@< 08: raw_offset=999988 raw_gain=05d54 offset=-000012 gain=1.004754 crc=ff  300 Ohm 2W/4W
000076 49 49 49 49 49 49 40 45 4d 43 4d 4a 47  >IIIIII@EMCMJG< 09: raw_offset=999999 raw_gain=05d3d offset=-000001 gain=1.004727 crc=ff  3 kOhm 2W/4W
000083 49 49 49 49 49 48 40 45 4f 41 4f 4a 46  >IIIIIH@EOAOJF< 10: raw_offset=999998 raw_gain=05f1f offset=-000002 gain=1.004909 crc=ff  30 kOhm 2W/4W
000090 49 49 49 49 49 49 40 45 4f 42 40 4b 43  >IIIIII@EOB@KC< 11: raw_offset=999999 raw_gain=05f20 offset=-000001 gain=1.004920 crc=ff  300 kOhm 2W/4W
00009d 49 49 49 49 49 48 40 45 4e 43 41 4b 43  >IIIIIH@ENCAKC< 12: raw_offset=999998 raw_gain=05e31 offset=-000002 gain=1.004831 crc=ff  3 MOhm 2W/4W
0000aa 49 49 49 49 49 48 41 4c 4f 43 4c 49 4f  >IIIIIHALOCLIO< 13: raw_offset=999998 raw_gain=1cf3c offset=-000002 gain=1.005926 crc=ff  30 MOhm 2W/4W
0000b7 49 49 49 48 44 45 42 4d 4c 41 41 4b 46  >IIIHDEBMLAAKF< 14: raw_offset=999845 raw_gain=2dc11 offset=-000155 gain=1.016611 crc=ff  300 mA DC
0000c4 49 49 49 49 48 43 42 4c 44 42 43 4b 49  >IIIIHCBLDBCKI< 15: raw_offset=999983 raw_gain=2c423 offset=-000017 gain=1.016423 crc=ff  3A DC
0000d1 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 16: raw_offset=000000 raw_gain=00000 offset=+000000 gain=1.000000 crc=00  <not used>
0000de 49 49 47 49 43 42 42 40 43 40 42 4d 41  >IIGICBB@C@BMA< 17: raw_offset=997932 raw_gain=20302 offset=-002068 gain=1.020302 crc=ff  300 mA/3A AC
0000eb 40 40 40 40 40 40 40 40 40 40 40 40 40  >@@@@@@@@@@@@@< 18: raw_offset=000000 raw_gain=00000 offset=+000000 gain=1.000000 crc=00  <not used>
0000f8 40 40 40 40 40 40 40 40                 >@@@@@@@@< 19: <padding>


[fenugrec]

Specifically, the -d switch forgets to remove the end-of-line line-feed character (0x0A) before processing the line.

Hi, I got the notification on github concerning this issue, but I can't access my code and crap so I have nothing to suggest until I get back in ~2 weeks. Ping me again if you don't hear from me by then.
But if it's just a processing issue it should be trivial to patch in the meantime (I think we got the math part mostly perfect ? except maybe clamping the gain digits to [-4..+5] , I forget the details)