Electronics > Repair
HP 3478A: How to read/write cal SRAM
<< < (22/41) > >>
lmester:

--- Quote from: Miti on January 24, 2020, 12:49:38 am ---
Sure I'm interested. Even if you do this experiment, I still intend to do it in the weekend.

--- End quote ---

Download the file HP3478AGainCheck.exe from https://mesterhome.com/gpibsw/hp3478a/Software/

Replace your existing Hp3478a.exe

Go to edit configuration. Select the desired range. Enter 0 for the offset. Enter exactly 5 hex digits for the raw gain. Click update cal. Exit edit calibration data and click start measurement from the main window. The meter reading should change to reflect the new gain and offset settings.

Warning! Range and error checking are disabled. Make sure to enter exactly 5 hex digits for the gain.

Below is the data from three tests. The second test was the most stable. The starting and ending meter reading was the same.  I'm measuring  resistance. It'll be interesting to see if you get better stability when measuring voltage.


--- Code: ---initial reading .68309
3K range default calibration: offset -1 gain 1.0036  (04C00)

Start: and End: are the calibrated resistance readings from the beginning and end of each test


Start: .68309
RawG  Meter reading
00000 68064
00700 68112
00800 68009
00900 68016
00a00 68022
00f00 68056
90700 63346
77777 73356
88888 62013
99999 62769
End: .68307


Start: .68307
00000 68062
00700 68110
00800 68008
00900 68014
00a00 68021
00f00 68055
90700 63345
77777 73356
88888 62012
99999 62769
End: . 68307


Start: .68306
00000 68061
00700 68109
00800 68006
00900 68013
00a00 68020
00f00 68053
90700 63343
77777 73353
88888 62010
99999 62766
End: . 68304
--- End code ---





fenugrec:

--- Quote from: lmester on January 24, 2020, 01:26:22 am ---Below is the data from three tests. The second test was the most stable.

--- End quote ---

Very nice, thanks for those rigorous tests. Plugging the numbers :


--- Code: ---gain const reading 'x' decode(x) Un-apply gain
(= reading / decode(x))
TEST SERIES 1
00000 68064 1.000000 68064.0
00700 68112 1.000700 68064.4
00800 68009 0.999200 68063.5
00900 68016 0.999300 68063.6
00a00 68022 0.999400 68062.8
00f00 68056 0.999900 68062.8
90700 63346 0.930700 68062.7
77777 73356 1.077777 68062.3
88888 62013 0.911112 68063.0
99999 62769 0.922223 68062.7

TEST SERIES 2
00000 68062 1.000000 68062.0
00700 68110 1.000700 68062.4
00800 68008 0.999200 68062.4
00900 68014 0.999300 68061.6
00a00 68021 0.999400 68061.8
00f00 68055 0.999900 68061.8
90700 63345 0.930700 68061.7
77777 73356 1.077777 68062.3
88888 62012 0.911112 68061.9
99999 62769 0.922223 68062.7

--- End code ---

As you can see, the results fit nearly perfectly - as good as I could expect, down to the last digit, which definitely proves the quality of your experimental methodology.

The code I used in that spreadsheet is unfortunately an awful BASIC macro,


--- Code: --- gain = 1

for cur=1 to 5
temp = clng("&h" & mid(gainstring, cur, 1))    ### this converts a single digit from the "raw gain constant" text string to a numerical value.
if (temp >= 8) then
temp = temp - 16
endif
rem digit 1 (cur = 1) is 1/100)
gain = gain + (temp / (10^(cur + 1)))
next cur

--- End code ---

but that code, to the best of my knowledge, is equivalent to my original C code and Steve's, assuming he checks for "digit >= 8" too.

Now, for the reverse function, encode(gain), I'd have to think about this before I could say anything smart.

Again, thanks for that data. Always nice to validate theory.
Miti:
Here are my readings. First I tried on the 3V range, it goes out of range with raw gain 77777. Then I tried on 30V range.
Some misalignment in the data but you get the idea.



--- Code: ---Initial reading 3.00048V 3V DC Range 30V DC Range Gain reported Correct gain
Initial offset -2
Initial gain 1.027762

00000 2.91943 2.9198 1.00000 1.00000 2.91943 2.9198
00700 2.92148 2.9219 1.0007 1.0007 2.919436395 2.919856101
00800 2.9171 2.9175 1.0008 0.9992 2.919435548 2.919835869
00900 2.91739 2.9178 0.9993 0.9993 2.919433604 2.919843891
00a00 2.91768 2.9181 0.9994 0.9994 2.919431659 2.919851911
00f00 2.91914 2.9195 0.9999 0.9999 2.919431943 2.919791979
90700 2.71712 2.7175 0.9307 0.9307 2.919436983 2.919845278
77777 Over 3.1469 1.077777 1.077777 0.00000 2.919806231
88888 2.65993 2.6603 1.088888 0.911112 2.919432518 2.919838615
99999 2.69237 2.6927 0.922223 0.922223 2.919434887 2.919792718


--- End code ---
lmester:
Miti,

The gain unpack is also working with your data. Your calculated gain values are very close to the 2.91943 (gain = 1) value.


--- Code: ---
GainC is the compressed gain value used for the test
MeterDisp is the meter reading for the current GainC
GainNum is the uncompressed gain value
Calc-Gain is MeterDisp/GainNum. Should be 2.91943

GainC MeterDisp GainNum Calc-Gain
00000 2.91943 1.000000 2.91943
00700 2.92148 1.000700 2.919436
00800 2.9171 0.999200 2.919435
00900 2.91739 0.999300 2.919433
00a00 2.91768 0.999400 2.919431
00f00 2.91914 0.999900 2.919431
90700 2.71712 0.930700 2.919436
77777 OverRng 1.077777 ?
88888 2.65993 0.911112 2.919432
99999 2.69237 0.922223 2.919434


--- End code ---


Thanks to steve1515 and fenugrec I'm getting a better understanding of what's going on here.

That's the problem with using code that I don't fully understand. Debugging can be really hard!

My gain unpack code is currently using "digit>8" not "digit>=8". I assume that I should switch this to "digit>=8"? With my meter cal data there are no compressed cal digits equal to 8. The change makes no difference using my data.

Also, I'd still like to know what range check to use for the gain. What limits should I set for user entered gain values?



fenugrec:

--- Quote from: lmester on January 25, 2020, 05:20:38 am ---My gain unpack code is currently using "digit>8" not "digit>=8". I assume that I should switch this to "digit>=8"?

--- End quote ---
yes, otherwise the raw constant "88888" wouldn't decode/unpack properly. As you saw, it corresponds to a gain of 0.911112 which is validated by many tests now.

Maybe it would help to think about how the gain is applied by the stock firmware: it doesn't convert the constant to a fixed-point decimal number like what we're doing. That would be incredibly wasteful on that mcs-48 architecture with no hardware multiply. Here's roughly what it does, starting with a raw reading 'R' , with for example a gain constant K=90700. I'll call each digit Kn such that K1 = 9, K3 = 7, ...


--- Code: ---C = R;   //C will be the "almost calibrated" reading, just before the offset is applied.

K1 = 9;
// At the "1" digit, we're working at the 1/100 decimal position, so R is "decimal-shifted" 2 digits right.
// Since K1 >= 8, we need to subtract (16-9)=7 times the shifted raw reading.
C = C - ((16-9) * R/100);

K2 = 0;
//do nothing

K3 = 7:
// digit 3 : means "add 7 times", but now R is shifted 4 positions
C = C + (7 * R/10000);

//and the rest of the digits are 0, so do nothing.

//The equivalent operation is C = (1 * R) - (7 * 0.01 * R) + (7 * 0.00001 * R)
//can also be written as C = R * (1 - 0.07 + 0.00007) or
//of course, C = R * 0.9307

--- End code ---


The reverse process is sortof like a long division I guess ? I think it goes something like this. Assume all offsets are 0 for simplicity. The meter has a raw ADC reading 'R', and it knows the expected calibrated value 'C'.


* calculate error 'e = c - r' . Can be negative of course
* shift raw value right, i.e. divide by 10. First iteration needs to start at a 1/100 division. let's call it 'sr = r * (1/100)'
* how many times does 'sr' fit in 'e' ?
* if it fits 0 to 7 times, then that encodes directly to digit 1.
* if it fits -1 to -8 times, then "encode" that digit as (16 - <value>), so 0x0F for -1 to 0x08 for -8.
* if it fits 8 or 9 times, problem : those digits indicated negative values, so we need to cheat:
 carry 10 to the digit to the left (by incrementing by 1 - careful if the digit was already 7, you'll need to do the same manoeuvre recursively), and set the current digit to (value - 10) i.e. -2 or -1.
 In other words, use the fact that 8 * sr = (10 * sr) + (-2 * sr)
* if it fits -9 times, same idea : borrow 10 from digit on the left, adjust, and continue.
* calculate new 'e' value for next loop, i.e. remainder. Right shift sr again and continue
I think that should work, conceptually at least. Interesting consequence of the +7/-8 limit on digits : some gain values can be encoded in more than one way, for example 0001C and 00006 both correspond to a gain of 1.000006 .


--- Quote ---Also, I'd still like to know what range check to use for the gain. What limits should I set for user entered gain values?

--- End quote ---

That would be the two extremes we've tested,


--- Code: ---77777 1.077777
88888 0.911112
--- End code ---
Navigation
Message Index
Next page
Previous page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod