Products > Programming

Creating lookup tables from sets of samples

(1/2) > >>

JPortici:
Hi, i have a nonlinear block i have to replicate... Its behaviour is more or less a piecewise linear function, but because reasons i don't want to derive a function.. i want to replicate it 1:1
So i ramped the input and now i have a few thousand samples that should describe the output completely and i want them to become a lookup table.

Is there a way using maybe excel to do it automatically?

Problem 1: I didn't actually ramp the input, it's a potentiometer so i had channel 1 connected to the taper, channel 2 connected to the output. This means that the ramp is a little more of a piecewise ramp, but it's just a problem of sorting and averaging/removing duplicates, right?
Problem 2: for other reasons there are missing points (not going down to zero and up to 5V) and the generator should be able to produce a 0 -> 2^n - 1 ramp assuming 0 = 0V and 2^n -1 = 5V, filling the gap following the trend

I am doing it manually and i'm almost at the end, but it would be nice to know if a tool for creating LUT already exists

The process is: sort for channel 1 (the input ramp) ascending -> apply formula to convert to counts -> average/eliminate duplicates -> fill gaps -> bonus: C style array

capt bullshot:
I'd just write a simple C (or whatever language is your preference) to do that. Excel is a tool for bean counters, not for engineers.

ledtester:

--- Quote from: capt bullshot on July 12, 2019, 09:34:33 am ---Excel is a tool for bean counters, not for engineers.

--- End quote ---

One programmer has changed his mind about them:

https://news.ycombinator.com/item?id=20417967

JPortici:
Yeah, in the end it would have taken the same time to write and test a C program to do that.
However, while i'm not a bean counter i prefer Excel to matlab/octave unless i know already what to do: change the content on one cell -> automatically change on everything else, plus diagrams without having to re-run the last 10s of commands

Nominal Animal:
If you have Bash and Awk, you can use the following script, interpolate.bash:

--- Code: ---#!/bin/bash
DEFAULTFORMAT="%.06f"
FORMAT="$DEFAULTFORMAT"
MIN=""
MAX=""
RANGE=""
HELP=0
ARGS=()
while [ $# -ge 1 ]; do
    if [ "$1" = "-min" ]; then
        MIN="$2 $3"
        shift 3
    elif [ "$1" = "-max" ]; then
        MAX="$2 $3"
        shift 3
    elif [ "$1" = "-format" -o "$1" = "-fmt" ]; then
        FORMAT="$2"
        shift 2
    elif [ "$1" = "-out" -o "$1" = "-range" ]; then
        RANGE="$2 $3"
        shift 3
    else
        ARGS=("${ARGS[@]}" "$1")
        shift 1
    fi
done
set -- "${ARGS[@]}"
if [ $# -lt 1 -o "$1" = "-h" -o "$1" = "--help" ]; then
    exec >&2
    printf '\n'
    printf 'Usage: %s [ -h | --help ]\n' "$0"
    printf '       %s [ OPTIONS ] INPUTFILE(s) [ > OUTPUTFILE ]\n' "$0"
    printf 'Options:\n'
    printf '       -min MININDEX MINVALUE\n'
    printf '       -max MAXINDEX MAXVALUE\n'
    printf '       -out FROMINDEX TOINDEX\n'
    printf '       -format FORMAT\n'
    printf '\n'
    printf 'Both INPUTFILE and OUTPUTFILE will contain two columns per line.\n'
    printf 'First column contains the index (integer), and\n'
    printf 'the second column contains the value at that index.\n'
    printf 'If an index is specified multiple times, the value is\n'
    printf 'averaged.  Missing indexes are interpolated linearly.\n'
    printf '\n'
    printf 'The order of the rows in INPUTFILE does not matter.\n'
    printf 'The rows in OUTPUTFILE will be in ascending order of index.\n'
    printf '\n'
    printf 'MININDEX and MAXINDEX enforce lower and upper limits for the values.\n'
    printf 'FROMINDEX and TOINDEX select only a subrange to be output.\n'
    printf "FORMAT is the printf format for the values, default '%s'.\n" "$DEFAULTFORMAT"
    printf '\n'
    exit 1
fi
awk -v "format=$FORMAT" -v "mini=$MIN" -v "maxi=$MAX" -v "range=$RANGE" 'BEGIN { imin=1e300; imax=-1e300; }
    NF>=2 && $1 !~ /#/ {
        i=int($1); v[i]+=$2 ; n[i]++;
        if (imin>i) imin=i;
        if (imax<i) imax=i
    }
    END {
        for (i=imin; i<=imax; i++) if (n[i] > 1) { v[i] = v[i] / n[i]; n[i] = 1 }
        if (split(mini, temp)==2) { imin=int(temp[1]); v[imin]=temp[2]; n[imin]=1 }
        if (split(maxi, temp)==2) { imax=int(temp[1]); v[imax]=temp[2]; n[imax]=1 }
        o = imin
        for (i=imin+1; i<=imax; i++) {
            if (n[i] > 0) {
                for (k=o+1; k<i; k++) v[k] = (v[o]*(i-k) + v[i]*(k-o)) / (i-o)
                o = i
            }
        }
        omin=imin ; omax=imax
        if (split(range, temp)==2) { omin=int(temp[1]); omax=int(temp[2]); }
        if (omin<imin) omin=imin
        if (omax>imax) omax=imax
        for (i=omin; i<=omax; i++) printf "%d " format "\n", i, v[i]
    }' "$INPUTFILE"

--- End code ---
It takes an input with two values per line (use - for standard input), separated by whitespace: first the index, and then the value at that index.  The order does not matter. Output has the same format, except in ascending indexes.  Run without parameters to see help/usage.

If you want to ensure the value is 255 at 0, and zero at 256, and print only 0..255 as a comma-separated list of integers suitable for a C array definition, use
    ./interpolate.bash INPUTFILE -fmt '%.0f' -min 0 255 -max 256 0 -out 0 255 | awk '{ printf "%s, ", $2 } END { printf "\n" }'

I've got tons of these snippets, including one that does a least squares fitting to an arbitrary function you specify on the command line, and displays the original data and the fit as curves using gnuplot.  It is typical for most of the code in my scripts to be about printing the usage, but I've learned to not omit it because I soon forget, and later, I'll want to know what the script does.  It is easier to have the script tell me when I run it, than read the code.

Navigation

[0] Message Index

[#] Next page