EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: DiTBho on September 25, 2021, 01:15:05 pm

Title: [solved] why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 01:15:05 pm
bash script
Code: [Select]
function check_result()
{
   local ans="$?" # get the exit status from a terminated program
   local answ

   if [ "$ans" == "0" ] # 0=success, 1=failure
      then
          answ="success"
      else
          answ="failure (ans=$ans)"
      fi
   echo "$answ"
}

echo "AAAA"  >  A
cp A B
echo "BBBB"  >> B

diff -Naur A B
check_result

#       -N, --new-file: treat absent files as empty
diff -Naur A C    # C doesn't exist
check_result


Code: [Select]
--- A   2021-09-25 13:52:21.012000000 -0000
+++ B   2021-09-25 13:52:21.012000000 -0000
@@ -1 +1,2 @@
 AAAA
+BBBB
failure (ans=1)
failure (ans=1)
--- A   2021-09-25 13:52:21.012000000 -0000
+++ C   1970-01-01 00:00:00.000000000 -0000
@@ -1 +0,0 @@
-AAAA
failure (ans=1)
failure (ans=1)


Code: [Select]
NAME
       GNU diff - compare files line by line

SYNOPSIS
       diff [OPTION]... FILES

DESCRIPTION
       Compare FILES line by line.

       Mandatory arguments to long options are mandatory for short options too.

       --normal
              output a normal diff (the default)

       -q, --brief
              report only when files differ

       -s, --report-identical-files
              report when two files are the same

       -c, -C NUM, --context[=NUM]
              output NUM (default 3) lines of copied context

       -u, -U NUM, --unified[=NUM]
              output NUM (default 3) lines of unified context

       -e, --ed
              output an ed script

       -n, --rcs
              output an RCS format diff

       -y, --side-by-side
              output in two columns

       -W, --width=NUM
       --left-column
              output only the left column of common lines

       --suppress-common-lines
              do not output common lines

       -p, --show-c-function
              show which C function each change is in

       -F, --show-function-line=RE
              show the most recent line matching RE

       --label LABEL
              use LABEL instead of file name and timestamp (can be repeated)

       -t, --expand-tabs
              expand tabs to spaces in output

       -T, --initial-tab
              make tabs line up by prepending a tab

       --tabsize=NUM
              tab stops every NUM (default 8) print columns

       --suppress-blank-empty
              suppress space or tab before empty output lines

       -l, --paginate
              pass output through 'pr' to paginate it

       -r, --recursive
              recursively compare any subdirectories found

       --no-dereference
              don't follow symbolic links

       -N, --new-file
              treat absent files as empty

       --unidirectional-new-file
              treat absent first files as empty

       --ignore-file-name-case
              ignore case when comparing file names

       --no-ignore-file-name-case
              consider case when comparing file names

       -x, --exclude=PAT
              exclude files that match PAT

       -X, --exclude-from=FILE
              exclude files that match any pattern in FILE

       -S, --starting-file=FILE
              start with FILE when comparing directories

       --from-file=FILE1
              compare FILE1 to all operands; FILE1 can be a directory

       --to-file=FILE2
              compare all operands to FILE2; FILE2 can be a directory

       -i, --ignore-case
              ignore case differences in file contents

       -E, --ignore-tab-expansion
              ignore changes due to tab expansion

       -Z, --ignore-trailing-space
              ignore white space at line end

       -b, --ignore-space-change
              ignore changes in the amount of white space

       -w, --ignore-all-space
              ignore all white space

       -B, --ignore-blank-lines
              ignore changes where lines are all blank

       -I, --ignore-matching-lines=RE
              ignore changes where all lines match RE

       -a, --text
              treat all files as text

       --strip-trailing-cr
              strip trailing carriage return on input

       -D, --ifdef=NAME
              output merged file with '#ifdef NAME' diffs

       --GTYPE-group-format=GFMT
              format GTYPE input groups with GFMT

       --line-format=LFMT
              format all input lines with LFMT

       --LTYPE-line-format=LFMT
              format LTYPE input lines with LFMT

              These format options provide fine-grained control over the output

              of diff, generalizing -D/--ifdef.

       LTYPE is 'old', 'new', or 'unchanged'.
              GTYPE is LTYPE or 'changed'.

              GFMT (only) may contain:

       %<     lines from FILE1

       %>     lines from FILE2

       %=     lines common to FILE1 and FILE2

       %[-][WIDTH][.[PREC]]{doxX}LETTER
              printf-style spec for LETTER

              LETTERs are as follows for new group, lower case for old group:

       F      first line number

       L      last line number

       N      number of lines = L-F+1

       E      F-1

       M      L+1

       %(A=B?T:E)
              if A equals B then T else E

              LFMT (only) may contain:

       %L     contents of line

       %l     contents of line, excluding any trailing newline

       %[-][WIDTH][.[PREC]]{doxX}n
              printf-style spec for input line number

              Both GFMT and LFMT may contain:

       %%     %

       %c'C'  the single character C

       %c'\OOO'
              the character with octal code OOO

       C      the character C (other characters represent themselves)

       -d, --minimal
              try hard to find a smaller set of changes

       --horizon-lines=NUM
              keep NUM lines of the common prefix and suffix

       --speed-large-files
              assume large files and many scattered small changes

       --color[=WHEN]
              colorize the output; WHEN can be 'never', 'always', or 'auto' (the default)

       --palette=PALETTE
              the colors to use when --color is active; PALETTE is a colon-separated list of terminfo capabilities

       --help display this help and exit

       -v, --version
              output version information and exit

       FILES  are  'FILE1  FILE2'  or 'DIR1 DIR2' or 'DIR FILE' or 'FILE DIR'.  If --from-file or --to-file is given,
       there are no restrictions on FILE(s).  If a FILE is '-', read standard input.  Exit status is 0 if inputs  are
       the same, 1 if different, 2 if trouble.



I am working on a bash-script, and I am confused by the exit status of the tool "diff", because it always return "1", which should mean FAILURE and others programs return "0" to say "success".

So why does it always return "1" ?!? :-//
Title: Re: why does "diff" always return "1"?!?
Post by: magic on September 25, 2021, 01:19:58 pm
Maybe read TFM that you posted above? :P
Title: Re: why does "diff" always return "1"?!?
Post by: retiredfeline on September 25, 2021, 01:39:03 pm
I haven't looked at your commands but you are doing this the complicated way.

Code: [Select]
if diff [options] A B
then
   echo "No differences"
else
   echo "Differences"
fi
Title: Re: why does "diff" always return "1"?!?
Post by: thinkfat on September 25, 2021, 01:42:34 pm
diff behaves exactly as described in the man page your posted above.
Title: Re: why does "diff" always return "1"?!?
Post by: PKTKS on September 25, 2021, 01:51:19 pm
not on my shells  ???

0 = match
1 = mismatch
2 = error

Paul
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 03:28:36 pm
Maybe read TFM that you posted above? :P

where is it written that it always return "0" as exit status?

Code: [Select]
int main()
{
    ...
    /*
     * 0 success
     * 1 failure
     */
    return exit_status;
}
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 03:29:29 pm
diff behaves exactly as described in the man page your posted above.

the man page doesn't mention the return status
Title: Re: why does "diff" always return "1"?!?
Post by: Siwastaja on September 25, 2021, 03:30:35 pm
diff behaves exactly as described in the man page your posted above.

the man page doesn't mention the return status

You literally quoted it! Look at the very last sentence.
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 03:33:41 pm
Code: [Select]
echo "hAllo"
check_result

ls ~
check_result

ping 192.168.1.11 -c1
check_result

Code: [Select]
hAllo
success

Book-CompilerDesignInC.pdf
Xilinx-ISE-v14.7-1015-1-Linux.tar
success

PING 192.168.1.11 (192.168.1.11) 56(84) bytes of data.
64 bytes from 192.168.1.11: icmp_seq=1 ttl=64 time=0.720 ms
--- 192.168.1.11 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.720/0.720/0.720/0.000 ms
success

Title: Re: why does "diff" always return "1"?!?
Post by: Siwastaja on September 25, 2021, 03:37:12 pm
You seem to have some misunderstanding that there is some universal standard that denotes 0 = success, others = failure. This is not the case. Return values are per program, and defined in the manual. Some programs do not generate information which should be returned using return value, but only return error / success in which case it's easy to just decide 0 = success, others = failure codes. Some programs, like diff, return the result of the program (differs; no difference), and also have to return errors. They chose the codes as described on the manual you can't read. Some others have chosen negative codes for errors, 0 for success, and positive numbers for non-error status codes, which is quite nice. Whoever wrote diff didn't do that, but that's OK too.
Title: Re: why does "diff" always return "1"?!?
Post by: retiredfeline on September 25, 2021, 03:37:38 pm
The Posix man page diff (1p) says:

Code: [Select]
EXIT STATUS
       The following exit values shall be returned:

        0    No differences were found.

        1    Differences were found.

       >1    An error occurred.

The GNU diff man page refers you to info diff which says the same thing.
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 03:41:01 pm
Quote
Exit status is 0 if inputs  are the same, 1 if different, 2 if trouble.

damn, the man-page says it in the last line, I didn't read it.
This happen when you are really tired to fix bugs here and there.

anyway I always get "1" from diff even when inputs are the same, and never get "0" or "2", but always "1", which is wrong.

I will re-implement "diff" myself as built-in function for the application I am developing, I am tired to waste time with that ..
Title: Re: why does "diff" always return "1"?!?
Post by: retiredfeline on September 25, 2021, 03:46:18 pm
anyway I always get "1" from diff even when inputs are the same, and never get "0" or "2", but always "1", which is wrong.

There's something wrong with your script then. What do you get when you do:

Code: [Select]
diff /etc/issue /etc/issue
echo $?
Title: Re: why does "diff" always return "1"?!?
Post by: PKTKS on September 25, 2021, 03:50:57 pm
anyway I always get "1" from diff even when inputs are the same, and never get "0" or "2", but always "1", which is wrong.

There's something wrong with your script then. What do you get when you do:

Code: [Select]
diff /etc/issue /etc/issue
echo $?


Yes there is a very optimistic assumption that the function will magically return that code...
It will not happen that way...

Code: [Select]

<EXECUTE DIFF HERE>
err_cod=$?  # grab returned error code wo quotes asap

if [ ${err_cod} -eq 0  ]; then  # 0=success, 1=failure


that is the proper modern bash syntax

Paul
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 04:15:17 pm
There's something wrong with your script then. What do you get when you do:

Code: [Select]
diff /etc/issue /etc/issue
echo $?

always "1", and it makes no sense.
Title: Re: why does "diff" always return "1"?!?
Post by: PKTKS on September 25, 2021, 04:30:39 pm
There's something wrong with your script then. What do you get when you do:

Code: [Select]
diff /etc/issue /etc/issue
echo $?

always "1", and it makes no sense.

This is wrong.

Copy paste a shot w/your shell bash version and diff version and this code executed.

Then recompile your bash shell making dead sure your are NOT using bash malloc.
There are known issues and bugs using that option

Paul
Title: Re: why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 04:37:27 pm
Yes there is a very optimistic assumption that the function will magically return that code...
It will not happen that way...

well, I have
Code: [Select]
# wc -l lib_patch_core.sh
1001 lib_patch_core.sh
1000 lines of bash code that works perfectly. It suffices to prove that it is not "bad bash syntax", but rather it must be something specifically wrong with "diff".

So, I tested the same bash-script on a different Linux-box (mac-mini/intel), and this time diff returned { 0 1 } as described in the manual.

So I re-emerged "sys-apps/diffutils" on the first Linux box (mac-mini/ppc) I have been working for days and upgraded the diffutils package to v3.7-r2.
I tried the bash script with the new /usr/bin/diff and this time diff worked as expected :o :o :o


Computers  :-//
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 04:43:23 pm
by updating diffutils it also updated bash from v4.1 to v5.0
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: PKTKS on September 25, 2021, 05:25:09 pm
  • mac-mini/ppc bash v4.1
  • mac-mini/intel bash v5.0
by updating diffutils it also updated bash from v4.1 to v5.0

latest bash fix is 5.1.8 - ded sure WITHOUT bash_malloc
latest diff is 3.8

it should by all means give the proper result as below
otherwise you have a broken system wide lib set

Paul
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: thinkfat on September 25, 2021, 05:37:51 pm
Ppc always is full of headaches. In my time as head of a commercial embedded Linux distribution, 90% of the really odd bugs we found were on that architecture.
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: DiTBho on September 25, 2021, 08:24:35 pm
Ppc always is full of headaches

Right, speaking of the kernel, we have a lot of quirks for Apple controllers, what's another? User space quirks and headaches couldn't really miss the party :D
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: thinkfat on September 26, 2021, 06:17:44 pm
I remember some python tests failing, I think Unicode handling was somehow broken, also some floating point stuff. But that is only the tip of the iceberg :)
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: bson on October 30, 2021, 07:30:39 am
Use cmp(1) if you just want to see if two files are identical.
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: DiTBho on October 30, 2021, 10:51:23 pm
Use cmp(1) if you just want to see if two files are identical.

Thanks. Diff is used in a complex Bash script that needs to check if two files are equal or not, and in case not it needs to report differences. My script allocates the output of Diff in a variable to have it ready.

But /usr/bin/cmp is something I didn't have on my system. Now I have it. It's part of the diffutils-v3.7-r1  ;D
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: magic on October 31, 2021, 06:59:33 am
Note that cmp has similarly "problematic" exit codes as diff :P
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: DiTBho on October 31, 2021, 09:19:35 am
yup, but at least it's not buggy (well, ok, because - it, bash, and their friends - have been already updated) ;D
Title: Re: [solved] why does "diff" always return "1"?!?
Post by: bson on November 01, 2021, 03:52:54 am
Thanks. Diff is used in a complex Bash script that needs to check if two files are equal or not, and in case not it needs to report differences. My script allocates the output of Diff in a variable to have it ready.
cmp is faster and if it were me I'd use a pattern like:

Code: [Select]
if ! cmp $FILE1 $FILE2 ; then
  echo '***' $FILE1 differs from $FILE2 '***'
  diff $FILE1 $FILE2
  exit 1
fi
This way, in the normal case where they don't differ you can quickly and silently move on and do nothing.  Then use diff to produce the actual message to print if they do differ.