Comparing source files

[AussieBruce]

I started to write my own solution but this is a universal problem. When comparing two text files, if there are line inserts or deletes, it derails the process. Can anyone suggest a - preferably free - compare tool that will deal effectively with this? Even one that will start off at specified line nos would be a big help. Thanks for any advice.

[Psi]

I'm not entirely sure I understand your issue, but have you tried 'winmerge' for comparing text files

[brucehoult]

I'm not entirely sure I understand your issue, but have you tried 'winmerge' for comparing text files

That sounds like something specific to a particular operating system that not everyone uses (and the OP may not).

The standard command line utility 'diff' has existed since AT&T Unix version 6 and appears with minor variations in BSD (e.g. MacOS) and GNU/Linux. Windows users can access it in WSL.

Often "diff3" is more useful, when someone has made a branch from some version of a program, and meanwhile more development has been done on the trunk. When it comes time to merge your changes back to trunk it is better do do a 3-way compare between the current head of trunk, your branch, and the common ancestor.

I think diff3 originates with AT&T version 7, and it's available everywhere also.

git always does 3-way diffs (if one of the versions is not a direct ancestor of the other)

[RoGeorge]

Search for 'diff tool'.
I'm using Meld.

[DiTBho]

Beyond Compare
it's commercial software for Wintel, but it's light years better than any open source alternative.

It compares files, directories, even trees, has multi-user locks, so you avoid making a mess on files that are opened by others, and your changes remain local, and can be "merged" later, under control (Git, Mercury, Svn, ... Doors-itself) anyway  ... and ' scriptable, it can be used to generate patches, and has plug-ins for integration into Understood{C, C++, Ada}, Doors and Stood AADL, AdaMulti, which are "versioned" professional work environments designed for team-working.

When I was a student, I got my Educational version of Beyond Compare for 50 euro. Still usable for personal stuff.

[tszaboo]

git will do that for you, just version control the file.

[DiTBho]

Git allows you to do versioning, but needs to use external tools for the interface as soon as you need advance features like "merge", and it certainly does integrate with neither Undertstood nor with Stood AADL, as similar open source tools do not exist.

Think that I even wrote a text console tool that does the same job as Meld (which has too many python dependencies for my taste), but inside an ncurses text-editor.

I've written both, and I like to run them on a serial console hardware VT220. Somehow they can also be used for real hobby projects. But I would never dream of using them for work things: because 'productivity' drops by at least 100 points.

For example, when you are in a team, and you need to analyze a source, without leaving the work environment, it is damn productive to open Understood and Beyond Compare at the same time (oh, even beetter without closing Doors!!!!), to see in real time what impact the changes have on the metrics.

It savev 5 minutes per module. Multiplied by 100-200 modules, a nice day makes the difference between being able to leave work at 4pm and being able to take the racing bike for a workout, or having to stay in the office or laboratory until 6pm when they then kick you out because they have to close.

Yup 

[srb1954]

Beyond Compare
it's commercial software for Wintel, but it's light years better than any open source alternative.

Also available for the Mac.

Highly recommended. It is one of the best value commercial S/W packages that I have ever bought.

[teb]

If you're using git, you can set your merge diff tool: https://git-scm.com/docs/merge-config

Otherwise for most systems I really like kdiff3: https://kdiff3.sourceforge.net/. Especially for multi-way comparisons and it handles whitespace better than git and unix's diff implementations in my experience.

[DavidAlfa]

Notepad++ (Win only) also has pretty decent compare plugins.

[Nominal Animal]

Note that standard diff supports multiple formats for describing the differences, including --side-by-side.  I prefer the unified format (-u); you can adjust how many lines of context (common lines preceding or succeeding a change) is displayed by using -U N.  I also like to use -ab, i.e. 'assume input is text' and 'ignore changes in the amount of white space'.  It is not just an "one size fits all" solution at all, it has features and output format you can adjust to fit your needs.
In a terminal, diff -Nabur --color=always file1 file2 | less -r produces paged, colored output.

It turns out that this June, diff will be 50 years old!

[eutectique]

+1 for meld

To have a scrollable file with colours, say diff file1.c file2.c | gvim -R -

[RAPo]

I use meld and ultra compare (https://www.ultraedit.com/products/ultracompare/8

[Nominal Animal]

Forgot to mention,
    diff -Nabur --color=always dir1/ dir2/ | less -r
compares all files in the two directories, producing unified diff output with (default) three lines of context.

The nice thing about such diffs is that you can apply the changes obtained via
    diff -Nabur dir1/ dir2/ > name.patch
by running
    (cd dir3/ && patch -p1 -i ../name.patch)
and revert the same changes by running
    (cd dir3/ && patch -p1 -R -i ../name.patch)

[SiliconWizard]

I mostly use Meld these days for interactive compare/merge and diff for automated compare.
Note that these are based on pretty much the same algorithms and still don't handle *interleaved* changes over multiple lines all that well, they tend to group lines to minimize the number of different blocks and often require manual editing for merging when you have interleaved changes.

[brucehoult]

Note that standard diff supports multiple formats for describing the differences, including --side-by-side.  I prefer the unified format (-u); you can adjust how many lines of context (common lines preceding or succeeding a change) is displayed by using -U N.  I also like to use -ab, i.e. 'assume input is text' and 'ignore changes in the amount of white space'.  It is not just an "one size fits all" solution at all, it has features and output format you can adjust to fit your needs.
In a terminal, diff -Nabur --color=always file1 file2 | less -r produces paged, colored output.

It turns out that this June, diff will be 50 years old!

"Any Colour You Like"

(50 years old, this time last year. You missed the starting gun.)

[Nominal Animal]

"Any Colour You Like"

(50 years old, this time last year. You missed the starting gun.)

Did I?  I thought the initial release was in June 1974; not in 1973 like the Pink Floyd album.

[brucehoult]

"Any Colour You Like"

(50 years old, this time last year. You missed the starting gun.)

Did I?  I thought the initial release was in June 1974; not in 1973 like the Pink Floyd album.

I meant the music is 51. diff is only 50. Just punning around... Strangers passing in the street By chance two separate glances meet And I am you and what I see is me. (even older)

[Nominal Animal]

I'm a couple of years younger myself, born in '76.

As to colors, unified format diff uses one for file names, one for the change header, one for the common lines, and one for the deleted and one for added lines.  So it's not that useful, although you can set the colors using --palette= option.  Piping (without --color) to an editor that supports syntax highlighting for whatever language is being edited often yields more useful color highlighting, as suggested by eutectique above.

Using command-line tools in chains also makes it easy to look for something specific in the output, for example by piping the diff -Nabur output to grep -e pattern, to output only lines that contain or match POSIX basic regular expression pattern.
For anything repetitive, I create a (bash or dash) shell script, Makefile, or an awk filter or script.  Or write a throwaway tool program.

This way, I've got much wider palette of tools to combine to do what I want.  I don't want to learn a new graphical tool for every little task I have, so I learned how to combine simpler ones to do what I want with minimum cognitive effort, as suggested by the Unix philosophy and minimalist modular approach.

[AussieBruce]

Thanks, that solved my problem in one, the functionality of that program is way beyond my needs.

[DiTBho]

I mostly use Meld these days for interactive compare/merge and diff for automated compare.
Note that these are based on pretty much the same algorithms and still don't handle *interleaved* changes over multiple lines all that well, they tend to group lines to minimize the number of different blocks and often require manual editing for merging when you have interleaved changes.

that's precisely +1 for Beyound Compare, as it does it, and it saves your time!

[radiolistener]

I started to write my own solution but this is a universal problem. When comparing two text files, if there are line inserts or deletes, it derails the process. Can anyone suggest a - preferably free - compare tool that will deal effectively with this? Even one that will start off at specified line nos would be a big help. Thanks for any advice.

I'm using Meld. It allows to compare/merge/synchronize files and folders and also allows to see github status for files.
It very similar to old Araxis Merge, but Araxis is paid and Meld is free.

[DiTBho]

my-c now has its own dedicated "diff" tool ("/usr/bin/myc-diff") for comparing two sources.

Instead of processing two text files as an "ascii" stream, the tool calls the my-c lexifier and creates two ASTs, one for each source, then compares the ASTs.

This way
- as a side effect, when you diff two my-c files, you also "format" them
- the comparison is totally immune to tabulations, spaces, interleaving...

Not exactly perfect, as it misses a graphic interface, syntax color highlighting and all the extra features you can find in { meld, beyond compare, ..}; it only works with curses and vt220 (directly driven), but I'm proud of it 

[SiliconWizard]

That's an interesting approach, although obviously quite different. As I understand it, you could have two "completely" different source files, if the corresponding ASTs are the same, it would show no difference. It would then miss any code style difference, comments, etc. That's interesting, but I personally think code style and comments are important, so missing them may not be all that good. But that's just me.

[RoGeorge]

Worked once for a company that was using Perforce (git was pretty new back then).  IIRC Perforce was a proprietary and payed only CVS (Control Versioning System), however, their merge tool was free, called P4Merge.  Diffing source files with P4Merge was looking very eye candy, easy and intuitive to work with.

https://www.perforce.com/products/helix-core-apps/merge-diff-tool-p4merge

[DiTBho]

That's an interesting approach, although obviously quite different. As I understand it, you could have two "completely" different source files, if the corresponding ASTs are the same, it would show no difference.

it only works with "my-c" sources, refuses "c" sources, and the first thing it chckes is the "level", so it can only compares

Code: [Select]

{
   { source0:level0, source1:level0 } <--------- this is 90% compatible with C/89, but not 100%
   { source0:level1, source1:level1 }
   { source0:level2, source1:level2 }
   { source0:level3, source1:level3 }
}


It would then miss any code style difference, comments, etc.

Yup, this is a defect. There must be Only one style.

To reduce the things I have to develop, I forced only one style, which is imposed to the user, and whatever the user writes (spaces, tabs, personal indentation), the tool always re-formats the source with the imposed style.

Comments are special-cases. A comment is recognized as "token_comment", managed as block, and its context is handled by the traditional diff algorithm, including CR.

Tabs are not allowed, and automatically converted into 4 spaces, as well as CR LF are automatically converted into CR LF (DOS --> Unix)
So, comments keep CRs LFs, you can see differences between the same comment, e.g. { typos fixed, more details added &| removed, ... } in two different sources.

edit:
fixed

[cfbsoftware]

CR LF are automatically converted into CR (DOS --> Unix)

Shouldn't that be:

CR LF are automatically converted into LF (DOS --> Unix)

[Nominal Animal]

Yup.  The four newline conventions, listed by the most well-known OSes that used them:

• LF ("\n"): Unix, BSDs, Linux, Android, OS X.
   
• CR ("\r"): Old Mac OS prior to X.
   
• CR LF ("\r\n"): DOS, Windows.
   
• LF CR ("\n\r"): An error, but invisible on most systems, behaving like a normal newline, and is thus occasionally encountered in the wild.  Can also occur in Unix/BSD/Linux/etc. systems when using tty devices (terminals, serial ports) with termios INLCR and ICRNL set on input with the other side using CR LF newlines.

My own conception for "universal newlines" is supporting any mix of the four, optionally with NUL ("\0") also considered a newline.

[Picuino]

+1 for Beyond Compare.
There are versions for Win, Mac and Linux, and with trial versions.
https://www.scootersoftware.com/download

[SiliconWizard]

Yup.  The four newline conventions, listed by the most well-known OSes that used them:

• LF ("\n"): Unix, BSDs, Linux, Android, OS X.
   
• CR ("\r"): Old Mac OS prior to X.
   
• CR LF ("\r\n"): DOS, Windows.
   
• LF CR ("\n\r"): An error, but invisible on most systems, behaving like a normal newline, and is thus occasionally encountered in the wild.  Can also occur in Unix/BSD/Linux/etc. systems when using tty devices (terminals, serial ports) with termios INLCR and ICRNL set on input with the other side using CR LF newlines.

My own conception for "universal newlines" is supporting any mix of the four, optionally with NUL ("\0") also considered a newline.

Yes.

[DiTBho]

Fixed 

LF CR ("\n\r"): An error, but invisible on most systems, behaving like a normal newline, and is thus occasionally encountered in the wild.  Can also occur in Unix/BSD/Linux/etc. systems when using tty devices (terminals, serial ports) with termios INLCR and ICRNL set on input with the other side using CR LF newlines.

umm, speaking of being invisible, I never noticed it on the DEC VT525(1994) hardware terminal with /usr/bin/screen on /dev/ttyS0
it shouldn't have worked 

[Nominal Animal]

LFCR "\n\r" works on Unix, BSD/Linux/etc., and VT terminals because while LF "\n" is a full newline, CR "\r" only returns the cursor to the beginning of the current line.  Thus, the effect of LF CR "\n\r" is exactly the same as LF "\n" only.  Invisible, like I said!

This is often exploited by POSIXy command-line programs that report progress using e.g.
    fprintf(stderr, "\r%.2f%% done", 100.0*i/n);
    fflush(stderr);  // Optional; normally/by default, this is not needed
followed by
    fprintf(stderr, "\r                   \r"); // or "\r\022[K"
    fflush(stderr);  // Optional; normally/by default, this is not needed
where "\033[K" (or "\e[K" where \e represents ASCII/ISO Latin/UTF-8 code 27 = 0x1B = ESC) is the ANSI Control Sequence for clearing the current line from the current cursor position to the end of the line without moving the cursor, when the maximum number of characters needed to "erase" the progress output is unknown.

If the progress is reported to standard output, or there is more information printed to standard error that may have been redirected to a file, often the followup is just a simple fprintf(stderr, "\n"); or fprintf(stderr, "\rCompleted.     \n");, as that way the progress information will look like a single line in the file, but won't otherwise mess up the output.

[dobsonr741]

Araxis Merge. They are in this business was for a long time, and for a reason.

[bobcat2000]

I, too, use Notepad++ with the compare plugin if you just want to look for differences.  It will do its best to line up the codes and color codes the differences.  Open the 1st file, open the 2nd file, and then click the compare button.  Nothing simpler than that.

Unless you want to merge the 2 files. I am using DiffMerge for that.

[westfw]

CR LF ("\r\n"): DOS, Windows.

Also most old DEC OSes, and "NVT ASCII", supposedly used by many internet protocols (telnet, specifically.  But also SMTP, etc.)
This means you also sometimes see CR NUL, which is supposedly a "bare CR" in NVT ASCII.