Making backups using rsync

[ve7xen]

this script mimics the mac time machine, which does incremential backups
https://github.com/samdoran/rsync-time-machine

Time Machine has a big advantage, because Apple modified their file system to support hard linking directories.

With other file systems you either have to recreate the entire directory tree in each incremental backup and hard link only the unchanged files, or else symlink the unchanged directories, which complicates selective pruning of backups later.

Or store the directory tree in some other way such as using a normal file to store each directory as git does. Or a database.

Most modern filesystems support mounted snapshots. Use btrfs or ZFS and you can accomplish basically identical behaviour. The openSUSE 'snapper' tool does a good job of automatically managing the snapshots. There are some flashy GUI tools too, but I don't find any actual value in those.

[brucehoult]

this script mimics the mac time machine, which does incremential backups
https://github.com/samdoran/rsync-time-machine

Time Machine has a big advantage, because Apple modified their file system to support hard linking directories.

With other file systems you either have to recreate the entire directory tree in each incremental backup and hard link only the unchanged files, or else symlink the unchanged directories, which complicates selective pruning of backups later.

Or store the directory tree in some other way such as using a normal file to store each directory as git does. Or a database.

Most modern filesystems support mounted snapshots. Use btrfs or ZFS and you can accomplish basically identical behaviour. The openSUSE 'snapper' tool does a good job of automatically managing the snapshots. There are some flashy GUI tools too, but I don't find any actual value in those.

FUSE lets you write a driver to make virtually anything look like a mounted drive. So, yes, you can make a snapshot stored with dedup in a database look like it's in a filesystem.

That's not the same thing as being able to hard link directories in your normal filesystem.

[ve7xen]

this script mimics the mac time machine, which does incremential backups
https://github.com/samdoran/rsync-time-machine

Time Machine has a big advantage, because Apple modified their file system to support hard linking directories.

With other file systems you either have to recreate the entire directory tree in each incremental backup and hard link only the unchanged files, or else symlink the unchanged directories, which complicates selective pruning of backups later.

Or store the directory tree in some other way such as using a normal file to store each directory as git does. Or a database.

Most modern filesystems support mounted snapshots. Use btrfs or ZFS and you can accomplish basically identical behaviour. The openSUSE 'snapper' tool does a good job of automatically managing the snapshots. There are some flashy GUI tools too, but I don't find any actual value in those.

FUSE lets you write a driver to make virtually anything look like a mounted drive. So, yes, you can make a snapshot stored with dedup in a database look like it's in a filesystem.

That's not the same thing as being able to hard link directories in your normal filesystem.

I have no idea what you're on about. Nothing in this post is relevant. I didn't mention FUSE or databases, and snapshots have nothing to do with dedup.

You're right that snapshots aren't the same thing as hard links, but for periodically snapshotting (heh) the system, they are better. The semantics with respect to what Time Machine does are the same, but the implementation depends on a filesystem primitive instead of a userland process creating and maintaining a bunch of hardlinks. All you need the script for is to schedule the snapshots and optionally mount them so you can have a /backups/2020-03-03T01:00:00 kind of hierarchy.