Incremental Backup Solution for Small Business and Professionals

[kaevee]

Hi,

Recently one of my client's computers was hijacked by ransomware. I had to revamp their network security and file services. They wanted an easy-to-access backup system for their data as they don't have a dedicated administrator. I wrote a program to take incremental backups from Windows/Mac/Linux systems to NAS.

Key Features

• User defined backup intervals
• User defined backup rotations per interval
• Uses hard links to save space across backup intervals
• Cross Platform (Windows, Linux, MacOs, FreeBSD). Requires CygWin for windows

The backups can be reviewed/access without any special tools.

Check out https://github.com/kaevee/NgBackup for more information.

[Karel]

Why didn't you use something like rdiff-backup?

https://rdiff-backup.net/

[kaevee]

Why didn't you use something like rdiff-backup?

https://rdiff-backup.net/

As I wrote in my initial post, the client does not have an in-house administrator. So, we needed an incremental backup system accessible to users as a shared drive. A typical tree will be

daily
--- 20092021
--- 21092021
--- 22092021
--- 23092021
--- 24092021
--- 25092021

Similarly, there will be an increments tree under hourly, weekly, monthly, and yearly folders.

if one wants to get a file as of 22092021, all they need to do is copy the file from the 22092021 folder.

[Benta]

I'm running Linux, so I can't say anything about Mac or Win.

I split my backups in two:

One for system backups. This is only run just before or after system changes or updates. For this I use Timeshift. This is for the sysadm.

Another for user data (/home/user) backups. Here I use BackInTime and can let it run automatically on preset intervals.

Both are rsync-based incremental backups with hard links and are extremely stable. The main differences are the GUI front-ends and their setup. I have not been in a situation where recovery wasn't possible.

Splitting the two backups makes sense to me, but perhaps not to everyone.

[kaevee]

I'm running Linux, so I can't say anything about Mac or Win.

I split my backups in two:

One for system backups. This is only run just before or after system changes or updates. For this I use Timeshift. This is for the sysadm.

Another for user data (/home/user) backups. Here I use BackInTime and can let it run automatically on preset intervals.

Both are Rsync-based incremental backups with hard links and are extremely stable. The main differences are the GUI front-ends and their setup. I have not been in a situation where recovery wasn't possible.

Splitting the two backups makes sense to me, but perhaps not to everyone.

I did explore many open-source tools and most of them are designed for mainstream users on Linux servers. TimeShift, BackInTime look very good and I am sure they will meet the needs of almost all users.

I am using TrueNAS Core based on FreeBSD. ZFS in NAS has a native snapshot feature. The hourly, daily, weekly, monthly, yearly snapshots have been set up in NAS. I need to compliment the snapshots with an easy-to-use restore on their own feature where only a file or small subset of data is needed by users. Furthermore, I need to push some files directly from Windows system(s) as some data is stored locally.

With ZFS one needs an administrator to restore data. My approach will allow users to recover a file or directory on their own. If there is a need to recover the full data, ZFS snapshots will be used.

[bson]

Why not use the best tool for each platform?  On Windows, Macrium Reflect is excellent (and free unless you want enterprise type features).  It can image a bunch of partitions to a single backup, or do incrementals, or both.  So you can say image the system once a week, maybe Sunday AM if people typically don't work weekends, and then a nightly incremental.  If their computer fails (business PCs aren't exactly the pinnacle of reliability) fixing it is as easy as replacing their entire system and loading up their last image using a bootable recovery stick.  It'll happily write to NAS.  Having a complete tamper-proof restorable image lets you recover from ransomware, randomware (Windows becoming unbootable for example), garbageware (yeah sure, these Win7 drivers work just fine on Win10 - famous last words), Windows just barfing all over itself like a drunken sailor, or smoke and fire.  Different systems, different needs...

[kaevee]

Why not use the best tool for each platform?  On Windows, Macrium Reflect is excellent (and free unless you want enterprise type features).  It can image a bunch of partitions to a single backup, or do incrementals, or both.  So you can say image the system once a week, maybe Sunday AM if people typically don't work weekends, and then a nightly incremental.  If their computer fails (business PCs aren't exactly the pinnacle of reliability) fixing it is as easy as replacing their entire system and loading up their last image using a bootable recovery stick.  It'll happily write to NAS.  Having a complete tamper-proof restorable image lets you recover from ransomware, randomware (Windows becoming unbootable for example), garbageware (yeah sure, these Win7 drivers work just fine on Win10 - famous last words), Windows just barfing all over itself like a drunken sailor, or smoke and fire.  Different systems, different needs...

I could have used platform-specific tools. As noted in my initial post, access to previous data using a share was one of the primary requirements. Furthermore, we were not looking to restore the complete system.

[Foxxz]

https://borgbackup.readthedocs.io/en/stable/

I'm using this with great results.

[Halcyon]

I use both ZFS snapshots and regular offline backups.

The snapshots allow you to quickly and easily restore data that has been changed or deleted. You can define them at any interval you like, even hourly if you really want. This is native to ZFS and TrueNAS.

For proper backups, I use LTO tape.

[kaevee]

https://borgbackup.readthedocs.io/en/stable/

I'm using this with great results.

This looks very good. Probably I will consider using it once they release windows binaries.

[kaevee]

I use both ZFS snapshots and regular offline backups.

The snapshots allow you to quickly and easily restore data that has been changed or deleted. You can define them at any interval you like, even hourly if you really want. This is native to ZFS and TrueNAS.

For proper backups, I use LTO tape.

We are using TrueNAS and periodic ZFS snapshots are enabled. The Python script is complimenting the ZFS snapshots in TrueNAS, by providing easy access to backups for users. As I noted in my initial post, we don't have a full-time administrator to restore data on demand.

[PaulAm]

I've used backuppc for both my self and a number of clients.  It's an rsync based platform that runs on Linux, but clients work fine with windows.  Great for incremental backups and file/directory retrieval, not so much for entire systems.  I do full dumps for that.

Had one client hit with ransomware and it was relatively easy to rebuild his system.

For a backup solution to be effective against ransomware it pretty much has to be an out of band solution.  If a backup share is visible to a system it's just as likely to be encrypted as anything else.

Whatever solution you use, be sure to do a test restore before you actually need to.  Don't assume everything will just work fine; 9 times out of 10 it won't.  I've seen everything from individual users to major data centers caught by that.

[Halcyon]

I use both ZFS snapshots and regular offline backups.

The snapshots allow you to quickly and easily restore data that has been changed or deleted. You can define them at any interval you like, even hourly if you really want. This is native to ZFS and TrueNAS.

For proper backups, I use LTO tape.

We are using TrueNAS and periodic ZFS snapshots are enabled. The Python script is complimenting the ZFS snapshots in TrueNAS, by providing easy access to backups for users. As I noted in my initial post, we don't have a full-time administrator to restore data on demand.

Snapshots in TrueNAS also support the Windows volume shadow copy service. That way if a user deletes/overwrites a folder or file by accident, they can just right-click and go to the "Previous versions" tab and restore it themselves from their own machines. No scripts needed.

[kaevee]

Snapshots in TrueNAS also support the Windows volume shadow copy service. That way if a user deletes/overwrites a folder or file by accident, they can just right-click and go to the "Previous versions" tab and restore it themselves from their own machines. No scripts needed.

Of course, we have enabled shadow copy. It meets almost all the needs except for a remote backup. Remote backups with shadow copy can be achieved if we have a second TrueNAS replicating the data from the primary server. We are yet to set up a failover system for the TrueNAS server.