Seeing as there was some interest in LTO on the forum i will start a thread with my journey using it. All costs include 25% VAT and transport as i live in Sweden. Almost everything bought on ebay.
I recently bought a refurbished tape library (autoloader) Dell PowerVault TL2000 on ebay for $194 and a used LTO-5 SAS tape drive for $199.
First problem: the drive is a HP drive and due to vendor lock in HP drives only work on HP libraries. My options were now to buy a HP library or a IBM drive (resold by Dell and Lenovo).
For some reason IBM drives cost 3 times as much as HP drives with equivalent specifications. So i bought a refurbished HP Storever MSL2024 library for $487 (starting to go downhill here :D) which was cheaper than a refurbished IBM LTO-5 drive at $800. The refurbished libraries came with 6 months warranty.
Both of these libraries are almost identical (dell one has a 4 line graphical display, hp has text 2 lines) with 24 slots for tapes split into 2st 12 bay sides that can be easily removed for bulk changing of tapes. There are also 4U models with 48 slots for just a bit more money.
Both my HP and Dell libraries have good web UIs. I suggest separating them with firewalls and/or VLAN as they probably have security holes. Or just don't connect the network cable for normal use.
If you want to spend way more money LTO-6 drives can be had for around $580 for a used HP and $1470 for IBM. They only store 1TB more than than LTO-5, 1.5TB vs 2.5TB, and are not worth it in my opinion.
I you somehow get your hands on a LTO-7 drive for less than a kidney they can store 6TB and have twice the speed (300MB/s) as LTO-5/6 (140/160MB/s).
For tapes i bought a new cleaning tape for $22 (50 uses), 10st new LTO-5 tapes from china for $225 and 100 used LTO-5 tapes for $490.
Each LTO-5 tape is guaranteed to work for at least 200 full writes.
My use case is monthly rotating backups on the new tapes, and archival backups on the used tapes (i.e. write a few tapes once a year and never reuse).
Archival backups are safe from crypto viruses and accidental deletes, rotating backups are mostly safe from crypto viruses and accidental deletes.
For tape barcodes the new tapes came with a sheet (100x96mm) of 4st empty stickers each. Used tapes already had stickers. Using the program glabels on linux i was able to print onto these with my laser printer (just about minimum paper size it works with, must go short side in first or it gets stuck). Use barcode style 39 in glabels, barcode size is 65.36x11.1mm, sticker size is 78x16mm, center barcode horizonally at the bottom and put whatever text you want above. Printing settings were paper type labels and monochrome i.e. black only as the text did not stick well if it used color. Glabels supports generating from csv files which makes it easy to print many.
I also bought an old 3G/s SAS 2xSFF-8088 PCIe x4 card for $46 and a 1m SAS cable for $30. 3G/s should be good up to LTO-6. LTO-7 is much faster and needs a 6G card to work at full speed.
The LTO drives have incredibly noisy tiny fans, i changed mine to a larger quieter fan using a 3D printed adapter, see
https://www.eevblog.com/forum/repair/replacing-the-cooling-fan-in-an-lto-2-tape-drive/msg4021672/#msg4021672 . Fan noise is barely audible now. Drive makes some noise when running but quite pleasant tones so i don't mind :D
Firmware for these things is a bit difficult to get.
IBM requires a service account but dell and lenovo use the exact same drive firmware and have it for free. My refurbished libraries were already up to date.
HP has a few years old firmware for free but i found the latest one on a shared google drive.
PM me if anyone wants copies, i got libraries and drives LTO-5 to LTO-8.
Programs i used on arch linux: mtx-git mt-st-git mbuffer amanda hpltt (itdt manually downloaded for IBM drives) tapestat (from package sysstat) maminfo-git
mt controls the drive and tape positioning, only useful if you want to do things manually. Point environment variable TAPE to your tape drive to ease mt usage.
mtx controls the library and allows listing tapes and loading/unloading into the tape drive.
mbuffer provides large RAM buffering so the drive does not stall if disk IO is intermittently slow.
hpltt or itdt is used for drive diagnostics and firmware updating (firmware can also be updated from the libraries web UI).
tapestat -m -1 shows tape read and write speeds.
maminfo shows tape cartridge stats like times loaded and total data written/read. Good if you buy used tapes to know how much life they have left.
Basic usage:
https://www.cyberciti.biz/hardware/unix-linux-basic-tape-management-commandsI access my drive via /dev/tape/by-id/scsi-HUE2530E22-nst and the library via /dev/tape/by-id/scsi-DEC8320699 .
You can also use /dev/nst0 for drive and /dev/sgNumber for the library (if you see /dev/chNumber instead of sg then blacklist module ch).
For manual backups and restore i use:
backup: tar cf - /aDirectory | zstd -qc -3 -T0 | openssl enc -pbkdf2 -e -aes-256-ctr -salt -pass fd:3 3< passwordFile | mbuffer -f --tapeaware -m 2G -L -P 80 -o $TAPE
restore: mbuffer -i $TAPE -s 1024k -b 2048 -L | openssl enc -pbkdf2 -d -aes-256-ctr -salt -pass fd:3 3< passwordFile | zstd -dqcf | tar -xf -
Amanda is a pretty old software (development started in the 80s) for tapes which is very noticable in that all the default settings are absolute garbage for modern hardware. The wiki is a mess but man pages are good.
But the software works well, it can do multi tape backups, networked backups (its own client or samba), (multi level) incremental backups, has good reporting and keeps a local file list so you can easily find the correct tape to restore from if you only need to restore a few files.
I will attach sensible settings that use aes-256-ctr and zstd compression. Put them in /etc/amanda/
A temporary fast storage disk can be useful if the disk you are backuping from is way too slow to write straight to tape (my LTO-5 drive seems to handle write speeds down to 50% of max speed just fine) or if you are doing many small backup sets that would make the tape drive start and stop a lot between them.
To start a backup i run "sudo -u amanda amdump archive localhost" or in user amanda's crontab "/usr/bin/amdump backup localhost; /usr/bin/amtape backup eject".
When it is done i get a email with which tapes it wrote to and other details. For the archive backups i then remove the right 12 slot bay and remove the tapes it used and put in new ones.
To register the new tapes i run:
single tape: sudo -u amanda amlabel --barcode ROT001L5 backup ROT001
multiple tapes: for i in {2..9}; do sudo -u amanda amlabel --barcode ROT00${i}L5 backup ROT00$i; done
Replace ROT001 with whatever the new tapes are called. My own printed tapes i use ROT01 to ROT09. My used ones all start with A and then 5 numbers.
This writes a small header to file 0 on the tapes marking it as a amanda tape registering it to a specific backup config so it does not accidentally get overwritten by amanda if you do something stupid.
Add a -f if a tape was previously used in an amanda config and you are sure nothing important is on it.
Amanda backups can be recovered easily without the software itself.
Position tape at the correct file using "mt fsf and mt status", file 1 if you only did a single backup set (1 DLE).
First block is text information from amanda about what this backup is and how to restore it. Either skip this block with mt fsr or read it "dd if=$TAPE bs=1024k count=1 of=header".
You should now be at block 1 according to mt status, everything following is the backup data, assuming you used my settings you can restore with the same line as up above or:
mbuffer -i $TAPE -s 1024k -b 2048 -L | /etc/amanda/encrypt -d | /etc/amanda/zstd-compression -d | tar -xf -
which uses the password stored in /var/lib/amanda/
Skip zstd-compression if you did not compress this backup (the header will tell you if you did).
Slight security warning here is that the header leaks some details about your backup like the backup name (DLE name) and settings used. So maybe don't label the backup my-banking-passwords if you care about not making it easier for physical attackers.
Another alternative to amanda is bakula but i have not tried it.
Assuming i used all my tape storage, 150TB used tapes and 15TB new, the cost is $11.14/TB including the cost for the extra library i don't use. Sum total i have spent is left as an exercise to the reader as i dare not write it T_T. Planning to allow friends to do archival on the used tapes if they want for a small cost.
-edit-
Had a parsing error in amanda.zip/backup/disklist on line 3, moved "tar-comp" down a line. Zip updated.
-edit2-
Updated config to include amanda-client.conf used for restores via amrecover. See
https://www.eevblog.com/forum/general-computing/lto-tape-usage-(modern-tape-drives)/msg4025251/#msg4025251 for usage.
Added tapestat and maminfo to list of useful programs.
-edit3-
Fix wrong dumptype used for raw disks in example config.
-edit4-
I was getting bad performance when backing up ZFS and the culprit is amanda using sparse file detection on tar, disable it by adding "property "SPARSE" "NO" to amgtar application definition.