Tommy Ku Photography

My Photo Storage

I have gone dark for nearly a year on this site because in the past year I have spent a gread deal of my time on my watch making/repair/collection site.

This year I thought I should pick up photography again, so I hope to be updating this site more frequently.

Since I haven't got much going on, I want to share my current photo storage workflow instead.

Some time back, Google started charging Google Photo product after years of being free and having received popular adoption, a cheap move, but a reasonable one as storage and the power to maintain them aren't free. So once Google had done whatever they wanted to do with people's privacy they started charging rent.

Before you read on, if you're not a tech savvy person, go for Google Photo or Amazon or OneDrive or whatever that provide convenience with a price tag. Whatever I am sharing here isn't suitable for anyone who can't administrate a Linux server. With hardware, electricity, cloud storage and maintenance cost it's likely not cheaper than those options anyways.

3-2-1 Backup

My workflow is based on the famous 3-2-1 Backup approach, keeping 2 on-site backups and 1 off-site backup to safeguard my data from data loss due to hardware failure (happens all the time) or losing access to the site (during travel/local copy failure).

My backup solution consists of 4 components:

Including the SD card/film medium that produced the original photo, actually this is a 4-3-1 approach, but here let's assume the SD card/film are not accessible or may be wiped to save space, and we'd still have a 3-2-1.

Workflow

NAS Backup

NAS stands for network attached storage which maybe bought or converted from a home PC, a laptop, or even a single-board computer (really risky though).

My NAS is a home PC-converted NAS running on TrueNAS CORE. My primary storage pool consists of 4 × 4TB running on RAIDZ2 configuration. That means I have 2-drive redundancy, keeping the original data intact even with 2 of the 4 hard disks failed.

In reality, there are usually at least 1 drive that fails every year. In that case I can still feel safe about my data because I have another redundancy drive to defend against total data loss.

This stage of photo backup means copying the photos or lab film scan to the NAS. I manually copy my film scan and photos in camera SD card to NAS using rlcone, which ensure data integrity during transmission. Photos taken on my phone are synced to the NAS using Nextcloud or Syncthing.

Now, there are 2 copies.

External HDD Backup

Once the data are safely loaded into the NAS, I can leverage its fast RAIDZ2 read speed to copy the data into an external HDD backup drive. Again, rlcone is used but due to the data size, I rarely reconcile the data content, and opt for simply rlcone copy --size-only such that only the file size are checked after copying.

Now, there are 3 copies.

Cloud Backup

Having 3 copies, all of which local, are not enough.

I encrypt and sync my photo collection to my cloud storage provider using duplicati, keeping every historical snapshots of my collection. It isn't cheap, but gives me the peace of mind that in case the 2-3 other copies failed, the cloud is at least less likely to fail (not that they don't, though).

Unless I have lost the Internet connection to the cloud provider too.

Now I have 4 copies.

Viewer

My data are safe.

But at this point it's not very human-friendly to view and retrieve. Certainly Nextcloud which I use to backup only my phone photos has a very slow and limited photo viewer, the experience is just nowhere as good as Google Photo.

There are many FOSS photo libraries and I ended up choosing Immich after running with Photoview for some time.

My Immich runs on docker, which in turns runs on an VM on the NAS. Immich supports docker well but I want to keep my NAS clean so using an VM instead of a FreeBSD Jail.

If I could migrate my TrueNAS from CORE to SCALE I hope to run docker directly from the OS instead of having to insteall an VM to use docker.

Immich works fine and have usable machine learning photo categorization and face recognition with even the slow i3 I am running my NAS on, all virtualization overheads considered.

It even has a mobile app on iOS and Android. Good stuff.

Sharer

Immich does it and works just like Google Photo.

My girlfriend can use it.

My family is still learning.

I pay for my mom's Google Photo subscription.

Bottom Line

My workflow is obviously not the best way out there.

If you have the money, paying for a cloud provider may be the cheapest way as they have the economies of scale. Even though they may suddenly choose to kill the entire product line and leaving clueless users in panic. Google Photo did this, somewhat anyways, albeit just to their free offering, for now.

Or, you want to keep the data to your own, easily accessible, then a simple 2-bay NAS may be a more user-friendly solution minus the maintenance cost.