Document setup, running and what lands in immich

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

README.md

@@ -0,0 +1,157 @@
+# rgl-photo-backup
+
+Back up the photos the [Randomness Guide to London](https://london.randomness.org.uk/)
+shows on its pages. Nearly all of them live on Flickr, which is a single
+point of failure. This fetches the largest available version of each into a
+directory, alongside a JSON sidecar of what the wiki and Flickr know about
+it, then mirrors the directory into an [Immich](https://immich.app) album
+with the source, photographer, licence and wiki pages carried into each
+asset's description, tags, capture date and location.
+
+## Requirements
+
+Python 3 and nothing else.
+
+## Setup
+
+Two credentials, both reused from other tools if you already have them:
+
+- `~/.config/flickrtouchr/auth.yml` with a `key:` line holding a Flickr API
+  key. Public photos need no OAuth, so the key alone is enough. Create one at
+  flickr.com/services/apps/create if you have none.
+- `~/.config/immich/auth.yml` with `url:` (your Immich API base, ending in
+  `/api`) and `key:` (an API key from Immich's account settings).
+
+Everything else can go in `~/.config/rgl-photo-backup/settings.yml`:
+
+```yaml
+directory: /srv/backup/rgl
+pace: 1
+quiet: true
+```
+
+Make the credential files `chmod 600`.
+
+## Running
+
+```
+./rgl-photo-backup /srv/backup/rgl            # fetch, then sync
+./rgl-photo-backup --dry-run /srv/backup/rgl  # report only
+./rgl-photo-backup --fetch-only --limit 5 DIR # a trial
+./rgl-photo-backup --sync-only DIR            # disk to Immich only
+./rgl-photo-backup --settings other.yml DIR   # a different settings file
+./rgl-photo-backup --debug DIR                # log every request to stderr
+./rgl-photo-backup -q DIR                     # same as --quiet
+```
+
+The first run downloads about 4800 photos, one Flickr info call and one
+sizes call each, so at the default one-second pace it takes a few hours.
+The pace applies to Flickr alone; Immich and the wiki index are not slowed.
+Later runs skip everything already on disk, and in Immich only write an
+album membership or a description that has actually changed.
+
+Exit status is 0 for a clean run, 1 if any photo failed, 2 for a
+configuration problem, and 130 if you interrupt it.
+
+### The first run
+
+Take it in stages, so a misconfiguration costs a few photos rather than
+thousands:
+
+1. `--fetch-only` on its own, to fill the backup directory.
+2. `--sync-only --limit 20`, and look at the twenty assets in Immich.
+3. the full run.
+
+Immich queues its own metadata extraction after an upload, so on a first
+run a description written straight afterwards can be overwritten by the
+extraction finishing. This tool writes descriptions after everything else
+for that reason, but if any of them come out wrong, run it again: the next
+run compares against what Immich holds and puts them right.
+
+Three things worth checking with curl against your own Immich version
+before leaving it unattended, since the API has moved before:
+
+- `POST /api/search/metadata` accepts `withExif`, `withArchived` and `size`
+- `POST /api/assets` honours the `x-immich-checksum` header
+- `PUT /api/tags` returns `value` as the full slash-separated path
+
+## What lands in Immich
+
+- An album named **RGL** holding every photo. Membership is read first, so
+  only photos not already in it are added.
+- Tags `rgl`, `rgl/licence/<licence>` and `rgl/photographer/<name>`. Unlike
+  the album and the description, tag assignments are sent every run: Immich
+  answers an asset that already carries the tag with `duplicate`, and those
+  are reported as already tagged rather than treated as a failure.
+- A description block, appended below anything already there and refreshed
+  in place on later runs:
+
+  ```
+  Randomness Guide to London photo
+  Photo: <Flickr title>
+  Source: <Flickr photo page>
+  Photographer: <name>
+  Licence: CC BY-SA 2.0 <licence url>
+  Used on:
+    <page name> <page url>
+  ```
+
+- The capture date from Flickr and the page's coordinates, only where the
+  asset has none already.
+
+A photo already in your library, for instance one of your own brought in by
+another backup, is recognised by checksum: it joins the album and gets the
+block appended rather than being uploaded twice.
+
+## The backup directory
+
+One file per photo, named by Flickr id (`3750465402.jpg`), beside a sidecar
+(`3750465402.json`) holding the wiki fields, the Flickr info and the pages
+that use it. The twenty-odd photos not on Flickr have no id to use, so they
+are named `url-` and the first sixteen hex characters of the SHA1 of their
+image URL (`url-0f4b1c2d3e4f5a6b.jpg`), with a matching sidecar. The
+directory is a complete backup on its own. A `.part` file is an interrupted
+download and is refetched next run.
+
+## Running under cron
+
+```cron
+23 4 * * *  /home/you/bin/rgl-photo-backup
+```
+
+With `quiet: true` a clean run prints nothing, so cron only mails when a
+photo failed. A quiet run that failed ends with a single `N failed` line
+after the individual failures. Failures list the photo and the wiki pages
+using it, which is how you learn a Flickr photo has gone.
+
+## Settings
+
+A flag beats the environment, which beats the file, which beats the default.
+
+| Key | Flag | Environment | Default |
+|---|---|---|---|
+| `directory` | positional | | required |
+| `flickr_auth` | `--flickr-config FILE` | | `~/.config/flickrtouchr/auth.yml` |
+| `immich_auth` | `--immich-config FILE` | | `~/.config/immich/auth.yml` |
+| `index_url` | `--index-url URL` | | the RGL index endpoint |
+| `fetch` | `--sync-only` sets false | | `true` |
+| `sync` | `--fetch-only` sets false | | `true` |
+| `dry_run` | `--dry-run` | | `false` |
+| `limit` | `--limit N` | | unlimited |
+| `pace` | `--pace SECONDS` | `RGL_PHOTO_BACKUP_PACE` | `1` |
+| `quiet` | `--quiet`, `-q` | | `false` |
+| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `6` |
+| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `5` |
+| `timeout` | | `RGL_PHOTO_BACKUP_TIMEOUT` | `60` |
+| `user_agent` | | `RGL_PHOTO_BACKUP_USER_AGENT` | `rgl-photo-backup/1.0` |
+| `debug` | `--debug` | `RGL_PHOTO_BACKUP_DEBUG` | `false` |
+
+`--settings FILE` reads a different settings file; it is a flag only, since
+it says where the rest of the settings come from.
+
+## Tests
+
+```
+make test    # offline unit tests
+make lint
+```