Record an album or tag write immich refused

immich answers a bulk album or tag write with HTTP 200 and one result per
asset, so a key without album permission produced "filing 0 asset(s)",
no recorded failure and exit 0. Refusals other than duplicate now come
back from the client and are recorded against the photo and its wiki
pages. The album and tag stages also get separate error handling, so a
failed album write no longer skips tagging for the run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

README.md

@@ -0,0 +1,170 @@
+# rgl-photo-backup
+
+Back up the photos the [Randomness Guide to London](https://london.randomness.org.uk/)
+shows on its pages. Nearly all of them live on Flickr, which is a single
+point of failure. This fetches the largest available version of each into a
+directory, alongside a JSON sidecar of what the wiki and Flickr know about
+it, then mirrors the directory into an [Immich](https://immich.app) album
+with the source, photographer, licence and wiki pages carried into each
+asset's description, tags, capture date and location.
+
+## Requirements
+
+Python 3 and nothing else.
+
+## Setup
+
+Two credentials, both reused from other tools if you already have them:
+
+- `~/.config/flickrtouchr/auth.yml` with a `key:` line holding a Flickr API
+  key. Public photos need no OAuth, so the key alone is enough. Create one at
+  flickr.com/services/apps/create if you have none.
+- `~/.config/immich/auth.yml` with `url:` (your Immich API base, ending in
+  `/api`) and `key:` (an API key from Immich's account settings).
+
+Everything else can go in `~/.config/rgl-photo-backup/settings.yml`:
+
+```yaml
+directory: /srv/backup/rgl
+pace: 1
+quiet: true
+```
+
+Make the credential files `chmod 600`.
+
+## Running
+
+```
+./rgl-photo-backup /srv/backup/rgl            # fetch, then sync
+./rgl-photo-backup --dry-run /srv/backup/rgl  # report only
+./rgl-photo-backup --fetch-only --limit 5 DIR # a trial
+./rgl-photo-backup --sync-only DIR            # disk to Immich only
+./rgl-photo-backup --settings other.yml DIR   # a different settings file
+./rgl-photo-backup --debug DIR                # log every request to stderr
+./rgl-photo-backup -q DIR                     # same as --quiet
+./rgl-photo-backup --report DIR               # which pages have no photo on disk, and why
+```
+
+The first run downloads about 4800 photos, one Flickr info call and one
+sizes call each, so at the default one-second pace it takes a few hours.
+The pace applies to Flickr alone; Immich and the wiki index are not slowed.
+A 429 from Flickr doubles the pace for the rest of the run, up to ten seconds
+between calls, and it relaxes again after twenty untroubled requests. The
+photo that hit the limit is retried with backoff starting at fifteen seconds
+and doubling, up to eight attempts.
+Later runs skip everything already on disk, and in Immich only write an
+album membership or a description that has actually changed.
+
+Exit status is 0 for a clean run, 1 if any photo failed, 2 for a
+configuration problem, and 130 if you interrupt it.
+
+### The first run
+
+Take it in stages, so a misconfiguration costs a few photos rather than
+thousands:
+
+1. `--fetch-only` on its own, to fill the backup directory.
+2. `--sync-only --limit 20`, and look at the twenty assets in Immich.
+3. the full run.
+
+Immich queues its own metadata extraction after an upload, so on a first
+run a description written straight afterwards can be overwritten by the
+extraction finishing. This tool writes descriptions after everything else
+for that reason, but if any of them come out wrong, run it again: the next
+run compares against what Immich holds and puts them right.
+
+Three things worth checking with curl against your own Immich version
+before leaving it unattended, since the API has moved before:
+
+- `POST /api/search/metadata` accepts `withExif`, `withArchived` and `size`
+- `POST /api/assets` honours the `x-immich-checksum` header
+- `PUT /api/tags` returns `value` as the full slash-separated path
+
+## What lands in Immich
+
+- An album named **RGL** holding every photo. Membership is read first, so
+  only photos not already in it are added.
+- Tags `rgl`, `rgl/licence/<licence>` and `rgl/photographer/<name>`. Unlike
+  the album and the description, tag assignments are sent every run: Immich
+  answers an asset that already carries the tag with `duplicate`, and those
+  are reported as already tagged rather than treated as a failure.
+- A description block, appended below anything already there and refreshed
+  in place on later runs:
+
+  ```
+  Randomness Guide to London photo
+  Photo: <Flickr title>
+  Source: <Flickr photo page>
+  Photographer: <name>
+  Licence: CC BY-SA 2.0 <licence url>
+  Used on:
+    <page name> <page url>
+  ```
+
+- The capture date from Flickr and the page's coordinates, only where the
+  asset has none already.
+
+A photo already in your library, for instance one of your own brought in by
+another backup, is recognised by checksum: it joins the album and gets the
+block appended rather than being uploaded twice.
+
+## The backup directory
+
+One file per photo, named by Flickr id (`3750465402.jpg`), beside a sidecar
+(`3750465402.json`) holding the wiki fields, the Flickr info and the pages
+that use it. The twenty-odd photos not on Flickr have no id to use, so they
+are named `url-` and the first sixteen hex characters of the SHA1 of their
+image URL (`url-0f4b1c2d3e4f5a6b.jpg`), with a matching sidecar. The
+directory is a complete backup on its own. A `.part` file is an interrupted
+download and is refetched next run.
+
+`.failures.json` records the last reason each photo failed, with the pages
+that use it. A photo that later arrives drops out of it. `--report` reads the
+wiki index and that file, and lists every page whose photo is not on disk
+with the reason, or `not yet attempted`, without calling Flickr or Immich, so
+it is the way to find photos that have gone from Flickr and need replacing
+on the wiki.
+
+## Running under cron
+
+```cron
+23 4 * * *  /home/you/bin/rgl-photo-backup
+```
+
+With `quiet: true` a clean run prints nothing, so cron only mails when a
+photo failed. A quiet run that failed ends with a single `N failed` line
+after the individual failures. Failures list the photo and the wiki pages
+using it, which is how you learn a Flickr photo has gone.
+
+## Settings
+
+A flag beats the environment, which beats the file, which beats the default.
+
+| Key | Flag | Environment | Default |
+|---|---|---|---|
+| `directory` | positional | | required |
+| `flickr_auth` | `--flickr-config FILE` | | `~/.config/flickrtouchr/auth.yml` |
+| `immich_auth` | `--immich-config FILE` | | `~/.config/immich/auth.yml` |
+| `index_url` | `--index-url URL` | | the RGL index endpoint |
+| `fetch` | `--sync-only` sets false | | `true` |
+| `sync` | `--fetch-only` sets false | | `true` |
+| `dry_run` | `--dry-run` | | `false` |
+| `report` | `--report` | | `false` |
+| `limit` | `--limit N` | | unlimited |
+| `pace` | `--pace SECONDS` | `RGL_PHOTO_BACKUP_PACE` | `1` |
+| `quiet` | `--quiet`, `-q` | | `false` |
+| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `8` |
+| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `15` |
+| `timeout` | | `RGL_PHOTO_BACKUP_TIMEOUT` | `60` |
+| `user_agent` | | `RGL_PHOTO_BACKUP_USER_AGENT` | `rgl-photo-backup/1.0` |
+| `debug` | `--debug` | `RGL_PHOTO_BACKUP_DEBUG` | `false` |
+
+`--settings FILE` reads a different settings file; it is a flag only, since
+it says where the rest of the settings come from.
+
+## Tests
+
+```
+make test    # offline unit tests
+make lint
+```