Slow the whole run down when flickr answers 429

Flickr's limit is an hourly window, so backing off one photo for a couple
of minutes and then hitting the next photo at the old rate just fails
again, which is how a full fetch gave up on photos after six attempts. A
429 on a paced request now doubles the pace for every later flickr call,
up to ten seconds, and the pace relaxes back to the configured value only
after twenty untroubled requests. The per-request backoff also starts at
fifteen seconds instead of five, caps at fifteen minutes, and gets eight
attempts, so a single photo can wait out a short window on its own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
bobandClaude Fable 5.1 committed 2026-09-10 16:37:08 +01:00
1 parent 07db2739ca
commit 36c974e843
3 files changed
+74 -14

No files matched your search

+6 -2
View File
@@ -47,6 +47,10 @@ Make the credential files `chmod 600`.
The first run downloads about 4800 photos, one Flickr info call and one
sizes call each, so at the default one-second pace it takes a few hours.
The pace applies to Flickr alone; Immich and the wiki index are not slowed.
A 429 from Flickr doubles the pace for the rest of the run, up to ten seconds
between calls, and it relaxes again after twenty untroubled requests. The
photo that hit the limit is retried with backoff starting at fifteen seconds
and doubling, up to eight attempts.
Later runs skip everything already on disk, and in Immich only write an
album membership or a description that has actually changed.
@@ -140,8 +144,8 @@ A flag beats the environment, which beats the file, which beats the default.
| `limit` | `--limit N` | | unlimited |
| `pace` | `--pace SECONDS` | `RGL_PHOTO_BACKUP_PACE` | `1` |
| `quiet` | `--quiet`, `-q` | | `false` |
| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `6` |
| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `5` |
| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `8` |
| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `15` |
| `timeout` | | `RGL_PHOTO_BACKUP_TIMEOUT` | `60` |
| `user_agent` | | `RGL_PHOTO_BACKUP_USER_AGENT` | `rgl-photo-backup/1.0` |
| `debug` | `--debug` | `RGL_PHOTO_BACKUP_DEBUG` | `false` |