Slow the whole run down when flickr answers 429
Flickr's limit is an hourly window, so backing off one photo for a couple of minutes and then hitting the next photo at the old rate just fails again, which is how a full fetch gave up on photos after six attempts. A 429 on a paced request now doubles the pace for every later flickr call, up to ten seconds, and the pace relaxes back to the configured value only after twenty untroubled requests. The per-request backoff also starts at fifteen seconds instead of five, caps at fifteen minutes, and gets eight attempts, so a single photo can wait out a short window on its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
3 files changed
+74
-14
No files matched your search
@@ -47,6 +47,10 @@ Make the credential files `chmod 600`.
|
||||
The first run downloads about 4800 photos, one Flickr info call and one
|
||||
sizes call each, so at the default one-second pace it takes a few hours.
|
||||
The pace applies to Flickr alone; Immich and the wiki index are not slowed.
|
||||
A 429 from Flickr doubles the pace for the rest of the run, up to ten seconds
|
||||
between calls, and it relaxes again after twenty untroubled requests. The
|
||||
photo that hit the limit is retried with backoff starting at fifteen seconds
|
||||
and doubling, up to eight attempts.
|
||||
Later runs skip everything already on disk, and in Immich only write an
|
||||
album membership or a description that has actually changed.
|
||||
|
||||
@@ -140,8 +144,8 @@ A flag beats the environment, which beats the file, which beats the default.
|
||||
| `limit` | `--limit N` | | unlimited |
|
||||
| `pace` | `--pace SECONDS` | `RGL_PHOTO_BACKUP_PACE` | `1` |
|
||||
| `quiet` | `--quiet`, `-q` | | `false` |
|
||||
| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `6` |
|
||||
| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `5` |
|
||||
| `attempts` | | `RGL_PHOTO_BACKUP_ATTEMPTS` | `8` |
|
||||
| `backoff` | | `RGL_PHOTO_BACKUP_BACKOFF` | `15` |
|
||||
| `timeout` | | `RGL_PHOTO_BACKUP_TIMEOUT` | `60` |
|
||||
| `user_agent` | | `RGL_PHOTO_BACKUP_USER_AGENT` | `rgl-photo-backup/1.0` |
|
||||
| `debug` | `--debug` | `RGL_PHOTO_BACKUP_DEBUG` | `false` |
|
||||
|
||||
Reference in new issue
Block a user