Re: [Wikitech-l] Best practices for read/write vs read-only requests, and our multi-DC future

Over in TimedMediaHandler extension, we've had a number of cases where old code did things that were convenient in terms of squishing read-write operations into data getters, that got removed due to problems with long running transactions or needing to refactor things to support future-facing multi-DC work where we want requests to be able to more reliably distinguish between read-only and read-write. And we sometimes want to put some of those clever hacks back and add more. ;) For instance in https://gerrit.wikimedia.org/r/284368 we'd like to remove transcode derivative files of types/resolutions that have been disabled automatically when we come across them. But I'm a bit unsure it's safe to do so. Note that we could fire off a job queue background task to do the actual removal... But is it also safe to do that on a read-only request?

seems to indicate job queueing will be safe, but would like to confirm that. :)

Similarly in https://gerrit.wikimedia.org/r/#/c/284269/ we may wish to trigger missing transcodes to run on demand, similarly. The actual re encoding happens in a background job, but we have to fire it off, and we have to record that we fired it off so we don't duplicate it... (This would require a second queue to do the high-priority state table update and queue the actual transcoding job; we can't put them in one

because a backup of transcode jobs would prevent the high priority job

running in a timely fashion.)

A best practices document on future-proofing for multi DC would be pretty awesome! Maybe factor out some of the stuff from the RfC into a nice dev doc page... -- brion _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l