Forwarding. (:
Pine
---------- Forwarded message ----------
From: Brion Vibber <bvibber(a)wikimedia.org>
Date: Thu, Feb 9, 2017 at 12:00 PM
Subject: Re: [Wikitech-l] [Wikivideo-l] Video transcode prioritization
changes in progress
To: Opportunities for video in the Wikimedia universe - tech help <
wikivideo-l(a)lists.wikimedia.org>
Cc: Wikimedia-tech list <wikitech-l(a)lists.wikimedia.org>
Ok, the video scaler queue change is now live: SD and low-resolution video
output runs on a new, high-priority queue, with HD video output and
transcodes of files over 15 minutes in length run on the old queue.
Note that existing queued conversions will continue to run on the
low-priority queue until it runs out, but most new uploads from here out
will have web-ready output much faster.
(Load on the servers may be unbalanced for a bit, and may require further
tweaking to best balance responsiveness and throughput, so we'll keep an
eye on the graphs.)
-- brion
On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber <bvibber(a)wikimedia.org> wrote:
> We're in progress of deploying a small change to the video scalers[1],
> which should improve availability of newly uploaded video and audio files.
>
> The queue will now be split in two, one which covers low-resolution
> conversions for relatively short files, and one which covers long files
and
> high-resolution conversions. When there's a flood of large uploads, other
> new files should still go through the high-priority queue while the large
> uploads and HD conversions may back up on the low-priority queue.
>
> The queue-runner side is updated now (done during the 'puppet SWAT'
> deployment window), with the MediaWiki side ready to roll around 19:00 UTC
> (11am Pacific time, regular SWAT deployment window).
>
> [1] https://gerrit.wikimedia.org/r/#/c/336846/
>
> -- brion
>
> _______________________________________________
> Wikivideo-l mailing list
> Wikivideo-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Quick update -- I tested some batch-reencoding of missing low-res files and
have flooded the high-priority queue. :) Thanks for your patience as we get
the balance right.
I'll add a throttle to TimedMediaHandler's requeueTranscodes.php so I can
"fire-and-forget" on the background jobs without disturbing things...
-- brion
On Thu, Feb 9, 2017 at 12:00 PM, Brion Vibber <bvibber(a)wikimedia.org> wrote:
> Ok, the video scaler queue change is now live: SD and low-resolution video
> output runs on a new, high-priority queue, with HD video output and
> transcodes of files over 15 minutes in length run on the old queue.
>
> Note that existing queued conversions will continue to run on the
> low-priority queue until it runs out, but most new uploads from here out
> will have web-ready output much faster.
>
> (Load on the servers may be unbalanced for a bit, and may require further
> tweaking to best balance responsiveness and throughput, so we'll keep an
> eye on the graphs.)
>
> -- brion
>
>
> On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber <bvibber(a)wikimedia.org>
> wrote:
>
>> We're in progress of deploying a small change to the video scalers[1],
>> which should improve availability of newly uploaded video and audio files.
>>
>> The queue will now be split in two, one which covers low-resolution
>> conversions for relatively short files, and one which covers long files and
>> high-resolution conversions. When there's a flood of large uploads, other
>> new files should still go through the high-priority queue while the large
>> uploads and HD conversions may back up on the low-priority queue.
>>
>> The queue-runner side is updated now (done during the 'puppet SWAT'
>> deployment window), with the MediaWiki side ready to roll around 19:00 UTC
>> (11am Pacific time, regular SWAT deployment window).
>>
>> [1] https://gerrit.wikimedia.org/r/#/c/336846/
>>
>> -- brion
>>
>> _______________________________________________
>> Wikivideo-l mailing list
>> Wikivideo-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>>
>>
>
> _______________________________________________
> Wikivideo-l mailing list
> Wikivideo-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>
>