The capacity scheduler has been deployed, we'll add the GPU labels tomorrow
as a second step (to avoid monitoring too many changes at once).
Luca
On Mon, Apr 26, 2021 at 8:50 AM Luca Toscano <ltoscano(a)wikimedia.org> wrote:
Hi everybody,
We are going to change the Yarn scheduler in a bit, moving from Fair to
Capacity. More info in
https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop#Queues
It shouldn't impact ongoing jobs, but there will probably be some
tuning/adjustment to do during the next few days. If you see any weird
behavior please ping us or contact us in T277062.
A gift for your patience: this change will allow users to select hadoop
nodes with GPUs when launching jobs :) (we have 6, running the same GPUs as
stat100[5,8]).
Luca