Can a Toolforge job get access to about 10G of reasonably fast, ephemeral
scratch space?
My job crunches /public/dumps to build a file (an LMDB database) which it
publishes on behalf of other tools. However, on Toolforge Kubernetes, the
tool runs about 20 times slower than on my laptop or in a commercial cloud.
I wonder how come Toolforge is so awfully slow, and I/O seems like an
obvious reason. Basically, my code inserts many millions of keys in random
order into a btree in an mmap-ed file. This works super well on local disk
or also on a remote Cinder or Ceph volume, but not so well over NFS.
— Sascha