[Labs-l] Building our ZIM farm @wmflabs

[Federico Leva (Nemo)]

Any news on this?

 > == Next steps ==
 >
 > We want to complete our effort and mirror the biggest Wikipedia
 > projects. Unfortunately, we have reached the limits of a traditional
 > usage of wmflabs. We need more quota and to experiment with the NFS
 > storage because an x-large instance in not able to mirror more than
 > 1.5 millions of articles at a time. How might that be made possible?

Per https://phabricator.wikimedia.org/T91853#1113701 and 
https://phabricator.wikimedia.org/T91853#1129560 , francium seems a 
storage server rather than a processing server: it can't be used for 
mwoffliner tasks. https://phabricator.wikimedia.org/T57503 is a separate 
task.
	Emmanuel, I think the Labs instances are only meant to store the data 
they're currently processing, while the ZIM files as such keep ending up 
on http://download.kiwix.org/ + mirrors: right?
	Andrew, if so the best option IMHO is to increase the local disk quota 
of (at least) one of the instances from 160 to 500 GB. Then Emmanuel can 
start testing an en.wiki run and see how it goes. virt1011, currently 
used by mwoffliner2, seems to afford it. 
https://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Virtualization%20cluster%20eqiad&h=virt1011.eqiad.wmnet&r=hour&z=default&jr=&js=&st=1427785147&v=1188.316&m=disk_free&vl=GB&ti=Disk%20Space%20Available&z=large

Nemo