<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Oct 21, 2016 at 12:01 PM, Martin Urbanec <span dir="ltr"><<a href="mailto:martin.urbanec@wikimedia.cz" target="_blank">martin.urbanec@wikimedia.cz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Okay, I understood. Just for clarify, read-only access will be available? I.e. will I be able to download source code and change it locally? <div><br></div></div></blockquote><div>Yes, read only access to data will be available and yes, you should be able to download any code available in your home or project directories.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Martin</div></div><br><div class="gmail_quote"><div dir="ltr">pá 21. 10. 2016 v 20:59 odesílatel Madhumitha Viswanathan <<a href="mailto:mviswanathan@wikimedia.org" target="_blank">mviswanathan@wikimedia.org</a>> napsal:<br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="m_-2452589514509425118gmail_msg"><div class="gmail_extra m_-2452589514509425118gmail_msg"><div class="gmail_quote m_-2452589514509425118gmail_msg">On Fri, Oct 21, 2016 at 11:49 AM, Martin Urbanec <span dir="ltr" class="m_-2452589514509425118gmail_msg"><<a href="mailto:martin.urbanec@wikimedia.cz" class="m_-2452589514509425118gmail_msg" target="_blank">martin.urbanec@wikimedia.cz</a>></span> wrote:<br class="m_-2452589514509425118gmail_msg"><blockquote class="gmail_quote m_-2452589514509425118gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="m_-2452589514509425118gmail_msg">If 48 hours are required for this, maybe we can offer backup solution which will work as before. So there will be no outage, all will be setup at another virtuals and then only swith them. If there will be problem, switch again, solve it and switch again. <div class="m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"></div><div class="m_-2452589514509425118gmail_msg">Is this possible?</div><span class="m_-2452589514509425118m_-6752945025179250134HOEnZb m_-2452589514509425118gmail_msg"><font color="#888888" class="m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"></div></font></span></div></blockquote></div></div></div><div dir="ltr" class="m_-2452589514509425118gmail_msg"><div class="gmail_extra m_-2452589514509425118gmail_msg"><div class="gmail_quote m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118gmail_msg">This is not possible. At some point we have to stop new data being written into, sync the latest changes and then switch everyone over to the new servers. Not doing this will cause data loss and inconsistency, and not something we'd do.</div></div></div></div><div dir="ltr" class="m_-2452589514509425118gmail_msg"><div class="gmail_extra m_-2452589514509425118gmail_msg"><div class="gmail_quote m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"></div><blockquote class="gmail_quote m_-2452589514509425118gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="m_-2452589514509425118gmail_msg"><span class="m_-2452589514509425118m_-6752945025179250134HOEnZb m_-2452589514509425118gmail_msg"><font color="#888888" class="m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118gmail_msg"></div><div class="m_-2452589514509425118gmail_msg">Martin</div></font></span><div class="m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"></div><div class="m_-2452589514509425118gmail_msg">PS: I can live without toollabs for 48 hours but a lot of tools depends on availability so I strongly prefer as short window as possible. </div></div><br class="m_-2452589514509425118gmail_msg"><div class="gmail_quote m_-2452589514509425118gmail_msg"><div dir="ltr" class="m_-2452589514509425118gmail_msg">pá 21. 10. 2016 v 20:29 odesílatel Martin Domdey <<a href="mailto:animalia@gmx.net" class="m_-2452589514509425118gmail_msg" target="_blank">animalia@gmx.net</a>> napsal:<br class="m_-2452589514509425118gmail_msg"></div><div class="m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118m_-6752945025179250134h5 m_-2452589514509425118gmail_msg"><blockquote class="gmail_quote m_-2452589514509425118gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div style="font-family:Verdana;font-size:12.0px" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Why do you need 48 hours for that?</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">I'm submitting very many cron jobs the day to deliver much stuff and services to a lot of users in dewiki and other wikis. An outage window of 48 hours (!) is simply not possible.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Please suggest a solution how I can work on during the outage window or at least a crontab that can handle the data and files on tools.taxonbot. You maybe can install a NFS redundancy for at least that time.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Thank you</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Martin ...</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div name="quote" style="margin:10px 5px 5px 10px;padding:10px 0 10px 10px;border-left:2px solid #c3d9e5;word-wrap:break-word" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div style="margin:0 0 10px 0" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><b class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Gesendet:</b> Freitag, 21. Oktober 2016 um 20:00 Uhr<br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<b class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Von:</b> "Madhumitha Viswanathan" <<a href="mailto:mviswanathan@wikimedia.org" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">mviswanathan@wikimedia.org</a>><br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<b class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">An:</b> "Wikimedia Labs" <<a href="mailto:labs-l@lists.wikimedia.org" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">labs-l@lists.wikimedia.org</a>>, <a href="mailto:labs-announce@lists.wikimedia.org" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">labs-announce@lists.wikimedia.<wbr>org</a><br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<b class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Betreff:</b> [Labs-l] Disruptive Tools NFS maintenance on 11/2/2016</div></div></div></div></div></div><div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div style="font-family:Verdana;font-size:12.0px" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"><div name="quote" style="margin:10px 5px 5px 10px;padding:10px 0 10px 10px;border-left:2px solid #c3d9e5;word-wrap:break-word" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div name="quoted-content" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">As the next step in our storage redundancy and reliability efforts for Labs, we have a significant migration coming up on 11/2 starting 08:00 PST(15:00 UTC) involving the tools NFS share. The maintenance window can be up to 48h long, and will affect most running tools. At the end of the migration, everything (except transient jobs) should ideally be working the same way as they were before the migration, but better.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Here's what to expect during the maintenance window:</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">* The tools NFS share (/data/project and /home) will be read-only for the duration of the maintenance, so no new data or logs will get written to it.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">* New jobs cannot be submitted for the whole maintenance window - this means submitting jobs through cron or tools-mail will not function, although tools-mail can continue to send emails. </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">* Current jobs might keep running, but won't get rescheduled if they die. If they do not die and aren't writing to NFS they should be fine. </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">* All exec nodes will get depooled, rebooted and repooled and jobs that don't get rescheduled automatically will have died and need manual restarts.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Do let us know if you have any questions or concerns on the lists or on #wikimedia-labs.</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg"> </div>
--
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Madhumitha Viswanathan</div>
<div class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">Operations Engineer, Wikimedia Labs</div>
</div>
______________________________<wbr>_________________ Labs-l mailing list <a href="mailto:Labs-l@lists.wikimedia.org" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a> <a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a></div>
</div></div></div></div></div>
______________________________<wbr>_________________<br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
Labs-l mailing list<br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<a href="mailto:Labs-l@lists.wikimedia.org" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a><br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br class="m_-2452589514509425118m_-6752945025179250134m_-269486215696279122gmail_msg m_-2452589514509425118gmail_msg">
</blockquote></div></div></div>
<br class="m_-2452589514509425118gmail_msg">______________________________<wbr>_________________<br class="m_-2452589514509425118gmail_msg">
Labs-l mailing list<br class="m_-2452589514509425118gmail_msg">
<a href="mailto:Labs-l@lists.wikimedia.org" class="m_-2452589514509425118gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a><br class="m_-2452589514509425118gmail_msg">
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" class="m_-2452589514509425118gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br class="m_-2452589514509425118gmail_msg">
<br class="m_-2452589514509425118gmail_msg"></blockquote></div></div></div><div dir="ltr" class="m_-2452589514509425118gmail_msg"><div class="gmail_extra m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"><br clear="all" class="m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118gmail_msg"><br class="m_-2452589514509425118gmail_msg"></div>-- <br class="m_-2452589514509425118gmail_msg"><div class="m_-2452589514509425118m_-6752945025179250134gmail_signature m_-2452589514509425118gmail_msg" data-smartmail="gmail_signature"><div dir="ltr" class="m_-2452589514509425118gmail_msg">--Madhu :)</div></div>
</div></div>
______________________________<wbr>_________________<br class="m_-2452589514509425118gmail_msg">
Labs-l mailing list<br class="m_-2452589514509425118gmail_msg">
<a href="mailto:Labs-l@lists.wikimedia.org" class="m_-2452589514509425118gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a><br class="m_-2452589514509425118gmail_msg">
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" class="m_-2452589514509425118gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br class="m_-2452589514509425118gmail_msg">
</blockquote></div></div></div>
<br>______________________________<wbr>_________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Madhu :)</div></div>
</div></div>