<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">On 2/23/17 12:30 PM, Yuvi Panda wrote:<br>
    </div>
    <blockquote
cite="mid:CAFw=ySheg4xjtq=LaiNaVPkMdpWJaWP8BrgZ5NH+EKZFiLEd=Q@mail.gmail.com"
      type="cite">
      <div dir="auto">Do we have the human bandwidth to commit to doing
        this as a team? GPUs are fickle beasts.</div>
    </blockquote>
    Noooo!  I'm not telling them 'we will do this' only 'this might be
    technically possible'<br>
    <br>
    <blockquote
cite="mid:CAFw=ySheg4xjtq=LaiNaVPkMdpWJaWP8BrgZ5NH+EKZFiLEd=Q@mail.gmail.com"
      type="cite">
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Feb 23, 2017 5:31 AM, "Andrew
          Bogott" <<a moz-do-not-send="true"
            href="mailto:abogott@wikimedia.org">abogott@wikimedia.org</a>>
          wrote:<br type="attribution">
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div bgcolor="#FFFFFF" text="#000000">
              <p>FYI, here is a thing that reading/ores people are
                daydreaming about.  I've made it clear that
                bare-metal-in-labs is a nonstarter; having special
                hardware with special VM types is not necessarily all
                that much work.  I bet a labvirt with a GPU will be
                super expensive though, and I don't know a damn thing
                about how GPU resource-contention would be handled.<br>
              </p>
              <p>-A</p>
              <br>
              <div class="m_6847163355653223882moz-forward-container">--------
                Forwarded Message --------
                <table
                  class="m_6847163355653223882moz-email-headers-table"
                  border="0" cellpadding="0" cellspacing="0">
                  <tbody>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Delivered-To: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:andrewbogott@gmail.com"
                          target="_blank">andrewbogott@gmail.com</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>by 10.55.130.1 with SMTP id e1csp77787qkd;
                        Wed, 22 Feb 2017 08:57:16 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Received: </th>
                      <td>by 10.237.36.116 with SMTP id
                        s49mr18754207qtc.128.<wbr>1487782636280; Wed, 22
                        Feb 2017 08:57:16 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Return-Path: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:abogott+caf_=andrewbogott=gmail.com@wikimedia.org"
                          target="_blank"><abogott+caf_=andrewbogott=<wbr>gmail.com@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>from <a moz-do-not-send="true"
                          href="http://mail-qk0-f173.google.com"
                          target="_blank">mail-qk0-f173.google.com</a> (<a
                          moz-do-not-send="true"
                          href="http://mail-qk0-f173.google.com"
                          target="_blank">mail-qk0-f173.google.com</a>.
                        [209.85.220.173]) by <a moz-do-not-send="true"
                          href="http://mx.google.com" target="_blank">mx.google.com</a>
                        with ESMTPS id q53si660238qtf.337.2017.02.22.<wbr>08.57.16
                        for <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:andrewbogott@gmail.com"
                          target="_blank"><andrewbogott@gmail.com></a>
                        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-<wbr>SHA256
                        bits=128/128); Wed, 22 Feb 2017 08:57:16 -0800
                        (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received-SPF: </th>
                      <td>pass (<a moz-do-not-send="true"
                          href="http://google.com" target="_blank">google.com</a>:
                        domain of <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
href="mailto:abogott+caf_=andrewbogott=gmail.com@wikimedia.org"
                          target="_blank">abogott+caf_=andrewbogott=<wbr>gmail.com@wikimedia.org</a>
                        designates 209.85.220.173 as permitted sender)
                        client-ip=209.85.220.173;</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Authentication-Results: </th>
                      <td><a moz-do-not-send="true"
                          href="http://mx.google.com" target="_blank">mx.google.com</a>;
                        dkim=pass <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:header.i=@wikimedia.org"
                          target="_blank">header.i=@wikimedia.org</a>;
                        spf=pass (<a moz-do-not-send="true"
                          href="http://google.com" target="_blank">google.com</a>:
                        domain of <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
href="mailto:abogott+caf_=andrewbogott=gmail.com@wikimedia.org"
                          target="_blank">abogott+caf_=andrewbogott=<wbr>gmail.com@wikimedia.org</a>
                        designates 209.85.220.173 as permitted sender) <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
href="mailto:smtp.mailfrom=abogott+caf_=andrewbogott=gmail.com@wikimedia.org"
                          target="_blank">smtp.mailfrom=abogott+caf_=<wbr>andrewbogott=gmail.com@<wbr>wikimedia.org</a>;
                        dmarc=pass (p=NONE sp=NONE dis=NONE)
                        header.from=<a moz-do-not-send="true"
                          href="http://wikimedia.org" target="_blank">wikimedia.org</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>by <a moz-do-not-send="true"
                          href="http://mail-qk0-f173.google.com"
                          target="_blank">mail-qk0-f173.google.com</a>
                        with SMTP id n127so300qkf.0 for <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:andrewbogott@gmail.com"
                          target="_blank"><andrewbogott@gmail.com></a>;
                        Wed, 22 Feb 2017 08:57:16 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Google-DKIM-Signature: </th>
                      <td>v=1; a=rsa-sha256; c=relaxed/relaxed; d=<a
                          moz-do-not-send="true" href="http://1e100.net"
                          target="_blank">1e100.net</a>; s=20161025;
                        h=x-gm-message-state:<wbr>delivered-to:dkim-signature:<wbr>mime-version
                        :in-reply-to:references:from:<wbr>date:message-id:subject:to:cc;
                        bh=<wbr>geU9E0JuTBp45fwea4nD9VRvS20wnO<wbr>pfRwfENW1LjhU=;
                        b=<wbr>JCBzeI8wEoxwlF8L9N02zAe4B6yM5b<wbr>hHph//6v2nv99TlzVWmYMh+<wbr>M5YysH03KFmox
                        Pv6DRB9RYw7g+<wbr>3w6On4goiSqbRlOanasU9gEBuOqLai<wbr>vY/4ictoMjr0mfCZynFxGwwcO
                        nfgiGna0f9LVgrElZZDd01HweqTJ0u<wbr>BW7tPPJDxt74YLf7I+tysJReVuvW/<wbr>e4GzGtMHG
                        B801Icgxf5WNdjQYUImCtXXHQFxlDc<wbr>W+<wbr>PydqDPQMH5Tyg8ssKWUQwFqCKe0PVd<wbr>tQsCY9
                        HLYgur5owYC+<wbr>yZr2D1CNR3iUMH2jcHM0IdsuLmiKL6<wbr>kSCy+x6oslXO4kJf3Hj3O7sFEy
                        3ZFw==</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Gm-Message-State: </th>
                      <td>AMke39mzi2oJ/<wbr>S99gBkx9KxBafTj2nbLxmj5JK4xkqZ<wbr>1ij7Mt9aq5oAT2u0MCGpmwqclQKf7e<wbr>+yX+X2zHORrO8Cpp0HTeDQVCA==</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Received: </th>
                      <td>by 10.55.94.6 with SMTP id
                        s6mr34227257qkb.166.<wbr>1487782635865; Wed, 22
                        Feb 2017 08:57:15 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Forwarded-To: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:andrewbogott@gmail.com"
                          target="_blank">andrewbogott@gmail.com</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Forwarded-For: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank">abogott@wikimedia.org</a> <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:andrewbogott@gmail.com"
                          target="_blank">andrewbogott@gmail.com</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Delivered-To: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank">abogott@wikimedia.org</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>by 10.140.99.34 with SMTP id p31csp954099qge;
                        Wed, 22 Feb 2017 08:57:14 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Received: </th>
                      <td>by 10.237.50.229 with SMTP id
                        z92mr30222596qtd.182.<wbr>1487782634693; Wed, 22
                        Feb 2017 08:57:14 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Return-Path: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abaso@wikimedia.org"
                          target="_blank"><abaso@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>from <a moz-do-not-send="true"
                          href="http://mx1001.wikimedia.org"
                          target="_blank">mx1001.wikimedia.org</a> (<a
                          moz-do-not-send="true"
                          href="http://mx1001.wikimedia.org"
                          target="_blank">mx1001.wikimedia.org</a>.
                        [208.80.154.76]) by <a moz-do-not-send="true"
                          href="http://mx.google.com" target="_blank">mx.google.com</a>
                        with ESMTPS id 52si1315621qtw.95.2017.02.22.<wbr>08.57.14
                        for <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank"><abogott@wikimedia.org></a>
                        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-<wbr>SHA256
                        bits=128/128); Wed, 22 Feb 2017 08:57:14 -0800
                        (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received-SPF: </th>
                      <td>pass (<a moz-do-not-send="true"
                          href="http://google.com" target="_blank">google.com</a>:
                        domain of <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:abaso@wikimedia.org"
                          target="_blank">abaso@wikimedia.org</a>
                        designates 2a00:1450:400c:c0c::22a as permitted
                        sender) client-ip=2a00:1450:400c:c0c::<wbr>22a;</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>from <a moz-do-not-send="true"
                          href="http://mail-wr0-x22a.google.com"
                          target="_blank">mail-wr0-x22a.google.com</a>
                        ([2a00:1450:400c:c0c::22a]:<wbr>33057) by <a
                          moz-do-not-send="true"
                          href="http://mx1001.wikimedia.org"
                          target="_blank">mx1001.wikimedia.org</a> with
                        esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_<wbr>SHA256:128)
                        (Exim 4.84_2) (envelope-from <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abaso@wikimedia.org"
                          target="_blank"><abaso@wikimedia.org></a>)
                        id 1cgaE4-0000fA-8M for <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-abbreviated"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank">abogott@wikimedia.org</a>;
                        Wed, 22 Feb 2017 16:57:14 +0000</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>by <a moz-do-not-send="true"
                          href="http://mail-wr0-x22a.google.com"
                          target="_blank">mail-wr0-x22a.google.com</a>
                        with SMTP id 97so6296811wrb.0 for <a
                          moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank"><abogott@wikimedia.org></a>;
                        Wed, 22 Feb 2017 08:57:12 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">DKIM-Signature: </th>
                      <td>v=1; a=rsa-sha256; c=relaxed/relaxed; d=<a
                          moz-do-not-send="true"
                          href="http://wikimedia.org" target="_blank">wikimedia.org</a>;
                        s=google; h=mime-version:in-reply-to:<wbr>references:from:date:message-<wbr>id:subject:to
                        :cc; bh=<wbr>geU9E0JuTBp45fwea4nD9VRvS20wnO<wbr>pfRwfENW1LjhU=;
                        b=WFFw+<wbr>gGuZwu1fwbD30zuj8lNJi5A1g7aoIe<wbr>2iVvXiXk2ePZe0PpxBVjGtiO52g9Dm<wbr>K
                        ivzQn2KiDrcgF1X3pHaYSUiINhiR5N<wbr>/<wbr>IFRcmPznm0HESdJyycHRUySCxgRVHM<wbr>6Hlt2a7
                        YflNt6jxThtJE3Hygnodo53N7EPKxH<wbr>Wtj4NV0=</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">X-Received: </th>
                      <td>by 10.223.172.66 with SMTP id
                        v60mr25236742wrc.77.<wbr>1487782631305; Wed, 22
                        Feb 2017 08:57:11 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">MIME-Version: </th>
                      <td>1.0</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Received: </th>
                      <td>by 10.80.148.203 with HTTP; Wed, 22 Feb 2017
                        08:56:30 -0800 (PST)</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">In-Reply-To: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:3a1992ee-44b4-8f8c-f946-15b8f6bad81c@wikimedia.org"
                          target="_blank"><3a1992ee-44b4-8f8c-f946-<wbr>15b8f6bad81c@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">References: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=NqwwtYKkgxEZWex5HxFso1xBZ+Y4swRzJzRzhwKGXrnVw@mail.gmail.com"
                          target="_blank"><CAB74=<wbr>NqwwtYKkgxEZWex5HxFso1xBZ+<wbr>Y4swRzJzRzhwKGXrnVw@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CALuiOYYEp34ZTR76x2cAEkmpd03njx+SZUEAbWP5cmqkp4ERaQ@mail.gmail.com"
                          target="_blank"><<wbr>CALuiOYYEp34ZTR76x2cAEkmpd03nj<wbr>x+SZUEAbWP5cmqkp4ERaQ@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=NpEkj9r9SdGfsn=VFz44DxRz+Ch5gKrrZJeoEhdDBz5xw@mail.gmail.com"
                          target="_blank"><CAB74=NpEkj9r9SdGfsn=<wbr>VFz44DxRz+<wbr>Ch5gKrrZJeoEhdDBz5xw@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAO8=cz3j_A018QCU9iry+UjWEFQZ5AcV4AD942RhEiFkXvqS=Q@mail.gmail.com"
                          target="_blank"><CAO8=cz3j_A018QCU9iry+<wbr>UjWEFQZ5AcV4AD942RhEiFkXvqS=Q@<wbr>mail.gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAKP=3Wz0R5DpsA8-b+PmJHuUNYggisyN5_4yUStN6cga7ftApA@mail.gmail.com"
                          target="_blank"><CAKP=3Wz0R5DpsA8-b+<wbr>PmJHuUNYggisyN5_<wbr>4yUStN6cga7ftApA@mail.gmail.<wbr>com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAH8ZkEggG2WvTeGrZVVyMVKyThe7rZe+TqQsWfOkboW5HtsY7w@mail.gmail.com"
                          target="_blank"><<wbr>CAH8ZkEggG2WvTeGrZVVyMVKyThe7r<wbr>Ze+TqQsWfOkboW5HtsY7w@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAKP=3WwvZ3wfL1XH1aeszPN5Ja3sua_brFifzzSS5SHD0UzpEA@mail.gmail.com"
                          target="_blank"><CAKP=<wbr>3WwvZ3wfL1XH1aeszPN5Ja3sua_<wbr>brFifzzSS5SHD0UzpEA@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=NoUBJG_f31nb0ACzKyv5JZhnnwqK7N0LVoVeHssozTaEg@mail.gmail.com"
                          target="_blank"><CAB74=NoUBJG_<wbr>f31nb0ACzKyv5JZhnnwqK7N0LVoVeH<wbr>ssozTaEg@mail.gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=NrrdD46n8nhY3rBmXNNkwkWxyiBiAb9xYNb8XzkUuHBOw@mail.gmail.com"
                          target="_blank"><CAB74=<wbr>NrrdD46n8nhY3rBmXNNkwkWxyiBiAb<wbr>9xYNb8XzkUuHBOw@mail.gmail.<wbr>com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAKP=3WxiqnJZjc=2hPcCe5U7cdbxBMqkdzr91SAOdO7+ns9m0w@mail.gmail.com"
                          target="_blank"><CAKP=3WxiqnJZjc=<wbr>2hPcCe5U7cdbxBMqkdzr91SAOdO7+<wbr>ns9m0w@mail.gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=Nq09YL-PY_nbq-oir_7dReYjzLfVz-FgqvPYmD5U=Nqow@mail.gmail.com"
                          target="_blank"><CAB74=Nq09YL-PY_nbq-oir_<wbr>7dReYjzLfVz-FgqvPYmD5U=Nqow@<wbr>mail.gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAKP=3Wxy4dncDCQJo4bgGwAta-4+Y_xJ0iQHeC1GuPfH6y=RpQ@mail.gmail.com"
                          target="_blank"><CAKP=3Wxy4dncDCQJo4bgGwAta-4+<wbr>Y_xJ0iQHeC1GuPfH6y=RpQ@mail.<wbr>gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:40905b9f-e1e6-1ab2-5d9b-baa5a1dbd9ac@wikimedia.org"
                          target="_blank"><40905b9f-e1e6-1ab2-5d9b-<wbr>baa5a1dbd9ac@wikimedia.org></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=Npe3UwzPZv+ditB1bmNASvNuXgK00ZanM65E7-XySX_pg@mail.gmail.com"
                          target="_blank"><CAB74=Npe3UwzPZv+<wbr>ditB1bmNASvNuXgK00ZanM65E7-<wbr>XySX_pg@mail.gmail.com></a>
                        <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:3a1992ee-44b4-8f8c-f946-15b8f6bad81c@wikimedia.org"
                          target="_blank"><3a1992ee-44b4-8f8c-f946-<wbr>15b8f6bad81c@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">From: </th>
                      <td>Adam Baso <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abaso@wikimedia.org"
                          target="_blank"><abaso@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Date: </th>
                      <td>Wed, 22 Feb 2017 10:56:30 -0600</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Message-ID: </th>
                      <td><a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
href="mailto:CAB74=Nq+aSspOt2bp1Hno58xjcERf3P8S_c_HPeaBgFWiY_NFw@mail.gmail.com"
                          target="_blank"><CAB74=Nq+<wbr>aSspOt2bp1Hno58xjcERf3P8S_c_<wbr>HPeaBgFWiY_NFw@mail.gmail.com></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Subject: </th>
                      <td>Re: Servers with GPUs</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">To: </th>
                      <td>Andrew Bogott <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:abogott@wikimedia.org"
                          target="_blank"><abogott@wikimedia.org></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Cc: </th>
                      <td>Aaron Halfaker <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:ahalfaker@wikimedia.org"
                          target="_blank"><ahalfaker@wikimedia.org></a>,
                        Dario Taraborelli <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:dtaraborelli@wikimedia.org"
                          target="_blank"><dtaraborelli@wikimedia.org></a>,
                        Ellery Wulczyn <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:ewulczyn@wikimedia.org"
                          target="_blank"><ewulczyn@wikimedia.org></a>,
                        Andrew Otto <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:otto@wikimedia.org"
                          target="_blank"><otto@wikimedia.org></a>,
                        Corey Floyd <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:cfloyd@wikimedia.org"
                          target="_blank"><cfloyd@wikimedia.org></a>,
                        Andrew Otto <a moz-do-not-send="true"
                          class="m_6847163355653223882moz-txt-link-rfc2396E"
                          href="mailto:acotto@gmail.com" target="_blank"><acotto@gmail.com></a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Content-Type: </th>
                      <td>multipart/alternative; boundary=<wbr>f403045cf1523e23fe05492163db</td>
                    </tr>
                  </tbody>
                </table>
                <br>
                <br>
                <div dir="ltr">I feel comfortable saying we want one of
                  these options. I'm trying to gchat Ryan Lane to see if
                  has insight on the Nova support. I just realized when
                  I checked action=history he was one of the editors of <a
                    moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport"
                    style="font-size:12.8px" target="_blank">HeterogeneousGpuAccelerator<wbr>Support</a>!
                  <div><br>
                  </div>
                  <div>-Adam<br>
                    <div class="gmail_extra"><br>
                      <div class="gmail_quote">On Tue, Feb 21, 2017 at
                        2:16 PM, Andrew Bogott <span dir="ltr"><<a
                            moz-do-not-send="true"
                            href="mailto:abogott@wikimedia.org"
                            target="_blank">abogott@wikimedia.org</a>></span>
                        wrote:<br>
                        <blockquote class="gmail_quote"
                          style="margin:0px 0px 0px
                          0.8ex;border-left:1px solid
                          rgb(204,204,204);padding-left:1ex">
                          <div bgcolor="#FFFFFF"><span>
                              <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483moz-cite-prefix">On
                                2/21/17 1:36 PM, Adam Baso wrote:<br>
                              </div>
                              <blockquote type="cite">
                                <div dir="ltr">I think either that, or
                                  if it's easier, OpenStack-provisioned
                                  physical servers. Do you think the
                                  latter is doable? </div>
                              </blockquote>
                            </span> It's possible, although the last
                            time we visited that issue we swiftly
                            determined that even though people were
                            asking for it no one actually wanted it. 
                            The conclusion from that process is at <a
                              moz-do-not-send="true"
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483moz-txt-link-freetext"
href="https://wikitech.wikimedia.org/wiki/Labs_labs_labs/Bare_Metal"
                              target="_blank">https://wikitech.wikimedia.org<wbr>/wiki/Labs_labs_labs/Bare_Meta<wbr>l</a><br>
                            <br>
                            If GPU instance support is in nova and
                            actually maintained, that might be worth a
                            try.<span
                              class="m_6847163355653223882gmail-m_-5394071749873894282HOEnZb"><font
                                color="#888888"><br>
                                <br>
                                -A</font></span>
                            <div>
                              <div
                                class="m_6847163355653223882gmail-m_-5394071749873894282h5"><br>
                                <br>
                                <br>
                                <blockquote type="cite">
                                  <div dir="ltr">I gather the former
                                    would require different, if not
                                    deeper analysis (cf. <a
                                      moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport"
                                      target="_blank">HeterogeneousGpuAcceleratorSup<wbr>port</a>).
                                    <div><br>
                                    </div>
                                    <div>-Adam<br>
                                      <div class="gmail_extra"><br>
                                        <div class="gmail_quote">On Tue,
                                          Feb 21, 2017 at 12:08 PM,
                                          Andrew Bogott <span dir="ltr"><<a
                                              moz-do-not-send="true"
                                              href="mailto:abogott@wikimedia.org"
                                              target="_blank">abogott@wikimedia.org</a>></span>
                                          wrote:<br>
                                          <blockquote
                                            class="gmail_quote"
                                            style="margin:0px 0px 0px
                                            0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
                                            <div bgcolor="#FFFFFF">
                                              <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974moz-cite-prefix">Can
                                                y'all tell me a bit more
                                                about how this would
                                                relate to labs?  Is the
                                                idea that you want an
                                                option to create VMs
                                                with virtualized GPU
                                                hardware?  Or...
                                                something else?  (I'm
                                                not immediately clear on
                                                how that would work, but
                                                I'm largely ignorant on
                                                the subject.)<span
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709HOEnZb"><font
                                                    color="#888888"><br>
                                                    <br>
                                                    -A</font></span>
                                                <div>
                                                  <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709h5"><br>
                                                    <br>
                                                    <br>
                                                    On 2/21/17 12:02 PM,
                                                    Aaron Halfaker
                                                    wrote:<br>
                                                  </div>
                                                </div>
                                              </div>
                                              <div>
                                                <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709h5">
                                                  <blockquote
                                                    type="cite">
                                                    <div dir="auto">+9000
                                                      :D. (also + Andrew
                                                      bogott)
                                                      <div dir="auto"><br>
                                                      </div>
                                                      <div dir="auto">Would
                                                        love to have
                                                        this kind of
                                                        resource in labs
                                                        and openly
                                                        available.  I'm
                                                        personally
                                                        stoked to start
                                                        experimenting
                                                        but not ready to
                                                        invest in
                                                        production GPUs
                                                        yet.  A few of
                                                        my external
                                                        collaborators
                                                        have asked about
                                                        GPU resources in
                                                        labs too.</div>
                                                      <div dir="auto"><br>
                                                      </div>
                                                      <div dir="auto"><br>
                                                      </div>
                                                    </div>
                                                    <div
                                                      class="gmail_extra"><br>
                                                      <div
                                                        class="gmail_quote">On
                                                        Feb 21, 2017
                                                        11:52, "Adam
                                                        Baso" <<a
                                                          moz-do-not-send="true"
href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>>
                                                        wrote:<br
                                                          type="attribution">
                                                        <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Following
                                                          up on this
                                                          here current
                                                          thread, what
                                                          do you all
                                                          think about
                                                          doing the GPU
                                                          acceleration
                                                          in Labs first?
                                                          <div><br>
                                                          </div>
                                                          <div>I don't
                                                          know if it was
                                                          halfak or
                                                          marktraceur
                                                          who suggested
                                                          it first
                                                          (although
                                                          Aaron's
                                                          brought it up
                                                          a couple times
                                                          now), but
                                                          it's <i>probably</i>
                                                          less up front
                                                          architectural
                                                          overhead to
                                                          start out
                                                          with, even if
                                                          in the future
                                                          we'd have a
                                                          strict
                                                          requirement on
                                                          HA (Q1 FY
                                                          18-19 at the
                                                          very latest
                                                          under current
                                                          guess). As
                                                          Aaron has
                                                          rightly noted
                                                          doing this in
                                                          Labs also
                                                           lets us learn
                                                          plus give
                                                          community
                                                          members
                                                          greater access
                                                          to innovate
                                                          early, too.
                                                          The primary
                                                          downside of
                                                          not getting
                                                          this in
                                                          production
                                                          with HA up
                                                          front is if
                                                          funding dries
                                                          up for FY
                                                          18-19 we're
                                                          stuck or
                                                          pegged to
                                                          certain
                                                          workflows
                                                          later on. But
                                                          maybe it's not
                                                          worth worrying
                                                          about that too
                                                          much.
                                                          <div><br>
                                                          </div>
                                                          <div>I think
                                                          one very much
                                                          open question,
                                                          though, would
                                                          be if it's
                                                          possible to
                                                          have a machine
                                                          with the GPU
                                                          card installed
                                                          and
                                                          specifically
                                                          assigned in
                                                          Labs. Does
                                                          anyone know if
                                                          that's
                                                          actually
                                                          possible?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>-Adam</div>
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 16,
                                                          2017 at 4:56
                                                          PM, Aaron
                                                          Halfaker <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ahalfaker@wikimedia.org"
                                                          target="_blank">ahalfaker@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">+1
                                                          for looping me
                                                          in on that
                                                          thread &
                                                          discussion.  
                                                          I'd like to
                                                          loop in
                                                          someone from
                                                          Labs (probably
                                                          andrebogott)
                                                          about
                                                          purchasing
                                                          GPUs for Labs
                                                          so that we can
                                                          (1) run
                                                          prediction
                                                          models in Beta
                                                          and (2)
                                                          empower our
                                                          volunteers/external
                                                          researchers to
                                                          experiment
                                                          with us. <span
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045HOEnZb"><font
color="#888888">
                                                          <div><br>
                                                          </div>
                                                          <div>-Aaron</div>
                                                          </font></span></div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 16,
                                                          2017 at 12:11
                                                          PM, Adam Baso
                                                          <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">+Corey</div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 16,
                                                          2017 at 11:15
                                                          AM, Adam Baso
                                                          <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">We
                                                          have a need
                                                          for push
                                                          notification
                                                          servers
                                                          already, so
                                                          I've opened a
                                                          thread with
                                                          Mark and
                                                          Faidon about
                                                          getting those
                                                          servers and
                                                          putting in the
Nvidia-recommended cards for TensorFlow (ostensibly for machine vision),
                                                          for the sake
                                                          of simplifying
                                                          assumptions
                                                          about
                                                          hardware. I'm
                                                          awaiting their
                                                          feedback about
                                                          whether we
                                                          actually need
                                                          to split the
                                                          servers. If we
                                                          <i>do</i> need
                                                          to split the
                                                          servers for
                                                          separate
                                                          purposes, then
                                                          I think that
                                                          means we'd
                                                          push back the
                                                          online
                                                          computer
                                                          vision servers
                                                          and GPUs
                                                          purchase to
                                                          early Q4 FY
                                                          17-18 rather
                                                          than just
                                                          getting it
                                                          done Q1 FY
                                                          17-18 - which
                                                          is when we
                                                          need to be
                                                          moving
                                                          aggressively
                                                          on push
                                                          notification
                                                          so it would be
                                                          prudent to
                                                          just get it
                                                          done in one
                                                          fell swoop.
                                                          <div><br>
                                                          </div>
                                                          <div>Aaron, I
                                                          know last week
                                                          you had said
                                                          you'd be <i>open</i> to
                                                          collaborating
                                                          on this...and
                                                          I was quite
                                                          noncommittal!...But
                                                          I appreciate
                                                          your saying
                                                          you'd <i>like</i> to
                                                          collaborate
                                                          here. Would
                                                          you like if I
                                                          loop you on
                                                          that thread
                                                          with Mark and
                                                          Faidon? <b>Any
                                                          others who
                                                          should / would
                                                          like to join
                                                          that thread?</b>
                                                          Just be
                                                          prepared for
                                                          the thread to
                                                          be covering
                                                          two separate
                                                          use cases -
                                                          one on
                                                          cross-platform
                                                          push
                                                          notification
                                                          and one on
                                                          basic machine
                                                          vision.</div>
                                                          <span
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513HOEnZb"><font
color="#888888">
                                                          <div><br>
                                                          </div>
                                                          <div>-Adam</div>
                                                          </font></span></div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 16,
                                                          2017 at 10:57
                                                          AM, Aaron
                                                          Halfaker <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ahalfaker@wikimedia.org"
                                                          target="_blank">ahalfaker@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">OK
                                                          so I think
                                                          we'll want to
                                                          (1) get a GPU
                                                          in the stat
                                                          boxes ASAP and
                                                          (2) decide
                                                          whether we
                                                          want to plan
                                                          GPU resources
                                                          in Prod for
                                                          FY2018 or
                                                          FY2019.  
                                                          <div><br>
                                                          </div>
                                                          <div>For (2),
                                                          I don't think
                                                          my team's
                                                          current plans
                                                          will bring us
                                                          to using the
                                                          GPU in
                                                          production in
                                                          the next year,
                                                          but I suspect
                                                          that Reading
                                                          may want to
                                                          push some work
                                                          re. image
                                                          processing in
                                                          that time.  If
                                                          that's the
                                                          case, I want
                                                          my team to be
                                                          able to
                                                          collaborate
                                                          and support
                                                          getting that
                                                          deployed in
                                                          prod.  To do
                                                          this well, I
                                                          want GPU
                                                          resources in
                                                          Wikimedia Labs
                                                          too.  That
                                                          sounds like a
                                                          whole other
                                                          can of worms.
                                                           
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Wed, Feb 15,
                                                          2017 at 4:40
                                                          PM, Ellery
                                                          Wulczyn <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ewulczyn@wikimedia.org"
                                                          target="_blank">ewulczyn@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Having
                                                          GPUs for
                                                          training
                                                          should be
                                                          sufficient for
                                                          now, although
                                                          if we end up
                                                          getting a ton
                                                          of use, using
                                                          GPUs could be
                                                          a lot faster
                                                          and probably
                                                          cheaper than
                                                          provisioning
                                                          the same
                                                          compute amount
                                                          of CPUs.</div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 12:21
                                                          PM, Aaron
                                                          Halfaker <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ahalfaker@wikimedia.org"
                                                          target="_blank">ahalfaker@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">If
                                                          we only need
                                                          the GPU for
                                                          model
                                                          training,
                                                          it'll be OK to
                                                          use one stat
                                                          box.  If we
                                                          need the GPU
                                                          for
                                                          scoring/predictions,
                                                          we'll need a
                                                          whole new
                                                          hardware
                                                          plan. </div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 1:06
                                                          PM, Andrew
                                                          Otto <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:otto@wikimedia.org" target="_blank">otto@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Hm, a good rule of thumb
                                                          is: If it can
                                                          be offline or
                                                          not running
                                                          and not affect
                                                          end users,
                                                          then it is
                                                          probably fine
                                                          to use a stat
                                                          box.</div>
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
                                                          </div>
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
                                                          </div>
                                                          </div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 1:56
                                                          PM, Adam Baso
                                                          <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Got
                                                          it - I take
                                                          that to be the
                                                          case whether
                                                          it's batched
                                                          operation
                                                          (e.g., on
                                                          millions of
                                                          files) or it's
                                                          more of an
                                                          in-the-user-flow
                                                          sort of thing.
                                                          Is that right?</div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 10:55
                                                          AM, Andrew
                                                          Otto <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:acotto@gmail.com" target="_blank">acotto@gmail.com</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">I’d
                                                          say, if you
                                                          are going to
                                                          just to
                                                          analytics type
                                                           stuff, then
                                                          the single
                                                          stat machine
                                                          will do.  If
                                                          you want to
                                                          depend on a
                                                          GPU for a
                                                          end-user
                                                          production
                                                          thing, then
                                                          you’ll have to
                                                          work with ops
                                                          to find
                                                          another place
                                                          to run it. :/
                                                          :)</div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 11:40
                                                          AM, Aaron
                                                          Halfaker <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ahalfaker@wikimedia.org"
                                                          target="_blank">ahalfaker@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Ellery,
                                                          will we need
                                                          the GPUs in
                                                          order to use a
                                                          NN or will we
                                                          only need it
                                                          for training
                                                          models?</div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 10:21
                                                          AM, Adam Baso
                                                          <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">I
                                                          envision two
                                                          primary uses:
                                                          <div><br>
                                                          </div>
                                                          <div>1) Large
                                                          scale batch
                                                          offline
                                                          processing of
                                                          existing media
                                                          assets so that
                                                          the material
                                                          is ready for
                                                          curatorial
                                                          flows.</div>
                                                          <div>2) As
                                                          part of an end
                                                          user flow
                                                          where multiple
                                                          concurrent
                                                          users are
                                                          uploading
                                                          media and
                                                          verifying and
                                                          adding
                                                          structured
                                                          data on the
                                                          fly as part of
                                                          production
                                                          use.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Can both
                                                          of these be
                                                          done on stats
                                                          machines?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Ought we
                                                          have GPU
                                                          acceleration
                                                          in two
                                                          machines
                                                          instead of one
                                                          machine?</div>
                                                          <span
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639HOEnZb"><font
color="#888888">
                                                          <div><br>
                                                          </div>
                                                          <div>-Adam</div>
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </font></span></div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 8:15
                                                          AM, Andrew
                                                          Otto <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:otto@wikimedia.org" target="_blank">otto@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Oh ya!  If you have a use
                                                          case for this
                                                          too, all the
                                                          better!</div>
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
                                                          </div>
                                                          <div
                                                          class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Do you need it for
                                                          analytics type
                                                          work?  Or do
                                                          you need it to
                                                          process stuff
                                                          for a
                                                          production
                                                          feature?</div>
                                                          </div>
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639m_-4451679102871761861HOEnZb">
                                                          <div
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639m_-4451679102871761861h5">
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Thu, Feb 2,
                                                          2017 at 9:10
                                                          AM, Aaron
                                                          Halfaker <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:ahalfaker@wikimedia.org"
                                                          target="_blank">ahalfaker@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div>Hi Adam,</div>
                                                          <div><br>
                                                          </div>
                                                          <div>+ a bunch
                                                          of CCs</div>
                                                          <div><br>
                                                          </div>
                                                          Last I heard,
                                                          Dario thought
                                                          we might be
                                                          able to cover
                                                          the cost with
                                                          Research
                                                          budget.  Otto
                                                          thought that
                                                          we could get a
                                                          top of line
                                                          GPU and load
                                                          it into an
                                                          analytics
                                                          machine some
                                                          time in Q4 of
                                                          this year. 
                                                          Ellery was
                                                          planning to
                                                          use 3rd party
                                                          GPU processing
                                                          services until
                                                          it was ready.
                                                           
                                                          <div><br>
                                                          <div>
                                                          <div>See <a
                                                          moz-do-not-send="true"
href="https://phabricator.wikimedia.org/T148843" target="_blank">https://phabricator.wikime<wbr>dia.org/T148843</a></div>
                                                          </div>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>-Aaron</div>
                                                          </div>
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Wed, Feb 1,
                                                          2017 at 6:01
                                                          PM, Adam Baso
                                                          <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:abaso@wikimedia.org" target="_blank">abaso@wikimedia.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                          rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Aaron,
                                                          okay if I
                                                          schedule a 20
                                                          minute meeting
                                                          with you to
                                                          talk servers
                                                          with GPUs?
                                                          <div><br>
                                                          </div>
                                                          <div>Broadly,
                                                          I'm trying to
                                                          figure out
                                                          what server
                                                          CapEx I need
                                                          to ask of Mark
                                                          (e.g., for
                                                          TensorFlow
                                                          object
                                                          detection in
                                                          anticipation
                                                          of work later
                                                          in FY 17-18 /
                                                          earlier FY
                                                          18-19). I had
                                                          asked him the
                                                          other day
                                                          about when he
                                                          needs requests
                                                          for next FY,
                                                          and he
                                                          basically said
                                                          the sooner the
                                                          better.</div>
                                                          <span
class="m_6847163355653223882gmail-m_-5394071749873894282m_245415591453473483gmail-m_-4240006389916125709m_-1941634467393186974m_-4826646701549151045m_7327860486425803260m_-7609473567412279513m_-7468477604771789240m_-6986456113631074273m_5713288028632980946m_4719900974687532620m_6637965112908006474m_-7130635102425802995m_6867535296650873161m_-1614115059232508639m_-4451679102871761861m_-659134508675495638m_4072067741347120972HOEnZb"><font
color="#888888">
                                                          <div><br>
                                                          </div>
                                                          <div>-Adam</div>
                                                          </font></span></div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                        </blockquote>
                                                      </div>
                                                    </div>
                                                  </blockquote>
                                                  <p><br>
                                                  </p>
                                                </div>
                                              </div>
                                            </div>
                                          </blockquote>
                                        </div>
                                        <br>
                                      </div>
                                    </div>
                                  </div>
                                </blockquote>
                                <p><br>
                                </p>
                              </div>
                            </div>
                          </div>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            Labs-admin mailing list<br>
            <a moz-do-not-send="true"
              href="mailto:Labs-admin@lists.wikimedia.org">Labs-admin@lists.wikimedia.org</a><br>
            <a moz-do-not-send="true"
              href="https://lists.wikimedia.org/mailman/listinfo/labs-admin"
              rel="noreferrer" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-admin</a><br>
            <br>
          </blockquote>
        </div>
      </div>
    </blockquote>
    <p><br>
    </p>
  </body>
</html>