This one was real quick & smooth -- it certainly landed more quickly than I expected. Petr did a great job building the specialized Kafka consumer for this service, Jon built the actual trending logic, and Marko worked with ops on the service deployment. Another great collaboration, congratulations to all involved! I'm also looking forward to see how this data is going to be used creatively. Cheers, Gabriel On Tue, Dec 13, 2016 at 2:21 PM, Adam Baso <abaso(a)wikimedia.org> wrote: Yay! Great stuff (cc reading web team). Thank you Marko and Petr for making this a reality! Nice work, all! _______________________________________________ Services mailing list Services(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/services -- Gabriel Wicke Principal Engineer, Wikimedia Foundation

I just mention there are a few hiccups as can be expected and it's not quite behaving as we would hope... but it's live and only going to get better :) https://phabricator.wikimedia.org/T153122

On Tue, Dec 13, 2016 at 3:28 PM Corey Floyd <cfloyd(a)wikimedia.org> wrote: Gabriel already said it - ​But i​ t’s fantastic to see how everyone pulled together to get this out! Thanks again to Marko ​and​ Petr for all the support in making this happen. And Jon for ​ pushing ​so hard to take his idea from prototype to a RESTBase service. I spoke with Josh about the new service earlier today and iOS is pretty excited to start playing with this to see what they can do!​ ​ I already can’t wait to see what everyone builds next quarter! Congrats again! On Tue, Dec 13, 2016 at 6:07 PM Gabriel Wicke <gwicke(a)wikimedia.org> wrote: This one was real quick & smooth -- it certainly landed more quickly than I expected. Petr did a great job building the specialized Kafka consumer for this service, Jon built the actual trending logic, and Marko worked with ops on the service deployment. Another great collaboration, congratulations to all involved! I'm also looking forward to see how this data is going to be used creatively. Cheers, Gabriel On Tue, Dec 13, 2016 at 2:21 PM, Adam Baso <abaso(a)wikimedia.org> wrote: Yay! Great stuff (cc reading web team). Thank you Marko and Petr for making this a reality! Nice work, all! _______________________________________________ Services mailing list Services(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/services -- Gabriel Wicke Principal Engineer, Wikimedia Foundation _______________________________________________ Services mailing list Services(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/services

Very cool, good job everyone, and congratulations! On Thu, Dec 15, 2016 at 1:45 PM, Marko Obrovac <mobrovac(a)wikimedia.org> wrote: On 14 December 2016 at 20:39, Jon Robson <jrobson(a)wikimedia.org> wrote: I just mention there are a few hiccups as can be expected and it's not quite behaving as we would hope... but it's live and only going to get better :) https://phabricator.wikimedia.org/T153122 By switching the Kafka consumer to a different consuming mode the issue has been mitigated temporarily (see the ticket for a long-term solution). I feel confident that we are now good to go through the 4-week deployment freeze, technically speaking. Feel free to use this time to explore the service's results over that period and work on any product- or logic-related shortcomings (but keep in mind the no-deploy rule during the freeze). Jon, you mentioned a script to pull results from the service every 10 minutes. Beware that its output is cached for 30 minutes in Varnish. We started with a higher caching value to ensure stability. Let's revisit this interval after the freeze, i.e. after AllHands. Cheers, Marko On Tue, Dec 13, 2016 at 3:28 PM Corey Floyd <cfloyd(a)wikimedia.org> wrote: Gabriel already said it - ​But i​ t’s fantastic to see how everyone pulled together to get this out! Thanks again to Marko ​and​ Petr for all the support in making this happen. And Jon for ​ pushing ​so hard to take his idea from prototype to a RESTBase service. I spoke with Josh about the new service earlier today and iOS is pretty excited to start playing with this to see what they can do!​ ​ I already can’t wait to see what everyone builds next quarter! Congrats again! On Tue, Dec 13, 2016 at 6:07 PM Gabriel Wicke <gwicke(a)wikimedia.org> wrote: This one was real quick & smooth -- it certainly landed more quickly than I expected. Petr did a great job building the specialized Kafka consumer for this service, Jon built the actual trending logic, and Marko worked with ops on the service deployment. Another great collaboration, congratulations to all involved! I'm also looking forward to see how this data is going to be used creatively. Cheers, Gabriel On Tue, Dec 13, 2016 at 2:21 PM, Adam Baso <abaso(a)wikimedia.org> wrote: Yay! Great stuff (cc reading web team). Thank you Marko and Petr for making this a reality! Nice work, all! _______________________________________________ Services mailing list Services(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/services -- Gabriel Wicke Principal Engineer, Wikimedia Foundation _______________________________________________ Services mailing list Services(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/services -- Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation --

On Thu, Dec 15, 2016 at 4:45 AM Marko Obrovac <mobrovac(a)wikimedia.org> wrote: Yup we should definitely revisit this. I'm yet to find overlap with the results in https://trending.wmflabs.org/ which I think is due to this delay. Right now the API tells me http://en.m.wikipedia.org/wiki/Velupillai_Prabhakaran is the most edited but the last edit to that page was 50 minutes ago. Is it possible that this lag is higher than 30 in practice? Also, is the processing of old events (the recovery step) synchronous? I'm a little worried when we hit the API we're not processing the last 1 hour of edits before returning results. Sadly (although maybe not so much for the world) nothing has really trending over the last few days so it's hard to verify I'm seeing what I expect us to.