Hmm. Maybe it's easier to send the SM out and deal with the tech fine print by having people read a full write-up from the provided links?
I mainly wish that we could use some relatively safe, apolitical, uncontroversial article for the example.
Pine
On Fri, Jun 12, 2015 at 11:03 PM, Jeremy Baron jeremy@tuxmachine.com wrote:
On Jun 13, 2015 1:06 AM, "Pine W" wiki.pine@gmail.com wrote:
Perhaps we should take the discussion of how best to measure page
rendering performance to Wikitech. Would that be ok with you?
We could. Or maybe the research or analytics lists list would be better.
But should that block getting the SM out the door?
I agree that there is value in continuity, but remember that Wikipedia
articles change over time, so unless someone is using a specific rev for measuring every time that they make a change to how the page renders, then there is likely to be at least some unreliability in the measurement.
Obviously we could double check this but I'd wager that Obama's cite count would have trended upward in the last couple years. (so e.g. if we compared older HHVM vs. newer HHVM with constant Obama rev the gains would be more extreme than if we did older HHVM + older Obama vs. newer HHVM + newer Obama)
Anyway, it should be technically feasible to run benchmarks for old software again against the new revisions. In this case the author wasn't actually comparing to past numbers. (I think...) Only generating his own new numbers for a constant rev. And anyway, the comparison to old numbers wouldn't be meaningful (without rerunning them) because hardware's not constant.
Technical factors like bandwidth and geolocation may also be involved in
skewing the validity of comparisons.
I can't imagine a scenario where that's relevant. Does anyone benchmark specific articles over the public internet? vs. running the client on the same local network as the server.
For most citations, there appears to be a manually updated list here:
https://en.wikipedia.org/wiki/Wikipedia:Articles_with_the_most_references
not just manually updated but each entry has its own separate update date??? hrmmm, Obama is listed lower on that list than another article with Obama in title…
-Jeremy
P.S. the recently released slow parse logs may be useful for choosing articles to track over time. https://phabricator.wikimedia.org/T98563
Social-media mailing list Social-media@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/social-media