Hello,
Here's a quick status update about the rollout of HHVM.
* Since migrating
test.wikipedia.org to HHVM exactly one week ago, we've
had just one segfault (reported upstream: <
https://github.com/facebook/hhvm/issues/3438>)t;). That's very good.
* Giuseppe shared some benchmarks in an e-mail to wikitech: <
https://lists.wikimedia.org/pipermail/wikitech-l/2014-August/078034.html>gt;.
Also very good.
* Re-imaging an app server was surprisingly painful, in that Giuseppe and I
had to perform a number of manual actions to get the server up-and-running.
This sequence of steps was poorly automated: update server's salt key ->
synchronize Trebuchet deployment targets -> fetch scap scripts -> run
sync-common to fetch MediaWiki -> rebuild l10n cache. Doing this much
manual work per app server isn't viable. Giuseppe and I fixed some of this
but there's more work to be done.
* Mark submitted a series of patches (principally <
https://gerrit.wikimedia.org/r/#/c/152903/>) to create a service IP and
Varnish backend for an HHVM app server pool, with Giuseppe and Brandon
providing feedback and amending the change. Brandon thinks it looks good
and he may be able to deploy it some time next week.
* The patch routes requests that are tagged with a specific cookie to the
HHVM backends. Initially, we'll ask you (Wikimedia engineers and
technically savvy / adventurous editors) to opt-in to help with testing by
setting the cookie explicitly. The next step after that will be to divert a
fraction of general site traffic to those backends. When exactly that
happens will depend on how many bugs the next round of testing will uncover.
* We'll be adding servers to HHVM pool as we reimage them.
* Tim is looking at modifying the profiling feature of LuaSandbox to work
with HHVM. We currently disable it, due to <
https://bugzilla.wikimedia.org/show_bug.cgi?id=68413>gt;. (Feedback from users
re: how important is this feature to you would be useful).
* Giuseppe and Filippo noticed a memory leak on the HHVM job runner
(mw1053). Aaron is trying to isolate it. Tracked in bug <
https://bugzilla.wikimedia.org/show_bug.cgi?id=69428>gt;.
* Giuseppe is on vacation for the week of 8/18 - 8/22; Filippo is the
point-person for HHVM in TechOps.