Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options are:
1) Continue requiring that MediaWiki uses a common set of HHVM and Zend PHP. This, however, is a dead end and will make things progressively harder as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
2) Declare our loyalty to HHVM. This will result in most of our current users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
3) Revert WMF to Zend and forget about HHVM. This will result in performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
I personally think that 3) is the only viable option in the long run. What do you think?
---- [1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
On Sep 18, 2017, at 1:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options are:
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively harder as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?
I personally think that 3) is the only viable option in the long run. What do you think?
[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I think option 2 will be impossible. As that will decrease the user base of mediawiki by a lot. I think we should go with option 3 as php 7 has had increased in performances. On Monday, 18 September 2017, 22:03:20 BST, James Hare jamesmhare@gmail.com wrote:
On Sep 18, 2017, at 1:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options are:
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively harder as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
Do we have performance benchmarks on 5.3 vs. HHVM vs. 5.6 vs. 7?
I personally think that 3) is the only viable option in the long run. What do you think?
[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
I actually predicted this and wrote up my suggestions here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/...
I will *not* be making this my position statement at the dev summit -- only one per person, so many positions to take, this one didn't make the cut! -- but I would like to strongly encourage someone to submit a position statement relating to this issue (could be any of max's three positions) to the dev summit. I think it is an important issue for our foundation and community to discuss; certainly very relevant to the next 15 years of the project. --scott
On Mon, Sep 18, 2017 at 4:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
In particular, I've heard good things about PHP 7 performance. Someone less lazy than I am at 5pm might want to do some research on that though.
I can say that PHP 7 locally runs unit tests significantly faster than PHP 5.6, although that's not really a representative workload for running a website.
Hi!
I can say that PHP 7 locally runs unit tests significantly faster than PHP 5.6, although that's not really a representative workload for running a website.
PHP 7 is faster than 5.6, and probably be on almost any workload, from my experience (the degree of speedup would vary of course). As for comparison with hhvm, I heard various reports, but I think spending some time on seriously testing it (I mean creating proper production setup, and directing either captured/replayed or real traffic to it) is the best way to go.
On Monday, September 18, 2017, Max Semenik maxsem.wiki@gmail.com wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options are:
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively
harder
as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
while
5.6 and 7 provided nice performance improvements.
I personally think that 3) is the only viable option in the long run. What do you think?
[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Well i agree that 3 seems likely the best long term option, we will probably be doing 1 in the short term. However I think it would be prudent to wait and see how things turn out in the short term before comitting to any path. The landscape can still shift quite a lot before the time comes when its impractical to continue doing 1.
-- bawolff
Other tools we are using, such as Phabricator, will also be following HHVM to Hack (presumably). Facebook, as the largest (by engineering budget) production user of PHP, will certainly have an outsized influence on the direction of the platform and surrounding ecosystem. If we follow path #3 we're probably also committing to supporting Zend development long-term as the primary production user. --scott
On Mon, Sep 18, 2017 at 5:26 PM, Brian Wolff bawolff@gmail.com wrote:
On Monday, September 18, 2017, Max Semenik maxsem.wiki@gmail.com wrote:
Today, the HHVM developers made an announcement[1] that they have plans
of
ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options
are:
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively
harder
as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
while
5.6 and 7 provided nice performance improvements.
I personally think that 3) is the only viable option in the long run.
What
do you think?
[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Well i agree that 3 seems likely the best long term option, we will probably be doing 1 in the short term. However I think it would be prudent to wait and see how things turn out in the short term before comitting to any path. The landscape can still shift quite a lot before the time comes when its impractical to continue doing 1.
-- bawolff _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Mon, Sep 18, 2017 at 2:33 PM, C. Scott Ananian cananian@wikimedia.org wrote:
Other tools we are using, such as Phabricator, will also be following HHVM to Hack (presumably).
To the contrary, Phabricator has never supported HHVM.
Hi!
Other tools we are using, such as Phabricator, will also be following HHVM to Hack (presumably).
I am not sure this is the case. Mozilla recently declared they want to use Phabricator[1], but I heard no mention about HHVM. Which makes me think that one stays on PHP. Also, Phabricator is now independent from Facebook, afaik, since it's developers have separate company, Phacility.
[1] https://wiki.mozilla.org/Phabricator
Hi!
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively harder as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
This will probably ultimately happen, but given the PHP version stats, e.g. here: https://seld.be/notes/php-versions-stats-2017-1-edition
I think we have several years at least before that starts becoming an issue. Realistically, if you write distributable PHP code now, targeting 7.1 gets you only 17% of the users, so you'd do either 7.0 (gets you about half) or more likely even 5.6. Extending this trend (I know, dangerous, but let's assume) if 7.2 is released somewhere around Dec 2017-Jan 2018, 7.3 would probably not happen before around 2019. If that would have features not supported in HHVM, that means we'd have to worry around 2021 when people would start releasing components targeting it. So we have about 3 years to get the solution - *if* 7.3 has features not supported by HHVM.
Note that this statistics is for Composer users, which means it is probably skewed towards modern versions, since people using PHP 5.3 probably don't use Composer too much in general. OTOH, since we do use Composer, that appears to be appropriate for our case.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
I don't think this is a good idea, for reasons that seem obvious to me (but I can elaborate if necessary).
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
I think we should evaluate 7.1 or 7.2 (provided we don't have any runtime issues with them) and see how performance looks like ASAP (with opcache, of course). Of there's some help needed, or there are some specific issues that are blockers, I think Zend team would be glad to talk to us. If needed, I could probably help with establishing the contacts.
Hey,
Going with option 2 would be a rather big f***-*** to the non-WMF part of the MediaWiki community. Furthermore, it would limit usage of common PHP libraries, which sooner or later will end up using features not available in HHVM. This, together with the already outlined reasons, makes me concur with HHVM not being a viable long term option.
If only WMF was running PHP 7.1 already... guess I'll have more luck asking for a pony though :)
Cheers
-- Jeroen De Dauw | https://entropywins.wtf | https://keybase.io/jeroendedauw Software craftsmanship advocate ~=[,,_,,]:3
On 19/09/17 06:58, Max Semenik wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
The HHVM team did tell us privately that they were planning on changing their strategy, basically as you describe it above. The surprising things for me in this announcement were:
* The plan to also drop PHP 5 compatibility, on a short timeline (1 year). * Rather than "drifting away" from PHP, their top priority plans include removing core language features like references and destructors.
While this does not mean that we need to take an action immediately, eventually we will have to decide something.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
I personally think that 3) is the only viable option in the long run. What do you think?
Yes, I think it's the only viable option.
I'll run a benchmark, but I don't see how it could influence the decision. It'll be more for capacity planning.
-- Tim Starling
Hi!
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
Wow. I can see why they're doing it (those are sources of most complications ans security issues in the language, references being especially weird and tricky). But dropping those would certainly mean very heavy incompatibility with PHP, by which point it'd be completely separate language. Which probably excludes Max's #2 from consideration altogether.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
Should it be on the TechCom agenda and should we have some public discussion on IRC in RFC format for this soon?
On Mon, Sep 18, 2017 at 5:44 PM Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
Wow. I can see why they're doing it (those are sources of most complications ans security issues in the language, references being especially weird and tricky). But dropping those would certainly mean very heavy incompatibility with PHP, by which point it'd be completely separate language. Which probably excludes Max's #2 from consideration altogether.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
Should it be on the TechCom agenda and should we have some public discussion on IRC in RFC format for this soon?
I see zero reason for us to go through all the formalities, unless we want to really. I have yet to see anyone (on list, or on IRC anywhere at all today) where anyone suggested (2) was a good idea at all. It's a horrifically bad idea. I don't consider it remotely viable and would do everything possible to veto such a move.
(1) is impossible as a long-term goal.
So this basically means we're going the route of (3) which is the only way we can actually expect people to use MediaWiki outside of Wikimedia. And considering we never really implemented any HHVM/Hack specific features (as far as I know) it means there's no reason we have to continue to support HHVM at all once WMF has moved off of it.
It's been a fun experiment for the past couple of years, but it's time to move on.
-Chad
On Mon, Sep 18, 2017 at 9:51 PM, Chad innocentkiller@gmail.com wrote:
I see zero reason for us to go through all the formalities, unless we want to really. I have yet to see anyone (on list, or on IRC anywhere at all today) where anyone suggested (2) was a good idea at all. It's a horrifically bad idea.
Technically, I did outline the arguments for (2), earlier on this thread. It was a bit allegorical, though: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/... This should be seriously considered, not just dismissed.
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining control over our core runtime is important; I think we should at least discuss long-term contingencies if either goes down. Obviously, our future was most stable when we (briefly!) had a choice between two strong runtimes... but that opportunity seems to be vanishing. Practically speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is likely that the project with the most funding will continue to have the better performance.
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to follow PHP 7. Etsy also shifted to HHVM, do we know what their plans are? --scott
On Sep 19, 2017, at 8:21 AM, C. Scott Ananian cananian@wikimedia.org wrote:
On Mon, Sep 18, 2017 at 9:51 PM, Chad innocentkiller@gmail.com wrote:
I see zero reason for us to go through all the formalities, unless we want to really. I have yet to see anyone (on list, or on IRC anywhere at all today) where anyone suggested (2) was a good idea at all. It's a horrifically bad idea.
Technically, I did outline the arguments for (2), earlier on this thread. It was a bit allegorical, though: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/... This should be seriously considered, not just dismissed.
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining control over our core runtime is important; I think we should at least discuss long-term contingencies if either goes down. Obviously, our future was most stable when we (briefly!) had a choice between two strong runtimes... but that opportunity seems to be vanishing. Practically speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is likely that the project with the most funding will continue to have the better performance.
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to follow PHP 7. Etsy also shifted to HHVM, do we know what their plans are? --scott
This is what I want to know as well: who is the community of HHVM users outside of us? What are their own internal reactions? Keep in mind that businesses like Etsy don't have to support third party downstream users as we do; "I guess we'll use Hack now" is a more plausible response than it would be for us.
But if others are indeed like us and are balking at the plan to drop PHP, it would be worth looking into how we can pool resources to "save" HHVM, or create a runtime that maintains the performance improvements of HHVM while maintaining PHP support.
Incidentally, how much work has been done on incorporating HHVM's improvements back into Zend?
-- (http://cscott.net) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi!
Incidentally, how much work has been done on incorporating HHVM's improvements back into Zend?
Depends on which ones you're talking about. Syntax ones may or may not find its way into PHP, but performance ones would probably be completely different from HHVM - i.e. the resulting performance may or may not be on par or better, but reusing most of the performance work on HHVM in PHP would not be possible due to completely different engine internals.
So pretty much all that can be taken from HHVM into PHP would be "this syntax looks like a good idea, let's reimplement it".
On Tue, Sep 19, 2017 at 3:21 PM, C. Scott Ananian cananian@wikimedia.org wrote:
On Mon, Sep 18, 2017 at 9:51 PM, Chad innocentkiller@gmail.com wrote:
I see zero reason for us to go through all the formalities, unless we want to really. I have yet to see anyone (on list, or on IRC anywhere at all today) where anyone suggested (2) was a good idea at all. It's a horrifically bad idea.
Technically, I did outline the arguments for (2), earlier on this thread. It was a bit allegorical, though: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/... This should be seriously considered, not just dismissed.
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining control over our core runtime is important; I think we should at least discuss long-term contingencies if either goes down. Obviously, our future was most stable when we (briefly!) had a choice between two strong runtimes... but that opportunity seems to be vanishing. Practically speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is likely that the project with the most funding will continue to have the better performance.
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to follow PHP 7. Etsy also shifted to HHVM, do we know what their plans are? --scott
-- (http://cscott.net) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I disagree. I don't think its useful or possible to try to forecast technical trends 15 years out. 15 years from now, it is just as likely that facebook will be as relevant as myspace is today, as it is that facebook will go full cyberpunk dystopia on us and rule the world. I don't think we can realistically predict anything 15 years out.
On Tue, Sep 19, 2017 at 11:33 AM, bawolff bawolff+wn@gmail.com wrote:
I disagree. I don't think its useful or possible to try to forecast technical trends 15 years out. 15 years from now, it is just as likely that facebook will be as relevant as myspace is today, as it is that facebook will go full cyberpunk dystopia on us and rule the world. I don't think we can realistically predict anything 15 years out.
That's why I argued we should consider the code quality of both, and determine which we'd feel most comfortable supporting on our own in the case our crystal balls are wrong and we end up holding the bag for the runtime. (This is just one factor to consider; my argument is that we should carefully enumerate the various factors before making an evaluation, not just dismiss any of the options out of hand.) --scott
*Continue* to have the best performance? A quick search for benchmarks suggests that 7.1 outperforms HHVM more often than not.
Betting our future on Facebook because they have most money right now seems unwise. Which of the two has the larger developer community?
On 19 Sep 2017, at 16:21, C. Scott Ananian cananian@wikimedia.org wrote:
On Mon, Sep 18, 2017 at 9:51 PM, Chad innocentkiller@gmail.com wrote:
I see zero reason for us to go through all the formalities, unless we want to really. I have yet to see anyone (on list, or on IRC anywhere at all today) where anyone suggested (2) was a good idea at all. It's a horrifically bad idea.
Technically, I did outline the arguments for (2), earlier on this thread. It was a bit allegorical, though: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2018/Writing_Tips/... This should be seriously considered, not just dismissed.
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining control over our core runtime is important; I think we should at least discuss long-term contingencies if either goes down. Obviously, our future was most stable when we (briefly!) had a choice between two strong runtimes... but that opportunity seems to be vanishing. Practically speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is likely that the project with the most funding will continue to have the better performance.
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to follow PHP 7. Etsy also shifted to HHVM, do we know what their plans are? --scott
-- (http://cscott.net) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Sep 19, 2017 at 9:21 AM, C. Scott Ananian cananian@wikimedia.org wrote:
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to follow PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
Etsy 'experimented' with HHVM [0] and then eventually switched to PHP7 as their primary runtime. The blog posts about this are a little scattered, but Rasmus spoke about it [1] and Etsy started the phan project [2].
For what it's worth, my opinion is that PHP is an actual FLOSS software project with years of history and core contributions from Zend who make their living with PHP. HHVM is a well funded internal project from Facebook that has experimented with FLOSS but ultimately is controlled by the internal needs of Facebook. For me the choice here is obviously to back the community driven FLOSS project and help them continue to thrive.
[0]: https://codeascraft.com/2015/04/06/experimenting-with-hhvm-at-etsy/ [1]: https://codeascraft.com/speakers/rasmus-lerdorf-deploying-php-7/ [2]: https://github.com/phan/phan
Bryan
On Tue, Sep 19, 2017 at 11:34 AM, Bryan Davis bd808@wikimedia.org wrote:
On Tue, Sep 19, 2017 at 9:21 AM, C. Scott Ananian cananian@wikimedia.org wrote:
There are other big users of HHVM -- do we know what other members of the larger community are doing? We've heard that Phabricator intends to
follow
PHP 7. Etsy also shifted to HHVM, do we know what their plans are?
Etsy 'experimented' with HHVM [0] and then eventually switched to PHP7 as their primary runtime. The blog posts about this are a little scattered, but Rasmus spoke about it [1] and Etsy started the phan project [2].
I got confirmation on twitter: https://twitter.com/jazzdan/status/910162545805336576
For what it's worth, my opinion is that PHP is an actual FLOSS software project with years of history and core contributions from Zend who make their living with PHP. HHVM is a well funded internal project from Facebook that has experimented with FLOSS but ultimately is controlled by the internal needs of Facebook. For me the choice here is obviously to back the community driven FLOSS project and help them continue to thrive.
Fair enough. My point is just that we should stop and reflect that this is a major inflection point. Language choices are sticky, so this decision will have significant long-term implications. We should at least stop to evaluate PHP7 vs Hack and determine which is a better fit for our codebase, and do due diligence on both sides (count how many engineers, how many open source contributors, commit rates, etc). HHVM has been flirting with a LLVM backend, and LLVM itself has quite a large and active community. The PHP community has had issues with proper handling of security patches in the past. I'm suggesting to proceed cautiously and have a proper discussion of all the factors involved instead of over-simplifying this to "community" vs "facebook".
For example, the top-line github stats are: hhvm: 504 contributors (24,192 commits) php-src: 496 contributors (104,566 commits)
HHVM seems to have a larger community of contributors despite a much shorter active life. But note that the PHP github mirror has been broken since Jul 29 (!). In the past 6 days I count 8 distinct contributors to php-src, and 10 distinct contributors in the past *two days* to hhvm (one of whom contributed an OCAML frontend(!)). These are just hand-wavy figures; ideally we should try to determine how many of the recent contributors to each project are employed by Facebook and/or Zend.
I think there's room for a reasonable debate. --scott
On Tue, Sep 19, 2017 at 9:40 AM C. Scott Ananian cananian@wikimedia.org wrote:
Fair enough. My point is just that we should stop and reflect that this is a major inflection point. Language choices are sticky, so this decision will have significant long-term implications. We should at least stop to evaluate PHP7 vs Hack and determine which is a better fit for our codebase, and do due diligence on both sides (count how many engineers, how many open source contributors, commit rates, etc). HHVM has been flirting with a LLVM backend, and LLVM itself has quite a large and active community. The PHP community has had issues with proper handling of security patches in the past. I'm suggesting to proceed cautiously and have a proper discussion of all the factors involved instead of over-simplifying this to "community" vs "facebook".
I'm not trying to simplify this into community vs. facebook. And let's also be clear: we never chose HHVM for Hack. We don't use Hack. The one experiment I had at trying Hack never panned out. MediaWiki is in PHP, not Hack.
The *only* reason we're having a language discussion is because HHVM has announced that they're abandoning PHP in favor of Hack. If someone had some to the list last week and said "Hey let's abandon PHP for XYZLang" they would've been rightly laughed off.
The debate here is between runtimes for PHP, and on the long enough timescale there's only one option. PHP has a long-standing history of being a viable runtime. HHVM does not.
I don't see this as an A/B choice at all, but rather a clear path forward. So sure: let's have an RfC/TechComm meeting to work out the details, but let's not pretend that option #2 is even remotely viable. It is not.
-Chad
On Tue, Sep 19, 2017 at 1:17 PM, Chad innocentkiller@gmail.com wrote:
be clear: we never chose HHVM for Hack. We don't use Hack. The one experiment I had at trying Hack never panned out. MediaWiki is in PHP, not Hack.
To be super clear: MediaWiki is in *PHP5*. The choices are:
1) MediaWiki will always be in PHP5. 2) MediaWiki will eventually migrate to PHP7, or 3) MediaWiki will eventually migrate to Hack.
Is anyone arguing for #1?
So we've got two backwards-incompatible choices to make, eventually. --scott
On Tue, Sep 19, 2017 at 10:28 AM C. Scott Ananian cananian@wikimedia.org wrote:
On Tue, Sep 19, 2017 at 1:17 PM, Chad innocentkiller@gmail.com wrote:
be clear: we never chose HHVM for Hack. We don't use Hack. The one experiment I had at trying Hack never panned out. MediaWiki is in PHP,
not
Hack.
To be super clear: MediaWiki is in *PHP5*. The choices are:
- MediaWiki will always be in PHP5.
- MediaWiki will eventually migrate to PHP7, or
- MediaWiki will eventually migrate to Hack.
Is anyone arguing for #1?
So we've got two backwards-incompatible choices to make, eventually.
I see everyone saying #1 for now and #2 down the road. You're the only one here who seems to think #3 is even worth considering. The rest of us have--rightly--dismissed it already.
-Chad
On Tuesday, September 19, 2017, C. Scott Ananian cananian@wikimedia.org wrote:
On Tue, Sep 19, 2017 at 1:17 PM, Chad innocentkiller@gmail.com wrote:
be clear: we never chose HHVM for Hack. We don't use Hack. The one experiment I had at trying Hack never panned out. MediaWiki is in PHP,
not
Hack.
To be super clear: MediaWiki is in *PHP5*. The choices are:
- MediaWiki will always be in PHP5.
- MediaWiki will eventually migrate to PHP7, or
- MediaWiki will eventually migrate to Hack.
Is anyone arguing for #1?
So we've got two backwards-incompatible choices to make, eventually. --scott
-- (http://cscott.net)
No, MediaWiki is written in a subset of php5 which is forwards compatible with php7. The backwards incompatible things are by in large things no sane code would rely on (http://php.net/manual/en/migration70.incompatible.php) There are people who are using mediawiki with php7 right now without complaint. Presumably eventually we will move to needing php7, just like once we moved from php4 to php5, but thats really just a version requirement increment.
Sure there may be backwards incompatible things that come up, but thats nothing new. E.g. We used to have a class named Namespace. Then php 5.3 came along and its now MWNamespace. All signs point to php7 incompatibilities to be of this form. -- Brian
Hi,
On 09/19/2017 10:27 AM, C. Scott Ananian wrote:
To be super clear: MediaWiki is in *PHP5*. The choices are:
Well, kind of. MediaWiki has full support for PHP 7 while still maintaining PHP 5 support.
- MediaWiki will always be in PHP5.
- MediaWiki will eventually migrate to PHP7, or
- MediaWiki will eventually migrate to Hack.
Is anyone arguing for #1?
I don't think anyone has argued for anything besides #2.
So we've got two backwards-incompatible choices to make, eventually.
There's already an open RfC[1] about bumping the minimum PHP requirement to 7.x.
[1] https://phabricator.wikimedia.org/T172165
-- Legoktm
On Tue, Sep 19, 2017 at 9:40 AM, C. Scott Ananian cananian@wikimedia.org wrote:
Fair enough. My point is just that we should stop and reflect that this is a major inflection point. Language choices are sticky, so this decision will have significant long-term implications. We should at least stop to evaluate PHP7 vs Hack and determine which is a better fit for our codebase, and do due diligence on both sides (count how many engineers, how many open source contributors, commit rates, etc). HHVM has been flirting with a LLVM backend, and LLVM itself has quite a large and active community. The PHP community has had issues with proper handling of security patches in the past. I'm suggesting to proceed cautiously and have a proper discussion of all the factors involved instead of over-simplifying this to "community" vs "facebook".
PHP is an open-source language with mature tooling and major community buy-in. Facebook has *promised* to turn Hack into an open-source language with mature tooling and community buy-in; almost none of that exists currently. Once it already does, a worthwhile discussion might be had about switching to Hack. Right now it would be incredibly irresponsible.
Also, making PHP a viable language for third parties is the core business model of Zend. Making Hack a viable language for third parties has absolutely nothing to do with the business model of Facebook. At any point they might decide it is a distraction they don't need. Comparing commit numbers is not really meaningful without knowing what fraction of those committers can disappear overnight if Facebook reconsiders its priorities.
IMO the more interesting discussion to be had is how little we invest into the technology our whole platform is based on. You'd think the largest production user of PHP would pay at least one part-time PHP developer, or try to represent itself in standards and roadmap discussions, but we do not. Is that normal?
Hi!
IMO the more interesting discussion to be had is how little we invest into the technology our whole platform is based on. You'd think the largest production user of PHP would pay at least one part-time PHP developer, or try to represent itself in standards and roadmap discussions, but we do not. Is that normal?
I think this is a very good point. There are other ways to support PHP too. Like assisting in regular testing of upcoming versions. Helping writing the docs. Contributing to RFC discussions (we have a large codebase, heavily visited site, and a lot of experience dealing with it, surely there could be a thing or two we could contribute). Triaging the bugs (one of the most necessary, thankless, and under-appreciated jobs in an open-source project). Probably more...
On 20/09/17 02:40, C. Scott Ananian wrote:
For example, the top-line github stats are: hhvm: 504 contributors (24,192 commits) php-src: 496 contributors (104,566 commits)
There's a reason I've contributed loads of code to HHVM and hardly any to PHP. Although the HHVM folks were sometimes tied up in internal work and not available, when they were available, my interactions with them were very pleasant. They were enthusiastic about my contributions, and were happy to have long design discussions on IRC. Code review sometimes had a lot of back and forth, but at least they were positive and engaged. The people I dealt with in code review usually had it as their job to accept community contributions. My bug reports were respectfully treated.
It was a big contrast to my interactions with the PHP community, which were so often negative. For example, Jani's toxic behaviour on the bug tracker, closing bugs as "bogus" despite being serious and reproducible, usually because he didn't understand them technically. Even with other maintainers, I had to fight several times to keep serious bugs open. I had no illusions that they would ever be fixed, I just wanted them to be open for my reference and for the benefit of anyone hitting the same issue. I filed bugs as "documentation issues", requesting that undesired behaviour be documented in the manual, since they were more likely to stay open that way.
My interactions with Derick Rethans were quite unpleasant, he would not even consider accepting the code I wrote to fix a DoS vulnerability which was affecting us constantly. He wouldn't provide a code review, he just rejected it on principle. Instead he wrote his own version of it a couple of years later. He seemed to think that every line of code in the date module should be attributable to him.
Design discussions are apparently concentrated on the internals mailing list, where there is an incredible amount of negativity for any new idea. Developers really need a lot of energy to keep answering negative comments, over and over for a period of months, in order to get their RFCs accepted. Some language features which eventually made it into PHP, such as short arrays, were shot down many times on the mailing list before they found a champion who was sufficiently brave and influential.
Their code review practices were quite archaic, I don't know if they've improved. Their coding style is also dated.
Stas was great, a bright spot in a dismal field, which was why I was so keen to hire him.
So I'm not looking forward to returning to PHP. But at least we know what we are getting ourselves in for. Being community-driven means it has inertia, change is slow. Facebook have been inconsistent with HHVM, and have made it clear that they don't intend to cater to our needs.
-- Tim Starling
Hi!
It was a big contrast to my interactions with the PHP community, which were so often negative. For example, Jani's toxic behaviour on the bug tracker, closing bugs as "bogus" despite being serious and reproducible, usually because he didn't understand them technically. Even with other maintainers, I had to fight several times to keep serious bugs open. I had no illusions that they would ever be fixed, I just wanted them to be open for my reference and for the benefit of anyone hitting the same issue. I filed bugs as "documentation issues", requesting that undesired behaviour be documented in the manual, since they were more likely to stay open that way.
By your mention of Jani, I derive it was a long time ago :) Quite a lot changed since then, even though the internals list is still not the most friendly place on the nets. But the processes got better, I think, and the level of maturity of both discussion and participants increased.
Also, thank you for the kind words, and I'll be glad to help with what I can if we need something done in PHP core.
On 09/19/2017 10:34 AM, Bryan Davis wrote:
For what it's worth, my opinion is that PHP is an actual FLOSS software project with years of history and core contributions from Zend who make their living with PHP. HHVM is a well funded internal project from Facebook that has experimented with FLOSS but ultimately is controlled by the internal needs of Facebook. For me the choice here is obviously to back the community driven FLOSS project and help them continue to thrive.
+1.
As a practical choice, given the wide usage of PHP on the web, it makes a lot more sense to throw our weight behind Zend development since it is more likely to survive than HHVM which is more subject to the whims and fancies of FB's internal choices which we are not privy to and cannot influence in any meaningful way. I mean how big of a user besides Mediawiki serving Wikipedia sites does it have to be for FB to consider other user's needs?
Subbu.
Hi!
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM
15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and Linkedin did not exist and Slashdot, Livejournal, etc. were all the rage. We don't even know if Facebook as such would exist in 15 years or would have budget to support its own language.
would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
While (b) could happen, PHP project is not very dependent on Zend for its existence. Zend owns none of the infrastructure or processes, and while a lot of performance work on PHP 7 was conducted by Zend team (and they are still working on improvements AFAIK), there are plenty of community members that do not work for Zend and do not depend on Zend in any way. Of course, it is possible that the whole community would implode, but here we have many more stakeholders than in Hack case, where the stakeholder is mostly a single - albeit large and currently very successful - company.
speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is
Again, I do not think this is the right statement to make. The control of Zend Tech as a company over the future of PHP is much less than Facebook's one over Hack (which is pretty much absolute). PHP is guided by the community, decisions are taken by using community processes in which Zend does not have any special role, and PHP project could survive reasonably survive without Zend, even if with less resources. Most PHP infrastructure - Composer, debugging, IDEs, profiling, code quality, frameworks, etc. - are completely independent of Zend (which also has a number of tools, but it is not the only provider). So I do not think it is an adequate comparison.
I am not sure if Hack has an open-source community outside Facebook (if anybody has pointers to that, please share - commit numbers certainly don't tell much) - but it is pretty clear to me that Facebook is in absolute control over this platform. This is not the case with Zend and PHP.
Zend was testing against hhvm at one point, then decided they would stop testing against it. See https://www.phoronix.com/scan.php?page=news_item&px=PTS-PHP-7.1-Benchmar... On Tuesday, 19 September 2017, 19:24:49 BST, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
I agree the short timeline seems to push us toward reverting to Zend. But it is worth having a meaningful discussion about the long-term outlook. Which VM is likely to be better supported in 15 years' time? Which VM
15 years is a lot to predict. 15 years ago Facebook, Twitter, Reddit and Linkedin did not exist and Slashdot, Livejournal, etc. were all the rage. We don't even know if Facebook as such would exist in 15 years or would have budget to support its own language.
would we rather adopt and maintain indefinitely ourselves, if needed -- since in a 15 yr timeframe it's entirely possible that (a) Facebook could abandon Hack/HHVM, or (b) the PHP Zend team could implode. Maintaining
While (b) could happen, PHP project is not very dependent on Zend for its existence. Zend owns none of the infrastructure or processes, and while a lot of performance work on PHP 7 was conducted by Zend team (and they are still working on improvements AFAIK), there are plenty of community members that do not work for Zend and do not depend on Zend in any way. Of course, it is possible that the whole community would implode, but here we have many more stakeholders than in Hack case, where the stakeholder is mostly a single - albeit large and currently very successful - company.
speaking, it's not really a choice between "lock-in" and "no lock in" -- we have to choose to align our futures with either Zend Technologies Ltd or Facebook. One of these is *much* better funded than the other. It is
Again, I do not think this is the right statement to make. The control of Zend Tech as a company over the future of PHP is much less than Facebook's one over Hack (which is pretty much absolute). PHP is guided by the community, decisions are taken by using community processes in which Zend does not have any special role, and PHP project could survive reasonably survive without Zend, even if with less resources. Most PHP infrastructure - Composer, debugging, IDEs, profiling, code quality, frameworks, etc. - are completely independent of Zend (which also has a number of tools, but it is not the only provider). So I do not think it is an adequate comparison.
I am not sure if Hack has an open-source community outside Facebook (if anybody has pointers to that, please share - commit numbers certainly don't tell much) - but it is pretty clear to me that Facebook is in absolute control over this platform. This is not the case with Zend and PHP.
Hi,
On 09/18/2017 05:13 PM, Tim Starling wrote:
- The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
On Reddit[1], a member of the HHVM team clarified they plan on dropping support for destructors *from Hack* soon. (Not that I think it really makes any difference in what our long-term plan should be.)
[1] https://www.reddit.com/r/PHP/comments/70wtky/the_future_of_hhvm/dn6skdn/
-- Legoktm
On 19/09/17 12:30, Legoktm wrote:
Hi,
On 09/18/2017 05:13 PM, Tim Starling wrote:
- The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
On Reddit[1], a member of the HHVM team clarified they plan on dropping support for destructors *from Hack* soon. (Not that I think it really makes any difference in what our long-term plan should be.)
It's unclear how much difference that will make, since they are clear about wanting to make HHVM be purely a Hack runtime. They'll carry on supporting Composer and PHPUnit, but only until Hack has "its own ecosystem of core frameworks".
They "will not be targeting PHP software beyond such libraries after the 3.24 release", which presumably means that they will no longer run MediaWiki or PHP unit tests against HHVM.
Also, they said that they want to remove destructors in order to eliminate the performance overhead of reference counting, and I don't think it is possible to get that performance benefit unless you remove reference counts from the VM entirely. Maybe removing them from the Hack language will be a first step, but we can't expect them to keep them in the VM in the longer term.
-- Tim Starling
On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
On 19/09/17 06:58, Max Semenik wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
The HHVM team did tell us privately that they were planning on changing their strategy, basically as you describe it above. The surprising things for me in this announcement were:
- The plan to also drop PHP 5 compatibility, on a short timeline (1 year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
While this does not mean that we need to take an action immediately, eventually we will have to decide something.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
The next step would be the upgrade of the mw* fleet to Debian stretch while still using HHVM 3.18 (to minimise disruption since we've stabilised 3.18 in it's current build). That work is tracked at T174431. 3.18 will be supported by upstream for at least another six months (and if the migration drags further I can roll custom 3.18 security backports from later LTS releases)
Debian stretch ships PHP7, so that'd be a good stepstone to migrate back to Zend.
Cheers, Moritz
Should we have a TechComm-driven meeting about this ASAP?
Like others, I don't expect that there will be disagreement about the way to go, but there is a lot to discuss about what needs to be done, resourcing, etc.
It would be nice to have Ori around for it, to pick his brains about any undocumented or little-known knowledge about the HHVM migration that could bite us when migrating to PHP 7.x if we don't know about it.
On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff < mmuhlenhoff@wikimedia.org> wrote:
On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
On 19/09/17 06:58, Max Semenik wrote:
Today, the HHVM developers made an announcement[1] that they have
plans of
ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
The HHVM team did tell us privately that they were planning on changing their strategy, basically as you describe it above. The surprising things for me in this announcement were:
- The plan to also drop PHP 5 compatibility, on a short timeline (1
year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
While this does not mean that we need to take an action immediately, eventually we will have to decide something.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
The next step would be the upgrade of the mw* fleet to Debian stretch while still using HHVM 3.18 (to minimise disruption since we've stabilised 3.18 in it's current build). That work is tracked at T174431. 3.18 will be supported by upstream for at least another six months (and if the migration drags further I can roll custom 3.18 security backports from later LTS releases)
Debian stretch ships PHP7, so that'd be a good stepstone to migrate back to Zend.
Cheers, Moritz
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Sounds like a good idea. +Daniel for scheduling.
Best,
Victoria
On Sep 19, 2017, at 6:56 AM, Gilles Dubuc gilles@wikimedia.org wrote:
Should we have a TechComm-driven meeting about this ASAP?
Like others, I don't expect that there will be disagreement about the way to go, but there is a lot to discuss about what needs to be done, resourcing, etc.
It would be nice to have Ori around for it, to pick his brains about any undocumented or little-known knowledge about the HHVM migration that could bite us when migrating to PHP 7.x if we don't know about it.
On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff < mmuhlenhoff@wikimedia.org> wrote:
On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
On 19/09/17 06:58, Max Semenik wrote:
Today, the HHVM developers made an announcement[1] that they have
plans of
ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
The HHVM team did tell us privately that they were planning on changing their strategy, basically as you describe it above. The surprising things for me in this announcement were:
- The plan to also drop PHP 5 compatibility, on a short timeline (1
year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
While this does not mean that we need to take an action immediately, eventually we will have to decide something.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
The next step would be the upgrade of the mw* fleet to Debian stretch while still using HHVM 3.18 (to minimise disruption since we've stabilised 3.18 in it's current build). That work is tracked at T174431. 3.18 will be supported by upstream for at least another six months (and if the migration drags further I can roll custom 3.18 security backports from later LTS releases)
Debian stretch ships PHP7, so that'd be a good stepstone to migrate back to Zend.
Cheers, Moritz
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Sep 19, 2017, at 6:56 AM, Gilles Dubuc gilles@wikimedia.org wrote:
Should we have a TechComm-driven meeting about this ASAP?
Like others, I don't expect that there will be disagreement about the way to go, but there is a lot to discuss about what needs to be done, resourcing, etc.
It would be nice to have Ori around for it, to pick his brains about any undocumented or little-known knowledge about the HHVM migration that could bite us when migrating to PHP 7.x if we don't know about it.
On Tue, Sep 19, 2017 at 9:07 AM, Moritz Muehlenhoff < mmuhlenhoff@wikimedia.org> wrote:
On Tue, Sep 19, 2017 at 10:13:47AM +1000, Tim Starling wrote:
On 19/09/17 06:58, Max Semenik wrote:
Today, the HHVM developers made an announcement[1] that they have
plans of
ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
The HHVM team did tell us privately that they were planning on changing their strategy, basically as you describe it above. The surprising things for me in this announcement were:
- The plan to also drop PHP 5 compatibility, on a short timeline (1
year).
- Rather than "drifting away" from PHP, their top priority plans
include removing core language features like references and destructors.
While this does not mean that we need to take an action immediately, eventually we will have to decide something.
Actually, I think a year is a pretty short time for ops to switch to PHP 7. I think we need to decide on this pretty much immediately.
The next step would be the upgrade of the mw* fleet to Debian stretch while still using HHVM 3.18 (to minimise disruption since we've stabilised 3.18 in it's current build). That work is tracked at T174431. 3.18 will be supported by upstream for at least another six months (and if the migration drags further I can roll custom 3.18 security backports from later LTS releases)
Debian stretch ships PHP7, so that'd be a good stepstone to migrate back to Zend.
Cheers, Moritz
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 19/09/17 20:56, Gilles Dubuc wrote:
Should we have a TechComm-driven meeting about this ASAP?
We had an IRC meeting about this yesterday. Here is the log:
https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-09-27-21.03.log.html
We mostly talked about the migration plan for production, including the likely timeline.
-- Tim Starling
On 19/09/17 10:13, Tim Starling wrote:
I'll run a benchmark
I upgraded the test wiki container on my laptop from Ubuntu 14.04 to 16.04, which also necessitated a platform switch from schroot to systemd-nspawn. The benchmark is thus approximately native performance on a Core i5 4210U @ 1.7 GHz. I killed the usual CPU hogs so that everything was quiet in top. Then I ran benchmarkParse.php on a copy of the [[Australia]] article, including templates, with --loops 3.
The results were
PHP 7.0: 1.59 seconds HHVM 3.21: 1.75 seconds
So PHP 7 was significantly faster on this test.
Note that I ran HHVM with JIT enabled; total wall clock time including compilation and warmup was about 75 seconds, compared to 13 seconds for PHP 7.
The test wiki has Scribunto with LuaStandalone. Debug logging was disabled for this test.
-- Tim Starling
About third-parties.
* Regardless of whether PHP 7 (or 8, or 9) is more similar to PHP 5 than Hack is. * Regardless of whether upgrading from PHP 5 to 7 is more or less difficult than upgrading to Hack. * Regardless of whether Zend PHP is going to make big backwards-incompatible changes in the future (like HHVM is doing now in Hack, e.g. what if Zend PHP 8 or 9 were to also drop references and destructors?).
Regardless of these "our" costs, a site-admin must work with the options available. We can expect current PHP 5 deployments to also (soon) have PHP 7. Shared hosting, dedicated hosting (with shell but limited packages), and platform-as-a-service (PaaS), and other environments have, or will soon, provide an option/package/image for PHP 7.
But having them consider a new run-time (e.g. Node, Go, Perl, Hack) was always a good argument against "Let's rewrite MediaWiki in X".
On the other hand, I don't think we should consider it as something that can't happen. We shouldn't decide on the account of every single hosting provider. Some are probably still running PHP 4 and customers will have decided to switch to a more caring provider. If no providers add Hack, the loss is ours (leaving site-admins with no similar-cost options). If enough providers switch, the loss is the provider's (customer leaves or provider follows suit). I don't think a PHP-only provider is necessarily cheaper than a more diverse provider. They're just differently focussed.
The marketshare of sites runnable on pure "LAMP" seems to be shrinking with the other run-times gaining popularity, and new run-times being introduced. I'd expect a hosting service providing only Zend PHP to have a shrinking customer base in the long run. We might not be looking at "Will they add Hack as the first non-PHP run-time option", but rather "Will they add Hack among the other run-times". How do we feel about that? Does this seem like something that could happen, as well as something we could *help* make happen?
-- Timo
On Mon, Sep 18, 2017 at 9:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
Today, the HHVM developers made an announcement[1] that they have plans of ceasing to maintain 100% PHP7 compatibility and concentrating on Hack instead.
While this does not mean that we need to take an action immediately, eventually we will have to decide something. As I see it, our options are:
- Continue requiring that MediaWiki uses a common set of HHVM and Zend
PHP. This, however, is a dead end and will make things progressively harder as the implementations will diverge and various Composer libraries we use will start requiring some Zend-specific features.
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
I personally think that 3) is the only viable option in the long run. What do you think?
[1] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
- Declare our loyalty to HHVM. This will result in most of our current
users being unable to upgrade, eventually producing what amounts to a WMF-only product and lots of installations with outdated MediaWiki having security holes. At least we will be able to convert to Hack eventually. This is a very clean-cut case of vendor lock-in though, and if Facebook decides to switch their code base to something shinier, we'll be deep in trouble.
Hack has a couple interesting points, especially its async system, but isn't _hugely_ compelling at this point.
If they're going to be dropping destructors and references we'd have to retool our RAII patterns and our hook 'out-param' patterns to work with future-Hack, but that's possible if we *want* to make such a change.
However I don't see a lot of interest in such a change in this discussion or among TechCom, and buying into a single-vendor system is a risk both for us (if they change the language in ways we don't like or drop it) and others (harder to set up if few HHVM providers).
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated, while 5.6 and 7 provided nice performance improvements.
Migrating WMF's implementation to PHP 7 is probably the way to go. I leave it up to ops to figure out how to make the change. :)
-- brion
On Wed, Sep 20, 2017 at 10:56 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Mon, Sep 18, 2017 at 1:58 PM, Max Semenik maxsem.wiki@gmail.com wrote:
- Revert WMF to Zend and forget about HHVM. This will result in
performance degradation, however it will not be that dramatic: when we upgraded, we switched to HHVM from PHP 5.3 which was really outdated,
while
5.6 and 7 provided nice performance improvements.
Migrating WMF's implementation to PHP 7 is probably the way to go. I leave it up to ops to figure out how to make the change. :)
I think this is the more viable option too, mostly not to cause huge issues to non-Wikimedia users. The ease of installation and operation of PHP on most platforms compared to HHVM is incomparable. Just to make an example, I don't remember having to check the code of the VM to understand what an ini setting does with PHP (or even PHP-fpm), while with HHVM that has been a recurring pain, as options come and go without any warning between versions, not to mention the online docs that can even be plainly misleading. At the moment, there is no doubt PHP is a much friendlier environment for any third-party wiki.
But I think there is other value in going the PHP7 way; I had actually pitched the idea of switching to PHP7 for some time now, for various reasons, namely: - We use HHVM differently than what Facebook does. For instance, we use the fcgi server mode, and not the pure http one, and we don't run in repoAuth mode. This has brought us new bugs to solve every time we upgrade, as there is no battle testing in production for those code patterns at scale before we use it. - The recent, prolonged difficulty of interaction with the FLOSS community, although acknowledged by the HHVM team as something they're willing to fix, is worrying in itself. - PHP7 has shown performance comparable to HHVM for most PHP shops that migrated. So the single most compelling reason for which we migrated (performance) might not be a factor anymore. Using a runtime readily available (and security-patched) by the upstream distribution would make the ops team lives easier as well.
As for the actual migration:
I don't think there is any need to panic or rush to a decision, but the timeline is pretty set: by the end of 2018, when official support for HHVM 3.24 will end, any migration should be well underway within the WMF infrastructure. I expect a migration from HHVM to PHP 7 to be a less formidable undertaking than the switch from PHP 5.3 to HHVM - we did repay a good deal t of the tech debt in the Wikimedia Foundation installation back then, and we won't have to change radically the way we serve MediaWiki, as PHP 7 works as a FastCGI server as well. Still, it will take time and resources, and it needs to be planned in advance.
One important consequence of the announcement for Wikimedia is that we won't be able to use any version of MediaWiki not compatible with PHP 5.x until we transition to PHP7 (unless we decide to support both HHVM and PHP7). This might be important in steering the timing of the change in MediaWiki itself.
Cheers,
Giuseppe
wikitech-l@lists.wikimedia.org