Several people from the HipHop team at Facebook just met with several people from WMF. Also, in the last couple of days, I've been doing some research into what it would take to make MediaWiki support HipHop VM. The answer is: not very much.
There's two features that we use, mostly in extensions, that the Facebook people are not keen to implement due to their complexity: ArrayObject and filter_var(). It seems that it would be much easier for us to stop using them than for those features to be implemented in HipHop.
So I'd like to suggest that we refuse new code submissions in Gerrit that use these features, if they are targeted for WMF production, and that we start work on migrating away from the existing uses of those features.
There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop:
* CachingIterator * EmptyIterator * GlobIterator * InfiniteIterator * LimitIterator * MultipleIterator * NoRewindIterator * ParentIterator * RecursiveArrayIterator * RecursiveCachingIterator * RecursiveFilterIterator * RecursiveRegexIterator * RecursiveTreeIterator * RegexIterator * SplDoublyLinkedList * SplFixedArray * SplHeap * SplMaxHeap * SplMinHeap * SplPriorityQueue * SplQueue * SplStack * SplTempFileObject
We are not yet promising that we are indeed going to start using HipHop in WMF production, and we don't have any timetables. But HipHop has evolved to the point where supporting it is almost trivial, at least for test installations, so I think it makes sense to establish policies which will avoid making migration to HipHop more difficult.
-- Tim Starling
On Thu, 09 May 2013 18:10:53 -0700, Tim Starling tstarling@wikimedia.org wrote:
- SplDoublyLinkedList
- SplFixedArray
- SplHeap
- SplMaxHeap
- SplMinHeap
- SplPriorityQueue
- SplQueue
- SplStack
It would be nice if HHVM would support these. I'm not sure what is so complex about these that HHVM can implement array() but not these simple classes.
While working on my skin rewrite ideas, trying to implement the template syntax parsing was much MUCH more readable and less prone to mistakes (like those that would occur if php cloned the array() in a place it shouldn't cause I typed something wrong) when working with SplDoublyLinkedList, SplQueue, and SplStack.
-- Tim Starling
On 10/05/13 11:28, Daniel Friesen wrote:
On Thu, 09 May 2013 18:10:53 -0700, Tim Starling tstarling@wikimedia.org wrote:
- SplDoublyLinkedList
- SplFixedArray
- SplHeap
- SplMaxHeap
- SplMinHeap
- SplPriorityQueue
- SplQueue
- SplStack
It would be nice if HHVM would support these. I'm not sure what is so complex about these that HHVM can implement array() but not these simple classes.
While working on my skin rewrite ideas, trying to implement the template syntax parsing was much MUCH more readable and less prone to mistakes (like those that would occur if php cloned the array() in a place it shouldn't cause I typed something wrong) when working with SplDoublyLinkedList, SplQueue, and SplStack.
You can always implement the parts you need in pure PHP.
-- Tim Starling
On Thu, May 9, 2013 at 9:46 PM, Tim Starling tstarling@wikimedia.orgwrote:
You can always implement the parts you need in pure PHP.
True. It might be worthwhile to make some sort of Spl compatibility library that loads in PHP versions of those classes if they do not exists. That way we can use them without dropping HHVM support.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
On 10/05/13 11:48, Tyler Romeo wrote:
On Thu, May 9, 2013 at 9:46 PM, Tim Starling tstarling@wikimedia.orgwrote:
You can always implement the parts you need in pure PHP.
True. It might be worthwhile to make some sort of Spl compatibility library that loads in PHP versions of those classes if they do not exists. That way we can use them without dropping HHVM support.
If you can write a complete and accurate compatibility class in pure PHP, then it can be included in HipHop in the system/classes directory.
-- Tim Starling
On Thu, May 9, 2013 at 9:57 PM, Tim Starling tstarling@wikimedia.orgwrote:
If you can write a complete and accurate compatibility class in pure PHP, then it can be included in HipHop in the system/classes directory.
Interesting. I'll try and work on this.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
Hey,
I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library.
Question: Is it possible to use HipHop only for part of a codebase. For instance, if there is some MW extension that depends on a third party library we have no control over that makes uses of SPL, would this cause problems?
Suggestion: If such a policy is implemented, I'd be great to have a job that checks for violations of it on our CI.
Cheers
-- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. --
On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library.
Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term.
As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC).
Question: Is it possible to use HipHop only for part of a codebase. For instance, if there is some MW extension that depends on a third party library we have no control over that makes uses of SPL, would this cause problems?
I don't think so, as HipHop is a standalone implementation of PHP and doesn't even really know about the php binary/libraries.
Suggestion: If such a policy is implemented, I'd be great to have a job that checks for violations of it on our CI.
This is a very good idea, and something I'll take a stab at handling.
-Chad
On 10.05.2013 17:58, Chad wrote:
On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library.
Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term.
As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC).
System and development software, such as OS, compilers, language libraries, different kinds of VM's and so on are really huge and 2000+ lines of code actually is a *little* amount. It's not a framework or wiki. It's development software written in low-level language. Dmitriy
On Sat, May 11, 2013 at 3:31 AM, Dmitriy Sintsov questpc@rambler.ru wrote:
On 10.05.2013 17:58, Chad wrote:
On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library.
Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term.
As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC).
System and development software, such as OS, compilers, language libraries, different kinds of VM's and so on are really huge and 2000+ lines of code actually is a *little* amount. It's not a framework or wiki. It's development software written in low-level language.
Indeed, 2000 isn't really a lot, I was just trying to give an order of magnitude so people wouldn't think it was like <50 LOC or something.
-Chad
FWIW, here is what I have so far: http://pastebin.com/hUQ92DfB
I haven't tested it yet because my PHP environment is not behaving, and the only class I haven't implemented fully is SplHeap.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
On Sat, May 11, 2013 at 8:57 AM, Chad innocentkiller@gmail.com wrote:
On Sat, May 11, 2013 at 3:31 AM, Dmitriy Sintsov questpc@rambler.ru wrote:
On 10.05.2013 17:58, Chad wrote:
On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw <jeroendedauw@gmail.com
wrote:
Hey,
I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library.
Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term.
As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC).
System and development software, such as OS, compilers, language
libraries,
different kinds of VM's and so on are really huge and 2000+ lines of code actually is a *little* amount. It's not a framework or wiki. It's development software written in
low-level
language.
Indeed, 2000 isn't really a lot, I was just trying to give an order of magnitude so people wouldn't think it was like <50 LOC or something.
-Chad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 12.05.2013 1:18, Tyler Romeo wrote:
FWIW, here is what I have so far: http://pastebin.com/hUQ92DfB
I haven't tested it yet because my PHP environment is not behaving, and the only class I haven't implemented fully is SplHeap.
Perhaps you should send the link to HipHop developers (or to their list, if there's any).
On Sun, May 12, 2013 at 1:56 PM, Dmitriy Sintsov questpc@rambler.ru wrote:
Perhaps you should send the link to HipHop developers (or to their list, if there's any).
Once I finish it and it's working I plan on doing just that.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
On 10/05/13 22:05, Jeroen De Dauw wrote:
Question: Is it possible to use HipHop only for part of a codebase. For instance, if there is some MW extension that depends on a third party library we have no control over that makes uses of SPL, would this cause problems?
It would be possible to have a separate cluster running regular PHP, and to access it via internal API requests or by frontend routing, similar to how we do image scaling. But the justification would need to be pretty strong to justify that complexity.
-- Tim Starling
On 10.05.2013, 5:10 Tim wrote:
Several people from the HipHop team at Facebook just met with several people from WMF. Also, in the last couple of days, I've been doing some research into what it would take to make MediaWiki support HipHop VM. The answer is: not very much.
There's two features that we use, mostly in extensions, that the Facebook people are not keen to implement due to their complexity: ArrayObject and filter_var(). It seems that it would be much easier for us to stop using them than for those features to be implemented in HipHop.
So I'd like to suggest that we refuse new code submissions in Gerrit that use these features, if they are targeted for WMF production, and that we start work on migrating away from the existing uses of those features.
There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop:
What about namespaces? Last time I heard HH didn't support them - is it safe to assume that support for them is coming?
On Fri, May 10, 2013 at 9:49 AM, Max Semenik maxsem.wiki@gmail.com wrote:
On 10.05.2013, 5:10 Tim wrote:
Several people from the HipHop team at Facebook just met with several people from WMF. Also, in the last couple of days, I've been doing some research into what it would take to make MediaWiki support HipHop VM. The answer is: not very much.
There's two features that we use, mostly in extensions, that the Facebook people are not keen to implement due to their complexity: ArrayObject and filter_var(). It seems that it would be much easier for us to stop using them than for those features to be implemented in HipHop.
So I'd like to suggest that we refuse new code submissions in Gerrit that use these features, if they are targeted for WMF production, and that we start work on migrating away from the existing uses of those features.
There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop:
What about namespaces? Last time I heard HH didn't support them - is it safe to assume that support for them is coming?
Yes. As I understood it yesterday, this is just about to land in master.
-Chad
Le 10/05/13 03:10, Tim Starling a écrit :
So I'd like to suggest that we refuse new code submissions in Gerrit that use these features, if they are targeted for WMF production,
This can be done using CodeSniffer. We can write a specific style standard such as "WikimediaProduction" that will spurt error whenever some function / class is encountered.
Le 10/05/13 03:10, Tim Starling a écrit :
There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop:
<snip: various SPL classes>
I wish we actually used Spl :-] They are nice classes providing all kind of useful features: http://php.net/manual/en/book.spl.php
Although I fully agree we should not add more to be able to use HipHop, I think we should also look at having them implemented in HipHop.
On 10.05.2013 23:51, Antoine Musso wrote:
Le 10/05/13 03:10, Tim Starling a écrit :
There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop:
<snip: various SPL classes>
I wish we actually used Spl :-] They are nice classes providing all kind of useful features: http://php.net/manual/en/book.spl.php
Yes it is strange that Facebook does not need SPL. Especially because since PHP 5.3 SPL is not optional anymore.
wikitech-l@lists.wikimedia.org