2011/3/28 Tim Starling <tstarling(a)wikimedia.org>rg>: I don't really have a good idea of what would need to change to support HipHop, but if the changes aren't to intrusive I'm all for it. If we decide to do this, we should also decide when to start and when we want to have HPHP support working (1.18? 1.19?).

011/3/28 Douglas Gardner <douglas.gardner(a)wikinewsie.org>rg>: I don't think so. There's no technical reason for them to be on Commons. If there's some community/policy/whatever reason to have them there, ask on a community/policy/whatever list, not a technical list. The icons can be found at http://bits.wikimedia.org/skins-1.5/vector/images/external-link-ltr-icon.png and http://bits.wikimedia.org/skins-1.5/vector/images/external-link-rtl-icon.png Roan Kattouw (Catrope)

(ii) Also, it would be nice if developers could all have hiphop running on their test wikis, so that code that's broken on hiphop isn't committed in ignorance. The only problem is that, last time I checked, the dependency list for hiphop is very considerable.

... and Facebook ignores that and adds what it thinks would be useful?

We can use features from both, using function_exists(), like what we do now with PHP modules.

Yes, that's true, and that's part of the reason I'm flagging this change on the mailing list. Domas says that the HipHop team is working on PHP 5.3 support, so maybe the issue won't come up. But yes, in principle, I am saying that we should support HipHop even when it means not using new features from PHP. PHP 5.3 might be cool, but so is cutting our power usage by half (pun intended).

Plus, free C++ MediaWiki parser ;-) Seriously, there should be a way to turn the entire package into a (huge) library; maybe transpile it and then replace the C++ code for index.php with a manually written library interface?

Offline readers, scientific analysis tools, etc. could profit massively from an always-current, fast C++ library...

On 05/04/11 04:47, Tim Starling wrote: <snip> I have imported in my local wiki the english [[Barack Obama]] article with all its dependencies. I can not have it parsed under either 256MB max memory or 1 minute max execution time limits. Hiphop helps, but there is still a highly broken code somewhere in our PHP source code. No matter how much hacks we throw at bad code, the algorithm still need to get fixed.

<snip> As I understand it, hiphop is just a straight translator from PHP to C language but does not actually enhance the code. Just like you would use Google translator instead of Charles Baudelaire [1].

Your code above comes from Parse.php getVariableValue() which use a long switch() structure to map a string to a method call. If you manage to find unoptimized code in the translated code, fix it in the PHP source code :-b

For comparison: WYSIFTW parses [[Barak Obama]] in 3.5 sec on my iMac, and in 4.4 sec on my MacBook (both Chrome 12).

2011/4/5 Magnus Manske &lt;magnusmanske(a)googlemail.com&gt;om>: Seriously, the bulk of the time needed to parse these enwiki articles is for template expansion. If you pre-expand them, taking care that also the templates in <ref>...</ref> tags get expanded, MediaWiki can parse the article in a few seconds, 3-4 on my laptop.

Yes, it doesn't do template/variable replacing, and it's probably full of corner cases that break; OTOH, it's JavaScript running in a browser, which should make it much slower than a dedicated server setup running precompiled PHP.

On 5 April 2011 21:29, Aryeh Gregor &lt;Simetrical+wikilist(a)gmail.com&gt; wrote: So if we machine-translate the parser into JS, we can get the user to do the work and everyone wins! [*] (Magnus, did you do something like this for WYSIFTW?)

On 04/05/2011 05:47 AM, Tim Starling wrote: Hmm, does HipHop precompile regexen?

There is a list of things that differ here: https://github.com/facebook/hiphop-php/blob/master/doc/inconsistencies Unfortunately it seems to leave out the most important differences.

When I'm ready to write all this up properly, the following page will appear on mediawiki.org: http://www.mediawiki.org/wiki/HipHop

On 03/29/2011 10:48 AM, Platonides wrote: I've explored the issues and made some initial changes to my working copy. I'm now waiting for it to compile, and once it's tested, I'll commit it. There is a list of things that differ here: https://github.com/facebook/hiphop-php/blob/master/doc/inconsistencies Unfortunately it seems to leave out the most important differences. It seems incredible, and I'm hoping someone will correct me, but it seems that file inclusion has to be completely different in HipHop. Even the simplest script won't work. I put this in foo.php: <?php class Foo { static function bar() { print "Hello\n"; } } ?> And this in test.php: <?php include 'foo.php'; Foo::bar(); ?> This gives "HipHop Fatal error: Cannot redeclare class Foo" at runtime. All classes which are compiled exist from startup, and trying to declare them produces this error. This means that it is no longer possible to mix class and function declarations with code we want to execute. My working copy has fixes for the most important instances of this, such as in Setup.php and WebStart.php. There are two exceptions to this. One is the interpreter. HipHop has an interpreter, which is used for eval() and for include() on a file with a fully-qualified path. We can use this to allow us to change LocalSettings.php without recompiling. If you want to do include() and have it execute compiled code, you need to use a path which is relative to the base of the compiled code. My working copy has some functions which allow this to be done in a self-documenting way. The other exception is volatile mode, which unfortunately appears to be completely broken, at least in the RPMs that I'm using. It's so broken that calling class_exists() on a literal string will break the class at compile time, making it impossible to use, with no way to repair it. My working copy has a wrapper for class_exists() which doesn't suffer from this problem. Another undocumented difference is that HipHop does not use php.ini or anything like it, so most instances of ini_get() and ini_set() are broken. The functions exist, but only have stub functionality. HipHop has its own configuration files, but they aren't like php.ini. When I'm ready to write all this up properly, the following page will appear on mediawiki.org: http://www.mediawiki.org/wiki/HipHop -- Tim Starling _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On 03/29/2011 10:48 AM, Platonides wrote: I've explored the issues and made some initial changes to my working copy. I'm now waiting for it to compile, and once it's tested, I'll commit it.

On 04/05/2011 07:31 AM, Platonides wrote: You can't include the class files for compiled classes, they all exist at startup and you get a redeclaration error if you try. I explained this in the documentation page on mediawiki.org which has now appeared. http://www.mediawiki.org/wiki/HipHop The autoloader is not used. It doesn't matter where the class_exists() is. HipHop scans the entire codebase for class_exists() at compile time, and breaks any classes it finds, whether the or not the class_exists() is reachable.

On 03/30/2011 12:51 AM, Chad wrote: I saw that there are RPMs for CentOS, so I installed CentOS inside a chroot inside Ubuntu 10.10 x86-64. Surprisingly, this was quite easy.

I need to get back to work on reviews and deployments and other such strategic things. I hope the work I've done on HipHop support is enough to get the project started. Initial benchmarks are showing a 5x speedup for article parsing, which is better than we had hoped for. So there are big gains to be had here, both for small users and for large websites like Wikimedia and Wikia. There's a list of things that still need doing under the "to do" heading at: http://www.mediawiki.org/wiki/HipHop I'll be available to support any developers who want to work on this. -- Tim Starling