-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello,
In the past few days I've been making good progress on the ExtensionManager extension, and am at the stage where information regarding extensions needs to be stored and polled. In the long term I hope for these files to become widespread (perhaps in nearly all extensions), therefore I'd like to open a discussion about the format -- once implemented it will be difficult to remove features from it, and backwards compatibility would have to be retained indefinitely so a good starting point would be essential.
My idea is to use an XML file for each extension, this would be fairly easy to parse (XML support is a requirement for installing MediaWiki) and is easy to write. My suggestion as an example format is (using CategoryStepper as the example) is:
<extension> <name>CategoryStepper</name> <version>1.5</version> <description>categorystepper-desc</description> <mediawiki>1.11.0</mediawiki> </extension>
While this original version is quite concise it would be easy to expand upon if any further features were needed (for example, there is a possibility of compatibility verification by checking class names etc.). The extensions i18n file will also be loaded within the special page's scope so the <description> tag correlates to a message name allowing for internationalization. The <mediawiki> tag correlates to the minimum MediaWiki version required, the rest are probably self explanatory.
Any input on this matter is greatly appreciated.
Thank you, MinuteElectron.
On Thu, Mar 20, 2008 at 5:21 PM, MinuteElectron minuteelectron@googlemail.com wrote:
My idea is to use an XML file for each extension, this would be fairly easy to parse (XML support is a requirement for installing MediaWiki) and is easy to write.
How about we just use PHP, which is even easier for PHP to parse and even easier to write?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Simetrical wrote:
How about we just use PHP, which is even easier for PHP to parse and even easier to write?
Yeah, I just experimented with using PHP's XML parser, your idea is good.
MinuteElectron.
MinuteElectron wrote: <snip>
My idea is to use an XML file for each extension, this would be fairly easy to parse (XML support is a requirement for installing MediaWiki) and is easy to write. My suggestion as an example format is (using CategoryStepper as the example) is:
<extension> <name>CategoryStepper</name> <version>1.5</version> <description>categorystepper-desc</description> <mediawiki>1.11.0</mediawiki> </extension>
Hello Minute e-,
You might want to use XML Schema Description (XSD) to describe the content of an XML file. You can even get some kind of versioning in case we have to add or deprecate elements.
A simple .xsd could looks like :
<?xml version="1.0" encoding="UTF-8" ?> <schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:mwext="http://www.mediawiki.org/xml/extension-0.1/" targetNamespace="http://www.mediawiki.org/xml/extension-0.1/" elementFormDefault="qualified">
<element name="mwextension" />
<!-- See includes/SpecialVersion.php for valid extension credits --> <element type="string" name="name" use="required"/> <element type="string" name="version" /> <element type="string" name="author" /> <element type="anyURI" name="url" /> <element type="string" name="descriptionmsg" /> <element type="string" name="description" /> </schema>
This should be enough to support the current $wgExtensionCredits. The XSD above is based on Brion's XSD for the exporter : http://www.mediawiki.org/xml/export-0.3/
You might want to add the extension base path, php entry point and how i18n is handled. That could be done later in a 0.2 schema :)
While this original version is quite concise it would be easy to expand upon if any further features were needed (for example, there is a possibility of compatibility verification by checking class names etc.). The extensions i18n file will also be loaded within the special page's scope so the <description> tag correlates to a message name allowing for internationalization. The <mediawiki> tag correlates to the minimum MediaWiki version required, the rest are probably self explanatory.
As the i18n descriptions are in PHP, I am not sure you want to put them in statics XML files. That will just duplicate the information.
The difference between doing it in PHP and XML, is that using the XML setup, we can read in that data without requiring that any of that data be included or anything (ie: For extensions that are not installed), and use it on something like a list of extensions which are enabled, and ones which are available to be enabled.
Also, I've done a bit with this before (^_^ Actually for the most part, your idea of using an xml file is similar to what I had thought of ages ago the last time I tried it) Though I think some of that should be expanded: <mediawiki> <min>1.7.0</min> <max>1.12.0</max> </mediawiki> There are some extensions which start to break after certain versions, and shouldn't be used on them. Some of them may even end up creating fatal PHP errors which would cripple any wiki which tried to enable them, but we still want to make them available for older wiki.
We should probably also have a notion of dependencies: <dependencies> <dependency>Semantic MediaWiki</dependency> </dependencies> This will be important for extensions like the extensions which use SMW, ones which use DPL, or things like the BizzWiki extensions or other ones which require things like the SimpleRegex classes, or other extensions meant to simplify functionality for other extensions. Trying to install those without their dependencies would lead to fatal php errors.
Oh wait crap... That was something I was working on when the old Wiki-Tools was online... T_T I had a few entire schema files and Web Specs (Like the W3 ones) dedicated entirely to the specification of Extension XML Definition files.
Also, some notion of installation would be nice. Basically it would be good to offer the ability to easily do the installation of the needed SQL tables, and additionally upgrading of those tables.
~Daniel Friesen(Dantman) of: -The Gaiapedia (http://gaia.wikia.com) -Wikia ACG on Wikia.com (http://wikia.com/wiki/Wikia_ACG) -and Wiki-Tools.com (http://wiki-tools.com)
Ashar Voultoiz wrote:
MinuteElectron wrote:
<snip>
My idea is to use an XML file for each extension, this would be fairly easy to parse (XML support is a requirement for installing MediaWiki) and is easy to write. My suggestion as an example format is (using CategoryStepper as the example) is:
<extension> <name>CategoryStepper</name> <version>1.5</version> <description>categorystepper-desc</description> <mediawiki>1.11.0</mediawiki> </extension>
Hello Minute e-,
You might want to use XML Schema Description (XSD) to describe the content of an XML file. You can even get some kind of versioning in case we have to add or deprecate elements.
A simple .xsd could looks like :
<?xml version="1.0" encoding="UTF-8" ?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:mwext="http://www.mediawiki.org/xml/extension-0.1/" targetNamespace="http://www.mediawiki.org/xml/extension-0.1/" elementFormDefault="qualified">
<element name="mwextension" /> <!-- See includes/SpecialVersion.php for valid extension credits --> <element type="string" name="name" use="required"/> <element type="string" name="version" /> <element type="string" name="author" /> <element type="anyURI" name="url" /> <element type="string" name="descriptionmsg" /> <element type="string" name="description" />
</schema>
This should be enough to support the current $wgExtensionCredits. The XSD above is based on Brion's XSD for the exporter : http://www.mediawiki.org/xml/export-0.3/
You might want to add the extension base path, php entry point and how i18n is handled. That could be done later in a 0.2 schema :)
While this original version is quite concise it would be easy to expand upon if any further features were needed (for example, there is a possibility of compatibility verification by checking class names etc.). The extensions i18n file will also be loaded within the special page's scope so the <description> tag correlates to a message name allowing for internationalization. The <mediawiki> tag correlates to the minimum MediaWiki version required, the rest are probably self explanatory.
As the i18n descriptions are in PHP, I am not sure you want to put them in statics XML files. That will just duplicate the information.
On Thu, Mar 20, 2008 at 11:07 PM, DanTMan dan_the_man@telus.net wrote:
The difference between doing it in PHP and XML, is that using the XML setup, we can read in that data without requiring that any of that data be included or anything (ie: For extensions that are not installed)
What do you mean by "included" in this context? It would not be difficult, if desired, to have separate files to load info and to load the extension itself, or to use some kind of switch to achieve the same effects.
XML:
* Moderately easy to write by hand * Moderately easy and fast for most modern computer languages to read and manipulate. Can typically be read in and manipulated with a relatively simple (but not necessarily well-known) set of functions. Parsed forms can be cached in memory data stores, but extra manual checks are needed to make sure the files have not changed.
PHP:
* Very easy to write by hand (fewer delimiters, less redundancy, etc.) * Incredibly easy and extremely fast for PHP to read and manipulate. Loading and parsing takes basically no time at all with opcode cache, which every installation that's serious about performance has. Functions to manipulate data structures are extremely familiar, fast, and powerful because they're just regular PHP data structures. Opcode caches will automatically cache the files and maintain integrity with stat()s, but those can be easy avoided by a one-line config change if you use a script like scap that can purge the opcode cache. * Something of a pain for other languages to read and manipulate. Possible if it's really necessary, but slow and error-prone. Will need custom functions.
Given that MediaWiki is written in PHP, I don't see any reason to care about other languages. As a data storage format, PHP files seem to be much better than XML files in every way for our purposes.
On Fri, 21 Mar 2008 11:16:14 -0400, Simetrical wrote:
On Thu, Mar 20, 2008 at 11:07 PM, DanTMan dan_the_man@telus.net wrote:
The difference between doing it in PHP and XML, is that using the XML setup, we can read in that data without requiring that any of that data be included or anything (ie: For extensions that are not installed)
What do you mean by "included" in this context? It would not be difficult, if desired, to have separate files to load info and to load the extension itself, or to use some kind of switch to achieve the same effects.
XML:
- Moderately easy to write by hand
- Moderately easy and fast for most modern computer languages to read
and manipulate. Can typically be read in and manipulated with a relatively simple (but not necessarily well-known) set of functions. Parsed forms can be cached in memory data stores, but extra manual checks are needed to make sure the files have not changed.
PHP:
- Very easy to write by hand (fewer delimiters, less redundancy, etc.)
- Incredibly easy and extremely fast for PHP to read and manipulate.
Loading and parsing takes basically no time at all with opcode cache, which every installation that's serious about performance has. Functions to manipulate data structures are extremely familiar, fast, and powerful because they're just regular PHP data structures. Opcode caches will automatically cache the files and maintain integrity with stat()s, but those can be easy avoided by a one-line config change if you use a script like scap that can purge the opcode cache.
- Something of a pain for other languages to read and manipulate.
Possible if it's really necessary, but slow and error-prone. Will need custom functions.
Given that MediaWiki is written in PHP, I don't see any reason to care about other languages. As a data storage format, PHP files seem to be much better than XML files in every way for our purposes.
Is performance much of a consideration here? I'd expect a few short files, parsed infrequently.
XML:
*easy to convert to wiki markup, to include in documentation. *would force a consistent interface if this interface allows editing config settings, which I'd hope it would.
PHP:
* A simpler API; each extension could have an arbitrary configuration callback, and a test method, which would allow the installer to grey out & display a status message for unusable extensions.
On Fri, Mar 21, 2008 at 2:27 PM, Steve Sanbeg ssanbeg@ask.com wrote:
Is performance much of a consideration here? I'd expect a few short files, parsed infrequently.
Every extension must be loaded (at least partially) on every page view. Making ten extra calls to memcached is probably going to add a millisecond, although less if you retrieve them all at once (which memcached supports AFAIK). So not the end of the world. Having to actually parse the XML would be slower.
Overall this will probably not be a bottleneck or a major consideration, though, you're right. Simplicity and ease of use are more important, in PHP's favor. Frankly I don't know PHP's XML API.
Actually, come to think of it, since when do we require any kind of PHP support for XML? We have our own XML function library. I haven't seen us use built-in functions. I think I've seen us optionally use them for SVG validation, but IIRC it skips that part if they don't exist. Does this really not require an extra PHP module dependency?
XML:
*easy to convert to wiki markup, to include in documentation.
No harder than PHP. We convert PHP documentation for extensions to wikitext already, on Special:Version, although that's fairly minimal as documentation goes. What were you thinking would be the advantage here, in a little more detail?
*would force a consistent interface if this interface allows editing config settings, which I'd hope it would.
It seems to me we were talking about info files, not anything that users would be expected to edit. We do dynamically generate some PHP files (well, at least one: LocalSettings.php) with no problems. What do you mean by a consistent interface? Could you expand on this?
On Fri, 21 Mar 2008 14:44:10 -0400, Simetrical wrote:
On Fri, Mar 21, 2008 at 2:27 PM, Steve Sanbeg ssanbeg@ask.com wrote:
Is performance much of a consideration here? I'd expect a few short files, parsed infrequently.
Every extension must be loaded (at least partially) on every page view. Making ten extra calls to memcached is probably going to add a millisecond, although less if you retrieve them all at once (which memcached supports AFAIK). So not the end of the world. Having to actually parse the XML would be slower.
I was assuming this would only be used by a restricted special page to enable/configure available extensions, and that installed extensions would otherwise be the same as they are.
Overall this will probably not be a bottleneck or a major consideration, though, you're right. Simplicity and ease of use are more important, in PHP's favor. Frankly I don't know PHP's XML API.
Actually, come to think of it, since when do we require any kind of PHP support for XML? We have our own XML function library. I haven't seen us use built-in functions. I think I've seen us optionally use them for SVG validation, but IIRC it skips that part if they don't exist. Does this really not require an extra PHP module dependency?
I haven't seen fully compliant (i.e. to the point of marking CDATA) XML, although I haven't really looked for it. I guess that's mostly significant if we want to be able to include arbitrary code (i.e. a test function) and still have these be parseable outside of mediawiki.
XML:
*easy to convert to wiki markup, to include in documentation.
No harder than PHP. We convert PHP documentation for extensions to wikitext already, on Special:Version, although that's fairly minimal as documentation goes. What were you thinking would be the advantage here, in a little more detail?
It seems like some of the proposed fields replicate what's already in the on-wiki documentation. So it might be nice if there was a bot that could comb through all the info files on the CVS, run them through an XSL transform, and convert them into infoboxes to keep the docs up to date. Of course, it should be possible to something similar by getting the relevant parts from a PHP format.
*would force a consistent interface if this interface allows editing config settings, which I'd hope it would.
It seems to me we were talking about info files, not anything that users would be expected to edit. We do dynamically generate some PHP files (well, at least one: LocalSettings.php) with no problems. What do you mean by a consistent interface? Could you expand on this?
Say we have a special page that lists extensions, with an enable/disable and configure option, and that the configuration options are somehow encoded in the file. It could be an XML schema to represent them, so that the extension manager could generate a preferences form, then generate the extensions settings from that; or the extension manager use a callback from the extension to get a form, and let the extension take care of the details.
That's not to say that I have a strong feeling one way or the other, but rather that flexibility for things like these are probably a better deciding factor than raw performance in this case.
Ya, the extension data files only need to be loaded on the special pages where they are needed. ie: Extension lists, and installation/management of them.
The only thing we need to know on every page view, is if an extension is installed or not. And obviously that would be done using a schematically cleanly written, auto-generated PHP file which the manager would make modifications to. You could also offer use of memcached for it, for those who will complain "You must be file-system independent!!!"... However, I'd just screw anyone with that viewpoint, cause this is a complete bonus feature, and the filesystem already needs to be web writable to setup the extensions in the first place.
You don't need to add a PHP dependency to do XML. I looked at this all awhile back. And there is actually a very nice PHP XPATH project. It works on PHP 4 & 5, and has no extra library dependencies. So it could simply be included in MediaWiki and used to read out data from XML files. http://sourceforge.net/projects/phpxpath/
~Daniel Friesen(Dantman) of: -The Gaiapedia (http://gaia.wikia.com) -Wikia ACG on Wikia.com (http://wikia.com/wiki/Wikia_ACG) -and Wiki-Tools.com (http://wiki-tools.com)
Steve Sanbeg wrote:
On Fri, 21 Mar 2008 14:44:10 -0400, Simetrical wrote:
On Fri, Mar 21, 2008 at 2:27 PM, Steve Sanbeg ssanbeg@ask.com wrote:
Is performance much of a consideration here? I'd expect a few short files, parsed infrequently.
Every extension must be loaded (at least partially) on every page view. Making ten extra calls to memcached is probably going to add a millisecond, although less if you retrieve them all at once (which memcached supports AFAIK). So not the end of the world. Having to actually parse the XML would be slower.
I was assuming this would only be used by a restricted special page to enable/configure available extensions, and that installed extensions would otherwise be the same as they are.
Overall this will probably not be a bottleneck or a major consideration, though, you're right. Simplicity and ease of use are more important, in PHP's favor. Frankly I don't know PHP's XML API.
Actually, come to think of it, since when do we require any kind of PHP support for XML? We have our own XML function library. I haven't seen us use built-in functions. I think I've seen us optionally use them for SVG validation, but IIRC it skips that part if they don't exist. Does this really not require an extra PHP module dependency?
I haven't seen fully compliant (i.e. to the point of marking CDATA) XML, although I haven't really looked for it. I guess that's mostly significant if we want to be able to include arbitrary code (i.e. a test function) and still have these be parseable outside of mediawiki.
XML:
*easy to convert to wiki markup, to include in documentation.
No harder than PHP. We convert PHP documentation for extensions to wikitext already, on Special:Version, although that's fairly minimal as documentation goes. What were you thinking would be the advantage here, in a little more detail?
It seems like some of the proposed fields replicate what's already in the on-wiki documentation. So it might be nice if there was a bot that could comb through all the info files on the CVS, run them through an XSL transform, and convert them into infoboxes to keep the docs up to date. Of course, it should be possible to something similar by getting the relevant parts from a PHP format.
*would force a consistent interface if this interface allows editing config settings, which I'd hope it would.
It seems to me we were talking about info files, not anything that users would be expected to edit. We do dynamically generate some PHP files (well, at least one: LocalSettings.php) with no problems. What do you mean by a consistent interface? Could you expand on this?
Say we have a special page that lists extensions, with an enable/disable and configure option, and that the configuration options are somehow encoded in the file. It could be an XML schema to represent them, so that the extension manager could generate a preferences form, then generate the extensions settings from that; or the extension manager use a callback from the extension to get a form, and let the extension take care of the details.
That's not to say that I have a strong feeling one way or the other, but rather that flexibility for things like these are probably a better deciding factor than raw performance in this case.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Mar 21, 2008 at 8:21 PM, DanTMan dan_the_man@telus.net wrote:
You don't need to add a PHP dependency to do XML. I looked at this all awhile back. And there is actually a very nice PHP XPATH project. It works on PHP 4 & 5, and has no extra library dependencies. So it could simply be included in MediaWiki and used to read out data from XML files. http://sourceforge.net/projects/phpxpath/
I still don't see the point in bothering with XML when PHP is easier all around, unless someone foresees someone using another language. But it really doesn't matter much. Whoever implements it gets to decide.
Simetrical a écrit : <snip>
I still don't see the point in bothering with XML when PHP is easier all around, unless someone foresees someone using another language. But it really doesn't matter much. Whoever implements it gets to decide.
Maybe because an XML file can be used with any other language ? Having data in PHP files restrict its access to PHP. For example, you might want to write an XSL file to automaticly generate a website presenting the extensions. In my opinion, it is ten time easier to maintain and enhance than a PHP script.
We should probably also have a notion of dependencies:
<dependencies> <dependency>Semantic MediaWiki</dependency> </dependencies> This will be important for extensions like the extensions which use SMW, ones which use DPL, or things like the BizzWiki extensions or other ones which require things like the SimpleRegex classes, or other extensions meant to simplify functionality for other extensions. Trying to install those without their dependencies would lead to fatal php errors.
It would also be nice if we could list PHP dependencies, that could be checked by some core code, rather than by each extension. For instance, quite a few people have issues with the LDAP Authentication plugin becuase their PHP doesn't have LDAP support either compiled in, or included via a module.
V/r,
Ryan Lane
On Fri, Mar 21, 2008 at 11:43 AM, Lane, Ryan Ryan.Lane@ocean.navo.navy.mil wrote:
It would also be nice if we could list PHP dependencies, that could be checked by some core code, rather than by each extension. For instance, quite a few people have issues with the LDAP Authentication plugin becuase their PHP doesn't have LDAP support either compiled in, or included via a module.
Usually we just use function_exists().
Ya but tis a little counterintuitive...
Activate LDAP Plugin ---- The LDAP Plugin may be enabled. Are you sure you wish to enable it? Yes, No? ---- Error: PHP Does not include LDAP Support and so the LDAP Plugin cannot be used.
^_^ In a case like that, shouldn't the extension list page note "LDAP Support is not installed into php, you cannot enable this extension. Please recompile PHP with the proper support to be able to enable this extension."
~Daniel Friesen(Dantman) of: -The Gaiapedia (http://gaia.wikia.com) -Wikia ACG on Wikia.com (http://wikia.com/wiki/Wikia_ACG) -and Wiki-Tools.com (http://wiki-tools.com)
Simetrical wrote:
On Fri, Mar 21, 2008 at 11:43 AM, Lane, Ryan Ryan.Lane@ocean.navo.navy.mil wrote:
It would also be nice if we could list PHP dependencies, that could be checked by some core code, rather than by each extension. For instance, quite a few people have issues with the LDAP Authentication plugin becuase their PHP doesn't have LDAP support either compiled in, or included via a module.
Usually we just use function_exists().
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Mar 21, 2008 at 12:15 PM, DanTMan dan_the_man@telus.net wrote:
^_^ In a case like that, shouldn't the extension list page note "LDAP Support is not installed into php, you cannot enable this extension. Please recompile PHP with the proper support to be able to enable this extension."
Yes, I see your point. This dependency requirement could take the form of a list of functions that must be present.
On Fri, Mar 21, 2008 at 12:15 PM, DanTMan dan_the_man@telus.net wrote:
^_^ In a case like that, shouldn't the extension list page
note "LDAP
Support is not installed into php, you cannot enable this
extension.
Please recompile PHP with the proper support to be able to
enable this
extension."
Yes, I see your point. This dependency requirement could take the form of a list of functions that must be present.
That would make extension support quite a bit easier. As of now, I have to tell people to enable debug mode, and copy/paste the output, just to find out they don't have LDAP support.
V/r,
Ryan Lane
On Fri, Mar 21, 2008 at 12:49 PM, Lane, Ryan Ryan.Lane@ocean.navo.navy.mil wrote:
That would make extension support quite a bit easier. As of now, I have to tell people to enable debug mode, and copy/paste the output, just to find out they don't have LDAP support.
Well, your life could be a little easier than *that*.
if( !function_exists( 'ldap_connect' ) ) { throw new MWException( 'LDAP is not installed. Please either install it or disable the LDAP extension.' ); }
But having it checked at install time would be better. Right now there's no such *thing* as install time.
wikitech-l@lists.wikimedia.org