I have been working on the ResourceLoader branch, where I've ended up writing a CSSMin class which performs CSS minification, URI-remapping and data-URI in-lining. It got me thinking that this class would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
* Software we've ported to PHP ourselves like our native-PHP CDB implementation or CSSJanus are buried in our code-base, and make use of a couple of trivial wf* global functions, making it somewhat inaccessible to third-party users. Which sucks because third-party users are important! They use the code in their own systems, make improvements and potentially pass them back to us, however if we don't make these things more general-purpose the code will more likely get taken from our repository, tweaked and never passed back; if we don't make it more easily accessible the code will never be found and we won't be taking advantage of the entire PHP development community. Sadness... * Software we've borrowed from other projects like JSMin are also buried within our MediaWiki-proprietary code, and while these libraries can operate independently of MediaWiki, we need to make it clear that they should be kept in sync with their original sources both, upstream and down. * Software we've created is often potentially useful to other projects, but unfortunately tied to and buried within MediaWiki. In some of these cases, the ties to MediaWiki are trivial and could be either optional or removed entirely, and the component could be factored out to a more general-purpose library, available for re-use.
I don't have a very mature proposal for solving this completely, but as a first step, it seems like we should have a libraries folder which we can move things that can function in a stand-alone manner to. Initial candidates appear to be libraries that already function in a stand-alone way such as JSMin.php, CSSJanus.php, and CSSMin.php (in the resourceloader branch right now but will be coming to trunk soon). Additional software could be moved into this space after some un-tethering such as Cdb/Cdb_PHP, DjVuImage, etc.
Overall, I think it would be great if we could take a look at this and other ways to better share our work with non-MediaWiki projects, and give back to the open-source community.
I welcome your thoughts and input.
- Trevor
On Thu, Aug 26, 2010 at 1:57 PM, Trevor Parscal tparscal@wikimedia.org wrote:
I have been working on the ResourceLoader branch, where I've ended up writing a CSSMin class which performs CSS minification, URI-remapping and data-URI in-lining. It got me thinking that this class would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
There are already some files that are explicitly meant to be reusable, like IEContentAnalyzer.php and includes/normal/ (both written by Tim Starling). I think this is a nice idea, although I don't think anyone has taken us up on reusing either of those (maybe I'm wrong). For convenience, I guess we want to keep them under phase3/, but maybe putting them in a special folder like phase3/libraries/ would be a good idea. We could try advertising them a bit more, too.
Right - to be clear, I meant phase3/libraries, not moving things completely out of phase3.
- Trevor
On 8/26/10 11:50 AM, Aryeh Gregor wrote:
On Thu, Aug 26, 2010 at 1:57 PM, Trevor Parscaltparscal@wikimedia.org wrote:
I have been working on the ResourceLoader branch, where I've ended up writing a CSSMin class which performs CSS minification, URI-remapping and data-URI in-lining. It got me thinking that this class would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
There are already some files that are explicitly meant to be reusable, like IEContentAnalyzer.php and includes/normal/ (both written by Tim Starling). I think this is a nice idea, although I don't think anyone has taken us up on reusing either of those (maybe I'm wrong). For convenience, I guess we want to keep them under phase3/, but maybe putting them in a special folder like phase3/libraries/ would be a good idea. We could try advertising them a bit more, too.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Aug 26, 2010 at 2:53 PM, Trevor Parscal tparscal@wikimedia.org wrote:
Right - to be clear, I meant phase3/libraries, not moving things completely out of phase3.
Yeah -- moving them out of phase3/ would be attractive, so that they really look like separate projects, but probably not worth it.
Trevor Parscal wrote:
* Software we've ported to PHP ourselves like our native-PHP CDB implementation or CSSJanus are buried in our code-base, and make use of a couple of trivial wf* global functions, making it somewhat inaccessible to third-party users.
We should probably provide stub a stub version of those functions, so reusers don't have to declare a bunch of functions just because we call wfProfileIn, wfGetIP and set a couple of hooks.
Other problem would be our usage of wrappers like WebRequest.
You can also use things like function_exists() to conditionally call MediaWiki versions of things or native PHP stuff.
- Trevor
On 8/26/10 3:58 PM, Platonides wrote:
Trevor Parscal wrote:
* Software we've ported to PHP ourselves like our native-PHP CDB implementation or CSSJanus are buried in our code-base, and make use of a couple of trivial wf* global functions, making it somewhat inaccessible to third-party users.We should probably provide stub a stub version of those functions, so reusers don't have to declare a bunch of functions just because we call wfProfileIn, wfGetIP and set a couple of hooks.
Other problem would be our usage of wrappers like WebRequest.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 26 August 2010 16:13, Trevor Parscal tparscal@wikimedia.org wrote:
You can also use things like function_exists() to conditionally call MediaWiki versions of things or native PHP stuff.
With respect, this is the wrong way to go about it. Unconditionally remove all MediaWIki from within the library, and, if necessary, create a wrapper that provides the extra functionality we need. Obviously, if it'd need a huge wrapper compared to the size of the library, it's not worth extracting - the chances are that any re-users will need to create vast wrappers too. The current examples of deliberately MediaWIki free code are very good candidates for this as they provide complex functionality that's really hard to do right yet they have very simple interfaces.
Conrad
- Trevor
On 8/26/10 3:58 PM, Platonides wrote:
Trevor Parscal wrote:
* Software we've ported to PHP ourselves like our native-PHP CDB implementation or CSSJanus are buried in our code-base, and make use of a couple of trivial wf* global functions, making it somewhat inaccessible to third-party users.
We should probably provide stub a stub version of those functions, so reusers don't have to declare a bunch of functions just because we call wfProfileIn, wfGetIP and set a couple of hooks.
Other problem would be our usage of wrappers like WebRequest.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Aug 26, 2010 at 7:21 PM, Conrad Irwin conrad.irwin@gmail.com wrote:
On 26 August 2010 16:13, Trevor Parscal tparscal@wikimedia.org wrote:
You can also use things like function_exists() to conditionally call MediaWiki versions of things or native PHP stuff.
With respect, this is the wrong way to go about it. Unconditionally remove all MediaWIki from within the library, and, if necessary, create a wrapper that provides the extra functionality we need.
This.
The current examples of deliberately MediaWIki free code are very good candidates for this as they provide complex functionality that's really hard to do right yet they have very simple interfaces.
IEContentAnaylzer is probably the best example. CDB-related stuff too.
I went ahead and moved some stuff around in r71763.
-Chad
Chad wrote:
IEContentAnaylzer is probably the best example. CDB-related stuff too.
I went ahead and moved some stuff around in r71763.
-Chad
What about relicensing it at the same time? To BSD or so. I think IEContentAnaylzer is already OSI-compatible (the pieces added on r44505 from had been added in 44505) so it would be just changing the header notice. I don't think r63254 qualifies for copyright (case change for doxygen), but for extra correctness the ok by ialex would be nice.
On Fri, Aug 27, 2010 at 8:54 AM, Platonides Platonides@gmail.com wrote:
What about relicensing it at the same time? To BSD or so.
Why?
Aryeh Gregor wrote:
On Fri, Aug 27, 2010 at 8:54 AM, Platonides Platonides@gmail.com wrote:
What about relicensing it at the same time? To BSD or so.
Why?
a) To encourage reuse and b) Avoid debates about using the libraries with other non-GPL code (eg. under PHP license)
License of each file in libs would depend on each the module creator, of course.
On Fri, Aug 27, 2010 at 4:31 PM, Platonides Platonides@gmail.com wrote:
a) To encourage reuse and b) Avoid debates about using the libraries with other non-GPL code (eg. under PHP license)
The exact same considerations apply to MediaWiki as a whole, though, and there are also considerations in favor of GPL. Clearly some people prefer BSD, but what does this have to do with breaking code out into libraries? It seems orthogonal, so I was puzzled by you mentioning it in this particular context. (You can license your files as BSD if you like even if they're in MediaWiki core, or at least some people have done that.)
On 8/26/10 4:25 PM, Chad wrote:
On Thu, Aug 26, 2010 at 7:21 PM, Conrad Irwinconrad.irwin@gmail.com wrote:
On 26 August 2010 16:13, Trevor Parscaltparscal@wikimedia.org wrote:
You can also use things like function_exists() to conditionally call MediaWiki versions of things or native PHP stuff.
With respect, this is the wrong way to go about it. Unconditionally remove all MediaWIki from within the library, and, if necessary, create a wrapper that provides the extra functionality we need.
This.
The current examples of deliberately MediaWIki free code are very good candidates for this as they provide complex functionality that's really hard to do right yet they have very simple interfaces.
IEContentAnaylzer is probably the best example. CDB-related stuff too.
I went ahead and moved some stuff around in r71763.
-Chad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Following your (chad's) lead, I've now moved some more stuff to includes/libs -- r72678
* JSMin -- Ryan Grove's PHP port (http://github.com/rgrove/jsmin-php/) of Douglas Crockford's JSMin library (http://www.crockford.com/javascript/jsmin.html) for removing white-space and comments from JavaScript * CSSJanus -- Roan Kattouw's PHP port of Google's CSSJanus library (http://code.google.com/p/cssjanus/) for transforming left-to-right style-sheets into right-to-left orientation * CSSMin -- Trevor Parscal's (my) library for CSS minification, URL remapping and data-URL embedding
There are more Gems in MediaWiki which can likely be moved here too, but this is at least a good start.
- Trevor
* Platonides Platonides@gmail.com [Fri, 27 Aug 2010 00:58:25 +0200]:
Trevor Parscal wrote:
* Software we've ported to PHP ourselves like our native-PHP CDB implementation or CSSJanus are buried in our code-base, and
make
use of a couple of trivial wf* global functions, making it somewhat inaccessible to third-party users.We should probably provide stub a stub version of those functions, so reusers don't have to declare a bunch of functions just because we
call
wfProfileIn, wfGetIP and set a couple of hooks.
Other problem would be our usage of wrappers like WebRequest.
I like WebRequest. If there was an request router, in similar way to CakePHP and another frameworks, it would even be better. Sorry if that's a bit off-topic. Dmitriy
On 27/08/10 03:57, Trevor Parscal wrote:
I have been working on the ResourceLoader branch, where I've ended up writing a CSSMin class which performs CSS minification, URI-remapping and data-URI in-lining. It got me thinking that this class would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
You should put it in PEAR. Very few people are going to find it if it's just in the MediaWiki core, regardless of what directory you put it in.
-- Tim Starling
I wrote:
You should put it in PEAR. Very few people are going to find it if it's just in the MediaWiki core, regardless of what directory you put it in.
I wouldn't like to be accused of hypocrisy, so I released a new version of File_Ogg:
http://pear.php.net/package/File_Ogg
It's not too hard.
-- Tim Starling
So, you suggest we maintain it in 2 places (where we use it in MediaWiki core and where people have access to it), and ask that others push changes back to where?
This only solves part of the problem, unless we stop including these libraries in core and make MediaWiki dependent on some PEAR modules, which seems undesirable.
- Trevor
On 8/26/10 6:46 PM, Tim Starling wrote:
On 27/08/10 03:57, Trevor Parscal wrote:
I have been working on the ResourceLoader branch, where I've ended up writing a CSSMin class which performs CSS minification, URI-remapping and data-URI in-lining. It got me thinking that this class would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
You should put it in PEAR. Very few people are going to find it if it's just in the MediaWiki core, regardless of what directory you put it in.
-- Tim Starling
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 8/26/10 6:46 PM, Tim Starling wrote:
On 27/08/10 03:57, Trevor Parscal wrote:
It got me thinking that [the CSSmin class] would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
You should put it in PEAR. Very few people are going to find it if it's just in the MediaWiki core, regardless of what directory you put it in.
So, you suggest we maintain it in 2 places (where we use it in MediaWiki core and where people have access to it), and ask that others push changes back to where?
Tim's right...putting it in PEAR is a great way to signal "this is a reusable component; please reuse this".
PEAR isn't a source of truth; it's a distribution channel. In order to get discovered and used, libraries need to be in a distribution channel that people use. Many people use PEAR, other people use Linux distribution repositories.
svn.wikimedia.org would continue to be the original source repository for the library, but PEAR would be one of many distribution channels. The PEAR version would only get updated each time you release the library.
Rob
My understanding is that PEAR has a specific coding style and set of standards, which is different than that of MediaWiki. Also things like JSMin should not be hijacked and thrown into PEAR -- it came from http://github.com/rgrove/jsmin-php/ -- it already has a distribution chanel.
In other words, the idea of using PEAR works really well for things we took from PEAR or think we should specifically contribute to PEAR, but not for all cases that I originally presented. Moving stand-alone things into a specific place, such as what ^demon has done in r 71774, is still useful as at least a first step towards sharing and generally being better about giving back to the larger open source community.
I do however agree with Tim however that simply moving libraries to phase3/includes/libs, while introducing a more sanitary convention for 3rd party library inclusion, does not by itself raise awareness to the availability of the libraries - that part of the problem may need to be solved in multiple ways, only one of which is likely to be PEAR.
- Trevor
On 8/27/10 9:44 AM, Rob Lanphier wrote:
On 8/26/10 6:46 PM, Tim Starling wrote:
On 27/08/10 03:57, Trevor Parscal wrote:
It got me thinking that [the CSSmin class] would be pretty useful to non-MediaWiki projects too, but sadly we don't have a history of sharing in this way...
You should put it in PEAR. Very few people are going to find it if it's just in the MediaWiki core, regardless of what directory you put it in.
So, you suggest we maintain it in 2 places (where we use it in MediaWiki core and where people have access to it), and ask that others push changes back to where?
Tim's right...putting it in PEAR is a great way to signal "this is a reusable component; please reuse this".
PEAR isn't a source of truth; it's a distribution channel. In order to get discovered and used, libraries need to be in a distribution channel that people use. Many people use PEAR, other people use Linux distribution repositories.
svn.wikimedia.org would continue to be the original source repository for the library, but PEAR would be one of many distribution channels. The PEAR version would only get updated each time you release the library.
Rob _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Trevor Parscal tparscal@wikimedia.org writes:
My understanding is that PEAR has a specific coding style and set of standards, which is different than that of MediaWiki.
Coding style is flexible, especially if we're the ones doing the distribution.
By this, I mean that “putting it in PEAR” could just mean setting up a MediaWiki PEAR channel. PHPUnit, for example, moved http://pear.php.net/package/PHPUnit to its own PEAR channel http://pear.php.net/package/PHPUnit. Other packages related to PHPUnit are also maintained there.
And as Rob pointed out, putting these “sharable artifacts” in a PEAR channel doesn't mean that we can't continue to maintain them in SVN.
Setting up pear.mediawiki.org seems like a good solution to me.
Mark.
Mark A. Hershberger wrote:
Trevor Parscal tparscal@wikimedia.org writes:
My understanding is that PEAR has a specific coding style and set of standards, which is different than that of MediaWiki.
Coding style is flexible, especially if we're the ones doing the distribution.
By this, I mean that “putting it in PEAR” could just mean setting up a MediaWiki PEAR channel. PHPUnit, for example, moved http://pear.php.net/package/PHPUnit to its own PEAR channel http://pear.php.net/package/PHPUnit. Other packages related to PHPUnit are also maintained there.
I think the second should be http://pear.phpunit.de/
And as Rob pointed out, putting these “sharable artifacts” in a PEAR channel doesn't mean that we can't continue to maintain them in SVN.
Setting up pear.mediawiki.org seems like a good solution to me.
Mark.
That's an interesting idea. But I'm not sure if it would achieve the "make it discoverable" goal.
On 8/28/10 11:17 AM, Platonides wrote:
Mark A. Hershberger wrote:
Trevor Parscaltparscal@wikimedia.org writes:
My understanding is that PEAR has a specific coding style and set of standards, which is different than that of MediaWiki.
Coding style is flexible, especially if we're the ones doing the distribution.
By this, I mean that “putting it in PEAR” could just mean setting up a MediaWiki PEAR channel. PHPUnit, for example, moved http://pear.php.net/package/PHPUnit to its own PEAR channel http://pear.php.net/package/PHPUnit. Other packages related to PHPUnit are also maintained there.
I think the second should be http://pear.phpunit.de/
And as Rob pointed out, putting these “sharable artifacts” in a PEAR channel doesn't mean that we can't continue to maintain them in SVN.
Setting up pear.mediawiki.org seems like a good solution to me.
Mark.
That's an interesting idea. But I'm not sure if it would achieve the "make it discoverable" goal.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It does make it accessible though!
- Trevor
Platonides Platonides@gmail.com writes:
That's an interesting idea. But I'm not sure if it would achieve the "make it discoverable" goal.
“Make it discoverable” is more of a social problem, though. It can be solved by using the visibility of blogs, etc. Talking about the libraries and using them in other projects will help.
Mark.
On Mon, Aug 30, 2010 at 10:40 AM, Mark A. Hershberger mah@everybody.org wrote:
“Make it discoverable” is more of a social problem, though. It can be solved by using the visibility of blogs, etc. Talking about the libraries and using them in other projects will help.
Getting them in standard searchable, easy-to-use distribution channels like pear.php.net will also help a lot. I think it's fine if we reformat them to follow PEAR standards instead of MediaWiki standards for that purpose.
wikitech-l@lists.wikimedia.org