Hi all,
Recently, I've been working with the Symfony web framework [1]. One of the classes they include is called the sfFinder class [2], which is a fluid, easy-to-use file finder class. It searches for files or directories in the filesystem, using a fluid PHP 5 interface. It has no dependancies, so it should work fine with MediaWiki. After finding numerous instances of opendir(), readdir(), closedir(), etc. in MediaWiki, I thought that it would be a good idea to use one centralized class to do all file searching. There is only 1 potential issue I see, though. It is MIT licensed, which is GPL compatible, so it should be okay to implement it, but I'm not too clear on this issue.
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code is very similar though, so most of it should apply to the current version.
What would people think of a change like this. I would like to see this happen, but I'd like some more opinions before I look into implementing it.
-X!
[1] - http://www.symfony-project.org [2] - http://trac.symfony-project.org/browser/branches/1.4/lib/util/sfFinder.class... [3] - http://www.symfony-project.org/cookbook/1_2/en/finder
* Soxred93 soxred93@gmail.com [Mon, 20 Dec 2010 23:21:34 -0500]:
Hi all,
Recently, I've been working with the Symfony web framework [1]. One of the classes they include is called the sfFinder class [2], which is a fluid, easy-to-use file finder class. It searches for files or directories in the filesystem, using a fluid PHP 5 interface. It has
no
dependancies, so it should work fine with MediaWiki. After finding numerous instances of opendir(), readdir(), closedir(), etc. in MediaWiki, I thought that it would be a good idea to use one
centralized
class to do all file searching. There is only 1 potential issue I see, though. It is MIT licensed, which is GPL compatible, so it should be okay to implement it, but I'm not too clear on this issue.
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of
PHP
files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code
is
very similar though, so most of it should apply to the current
version.
What would people think of a change like this. I would like to see
this
happen, but I'd like some more opinions before I look into
implementing
it.
The optimal way to use a framework is to integrate it through the whole code base. For example, special pages, actions may use Symphony routing. I do agree that GlobalFunctions are somewhat outdated (although the calls are short and compact). Also some basic classes such as WebRequest or Xml or Http are not powerful enough. Perhaps some framework would be better to use (but slower?). Dmitriy
On Tue, Dec 21, 2010 at 7:37 AM, Dmitriy Sintsov questpc@rambler.ru wrote:
The optimal way to use a framework is to integrate it through the whole code base. For example, special pages, actions may use Symphony routing. I do agree that GlobalFunctions are somewhat outdated (although the calls are short and compact). Also some basic classes such as WebRequest or Xml or Http are not powerful enough. Perhaps some framework would be better to use (but slower?).
Before arguing whether or not we need a new framework, it is perhaps a good idea to talk about requirements. What is not powerful enough about MWHttp? What is outdated about GlobalFunctions? Could it be updated to meet web 2.0 TM requirements?
* Bryan Tong Minh bryan.tongminh@gmail.com [Tue, 21 Dec 2010 08:57:47 +0100]:
On Tue, Dec 21, 2010 at 7:37 AM, Dmitriy Sintsov questpc@rambler.ru wrote:
The optimal way to use a framework is to integrate it through the
whole
code base. For example, special pages, actions may use Symphony
routing.
I do agree that GlobalFunctions are somewhat outdated (although the calls are short and compact). Also some basic classes such as
WebRequest
or Xml or Http are not powerful enough. Perhaps some framework would
be
better to use (but slower?).
Before arguing whether or not we need a new framework, it is perhaps a good idea to talk about requirements. What is not powerful enough about MWHttp? What is outdated about GlobalFunctions? Could it be updated to meet web 2.0 TM requirements?
There was no http proxy support and no multipart post encoding support in Http class (I've had a hosting without php curl module). GlobalFunctions are compact and nice yet not object-oriented. Routing is not generalized enough, however is possible. Dmitriy
On Tue, Dec 21, 2010 at 5:21 AM, Soxred93 soxred93@gmail.com wrote:
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code is very similar though, so most of it should apply to the current version.
I personally dislike the chained syntax. A file system abstraction layer would be a good idea, but I believe the standard MediaWiki convention of passing arrays around rather than function chaining is much more elegant.
Finder::openFile( 'filename' ); Finder::findFile('7path', array( 'name' => '*.php'))->delete(); for ( Finder::findFiles($IP) as $file ) { $file->delete(); }
On 10-12-20 11:56 PM, Bryan Tong Minh wrote:
On Tue, Dec 21, 2010 at 5:21 AM, Soxred93soxred93@gmail.com wrote:
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code is very similar though, so most of it should apply to the current version.
I personally dislike the chained syntax. A file system abstraction layer would be a good idea, but I believe the standard MediaWiki convention of passing arrays around rather than function chaining is much more elegant.
Finder::openFile( 'filename' ); Finder::findFile('7path', array( 'name' => '*.php'))->delete(); for ( Finder::findFiles($IP) as $file ) { $file->delete(); }
*twitch* Yea, I don't mind chained syntax, I've moved on to JavaScript after all... but that api looks horrid to me... There should be a clear distinction between safe names, and things where fnmatch patterns are interpreted. Ruby gave me nothing but trouble in that regard... And it's a really bad idea to have to go and do complex wildcard matching and recursion for something that's really simple doesn't need wildcard matching or recursion and would actually end up with unexpected bugs if the name it passed was interpreted.
foreach ( Finder::openDirectory($wgStyleDirectory, array( 'type' => 'directory' )) as $dir ) { $file = $dir->file($dir->name() . '.php'); if ( $file->isReadable ) { require_once( $file->path() ); } else { $skFile = $dir->findFile("*.skin.php"); if ( $skFile->isReadable ) { $skinName = $skFile->basename("skin.php"); $wgValidSkinNames[$dir->name()] = $skinName; $wgAutoloadClasses["Skin".$skinName] = $skFile->path; } } }
Finder::findFiles($wgFileCacheDirectory, array( 'pattern' => '*.html' ))->delete();
On 21/12/10 15:21, Soxred93 wrote:
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories
You can use RecursiveDirectoryIterator to do that. There's a cute example in the online manual comments.
-- Tim Starling
You can. But the main advantage is less writing, more understanding. I see type('file')->name('*.php')->in('/path/') tp be easier to understand than RegexIterator(RecursiveIteratorIterator(RecursiveDirectoryIterator('path/to/project/')), '/^.+.php$/i', RecursiveRegexIterator::GET_MATCH);
BUt we all have our own styles. I would like to see this included, even if everyone doesn't use it. The current usage in MW isn't even RDI, it's just opendir(), readdir(), etc.
-X!
On Dec 21, 2010, at 7:22 AM, Tim Starling wrote:
On 21/12/10 15:21, Soxred93 wrote:
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories
You can use RecursiveDirectoryIterator to do that. There's a cute example in the online manual comments.
-- Tim Starling
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I hate these frameworks, so a big -1 from me.
-Chad
On Dec 20, 2010 11:22 PM, "Soxred93" soxred93@gmail.com wrote:
Hi all,
Recently, I've been working with the Symfony web framework [1]. One of the classes they include is called the sfFinder class [2], which is a fluid, easy-to-use file finder class. It searches for files or directories in the filesystem, using a fluid PHP 5 interface. It has no dependancies, so it should work fine with MediaWiki. After finding numerous instances of opendir(), readdir(), closedir(), etc. in MediaWiki, I thought that it would be a good idea to use one centralized class to do all file searching. There is only 1 potential issue I see, though. It is MIT licensed, which is GPL compatible, so it should be okay to implement it, but I'm not too clear on this issue.
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code is very similar though, so most of it should apply to the current version.
What would people think of a change like this. I would like to see this happen, but I'd like some more opinions before I look into implementing it.
-X!
[1] - http://www.symfony-project.org [2] - http://trac.symfony-project.org/browser/branches/1.4/lib/util/sfFinder.class... [3] - http://www.symfony-project.org/cookbook/1_2/en/finder _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'm not looking to integrate these frameworks entirely into MediaWiki; I'm just talking about this one single file with one single class. (to be fair, it's 3 classes, but they're all in that one file).
-X!
On Dec 21, 2010, at 9:09 AM, Chad wrote:
I hate these frameworks, so a big -1 from me.
-Chad
On Dec 20, 2010 11:22 PM, "Soxred93" soxred93@gmail.com wrote:
Hi all,
Recently, I've been working with the Symfony web framework [1]. One of the classes they include is called the sfFinder class [2], which is a fluid, easy-to-use file finder class. It searches for files or directories in the filesystem, using a fluid PHP 5 interface. It has no dependancies, so it should work fine with MediaWiki. After finding numerous instances of opendir(), readdir(), closedir(), etc. in MediaWiki, I thought that it would be a good idea to use one centralized class to do all file searching. There is only 1 potential issue I see, though. It is MIT licensed, which is GPL compatible, so it should be okay to implement it, but I'm not too clear on this issue.
The usage is simple: sfFinder::type('file')->name('*.php')->in('/path/to/dir'); //list of PHP files in directory and all subdirectories sfFinder::type('file')->name('*.php')->in('/path/to/dir')->recurse(0); //list of PHP files in that directory only sfFinder::type('dir')->name('foo')->in('/path/to/dir'); //list of directories with the name "foo" There is documentation at [3], but it's for an old version. The code is very similar though, so most of it should apply to the current version.
What would people think of a change like this. I would like to see this happen, but I'd like some more opinions before I look into implementing it.
-X!
[1] - http://www.symfony-project.org [2] - http://trac.symfony-project.org/browser/branches/1.4/lib/util/sfFinder.class... [3] - http://www.symfony-project.org/cookbook/1_2/en/finder _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Dec 21, 2010 at 10:16 AM, Soxred93 soxred93@gmail.com wrote:
I'm not looking to integrate these frameworks entirely into MediaWiki; I'm just talking about this one single file with one single class. (to be fair, it's 3 classes, but they're all in that one file).
-X!
Mmmmm flavour^H^H^H^H framework of the month.
On Tue, Dec 21, 2010 at 8:16 AM, Soxred93 soxred93@gmail.com wrote:
I'm not looking to integrate these frameworks entirely into MediaWiki; I'm just talking about this one single file with one single class. (to be fair, it's 3 classes, but they're all in that one file).
Before going into too much detail on the thread, consider what you actually need out of a fancy directory iterator. Offhand, I really can't think of many places where that even *happens* in MediaWiki... maybe when purging thumbnails?
Now, if you *do* find yourself writing lots of new code that does a lot of directory iteration, then feel free to use whatever convenient mechanism is nice and doesn't have security or licensing problems or otherwise takes more effort than just using things built into PHP (there's at least 3 or for directory iterator systems already, including the opendir() stuff, the object-oriented variant in the Dir class, and the SPL iterators).
Getting code done and working will be far, far more important than any list discussion about hypotheticals.
-- brion
Before going into too much detail on the thread, consider what you actually need out of a fancy directory iterator. Offhand, I really can't think of many places where that even *happens* in MediaWiki... maybe when purging thumbnails?
I count 10 instances of opendir() exactly in trunk.
takes more effort than just using things built into PHP (there's at least 3 or for directory iterator systems already, including the opendir() stuff, the object-oriented variant in the Dir class, and the SPL iterators).
The main part of my push for this is that it isn't a new feature, but an easier way to read and write code. Myself, at least, readdir(), opendir(), etc is very arcane and confusing (same reason I don't use fopen, etc). RDI and SPL are even more confusing. I'm just going from ym own experience here, but I'm just thinking that it would be a good addition.
-X!
Tell you what -- 10 uses of opendir shouldnt take too long to replace; write the code and let's see how it actually looks. Us all sitting in the list bikeshedding won't be much use, but if it makes the code cleaner, hey win!
-- brion On Dec 21, 2010 9:16 AM, "Soxred93" soxred93@gmail.com wrote:
Before going into too much detail on the thread, consider what you
actually
need out of a fancy directory iterator. Offhand, I really can't think of many places where that even *happens* in MediaWiki... maybe when purging thumbnails?
I count 10 instances of opendir() exactly in trunk.
takes more effort than just using things built into PHP (there's at least 3 or for directory iterator systems already, including the opendir() stuff, the object-oriented variant in the Dir class, and the SPL iterators).
The main part of my push for this is that it isn't a new feature, but an
easier way to read and write code. Myself, at least, readdir(), opendir(), etc is very arcane and confusing (same reason I don't use fopen, etc). RDI and SPL are even more confusing. I'm just going from ym own experience here, but I'm just thinking that it would be a good addition.
-X! _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Dec 21, 2010 at 12:36 PM, Brion Vibber brion@pobox.com wrote:
Tell you what -- 10 uses of opendir shouldnt take too long to replace; write the code and let's see how it actually looks. Us all sitting in the list bikeshedding won't be much use, but if it makes the code cleaner, hey win!
-- brion
Well it's in trunk now[0] (plus a few followups). It's shorter, yes. Some profiling before/after would probably be good.
-Chad
[0] http://www.mediawiki.org/wiki/Special:Code/MediaWiki/78903
On Thu, Dec 23, 2010 at 1:47 PM, Chad innocentkiller@gmail.com wrote:
Well it's in trunk now[0] (plus a few followups). It's shorter, yes. Some profiling before/after would probably be good.
It's shorter by only a handful of lines, at the expense of using an obscure library rather than built-in PHP functions. Actually, not only are opendir()/readdir()/closedir() built-in PHP functions, they've been the standard way to manipulate directories in Unix for decades. And even if you've never seen them before, they're very simple to understand. The longest segment of code replaced was six lines. I don't see this change as a win at all.
Put another way -- the total change is not shorter. It removes maybe 20 lines, and adds over 800. It's a drastic increase in overall complexity given how few calls we have of this type. If we were doing this type of stuff a lot, then it would definitely make sense to abstract the work out to a library to reduce errors -- but we're not.
Soxred93 wrote:
Before going into too much detail on the thread, consider what you actually need out of a fancy directory iterator. Offhand, I really can't think of many places where that even *happens* in MediaWiki... maybe when purging thumbnails?
I count 10 instances of opendir() exactly in trunk.
1 in language (listing languages), 3 in filerepo, 1 in the installer (listing extensions) and the other in 4 file in maintenance.
takes more effort than just using things built into PHP (there's at least 3 or for directory iterator systems already, including the opendir() stuff, the object-oriented variant in the Dir class, and the SPL iterators).
The main part of my push for this is that it isn't a new feature, but an easier way to read and write code. Myself, at least, readdir(), opendir(), etc is very arcane and confusing (same reason I don't use fopen, etc). RDI and SPL are even more confusing. I'm just going from ym own experience here, but I'm just thinking that it would be a good addition.
-X!
We are only using opendir for getting a full directory list. It's fine changing it to DirectoryIterator but I don't think that symfony sfFinder would be too useful here.
This is a bit of NIH, but a) We wouldn't be using its full potential. b) We will need to modify it (eg. usage of sfException) c) Our copy becoming outdated from upstream is bad. We could keep the changes to a minimum, but. d) How long do you think it will take until someone wants to change its underscore_methods to camelCase?
On 12/22/2010 12:16 AM, Platonides wrote:
We are only using opendir for getting a full directory list.
That's a good point. Perhaps what we need is simply a utility method to list all files in a directory.
In fact, I just realized that PHP already has one. It's called scandir(). Its only flaw IMO is that it doesn't automatically skip the current and parent dir entries, but you could always do something like
$files = array_diff( scandir( $dir ), array( '.', '..' ) );
to accomplish that cleanly (or use preg_grep() to remove all dotfiles if you prefer).
Glob works too I think.
-- brion On Dec 23, 2010 12:06 PM, "Ilmari Karonen" nospam@vyznev.net wrote:
On 12/22/2010 12:16 AM, Platonides wrote:
We are only using opendir for getting a full directory list.
That's a good point. Perhaps what we need is simply a utility method to list all files in a directory.
In fact, I just realized that PHP already has one. It's called scandir(). Its only flaw IMO is that it doesn't automatically skip the current and parent dir entries, but you could always do something like
$files = array_diff( scandir( $dir ), array( '.', '..' ) );
to accomplish that cleanly (or use preg_grep() to remove all dotfiles if you prefer).
-- Ilmari Karonen
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
In the HISTORY file:
* glob() is horribly unreliable and doesn't work on some systems, including free.fr shared hosting. No longer using it in Language::getLanguageNames()
-X!
On Dec 24, 2010, at 12:24 PM, Brion Vibber wrote:
Glob works too I think.
-- brion On Dec 23, 2010 12:06 PM, "Ilmari Karonen" nospam@vyznev.net wrote:
On 12/22/2010 12:16 AM, Platonides wrote:
We are only using opendir for getting a full directory list.
That's a good point. Perhaps what we need is simply a utility method to list all files in a directory.
In fact, I just realized that PHP already has one. It's called scandir(). Its only flaw IMO is that it doesn't automatically skip the current and parent dir entries, but you could always do something like
$files = array_diff( scandir( $dir ), array( '.', '..' ) );
to accomplish that cleanly (or use preg_grep() to remove all dotfiles if you prefer).
-- Ilmari Karonen
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Dec 24, 2010 at 10:27 AM, Soxred93 soxred93@gmail.com wrote:
In the HISTORY file:
- glob() is horribly unreliable and doesn't work on some systems, including
free.fr shared hosting. No longer using it in Language::getLanguageNames()
Was it replaced with.... opendir()? :)
(A sufficient implementation that gives you an array should only be a few lines, which may be easier to maintain than an 800-line fork from another project.)
-- brion
2010/12/21 Chad innocentkiller@gmail.com:
I hate these frameworks, so a big -1 from me.
Seconded. My opinion on these frameworks aside, I don't think it's worth pulling in a framework just do handle a few instances of "give me all files in this directory" where there's perfectly good alternatives that, while not as short, are not at all hard to implement.
Roan Kattouw (Catrope)
On 22/12/10 20:03, Roan Kattouw wrote: <snip>
Seconded. My opinion on these frameworks aside, I don't think it's worth pulling in a framework just do handle a few instances of "give me all files in this directory" where there's perfectly good alternatives that, while not as short, are not at all hard to implement.
Nonetheless, I think we should have a look at them. We could probably use a routing system that handles nice URL :
http://www.symfony-project.org/book/1_0/09-Links-and-the-Routing-System
That would help with the API, get ride of wgActionPaths or replace the "lame" implementation in CodeReview (which is a big switch() on the number of parameters with a ton of if / else).
wikitech-l@lists.wikimedia.org