In my spare time at Redwerks I've been working on a Short URL configuration tool:
http://shorturls.redwerks.org/
Our Short URL manual pages have been VERY bad for quite awhile. Every last one of them has bad practice on them and I only managed to fix one of them.
Considering how we have so many different manual pages simply because people have slightly different configuration requirements (eg: One for /w and /wiki/ another for / and /wiki/, another for /subpath/wiki, etc... One for .htaccess, another for Alias in Apache config, another for RewriteRule in Apache config, another for Nginx, etc...) rather than trying to fix I started writing a tool to build the configuration instead.
The tool tries to auto-detect as much as possible (practically everything in fact). Everything from the type of server, the sapi (mod_php vs. ?), your scriptpath, etc... it even tries to jump ahead of you and guess what kind of article path you were intending to use. It even has an early feature to try and preemptively detect if you're likely to have root access or no root access (The idea is to detect what kind of host you're on using the reverse dns for the server you're on).
The configuration generator actually isn't really a simple thing. There are a lot of conditionals involved in the tool. It can handle the special cases needed for root /$1 style urls. It knows how to add an extra rewrite when you use /w and /wiki and need / to redirect to your wiki. It uses Apache's %{DOCUMENT_ROOT} in RewriteRules but also lets you expand an absolute docroot when you use an Alias (you can't use %{DOCUMENT_ROOT} there). I've loaded the Nginx config full of deny rules and conditions that most people never bothered to properly configure (and yes, it can handle root urls, and even knows how to stop Nginx from executing php in uploaded files). Heck, this tool can actually handle TWN's wacky /w/i.php script path.
The tool is definitely beta right now. I have Apache and Nginx support written. I'll probably want to have some discussion with Lighttpd and ISS users to fix the configuration some of them are using and add code to the tool to support those types of servers. There are still some conditions it might not handle just yet. For example I haven't written the code to handle root style urls in root Apache config files.
Feel free to start using it when you setup a MediaWiki installation. I'd love to know when configuration doesn't work and what kind of tweaks I need to make to it. As well I'd like to see how the configurations handle different shared host setups, etc... I'd also like to see people on different hosts, both shared host users and VPS, Dedicated, etc... users running Apache jump in and make use of the yellow message's question on whether you're in a root or shared environment. If you use that specific feature it keeps track of reverse dns and the response, which I can use to try and preload rules to indicate what host's reverse dns patterns are for shared hosting users and what are for VPS servers and the like.
http://shorturls.redwerks.org/ should link to http://www.mediawiki.org/wiki/Manual:Short_URL#Advantages_.26_disadvantages . for a balanced view. I'll stick with my long URLs.
Thanks jidanni, I'm glad that we can always count on you for fair and balanced opinion. On Feb 24, 2012 8:48 PM, jidanni@jidanni.org wrote:
http://shorturls.redwerks.org/ should link to http://www.mediawiki.org/wiki/Manual:Short_URL#Advantages_.26_disadvantages. for a balanced view. I'll stick with my long URLs.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, 24 Feb 2012 17:48:15 -0800, jidanni@jidanni.org wrote:
http://shorturls.redwerks.org/ should link to http://www.mediawiki.org/wiki/Manual:Short_URL#Advantages_.26_disadvantages . for a balanced view. I'll stick with my long URLs.
Balanced? That section lists a single bad example of an /advantage/ for long urls. If your host offers you absolutely no way at all to do any sort of mapping (no .htaccess, etc...) then you are being ripped off by them. PERIOD! Not even a free host is worth that level of service. And I say that because most of those crap free hosting are capable of having short urls.
;) So naturally, if you're actually paying for hosting, and they don't let you do something that a free host lets you do, you're being ripped off.
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to configure your own software. The point is moot anyways, because the goal of the Short URL building tool is to make it so that you don't need to know. You just tell it where your wiki is. Tell it what path you want. And it'll guess your server setup, environment, and generate the config you need. You just then setup that config and poof.
I think the tool is great!
Would it be possible to house on labs or toolserver? My only hesitation in linking to it is then we're dependent on Redwerks generosity in hosting. I think we should leave due credit to them and such in place.
-Greg aka varnent
____________ Sent from my iPhone. Apologies for any typos. A more detailed response may be sent later.
On Feb 25, 2012, at 2:26 PM, "Daniel Friesen" lists@nadir-seen-fire.com wrote:
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to configure your own software. The point is moot anyways, because the goal of the Short URL building tool is to make it so that you don't need to know. You just tell it where your wiki is. Tell it what path you want. And it'll guess your server setup, environment, and generate the config you need. You just then setup that config and poof.
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Integrating short url config to the installer has been on my todo list for a long time.
Dan: would you be interested in putting this into the installer sometime in the future?
-Chad On Feb 25, 2012 3:06 PM, "Mr. Gregory Varnum" gregory.varnum@gmail.com wrote:
I think the tool is great!
Would it be possible to house on labs or toolserver? My only hesitation in linking to it is then we're dependent on Redwerks generosity in hosting. I think we should leave due credit to them and such in place.
-Greg aka varnent
Sent from my iPhone. Apologies for any typos. A more detailed response may be sent later.
On Feb 25, 2012, at 2:26 PM, "Daniel Friesen" lists@nadir-seen-fire.com wrote:
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr
wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop
team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to
configure your own software.
The point is moot anyways, because the goal of the Short URL building
tool is to make it so that you don't need to know.
You just tell it where your wiki is. Tell it what path you want. And
it'll guess your server setup, environment, and generate the config you need.
You just then setup that config and poof.
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
You might be able to adapt the tests I use in the tool and the rules for generating the config later on. Though as-is the actual code won't be as much help. It's in Ruby rather than PHP. And the code has to be intermixed with the config output because of how variable short url config can be. You may want to consider holding out on actually adding it to the installer until after the external tool has been in use for some time and we've added all the servers and made sure all the configs it outputs in edge cases work properly, and taken into account any issues with some shared hosts that might make it need tweaks.
As it is right now I haven't added the root urls in Apache root config yet (just the .htaccess ones), I still need to look into the best way to configure Lighttpd, and ISS, and perhaps some of the other small servers people seem to be using.
So it will probably have to be some distance in the future before it's robust enough for us to include into the installer. That is probably natural anyways. Since when you think about it, at this point we don't even have good manual pages for creating short urls. So if we don't even have good guides, it's unlikely that any code will be robust right away either.
Also, integrating it into the installer will have to be a spare time project. Mixed in with the pile of other things I want to do in that short time. (Like drop our Loggedout cookie in favor of better ETag support) I think it would be hard to swing that part as something I can use work hours for.
On Sat, 25 Feb 2012 12:26:17 -0800, Chad innocentkiller@gmail.com wrote:
Integrating short url config to the installer has been on my todo list for a long time.
Dan: would you be interested in putting this into the installer sometime in the future?
-Chad On Feb 25, 2012 3:06 PM, "Mr. Gregory Varnum" gregory.varnum@gmail.com wrote:
I think the tool is great!
Would it be possible to house on labs or toolserver? My only hesitation in linking to it is then we're dependent on Redwerks generosity in hosting. I think we should leave due credit to them and such in place.
-Greg aka varnent
Sent from my iPhone. Apologies for any typos. A more detailed response may be sent later.
On Feb 25, 2012, at 2:26 PM, "Daniel Friesen" lists@nadir-seen-fire.com wrote:
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr
wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/
for
long urls.
Another advantage of long URLs, is for such corner case when the
sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to
configure your own software.
The point is moot anyways, because the goal of the Short URL building
tool is to make it so that you don't need to know.
You just tell it where your wiki is. Tell it what path you want. And
it'll guess your server setup, environment, and generate the config you need.
You just then setup that config and poof.
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Sat, Feb 25, 2012 at 4:16 PM, Daniel Friesen lists@nadir-seen-fire.com wrote:
So it will probably have to be some distance in the future before it's robust enough for us to include into the installer. That is probably natural anyways. Since when you think about it, at this point we don't even have good manual pages for creating short urls. So if we don't even have good guides, it's unlikely that any code will be robust right away either.
Good point.
-Chad
I can forward that. Technically since I logged work hours to write that and the MW Skinning tutorial I don't own the copyright for them, so I have to get an ok for anything like that. Keep in mind that the premise under which I was allowed to write them under work hours was that putting things out for MediaWiki under the Redwerks name could help draw more big clients like BASESwiki and CCA to us asking for us to design a skin for their wiki.
Btw, for reference the tool is written in Ruby (just Rack, no Rails). Ruby's URL handling libraries were miles better than PHP's. And the net library was better (native https support even). And a few other libraries as well. So I used Ruby rather than php.
For that reason, and of course other probable issues with it, I'd say the toolserver is out. Labs only.
On Sat, 25 Feb 2012 12:05:22 -0800, Mr. Gregory Varnum gregory.varnum@gmail.com wrote:
I think the tool is great!
Would it be possible to house on labs or toolserver? My only hesitation in linking to it is then we're dependent on Redwerks generosity in hosting. I think we should leave due credit to them and such in place.
-Greg aka varnent
Sent from my iPhone. Apologies for any typos. A more detailed response may be sent later.
On Feb 25, 2012, at 2:26 PM, "Daniel Friesen" lists@nadir-seen-fire.com wrote:
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to configure your own software. The point is moot anyways, because the goal of the Short URL building tool is to make it so that you don't need to know. You just tell it where your wiki is. Tell it what path you want. And it'll guess your server setup, environment, and generate the config you need. You just then setup that config and poof.
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'm actually glad you brought that up - it's a general topic that's been bouncing around in my mind lately.
I can appreciate the marketing concern. :) I think we'll need to be flexible in allowing the look and logo to remain intact. As I've been reviewing extension pages, I've noticed a healthy number of organizations/companies plugging their involvement in the effort. I'm not sure I'd encourage it, but I don't think it should be discouraged either.
I'm not sure if WMF has an official stance on the topic, but I would certainly strongly advocating for allowing such promotion of companies/orgs supporting specific efforts. Especially in light of the conversation about seeking donations for developers, etc. I'm not sure I'd be as comfortable with a master list of orgs doing that or some type of "these folks work with WMF" appearance. However, I see a clear line between mention on a specific tool or extension and wiki-wide pages including it. Perhaps a mention when linked from a hub. "URL Short URL Builder developed by Redwerks" seems reasonable as long as Redwerks maintains it.
Basically, giving credit where credit is due while an entity is actively engaged only seems fair. If we house that work on WMF servers in case something ever happens to that entity or their development priorities, we don't lose the work altogether. I have the same general feeling about the skin tutorial. Give Redwerks credit for developing it - but housing a copy (even if just a PDF or protected historical archives page) on WMF servers seems like the responsible thing to do. :)
My two cents. -greg aka varnent
ps. Daniel, I'm not predicting any problems or anything for Redwerks - just the example at hand. :) I'm ecstatic they're supporting the work! But the "capacity building" side of me is thinking of 5 years down the road. Lots can happen in one year, let alone five. :) Once they amass a huge client list, supporting this may take a lower priority.
On Feb 25, 2012, at 3:55 PM, Daniel Friesen wrote:
I can forward that. Technically since I logged work hours to write that and the MW Skinning tutorial I don't own the copyright for them, so I have to get an ok for anything like that. Keep in mind that the premise under which I was allowed to write them under work hours was that putting things out for MediaWiki under the Redwerks name could help draw more big clients like BASESwiki and CCA to us asking for us to design a skin for their wiki.
Btw, for reference the tool is written in Ruby (just Rack, no Rails). Ruby's URL handling libraries were miles better than PHP's. And the net library was better (native https support even). And a few other libraries as well. So I used Ruby rather than php.
For that reason, and of course other probable issues with it, I'd say the toolserver is out. Labs only.
On Sat, 25 Feb 2012 12:05:22 -0800, Mr. Gregory Varnum gregory.varnum@gmail.com wrote:
I think the tool is great!
Would it be possible to house on labs or toolserver? My only hesitation in linking to it is then we're dependent on Redwerks generosity in hosting. I think we should leave due credit to them and such in place.
-Greg aka varnent
Sent from my iPhone. Apologies for any typos. A more detailed response may be sent later.
On Feb 25, 2012, at 2:26 PM, "Daniel Friesen" lists@nadir-seen-fire.com wrote:
On Sat, 25 Feb 2012 01:25:32 -0800, Antoine Musso hashar+wmf@free.fr wrote:
Daniel Friesen wrote:
Balanced? That section lists a single bad example of an /advantage/ for long urls.
Another advantage of long URLs, is for such corner case when the sysop team does not know about URL rewriting and has more important things to do than figuring it out. I have seen such cases in small company.
That's not really even an advantage. That's just not knowing how to configure your own software. The point is moot anyways, because the goal of the Short URL building tool is to make it so that you don't need to know. You just tell it where your wiki is. Tell it what path you want. And it'll guess your server setup, environment, and generate the config you need. You just then setup that config and poof.
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
If short URLs were so good, then one wouldn't need an 'after market' treatment to add them. I'll wait.
Plus, I might know how to maintain them now, but five years later will I still know? I'm not getting any younger.
Plus, what about all that spam in my mailbox about increasing the length of my URL without the need to see a URoLogist? It can't be all wrong.
On Mon, Feb 27, 2012 at 8:15 PM, jidanni@jidanni.org wrote:
If short URLs were so good, then one wouldn't need an 'after market' treatment to add them. I'll wait.
It's because it's pretty hard to detect this stuff "out of the box." We can try, but it's not always easy. I'd love to see this in the installer eventually, so the process of configuring it is easier.
The process is different for each webserver, which is why there's no "one-size-fits-all" answer to how to do this.
Plus, I might know how to maintain them now, but five years later will I still know? I'm not getting any younger.
I can't speak for nginx or IIS or lighttpd, but I know that Apache's mod_rewrite has been functioning the same way for a long time, so I can't imagine that configuration for that will change. On the MediaWiki side--we haven't changed the behavior of $wgArticlePath, $wgScriptPath and the rest in a very very long time (and I can't imagine we'd break it either).
Plus, what about all that spam in my mailbox about increasing the length of my URL without the need to see a URoLogist? It can't be all wrong.
Not everybody can figure it out for themselves. Having tools to simplify things are a Good Thing.
-Chad
Actually if you put it in the installer you are making a commitment to handhold them through thick and thin, though better and worse, till the death of their wiki do we part. I do (NOT!).
"C" == Chad innocentkiller@gmail.com writes:
Plus, I might know how to maintain them now, but five years later will I still know? I'm not getting any younger.
C> I can't speak for nginx or IIS or lighttpd, but I know that Apache's C> mod_rewrite has been functioning the same way for a long time, so I can't C> imagine that configuration for that will change. On the MediaWiki side--we C> haven't changed the behavior of $wgArticlePath, $wgScriptPath and the C> rest in a very very long time (and I can't imagine we'd break it either).
I'm talking about me. I can't even understand the comments I myself wrote in code six months ago.
C> The process is different for each webserver, which is why there's C> no "one-size-fits-all" answer to how to do this.
Anyways, I'll stick with my Big Ten Inch, http://www.youtube.com/watch?v=b_VqNuLdrGo&list=PL648DE656FFB7A7A7
On Feb 27, 2012 9:57 PM, jidanni@jidanni.org wrote:
Actually if you put it in the installer you are making a commitment to handhold them through thick and thin, though better and worse, till the death of their wiki do we part. I do (NOT!).
What.
"C" == Chad innocentkiller@gmail.com writes:
Plus, I might know how to maintain them now, but five years later will
I
still know? I'm not getting any younger.
C> I can't speak for nginx or IIS or lighttpd, but I know that Apache's C> mod_rewrite has been functioning the same way for a long time, so I
can't
C> imagine that configuration for that will change. On the MediaWiki
side--we
C> haven't changed the behavior of $wgArticlePath, $wgScriptPath and the C> rest in a very very long time (and I can't imagine we'd break it
either).
I'm talking about me. I can't even understand the comments I myself wrote in code six months ago.
Sounds like you need to write better comments then.
C> The process is different for each webserver, which is why there's C> no "one-size-fits-all" answer to how to do this.
Anyways, I'll stick with my Big Ten Inch, http://www.youtube.com/watch?v=b_VqNuLdrGo&list=PL648DE656FFB7A7A7
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I've added Lighttpd support to the shorturl tool now. Complete with a port of the IE6 XSS fix we have in images/.htaccess.
wikitech-l@lists.wikimedia.org