change of regular expression behaviour in rewrite scripts

List overview All Threads
Download

newer

older

check your home directory size

an idea

River Tarnell

19 Aug 2009 19 Aug '09

11:55 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, after the web server change on Wednesday, we noticed the following behaviour of regexp matching in rewrite scripts: when using a construct like "(a|b|)", the regexp will *not* match the empty string. as a workaround, you can write "(a|b)?". if you do not use this construct in your rewrite.script, this does not affect you. we have raised this issue with Zeus and are waiting for a resolution. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqMZtEACgkQIXd7fCuc5vJSBwCfUqPM5UnwGu2bkaE7QkMejUzq ljQAnR2Z9uPjlvi54LJd6D5Fp9qsim6J =ug1G -----END PGP SIGNATURE-----

Show replies by date

Martin Peeks

19 Aug 19 Aug

11:58 p.m.

New subject: change of regular expression behaviour in rewrite scripts

Hi, River Tarnell wrote:

...

Just out of curiosity, why does the toolserver use a webserver which so few will be familiar with, rather than apache/etc? Thanks, Martin

River Tarnell

20 Aug 20 Aug

12:03 a.m.

New subject: change of regular expression behaviour in rewrite scripts

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Peeks:

...

Just out of curiosity, why does the toolserver use a webserver which so few will be familiar with, rather than apache/etc?

we previously used Apache + mod_suphp, until CGI PHP became too slow. we explored several solutions and eventually settled on ZWS. in general users do not deal much with the web server directly; the main user-visible change was from mod_rewrite to rewrite.script. most users do not use rewrite rules, so we considered the inconvenience was acceptable. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqMaKAACgkQIXd7fCuc5vIijwCdHe/EQJ+FILE2bpp/j0Z5qHt0 GQgAoIQD4Oyd80nfiEzpWqAwJVn21erD =oIB4 -----END PGP SIGNATURE-----

Platonides

12:29 a.m.

New subject: change of regular expression behaviour in rewrite scripts

River Tarnell wrote:

...

Martin Peeks:

Just out of curiosity, why does the toolserver use a webserver which so few will be familiar with, rather than apache/etc?

I understand the problem was the process creation needed by mod_suphp (that's also why the switchserver was tried). How does Zeus run the scripts as different users? I thought Zeus loaded php as isapi.

River Tarnell

12:34 a.m.

New subject: change of regular expression behaviour in rewrite scripts

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Platonides:

...

I understand the problem was the process creation needed by mod_suphp (that's also why the switchserver was tried).

that is correct.

...

How does Zeus run the scripts as different users?

it starts a FastCGI process as the user when a request comes in. when the request is finished, it caches the fastcgi process for a few minutes until the next request for that user appears. if no request appears, it kills the process. this way, for users with popular tools, there is always a PHP process around to serve it without having to start one. for less popular users, a new process would have to be started on each request, but since the request rate is so low, that's not an issue. ZWS supports both ISAPI and NSAPI, and i suppose you could run PHP that way, but then everything would run as the web server user, just like mod_php. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqMb84ACgkQIXd7fCuc5vJfXgCeN/xjK1v9/c4Cw19PuI/La/DV vx4AoJ73sJVcJtGHj5c7plN8SzpjVAiu =eC4c -----END PGP SIGNATURE-----

Platonides

1:05 a.m.

New subject: change of regular expression behaviour in rewrite scripts

River Tarnell wrote:

...

Platonides:

I understand the problem was the process creation needed by mod_suphp (that's also why the switchserver was tried).

that is correct.

How does Zeus run the scripts as different users?

I see. It's an optimization by lazy killing the fastcgi processes.

Fahad Sadah

9:17 a.m.

New subject: change of regular expression behaviour in rewrite scripts

Would lighttpd/nginx be any better? I've had good experience with those. Fahad Sadah

Andrew Dunbar

2 Sep 2 Sep

1:59 a.m.

New subject: change of regular expression behaviour in rewrite scripts

2009/8/19 River Tarnell <river(a)loreley.flyingparchment.org.uk>uk>:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Platonides:

I understand the problem was the process creation needed by mod_suphp (that's also why the switchserver was tried).

that is correct.

How does Zeus run the scripts as different users?

Hmm is this when the setting was changed to kill FastCGI scripts after a minute or two when it used to be pretty much indefinite when wolfsbane was running on Linux?

...

this way, for users with popular tools, there is always a PHP process around to serve it without having to start one. for less popular users, a new process would have to be started on each request, but since the request rate is so low, that's not an issue.

Sadly this is causing some of my tools to thrash since they have expensive startups and are used on the English Wiktionary which is popular enough to keep starting them pretty frequently but of course nowhere near as popular as Wikipedia which most tools are for. I've filed a bug on JIRA for this: https://jira.toolserver.org/browse/TS-334 Andrew Dunbar (hippietrail)

...

ZWS supports both ISAPI and NSAPI, and i suppose you could run PHP that way, but then everything would run as the web server user, just like mod_php. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqMb84ACgkQIXd7fCuc5vJfXgCeN/xjK1v9/c4Cw19PuI/La/DV vx4AoJ73sJVcJtGHj5c7plN8SzpjVAiu =eC4c -----END PGP SIGNATURE-----

_______________________________________________ Toolserver-l mailing list (Toolserver-l(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette

-- http://wiktionarydev.leuksman.com http://linguaphile.sf.net

River Tarnell

4:06 a.m.

New subject: change of regular expression behaviour in rewrite scripts

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andrew Dunbar:

...

Hmm is this when the setting was changed to kill FastCGI scripts after a minute or two when it used to be pretty much indefinite when wolfsbane was running on Linux?

there was no time or date or event of any kind mentioned in the text you quoted, so i don't understand what you mean by "is this when". there has been no deliberate change to FastCGI settings recently that would cause the issue you have complained about. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqd00EACgkQIXd7fCuc5vJBYwCZASX8p6qZlvrKSaG1cTjwp2Rb tboAn00iokG0Ge9l10ioyiNn+CW8T9+P =qYv6 -----END PGP SIGNATURE-----

Peter Körner

20 Aug 20 Aug

2:26 p.m.

New subject: change of regular expression behaviour in rewrite scripts

...

we previously used Apache + mod_suphp, until CGI PHP became too slow. we explored several solutions and eventually settled on ZWS.

So then we should think about the setup on cassini, which uses Apache + mod_suphp, and maybe make the change before it's populated with tools. Peter

seth

11:29 a.m.

New subject: change of regular expression behaviour in rewrite scripts

Hi! On Wed, August 19, 2009 22:55, River Tarnell wrote:

...

after the web server change on Wednesday, we noticed the following behaviour of regexp matching in rewrite scripts: when using a construct like "(a|b|)", the regexp will *not* match the empty string. as a workaround, you can write "(a|b)?". if you do not use this construct in your rewrite.script, this does not affect you.

I'm not very experienced in rewrite-regexp. But in perl (and php's pcre, ...) this would not be a good work-around, because the capture buffer would sometimes be created and sometimes not. If one does not need capture buffers, he should write (?:a|b)? If one needs capture buffers but can't use (a|b|), he should use ((?:a|b)?) This will in any case make it possible to use a corresponding backreference. All this holds for perl, I'm not sure whether this is releveant for rewrite-regexp. If it does not, you may just call me clever shit, I guess. :-) prosit seth

River Tarnell

12:49 p.m.

New subject: change of regular expression behaviour in rewrite scripts

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 seth:

...

I'm not very experienced in rewrite-regexp. But in perl (and php's pcre, ...) this would not be a good work-around, because the capture buffer would sometimes be created and sometimes not.

i was not able to reproduce this behaviour, or else i misunderstood your meaning. i tested with the following script: #! /usr/bin/env perl if ($ARGV[0] =~ /^(.*\/)?(.*)$/) { print "1: $1\n2: $2\n"; } which produced this output: % ./test.pl foo 1: 2: foo % ./test.pl bar/foo 1: bar/ 2: foo even when the ?-expression was not matched, it created an empty backreference, so the following expression was always $2. - river.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqNHEUACgkQIXd7fCuc5vJJ9gCfWWbBLLgQftd2qMEJYHu28K2D 30IAoMDV7vwU4+E6Mllbo3REFJIj5V8a =FtnC -----END PGP SIGNATURE-----

seth

1:33 p.m.

New subject: change of regular expression behaviour in rewrite scripts

Hi! On Thu, August 20, 2009 11:49, River Tarnell wrote:

...

seth wrote:

I'm not very experienced in rewrite-regexp. But in perl (and php's pcre, ...) this would not be a good work-around, because the capture buffer would sometimes be created and sometimes not.

i was not able to reproduce this behaviour, or else i misunderstood your meaning. i tested with the following script: #! /usr/bin/env perl

use strict; use warnings;

...

if ($ARGV[0] =~ /^(.*\/)?(.*)$/) { print "1: $1\n2: $2\n"; } [...] which produced this output: % ./test.pl foo 1:

"Use of uninitialized value in concatenation (.) or string ..."

...

2: foo

...

even when the ?-expression was not matched, it created an empty backreference, so the following expression was always $2.

in one case $1 is an empty string, in the other case $1 is undef. prosit seth