I finally came to the realization that the time I've spent getting
to know the software enough to make the few changes I wanted would
be wasted if I didn't do a good reorganization that the code badly
needs. Hopefully that will also help us diagnose some of its
performance and scalability problems (for example, I've already
found that the code as it is now does 14 database queries to display
the front page!)
So, I'm refactoring the code and documenting as I go, but I want
to get an opinion from not only the tech folks but the list at
large about what the login/logout behavior should be.
Question 1: What is the behavior of the "remember password" user
option supposed to be? Please don't describe code details--I need
to know WHAT it's supposed to do, not HOW (because I'm changing
the latter). We already save the user's ID and some settings in
a cookie (we have to to handle prefs), and we delete the cookie
when the user explicitly logs out (which is good for security),
requiring him to log in and specify his password again. So what's
the difference between a user with this setting and a user without
it? Do we want to retain the user's password in a cookie even when
he logs out? We don't do that currently, so I'm just not clear
what the option is supposed to do.
Question 2: What's a reasonable cookie expiration? It's currently
a year, but I'm thinking a week or a month might be better for
security.
Question 3: Do we "trust" cookies for such issues as sysop access?
Some anonymous user could, for example, mockup the cookies of a
logged-in sysop and start deleting things. Should we take measures
to secure against that, or just be more relaxed and rely on making
frequent backups?
--
Lee Daniel Crocker <lee(a)piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
I'm curious: why is the Spanish text in wikiTextEs.php encoded
in utf-8? ISO-8859-1 has all the needed characters for Spanish;
The German wikiTextDe.php uses plain ISO, and the current
es.wikipedia.com is in ISO.
--
Lee Daniel Crocker <lee(a)piclab.com> <http://www.piclab.com/lee/>
"All inventions or works of authorship original to me, herein and past,
are placed irrevocably in the public domain, and may be used or modified
for any purpose, without permission, attribution, or notification."--LDC
On lun, 2002-05-06 at 15:21, Jason Richey wrote:
> Brion L. VIBBER wrote:
> > Could you double-check that there was '$wikiLanguage = "eo";' set in
> > wikiSettings.php before the conversion?
>
> It was set in wikiLocalSettings... That should be equivalent (or
> better), right?
Er, yes, that's what I meant. But clearly something didn't go right...
-- brion vibber (brion @ pobox.com)
When I started to get extremely long response times (> 300 seconds), I
thought perhaps these are tiemouts or other kinds of accidents, so I
started to log HTTP status codes and the length of the returned
document as well. But the status code is 200 (OK) and the full
document is returned, some 400-1800 seconds after the request was
sent.
Is anybody able to use the English Wikipedia now? These are the last
two hours:
min avg max (count) <2 2-5 5-15 >15 URL - response times in seconds
0.72 0.79 1.54 ( 15) 100% 0% 0% 0% http://www.wikipedia.com/wiki.png
10.02 100.24 920.39 ( 13) 0% 0% 30% 69% http://www.wikipedia.com/wiki/special:RecentChanges
17.83 281.00 1840.27 ( 12) 0% 0% 0% 100% http://www.wikipedia.com/wiki/Sweden
22.14 333.33 1599.76 ( 10) 0% 0% 0% 100% http://www.wikipedia.com/
--
Lars Aronsson
<lars(a)aronsson.se>
tel +46-70-7891609
http://aronsson.se/http://elektrosmog.nu/http://susning.nu/
Source code and discussion follow below.
Over the last 24 hours I've polled some URLs each 10 minutes and
measured the response time. This is a statistics report:
min avg max (count) <2 2-5 5-15 >15 URL - response times in seconds
0.75 2.00 11.79 ( 144) 68% 27% 4% 0% http://pl.wikipedia.com/wiki.cgi?Szwecja
0.56 2.70 89.78 ( 144) 95% 0% 0% 3% http://www.wikipedia.com/wiki.png
1.47 2.90 8.63 ( 144) 27% 64% 7% 0% http://pl.wikipedia.com/wiki.cgi?Ostatnie_zmiany
1.36 2.98 28.68 ( 144) 30% 62% 6% 0% http://pl.wikipedia.com/
0.83 3.21 91.86 ( 144) 94% 0% 0% 4% http://eo.wikipedia.com/vikio.png
1.00 4.45 140.53 ( 144) 63% 29% 2% 4% http://eo.wikipedia.com/wiki/Svedio
1.08 5.70 137.43 ( 144) 53% 34% 7% 4% http://eo.wikipedia.com/
3.40 7.09 68.23 ( 144) 0% 38% 56% 4% http://eo.wikipedia.com/wiki/Lastaj_Sxangxoj
3.35 13.59 203.39 ( 144) 0% 18% 63% 18% http://www.wikipedia.com/wiki/special:RecentChanges
1.61 28.38 411.43 ( 144) 4% 23% 33% 38% http://www.wikipedia.com/wiki/Sweden
3.86 45.16 359.17 ( 144) 0% 2% 38% 58% http://www.wikipedia.com/
The first three columns present the minimum, average, and maximum
response times. The rows are sorted on the average column. As you
can see the minimums are very good: 0.56 seconds roundtrip from Sweden
to San Diego is excellent. Even the English Wikipedia start page
(with the worst average in the list) has been served on 3.86 seconds,
which is quite good (this happened 6:10 am GMT). However, the
striking numbers are the maximum response times of several minutes.
The fourth column is the number of samples, which is 144 in 24 hours.
The following four columns present the statistical distribution of
samples in four categories: The percentage of samples that were less
than 2 seconds, those between 2 and 5 seconds, those between 5 and 15
seconds, and those in excess of 15 seconds. I thing usability gurus
like Jakob Nielsen has declared that 5 seconds is an acceptable
maximum for normal pages and most people can accept 15 seconds
response time for special functions such as searches and the recent
changes list.
The Polish Wikipedia has not a single sample above 15 seconds in these
24 hours. The majority of samples is in the lower two categories,
which is very good.
The Esperanto Wikipedia has a small number of samples above 15
seconds, which is sad but perhaps not alarming. The "recent changes"
has 56 % of its samples in the high 5-15 seconds response time
interval, which is a little high. Perhaps this could be fixed by
setting the default "recent changes" list from 7 or 3 days. Who can
change this setting? Tell me when you change, and I will report how
the response time changed. Almost all other samples are in the 0-2
and 2-5 categories, which is very good.
For the English Wikipedia, the static logotype image is served in less
than 2 seconds in 95 % of the samples. For this URL, there are no
samples in the 2-5 or 5-15 intervals, but a few samples have very long
response times. Perhaps the entire server was put on hold by some
other event? The last three lines of the report are depressing.
Almost none of the samples fall in the 0-2 or 2-5 categories. This
has to be analysed further by instrumenting the source code to report
where the delay is introduced.
I now have a running copy of the Wikipedia on my computer and have
started to experiment with this instrumentation. It's really straight
forward. In version 1.14 of wiki.phtml, Magnus Manske introduced the
function getmicrotime(), but the call to the function is commented
out. Just after getmicrotime(), I introduce a new function:
function trace($text){
global $startTime, $traceText;
$now = getmicrotime();
$elapsed = $now - $startTime;
if ($elapsed > 3.0) {
$traceText = "$traceText\nAfter $elapsed seconds: $text";
$startTime = $now;
}
}
Then at the beginning of the "main" program (where Magnus left a
commented-out first call to getmicrotime), I declare:
global $startTime, $traceText;
$startTime = getmicrotime();
$traceText = "";
Then at various points throughout the code, just after function calls
that I suspect are time bandits, I insert calls to my trace function:
trace("Just after updating the database");
These informative texts will accumulate in $traceText if the elapsed
time since the start is more than 3 seconds.
At the end of the "main" program comes the question, what should be
done with this $traceText? Should it be inserted into a new database
table? Or appended to the end of a text log file? Or inserted as an
HTML comment into the generated web page? Where can we best use this
information? The easiest but perhaps least useful is this:
trace("bottom of wiki.phtml main");
if ($traceText != "")
$out = "<!-- traceText:$traceText -->\n$out";
Who can implement this in the real source code? I'm not in the gang.
--
Lars Aronsson (lars(a)aronsson.se)
Aronsson Datateknik
Teknikringen 1e, SE-583 30 Linuxköping, Sweden
tel +46-70-7891609
http://aronsson.se/http://elektrosmog.nu/http://susning.nu/
I have put some links together on http://wikipedia.sourceforge.net,
the home page of our Sourceforge project. All members of the
Sourceforge Wikipedia group can add more stuff, as explained on the
page itself.
Axel
Ok, Jason switched the logging off; it doesn't seem to have made much
of a difference. A log of the latest long queries can be downloaded
from our sourceforge website:
http://wikipedia.sourceforge.net/long_queries.gz
Regarding the "INSERT DELAYED" idea: I'm not sure that it will help
much: the manual states that it's not that useful for the MyISAM
tables we are using, since those tables allow asynchronous updates
anyway. It's worth a try though.
Another idea is to switch off the query cache (query_cache_size=0):
our tables change so frequently that a caching of queries seems to be
a big loss.
Axel
>I'm getting incredibly slow responses from the main Wikipedia now and
>in the last few days. Why is this? Who is working to fix it? How
>can I help?
I don't think anybody is actively working on it right now. Jimbo has
said that he will soon move some other web sites off the wikipedia
server and will then give some of us user accounts on that machine, so
that we can better diagnose the problems. Right now, it is not clear
what we can do. For you, maybe the best would be to download the
current code from cvs, get it to work and familiarize yourself with
it; once we have access to the server we should be able to figure out
what's going on pretty quickly.
Axel
For anyone who is interested, Tim Perdue has performed a set of
benchmarks comparing MySQL with PostgreSQL
See his article at
http://www.phpbuilder.com/columns/tim20000705.php3?page=2
and
http://www.phpbuilder.com/columns/tim20000705-res.php3
for the raw results.
For read performance, they are comparable, with mySQL performing about
50% better.
In the 10% write case, however, Postgres kept going beyond the
client-concurrency load where MySQL falls over.
-- Neil
> It is logging slow requests... 320 Meg since April 19.
Assuming an average log message length of 100 bytes, this amounts to
between two and three logs per second, so this could conceivably be
the bottle neck. Could you chop off the last 10 megs or so, make them
available on www.wikipedia.com/tarballs, and switch off the logging?
Axel
