Wikitech-l May 2002

wikitech-l@lists.wikimedia.org

22 participants
89 discussions

Security issue--new software
by Lee Daniel Crocker 08 May '02

08 May '02

I finally came to the realization that the time I've spent getting to know the software enough to make the few changes I wanted would be wasted if I didn't do a good reorganization that the code badly needs. Hopefully that will also help us diagnose some of its performance and scalability problems (for example, I've already found that the code as it is now does 14 database queries to display the front page!) So, I'm refactoring the code and documenting as I go, but I want to get an opinion from not only the tech folks but the list at large about what the login/logout behavior should be. Question 1: What is the behavior of the "remember password" user option supposed to be? Please don't describe code details--I need to know WHAT it's supposed to do, not HOW (because I'm changing the latter). We already save the user's ID and some settings in a cookie (we have to to handle prefs), and we delete the cookie when the user explicitly logs out (which is good for security), requiring him to log in and specify his password again. So what's the difference between a user with this setting and a user without it? Do we want to retain the user's password in a cookie even when he logs out? We don't do that currently, so I'm just not clear what the option is supposed to do. Question 2: What's a reasonable cookie expiration? It's currently a year, but I'm thinking a week or a month might be better for security. Question 3: Do we "trust" cookies for such issues as sysop access? Some anonymous user could, for example, mockup the cookies of a logged-in sysop and start deleting things. Should we take measures to secure against that, or just be more relaxed and rely on making frequent backups? -- Lee Daniel Crocker <lee(a)piclab.com> <http://www.piclab.com/lee/> "All inventions or works of authorship original to me, herein and past, are placed irrevocably in the public domain, and may be used or modified for any purpose, without permission, attribution, or notification."--LDC

4 3

Spanish text in new software
by Lee Daniel Crocker 07 May '02

07 May '02

I'm curious: why is the Spanish text in wikiTextEs.php encoded in utf-8? ISO-8859-1 has all the needed characters for Spanish; The German wikiTextDe.php uses plain ISO, and the current es.wikipedia.com is in ISO. -- Lee Daniel Crocker <lee(a)piclab.com> <http://www.piclab.com/lee/> "All inventions or works of authorship original to me, herein and past, are placed irrevocably in the public domain, and may be used or modified for any purpose, without permission, attribution, or notification."--LDC

2 1

Re: [Wikitech-l] eo-php wiki
by Brion L. VIBBER 06 May '02

06 May '02

On lun, 2002-05-06 at 15:21, Jason Richey wrote: > Brion L. VIBBER wrote: > > Could you double-check that there was '$wikiLanguage = "eo";' set in > > wikiSettings.php before the conversion? > > It was set in wikiLocalSettings... That should be equivalent (or > better), right? Er, yes, that's what I meant. But clearly something didn't go right... -- brion vibber (brion @ pobox.com)

1 0

response times
by Lars Aronsson 06 May '02

06 May '02

When I started to get extremely long response times (> 300 seconds), I thought perhaps these are tiemouts or other kinds of accidents, so I started to log HTTP status codes and the length of the returned document as well. But the status code is 200 (OK) and the full document is returned, some 400-1800 seconds after the request was sent. Is anybody able to use the English Wikipedia now? These are the last two hours: min avg max (count) <2 2-5 5-15 >15 URL - response times in seconds 0.72 0.79 1.54 ( 15) 100% 0% 0% 0% http://www.wikipedia.com/wiki.png 10.02 100.24 920.39 ( 13) 0% 0% 30% 69% http://www.wikipedia.com/wiki/special:RecentChanges 17.83 281.00 1840.27 ( 12) 0% 0% 0% 100% http://www.wikipedia.com/wiki/Sweden 22.14 333.33 1599.76 ( 10) 0% 0% 0% 100% http://www.wikipedia.com/ -- Lars Aronsson <lars(a)aronsson.se> tel +46-70-7891609 http://aronsson.se/ http://elektrosmog.nu/ http://susning.nu/

1 0

response times
by Lars Aronsson 05 May '02

05 May '02

Source code and discussion follow below. Over the last 24 hours I've polled some URLs each 10 minutes and measured the response time. This is a statistics report: min avg max (count) <2 2-5 5-15 >15 URL - response times in seconds 0.75 2.00 11.79 ( 144) 68% 27% 4% 0% http://pl.wikipedia.com/wiki.cgi?Szwecja 0.56 2.70 89.78 ( 144) 95% 0% 0% 3% http://www.wikipedia.com/wiki.png 1.47 2.90 8.63 ( 144) 27% 64% 7% 0% http://pl.wikipedia.com/wiki.cgi?Ostatnie_zmiany 1.36 2.98 28.68 ( 144) 30% 62% 6% 0% http://pl.wikipedia.com/ 0.83 3.21 91.86 ( 144) 94% 0% 0% 4% http://eo.wikipedia.com/vikio.png 1.00 4.45 140.53 ( 144) 63% 29% 2% 4% http://eo.wikipedia.com/wiki/Svedio 1.08 5.70 137.43 ( 144) 53% 34% 7% 4% http://eo.wikipedia.com/ 3.40 7.09 68.23 ( 144) 0% 38% 56% 4% http://eo.wikipedia.com/wiki/Lastaj_Sxangxoj 3.35 13.59 203.39 ( 144) 0% 18% 63% 18% http://www.wikipedia.com/wiki/special:RecentChanges 1.61 28.38 411.43 ( 144) 4% 23% 33% 38% http://www.wikipedia.com/wiki/Sweden 3.86 45.16 359.17 ( 144) 0% 2% 38% 58% http://www.wikipedia.com/ The first three columns present the minimum, average, and maximum response times. The rows are sorted on the average column. As you can see the minimums are very good: 0.56 seconds roundtrip from Sweden to San Diego is excellent. Even the English Wikipedia start page (with the worst average in the list) has been served on 3.86 seconds, which is quite good (this happened 6:10 am GMT). However, the striking numbers are the maximum response times of several minutes. The fourth column is the number of samples, which is 144 in 24 hours. The following four columns present the statistical distribution of samples in four categories: The percentage of samples that were less than 2 seconds, those between 2 and 5 seconds, those between 5 and 15 seconds, and those in excess of 15 seconds. I thing usability gurus like Jakob Nielsen has declared that 5 seconds is an acceptable maximum for normal pages and most people can accept 15 seconds response time for special functions such as searches and the recent changes list. The Polish Wikipedia has not a single sample above 15 seconds in these 24 hours. The majority of samples is in the lower two categories, which is very good. The Esperanto Wikipedia has a small number of samples above 15 seconds, which is sad but perhaps not alarming. The "recent changes" has 56 % of its samples in the high 5-15 seconds response time interval, which is a little high. Perhaps this could be fixed by setting the default "recent changes" list from 7 or 3 days. Who can change this setting? Tell me when you change, and I will report how the response time changed. Almost all other samples are in the 0-2 and 2-5 categories, which is very good. For the English Wikipedia, the static logotype image is served in less than 2 seconds in 95 % of the samples. For this URL, there are no samples in the 2-5 or 5-15 intervals, but a few samples have very long response times. Perhaps the entire server was put on hold by some other event? The last three lines of the report are depressing. Almost none of the samples fall in the 0-2 or 2-5 categories. This has to be analysed further by instrumenting the source code to report where the delay is introduced. I now have a running copy of the Wikipedia on my computer and have started to experiment with this instrumentation. It's really straight forward. In version 1.14 of wiki.phtml, Magnus Manske introduced the function getmicrotime(), but the call to the function is commented out. Just after getmicrotime(), I introduce a new function: function trace($text){ global $startTime, $traceText; $now = getmicrotime(); $elapsed = $now - $startTime; if ($elapsed > 3.0) { $traceText = "$traceText\nAfter $elapsed seconds: $text"; $startTime = $now; } } Then at the beginning of the "main" program (where Magnus left a commented-out first call to getmicrotime), I declare: global $startTime, $traceText; $startTime = getmicrotime(); $traceText = ""; Then at various points throughout the code, just after function calls that I suspect are time bandits, I insert calls to my trace function: trace("Just after updating the database"); These informative texts will accumulate in $traceText if the elapsed time since the start is more than 3 seconds. At the end of the "main" program comes the question, what should be done with this $traceText? Should it be inserted into a new database table? Or appended to the end of a text log file? Or inserted as an HTML comment into the generated web page? Where can we best use this information? The easiest but perhaps least useful is this: trace("bottom of wiki.phtml main"); if ($traceText != "") $out = "\n$out"; Who can implement this in the real source code? I'm not in the gang. -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik Teknikringen 1e, SE-583 30 Linuxköping, Sweden tel +46-70-7891609 http://aronsson.se/ http://elektrosmog.nu/ http://susning.nu/

2 2

Wikipedia.sourceforge.net
by Axel Boldt 05 May '02

05 May '02

I have put some links together on http://wikipedia.sourceforge.net, the home page of our Sourceforge project. All members of the Sourceforge Wikipedia group can add more stuff, as explained on the page itself. Axel

1 0

Re: Slow Response
by Axel Boldt 05 May '02

05 May '02

Ok, Jason switched the logging off; it doesn't seem to have made much of a difference. A log of the latest long queries can be downloaded from our sourceforge website: http://wikipedia.sourceforge.net/long_queries.gz Regarding the "INSERT DELAYED" idea: I'm not sure that it will help much: the manual states that it's not that useful for the MyISAM tables we are using, since those tables allow asynchronous updates anyway. It's worth a try though. Another idea is to switch off the query cache (query_cache_size=0): our tables change so frequently that a caching of queries seems to be a big loss. Axel

1 0

Re: Slow Response
by Axel Boldt 05 May '02

05 May '02

>I'm getting incredibly slow responses from the main Wikipedia now and >in the last few days. Why is this? Who is working to fix it? How >can I help? I don't think anybody is actively working on it right now. Jimbo has said that he will soon move some other web sites off the wikipedia server and will then give some of us user accounts on that machine, so that we can better diagnose the problems. Right now, it is not clear what we can do. For you, maybe the best would be to download the current code from cvs, get it to work and familiarize yourself with it; once we have access to the server we should be able to figure out what's going on pretty quickly. Axel

2 1

"Real-world" comparison of MySQL with PostgreSQL
by Neil Harris 04 May '02

04 May '02

For anyone who is interested, Tim Perdue has performed a set of benchmarks comparing MySQL with PostgreSQL See his article at http://www.phpbuilder.com/columns/tim20000705.php3?page=2 and http://www.phpbuilder.com/columns/tim20000705-res.php3 for the raw results. For read performance, they are comparable, with mySQL performing about 50% better. In the 10% write case, however, Postgres kept going beyond the client-concurrency load where MySQL falls over. -- Neil

3 2

Re: Is the database logging?
by Axel Boldt 04 May '02

04 May '02

> It is logging slow requests... 320 Meg since April 19. Assuming an average log message length of 100 bytes, this amounts to between two and three logs per second, so this could conceivably be the bottle neck. Could you chop off the last 10 megs or so, make them available on www.wikipedia.com/tarballs, and switch off the logging? Axel

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2002