2009/12/16 Nicolas Dumazet nicdumz@gmail.com:
2009/12/16 ObsessiveMathsFreak obsessivemathsfreak@obsessivemathsfreak.org:
I found the problem. It was a problem with python socket library. Apparently the socket wasn't being buffered properly, or at least that's my understanding.
The problem was in the socket.py library, found in Ubuntu 9.10 under /usr/lib/python2.6/socket.py
At around line 500, replace this line for res in getaddrinfo(host, port, 0, SOCK_STREAM): with this on for res in getaddrinfo(host, port, AF_INET, SOCK_STREAM):
Apparently now the socket will be properly buffered. With the old line, wiki login took about 90 seconds for me. Now it takes less than a second.
Hope that helps someone else.
While it might be interesting for some pywikipedia users, I think that this error should be reported upstream. Can you please do it, explaining more thoroughly what the problem was and how you diagnosed it?
Looking again at this, I think that your fix is quite suspicious.
0 == AF_UNSPEC, and UNSPEC means "A value of AF_UNSPEC for ai_family means that the caller shall accept any address family." If create_connection(...) is significantly slower than manual ways to create a socket connections because of this generic parameter (aka finding with Address family should be used is a slow operation), then we should not rely on create_connection.
And if we dont call directly create_connection, then the library responsible for this issue should be modified (urllib/httplib/etc...)
But your fix will break your Python installation, I'm afraid. In particular, you wont be able to call create_connection() on UNIX sockets, which may or may not break several installed softwares.
So, again, can you tell us more about that issue? How did you narrow down the problem to that specific call? Can you include caller information?