I am seeing my tools very slow connecting to the sql servers, such as taking 33 or 79 seconds just for figuring out the right server and connecting to it, and even failing with «Can't connect to MySQL server on 'sql-s4-rr.toolserver.org'» (not only on this server)
Are sql servers overloaded? Is it a problem with willow? Should we give a kick to the tcp stack?
This anomaly is also visible from the command line, varying from immediate connection to slow one:
$ time mysql -h sql-s4-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m0.098s user 0m0.003s sys 0m0.009s
# Shouldn't it always so quick?
$ time mysql -h sql-s4-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m4.803s user 0m0.003s sys 0m0.009s
$ time mysql -h sql-s4-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m1.619s user 0m0.003s sys 0m0.009s
$ time mysql -h sql-s4-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m4.222s user 0m0.003s sys 0m0.009s
$ time mysql -h sql-s4-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m15.434s user 0m0.003s sys 0m0.008s
$ time mysql -h sql-s3-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m13.463s user 0m0.003s sys 0m0.009s
$ time mysql -h sql-s3-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m2.356s user 0m0.003s sys 0m0.009s
$ time mysql -h sql-s3-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m1.850s user 0m0.003s sys 0m0.008s
$ time mysql -h sql-s3-rr.toolserver.org -e 'SELECT VERSION()' +-----------+ | VERSION() | +-----------+ | 5.1.59 | +-----------+
real 0m24.724s user 0m0.003s sys 0m0.008s
toolserver-l@lists.wikimedia.org