Hello all,
the debian-security-folks updated php because of some security-problems (if
you like details, I added the report below). The update requires an update and
restart of apache too.
For this reason, I will update and restart php and apache on cassini tonight
(between 1 and 2 o'clock UTC) . The "downtime" of apache should be only a few
minute at maximum. You can see the progress at [1].
Sincerly,
DaB.
[1] https://jira.toolserver.org/browse/MNT-16
--- News for php5 (php-pear php5 php5-cgi php5-cli php5-common php5-curl php5-
gd php5-mysql php5-pgsql) ---
php5 (5.2.6.dfsg.1-1+lenny4) stable-security; urgency=high
* Maximum number of file uploads per request limited
To prevent Denial of Service attacks by exhausting the number of
available temporary file names, the max_file_uploads option
introduced in PHP 5.3.1 has been backported.
Due to the nature of this new option a default limit has been set
to 50, hoping it is sensible enough to not to cause disruptions on
existing services.
The value of this new limit can be changed in the php.ini file.
If you installed the php5-suhosin extension there was a limiting
mechanism in place already. In this case you may want to make sure
the new limit imposed by PHP itself is not smaller than suhosin's.
-- Raphael Geissert <geissert(a)debian.org> Sat, 21 Nov 2009 18:13:48 -0600
--- Changes for php5 (php-pear php5 php5-cgi php5-cli php5-common php5-curl
php5-gd php5-mysql php5-pgsql) ---
php5 (5.2.6.dfsg.1-1+lenny4) stable-security; urgency=high
* CVE-2009-2687: DoS via malformed JPEG images with invalid offset fields
(Closes: #535888)
* CVE-2009-2626: remote memory disclosure via ini_* functions
(Closes: #540605)
* CVE-2009-3292: multiple missing checks processing exif image data
* CVE-2009-3291: improper handling of nul character in CommonName fields
of X509 certificates
* max_file_uploads: prevent, by limiting, temporary files exhaustion DoS
* Add an entry to debian/NEWS about the new per-request file uploads limit
-- Raphael Geissert <geissert(a)debian.org> Sat, 21 Nov 2009 18:28:12 -0600
--
wp-blog.de
Hi
Atm. the load on cassini is around 20 and the diff imports are fighting
to stay up. I think it's cmarq's hillshade-generator takes the most
resources on cassini. top reports ~57.1% IO waiting so I think there's a
lot of disk going on.
I tried to do a "du -hs /mnt/user-store/osm_hillshading" but ir ran
loooong until I canceled it.
What I think is going on is that the generate_tiles script creates a
real huge number of single pngs. Unlike mod_tile it is not combining the
small tiles into larger chunks (mod_tile packs 8x8 tiles into a
metatile). Also unless mod_tile it renders each and every tile in each
and every zoomlevel, while mod_tile only renders tiles that are really
shown (e.g. it leaves out most of the atlantic at zoomlevel 18, as
nobody would view the empty sea at this zoomlevel.
Both problems together create a real huge number of files in a single
directory which is not good for most filesystems. To make things worse,
/mnt/user-store is connected to cassini via NFS, so there's another
bottleneck.
I don't know it this really is a problem, but I think it's worth talking
about it.
Peter
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
per previous discussions, we've now reinstalled ptolemy as the OSM
Toolserver database, and i'm now working on setting up PostgreSQL and
the mapnik database (which is what cassini has). cassini will remain up
for now, but will go offline once ptolemy is ready, since it will become
Wikimedia's production OSM database.
users cannot log into ptolemy. instead, it will be accessed from the
normal Toolserver.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAksLUf8ACgkQIXd7fCuc5vIhUgCgk1ATOtLNun34E8u8ZLyCfJ+f
jR8An1JRpjcUOpwrNqld1lhi+dXkfrkt
=Nvwn
-----END PGP SIGNATURE-----
Why does this happen?
---------- Forwarded message ----------
From: Cron Daemon <root(a)cassini.toolserver.org>
Date: Sat, Nov 14, 2009 at 11:35 AM
Subject: Cron <root@cassini> /sql/diffs/load-next
To: root(a)cassini.toolserver.org
ERROR: List of process IDs must follow -p.
********* simple selection ********* ********* selection by list *********
-A all processes -C by command name
-N negate selection -G by real group ID (supports names)
-a all w/ tty except session leaders -U by real user ID (supports names)
-d all except session leaders -g by session OR by effective group name
-e all processes -p by process ID
T all processes on this terminal -s processes in the sessions given
a all w/ tty, including other users -t by tty
g OBSOLETE -- DO NOT USE -u by effective user ID (supports names)
r only running processes U processes for specified users
x processes w/o controlling ttys t by tty
*********** output format ********** *********** long options ***********
-o,o user-defined -f full --Group --User --pid --cols --ppid
-j,j job control s signal --group --user --sid --rows --info
-O,O preloaded -o v virtual memory --cumulative --format --deselect
-l,l long u user-oriented --sort --tty --forest --version
-F extra full X registers --heading --no-heading --context
********* misc options *********
-V,V show version L list format codes f ASCII art forest
-m,m,-L,-T,H threads S children in sum -y change -l format
-M,Z security data c true command name -c scheduling class
-w,w wide output n numeric WCHAN,UID -H process hierarchy
Hi Ævar
I'm sorry I have to bother you so much. Can you please execute the
following commands on cassini?
mv /sql/diffs/state.txt /sql/diffs/state-old.txt
mv /sql/diffs/state-new.txt /sql/diffs/state.txt
rm /sql/diffs/stop
this will re-enable the updates from a state before the error, so we
won't loose changes. I patched osmosis with woodpacks patch [1] to cut
of relations with more than 32767 members, as those kill as well osmosis
as osm2pgsql.
The Owner of /sql/diffs/state.txt is root, so I'm not able to do it
myself. Maybe we should run the update script under a user that is part
of the mapper group.
I have to go now but I'll look at it tomorrow morning.
Peter
[1]
Source Patch:
http://www.remote.org/frederik/tmp/remove_excessive_members.diff
replace Character.MAX_VALUE with 32767 before patching
Binary Package:
http://www.remote.org/frederik/tmp/osmosis-patched-to-drop-excessive-member…
renderd is crashing regularly on cassini. Here's the renderd -f output:
"""
[...]
renderd[23606]: DEBUG: Got command Dirty fd(35) xml(en), z(10), x(528), y(338)
renderd[23606]: DEBUG: Connection 18, fd 40 closed, now 20 left
renderd[23606]: DEBUG: Got command Dirty fd(39) xml(en), z(10), x(528), y(339)
renderd[23606]: DEBUG: Connection 18, fd 35 closed, now 19 left
renderd[23606]: DEBUG: Connection 18, fd 39 closed, now 18 left
Warning 1: TIFFReadDirectory:/mnt/user-store/osm_hillshading/N45E010-N49E014/N45E010-N49E014_hill.tif:
Wrong "StripByteCounts" field, ignoring and calculating from
imagelength
Warning 1: TIFFReadDirectory:/mnt/user-store/osm_hillshading/N45E010-N49E014/N45E010-N49E014_hill.tif:
Wrong "StripByteCounts" field, ignoring and calculating from
imagelength
terminate called after throwing an instance of 'mapnik::datasource_exception'
what(): PSQL error:
ERROR: invalid input syntax for type numeric: " "
Full sql was: 'SELECT AsBinary("way",'NDR') AS
geom,"importance_real","name","natural" from (SELECT
*, (to_number(importance, '9.9')) as importance_real
FROM planet_osm_point WHERE "natural" IN
('peak')) AS peak WHERE "way" &&
SetSRID('BOX3D(1210762.528037192 5750510.511950379,1254790.256329453
5794538.240242641)'::box3d, 900913)'
Aborted
"""
And from the postgresql main log:
"""
2009-11-11 19:57:12 UTC ERROR: invalid input syntax for type numeric: " "
2009-11-11 19:57:12 UTC STATEMENT: SELECT AsBinary("way",'NDR') AS
geom,"importance_real","name","natural" from (SELECT
*, (to_number(importance, '9.9')) as importance_real
FROM planet_osm_point WHERE "natural" IN
('peak')) AS peak WHERE "way" &&
SetSRID('BOX3D(1210762.528037192 5750510.511950379,1254790.256329453
5794538.240242641)'::box3d, 900913)
2009-11-11 19:57:13 UTC LOG: unexpected EOF on client connection
"""
Don't have time to debug this now, meanwhile I'm restarting renderd
every 5 minutes in cron..
Hey ho!
Finally, after the last import got interrupted, we got a fresh planet
import on cassini with the additional fields. It took around 28 hours
because of the new text-fields, i think. As cassini has no java, I can't
run osmosis and start the diff imports, but as river & marcin agreed on
a plan to make ptolem the new toolserver-db, we should maybe wait for
this to happen before putting any more work in cassini.
The script I did the import with can be fetched from here:
http://cassini.toolserver.org/~mazder/planet-update
Maybe it will be helpful un ptolemy, too.
Peter
