Why does this happen?
---------- Forwarded message ----------
From: Cron Daemon <root(a)cassini.toolserver.org>
Date: Sat, Nov 14, 2009 at 11:35 AM
Subject: Cron <root@cassini> /sql/diffs/load-next
To: root(a)cassini.toolserver.org
ERROR: List of process IDs must follow -p.
********* simple selection ********* ********* selection by list *********
-A all processes -C by command name
-N negate selection -G by real group ID (supports names)
-a all w/ tty except session leaders -U by real user ID (supports names)
-d all except session leaders -g by session OR by effective group name
-e all processes -p by process ID
T all processes on this terminal -s processes in the sessions given
a all w/ tty, including other users -t by tty
g OBSOLETE -- DO NOT USE -u by effective user ID (supports names)
r only running processes U processes for specified users
x processes w/o controlling ttys t by tty
*********** output format ********** *********** long options ***********
-o,o user-defined -f full --Group --User --pid --cols --ppid
-j,j job control s signal --group --user --sid --rows --info
-O,O preloaded -o v virtual memory --cumulative --format --deselect
-l,l long u user-oriented --sort --tty --forest --version
-F extra full X registers --heading --no-heading --context
********* misc options *********
-V,V show version L list format codes f ASCII art forest
-m,m,-L,-T,H threads S children in sum -y change -l format
-M,Z security data c true command name -c scheduling class
-w,w wide output n numeric WCHAN,UID -H process hierarchy
Hi Ævar
I'm sorry I have to bother you so much. Can you please execute the
following commands on cassini?
mv /sql/diffs/state.txt /sql/diffs/state-old.txt
mv /sql/diffs/state-new.txt /sql/diffs/state.txt
rm /sql/diffs/stop
this will re-enable the updates from a state before the error, so we
won't loose changes. I patched osmosis with woodpacks patch [1] to cut
of relations with more than 32767 members, as those kill as well osmosis
as osm2pgsql.
The Owner of /sql/diffs/state.txt is root, so I'm not able to do it
myself. Maybe we should run the update script under a user that is part
of the mapper group.
I have to go now but I'll look at it tomorrow morning.
Peter
[1]
Source Patch:
http://www.remote.org/frederik/tmp/remove_excessive_members.diff
replace Character.MAX_VALUE with 32767 before patching
Binary Package:
http://www.remote.org/frederik/tmp/osmosis-patched-to-drop-excessive-member…
renderd is crashing regularly on cassini. Here's the renderd -f output:
"""
[...]
renderd[23606]: DEBUG: Got command Dirty fd(35) xml(en), z(10), x(528), y(338)
renderd[23606]: DEBUG: Connection 18, fd 40 closed, now 20 left
renderd[23606]: DEBUG: Got command Dirty fd(39) xml(en), z(10), x(528), y(339)
renderd[23606]: DEBUG: Connection 18, fd 35 closed, now 19 left
renderd[23606]: DEBUG: Connection 18, fd 39 closed, now 18 left
Warning 1: TIFFReadDirectory:/mnt/user-store/osm_hillshading/N45E010-N49E014/N45E010-N49E014_hill.tif:
Wrong "StripByteCounts" field, ignoring and calculating from
imagelength
Warning 1: TIFFReadDirectory:/mnt/user-store/osm_hillshading/N45E010-N49E014/N45E010-N49E014_hill.tif:
Wrong "StripByteCounts" field, ignoring and calculating from
imagelength
terminate called after throwing an instance of 'mapnik::datasource_exception'
what(): PSQL error:
ERROR: invalid input syntax for type numeric: " "
Full sql was: 'SELECT AsBinary("way",'NDR') AS
geom,"importance_real","name","natural" from (SELECT
*, (to_number(importance, '9.9')) as importance_real
FROM planet_osm_point WHERE "natural" IN
('peak')) AS peak WHERE "way" &&
SetSRID('BOX3D(1210762.528037192 5750510.511950379,1254790.256329453
5794538.240242641)'::box3d, 900913)'
Aborted
"""
And from the postgresql main log:
"""
2009-11-11 19:57:12 UTC ERROR: invalid input syntax for type numeric: " "
2009-11-11 19:57:12 UTC STATEMENT: SELECT AsBinary("way",'NDR') AS
geom,"importance_real","name","natural" from (SELECT
*, (to_number(importance, '9.9')) as importance_real
FROM planet_osm_point WHERE "natural" IN
('peak')) AS peak WHERE "way" &&
SetSRID('BOX3D(1210762.528037192 5750510.511950379,1254790.256329453
5794538.240242641)'::box3d, 900913)
2009-11-11 19:57:13 UTC LOG: unexpected EOF on client connection
"""
Don't have time to debug this now, meanwhile I'm restarting renderd
every 5 minutes in cron..
Hey ho!
Finally, after the last import got interrupted, we got a fresh planet
import on cassini with the additional fields. It took around 28 hours
because of the new text-fields, i think. As cassini has no java, I can't
run osmosis and start the diff imports, but as river & marcin agreed on
a plan to make ptolem the new toolserver-db, we should maybe wait for
this to happen before putting any more work in cassini.
The script I did the import with can be fetched from here:
http://cassini.toolserver.org/~mazder/planet-update
Maybe it will be helpful un ptolemy, too.
Peter
DaB. schrieb:
> Hello all,
>
> cassini was reboted because of a kernel-update.
>
> Sincerly,
> DaB.
>
Hum.. as discussed on Maps-L we were just in another planet-import that
would have taken some more hours.. As I can't connect via ssh atm. I'm
unsure if the import was complete at the time of reboot, but I suspect
it wasn't. Hopefully it left the Postgresql-Cluster in a clean state..
re-starting the import would be the minor problem, i think.
Peter
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
i'm the senior Unix administrator for the Toolserver project. i'm
mostly involved in the WMF side, and i haven't done much with the OSM
stuff. however, i have been observing the progress of the OSM
Toolserver (cassini), since that is ostensibly part of the Toolserver
cluster, and i have a few concerns.
firstly, i'd like to see proper integration between the OSM and WMF
parts of the Toolserver. for example, someone has configured a web
server on cassini, and people are already deploying tools at
<http://cassini.toolserver.org/~username/>. it should be clear why using
"cassini.toolserver.org" in a URL is a bad idea; but i don't see why
cassini needs to host user tools at all.
the Toolserver already has a web server which works quite well, and has
plenty of spare resources. by putting OSM user tools there, we can
combine resources and make efficient use of them; if/when the web server
is full, we can add additional resources and both WMF and OSM users will
immediately be able to take advantage of them. (i understand OSM needs
some special Apache module for serving tiles; keeping that on cassini
probably makes sense.)
it's also not clear to me whether cassini needs to be a login server at
all; the WMF databases at the TS don't allow user login, and users
connect to them remotely. is there something special about OSM tools
that prevents them from running on the existing TS login servers?
secondly, in the rush to deploy cassini as quickly as possible, it seems
that little planning has been done. one thing we've learnt from the
Toolserver is that it's a lot less effort to plan things properly from
the start, instead of deploying something, then discovering it has to be
redone later. might i suggest that people take a step back, consider
what exactly the "OSM Toolserver" is meant to be, and come up a plan
before going any further?
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAkr0AzYACgkQIXd7fCuc5vI9IgCgmrC/Y1klooI0DxmL8SziosO3
OcUAnAw30iUwysYLSnOD6fjTiGDukPW7
=Y5tA
-----END PGP SIGNATURE-----
Hello,
I make some extensions to my streetlist-script so that it gets nearly
the functionality of the old query-to-map script but is independent of
XAPI and GoogleMaps.
It now support point, line and area objects. As example you can see all
churches near Dresden:
http://cassini.toolserver.org/~kolossos/qtm2/queryinmap.php?BBOX=13.5333,50…
If you click on the objects you can get more informations about the object.
And you can get offcourse also a list of it:
http://cassini.toolserver.org/~kolossos/qtm2/featurelist.php?key=amenity&va…
More documention you can found in the wiki:
http://wiki.openstreetmap.org/wiki/Query-to-map
Before I want to promote it for to OSM-community I want to ask, if we
can get an update of the database? And could we get some additionally
columns to the database?
The following keys would be really usefull and are not too much:
cycleway, shop, lit, operator, surveillance, website and wikipedia.
Who can do this?
Greetings Kolossos
BTW: The OSM-Toolserver get a kind of competition:
http://wiki.openstreetmap.org/wiki/FOSSGIS/Server
This can be good for OSM and we can concentrate us perhaps to the
knowledge aspect of OSM and the interface to wikipedia.
<http://dict.leo.org/ende?lp=ende&p=DEdPgA&search=competition>
After a failure to install API database on ptolemy in reasonable amount
of time, I have come to conclusion that we might not need full API database
(ca. ~1 TB) for now, not at least for production Wikimedia use.
What Wikimedia might need is the Mapnik database (ca. 70GB for rendering)
- this is what we have now on Cassini (although outdated - but see neighbouring
thread).
Full API database - once *finally* imported - would have been probably more
useful for tool developers, hacking properties and some other features
of geodata (maybe even history).
However, only Ptolemy (currently WMF production server) has disk space
available for the whole API database; I do not see - but I might be wrong
- currently use for this database (except for genering mapnik DB out of it)
- for the WMF production environment.
So we might have add disk space to Cassini to have space for the full API
database, and start importing it there. But... Maybe we could just switch
servers? (i.e. current Cassini reconfigure as WMF-production Ptolemy and
current Ptolemy reinstall as a toolserver database).
Sure, it will require stopping tools for a while and do some IP (or maybe
even rack-juggling) but maybe it's the solution.
We might alternatively give up on full database API altogether
and just run Mapnik DB on the Ptolemy as we have.
What do you think?
--
<< Marcin Cieslak // saper(a)saper.info >>
