On 09/06/2011 01:29 AM, Peter Körner wrote:
Am 06.09.2011 07:27, schrieb Tim Alder:
Thanks for the answer.
I try now a "wget planet.openstreetmap.org" and this works fine.
So the problem seems on our side.
osm2pgsql is also running sometimes for short time.
But something seem confused.
I figured out that osm2pgsql is crashing with this error message:
Processing: Node(277k) Way(42k) Relation(0)get_way failed: ERROR: could
not access status of transaction 2920577984
DETAIL: Could not open file "pg_clog/0AE1": No such file or directory.
(7)
Arguments were: 70616681,
Error occurred, cleaning up
looks like a crashed postgres database. Maybe opening a ticket in jira
could help. Maybe.
As mentioned in the jira ticket, it is still crashing, and it is always
crashing at the exact same point: when trying to process way 70616681 (
http://www.openstreetmap.org/browse/way/70616681/history )
I have seen similar errors about "pg_clog" files missing a while ago
already. I think back then it was when manually sending queries to the
postgres db on ptolemy.
I tried to google for the error message and it seems to indicate a
corruption of the database.
With Postgresql 8.1 it sounds like it was a reasonably common issue due
to a software bug. I have seen some reports though of it in 8.3 as well,
but for those versions, I haven't read about a specific reason for the
corruption yet.
In this case, the data corruption (assuming that is what it is) occurred
presumably somehow in the planet_ways table.
Things people have suggested to try and fix it:
1) Run a vacuum full on the database.
What is the current strategy for vacuuming the database? Is Auto-vacuum
turned on, or manual vacuums run?
2) Create the missing pg_clog file as an empty file
dd if=/dev/zero of=/var/lib/pgsql/data/pg_clog/0AE1 bs=256K count=1
I think that might cause some data loss, although it might help to
recover the rest of the database (which in our case isn't necessary as
one can recreate it from other sources)
3) Dump the database and then do a full restore. In our case, it would
be probably easier to simply drop the db and do a fresh import via
osm2pgsql.
If 3) turns out to be the only possibility, it might be a good time to
upgrade to postgresql 9.0
Kai
Peter
_______________________________________________
Maps-l mailing list
Maps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l