[Toolserver-l] Troubles with reading Articles

[Stefan F. Keller]

On March 23, 2006 10:09 PM Gregory Maxwell wrote:
> The compression really isn't a problem, if the text access is ever
> restored I will help you write code to read the compressed data... it
> is easy.   

That sounds good! Thank you for helping out (soon?). 

> Yes, you can't access it directly in the database, but
> you'd need to parse the content anyways.

If we would have the means to use stored procedures, then we would be happy
get the access to it. 

Please understand our situation: Leo - and me as a humble sponsoring mentor
- are taking a step by step approach in order to extract georeferenced
content. We don't need real-time access (and no history); but we really need
to parse *all* available articles near real-time at least *once* (Step 1;
takes about 6 hours). After that, we need only those articles which have
changed say in a nightly run (Step 2; takes couple of minutes). 

We started experimenting on local dumps then moved to toolserver because
dumps are really outdated (currently dewiki is still 2005-Dec-11!). If dumps
would be available, say, at a daily or weekly basis, that would be a choice.

> For a geospatial project on toolserver I'd expect that you'd run into
> bigger problems with things like the lack of useful indexing for
> spatial objects in Innodb tables.

Right: If we never get access to all articles in the near future, we'll
never run into problems postprocessing it ;-> (just joking...) The data
maintained in Step 2 does'nt need to be spatially indexed yet (it's enough
e.g. for KML-export) as long as no geospatial queries are needed.

> (I had a postgresql install with the tiger data in my home directory,
> but I removed it after the loss of text access killed all my spatial
> wiki plans. :( )

Geospatial data types and related queries are supported in MySQL with
SPATIAL INDEX TYPE MyISAM or else we take our own Postgresql-DB types and
PostGIS.

--Stefan