On Fri, Jan 21, 2011 at 6:48 AM, Aryeh Gregor Simetrical+wikilist@gmail.com wrote:
Not to mention, the text table is immutable, so creating and publishing text table dumps incrementally should be trivial.
The problem there is deletion and oversight. The best solution if you didn't have to worry about that would be to have a database on the dump servers with only public data, which accesses a live feed (over the LAN). Then creating a dump would be as simple as pg_dump, and fancier incremental dumps could be made relatively simply as well.
Then again, if your live feed tells you which revisions to delete/oversight, that's still a viable solution.
On Thu, Jan 20, 2011 at 4:04 AM, Anthony wikimail@inbox.org wrote:
It wouldn't be trivial, but it wouldn't be particularly hard either. Most of the work is already being done. It's just being done inefficiently.
I'm glad to see you know what you're talking about here. Presumably you've examined the relevant code closely and determined exactly how you'd implement the necessary changes in order to evaluate the difficulty. Needless to say, patches are welcome.
Access to the servers is welcome. I can't possibly test and improve performance without it.
Alternatively, give me a free live feed, and I'll make a decent dump system here at home, and provide the source code when I'm done.