Hi,
I am writing to this list to inquire about possibility of setting up and maintaining up-to-date static HTML content of major Wikipedia/Wikimedia sites (for example http://download.wikimedia.org/static/).
I'd like to invite people from Wikimedia tech team to cooperate with the static content subcommittee of Special Projects Committee on technical aspects of its tasks. We need your help.
Please see: http://meta.wikimedia.org/wiki/Special_projects_committee http://meta.wikimedia.org/wiki/Static_content_subcommittee
Best regards, Kpjas
Krzysztof Jasiutowicz wrote:
Hi,
I am writing to this list to inquire about possibility of setting up and maintaining up-to-date static HTML content of major Wikipedia/Wikimedia sites (for example http://download.wikimedia.org/static/).
I'd like to invite people from Wikimedia tech team to cooperate with the static content subcommittee of Special Projects Committee on technical aspects of its tasks. We need your help.
I would be happy to assign a server for conversion duties. Originally srv31 (which hosts static.wikimedia.org) mostly did this; though it's been kind of taken over for additional things they can be moved.
-- brion vibber (brion @ pobox.com)
On this note, would it be possible to get a public pointer to the full skins, modifications, and so forth used in creating the latest static dumps (as at http://static.wikipedia.org)? Several projects I know of (including the OEPC project I'm working on) are looking for ways to create good static exports, and the November dumps come out looking excellent.
If it's more complicated than that, I'd be happy to contribute in any way I can to publicizing the export techniques. I would hope that this process would simplify the process of maintaining up-to-date static HTML dumps.
Thanks very much, Eric
-----Original Message----- From: wikitech-l-bounces@wikimedia.org [mailto:wikitech-l- bounces@wikimedia.org] On Behalf Of Krzysztof Jasiutowicz Sent: Monday, June 26, 2006 1:44 PM To: wikitech-l@wikimedia.org Subject: [Wikitech-l] Static content Wikipedia/Wikimedia
Hi,
I am writing to this list to inquire about possibility of setting up and maintaining up-to-date static HTML content of major Wikipedia/Wikimedia sites (for example http://download.wikimedia.org/static/).
I'd like to invite people from Wikimedia tech team to cooperate with the static content subcommittee of Special Projects Committee on technical aspects of its tasks. We need your help.
Please see: http://meta.wikimedia.org/wiki/Special_projects_committee http://meta.wikimedia.org/wiki/Static_content_subcommittee
Best regards, Kpjas -- Wikipedia - World's Greatest http://www.wikipedia.org _______________________________________________ Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
-- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.394 / Virus Database: 268.9.4/375 - Release Date: 6/25/2006
On 6/26/06, Eric Astor eastor1@swarthmore.edu wrote:
On this note, would it be possible to get a public pointer to the full skins, modifications, and so forth used in creating the latest static dumps (as at http://static.wikipedia.org)? Several projects I know of (including the OEPC project I'm working on) are looking for ways to create good static exports, and the November dumps come out looking excellent.
Shouldn't it be included in the sources?
http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/maintenance/dumpHTML....
Greetings, Mathias (who played around with dumpHTML.php a few months ago without getting into the details)
-----Original Message----- From: wikitech-l-bounces@wikimedia.org [mailto:wikitech-l- bounces@wikimedia.org] On Behalf Of Mathias Schindler Sent: Monday, June 26, 2006 2:17 PM To: Wikimedia developers Subject: Re: [Wikitech-l] Static content Wikipedia/Wikimedia
Shouldn't it be included in the sources?
http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/maintenance/dumpHTM L.php?revision=12476&view=markup
Not quite... I've done a dump, and the format comes out differently - not really appropriate for a static dump, since among other things, the Search button is present but fails. I suspect it's just a skin difference or something similarly small, but it'd be good to have a public statement of what's necessary to get the same format as the November static HTML dumps.
On Mon, 26 Jun 2006, Eric Astor wrote:
Not quite... I've done a dump, and the format comes out differently - not really appropriate for a static dump, since among other things, the Search button is present but fails. I suspect it's just a skin difference or something similarly small, but it'd be good to have a public statement of what's necessary to get the same format as the November static HTML dumps.
The November dumps also have working Category links. It seems hard to get these working properly.
Cheers, Andy!
Eric Astor wrote:
On this note, would it be possible to get a public pointer to the full skins, modifications, and so forth used in creating the latest static dumps (as at http://static.wikipedia.org)? Several projects I know of (including the OEPC project I'm working on) are looking for ways to create good static exports, and the November dumps come out looking excellent.
All the scripts used are public, they're at
http://static.wikipedia.org/scripts/
This is my actual working directory for constructing the November dumps, I decided that I didn't need to shut it away. The main entry point is do4thread. All patches and modifications were committed to CVS.
The skin used is at skins/disabled/HTMLDump.php, you need to move it to skins/HTMLDump.php for it to work.
In a second post, Eric Astor wrote:
Not quite... I've done a dump, and the format comes out differently - not really appropriate for a static dump, since among other things, the Search button is present but fails.
The JavaScript referenced by this skin is in skins/htmldump/, including lookup.js for the "go" button. It's not really a search button, since I haven't gotten around to writing a fully-fledged search engine in JavaScript just yet. It doesn't handle misses particularly gracefully.
Brion Vibber wrote:
I would be happy to assign a server for conversion duties. Originally srv31 (which hosts static.wikimedia.org) mostly did this; though it's been kind of taken over for additional things they can be moved.
srv31 would probably best, if it's got the hard drive space. dumpHTML is CPU-intensive, so it would be best if we could move other CPU-intensive tasks off it.
Actually what would really be nice is a server on VLAN 1. I wanted that last year but didn't get it. Instead we have srv31 serving webpages via squid, and downloads via NFS and benet. It would probably be better to do both directly.
-- Tim Starling
On Wed, 28 Jun 2006, Tim Starling wrote:
Eric Astor wrote:
On this note, would it be possible to get a public pointer to the full skins, modifications, and so forth used in creating the latest static dumps (as at http://static.wikipedia.org)? Several projects I know of (including the OEPC project I'm working on) are looking for ways to create good static exports, and the November dumps come out looking excellent.
All the scripts used are public, they're at
Thanks!
This is my actual working directory for constructing the November dumps, I decided that I didn't need to shut it away. The main entry point is do4thread. All patches and modifications were committed to CVS.
The skin used is at skins/disabled/HTMLDump.php, you need to move it to skins/HTMLDump.php for it to work.
I cannot reach this - it comes up empty. Is it being interpreted instead?
I cannot even get the HTMLDump.php.broken file.
In a second post, Eric Astor wrote:
Not quite... I've done a dump, and the format comes out differently - not really appropriate for a static dump, since among other things, the Search button is present but fails.
The JavaScript referenced by this skin is in skins/htmldump/, including lookup.js for the "go" button. It's not really a search button, since I haven't gotten around to writing a fully-fledged search engine in JavaScript just yet. It doesn't handle misses particularly gracefully.
Thanks.
Cheers, Andy!
On Wed, 28 Jun 2006, Andy Rabagliati wrote:
On Wed, 28 Jun 2006, Tim Starling wrote:
The skin used is at skins/disabled/HTMLDump.php, you need to move it to skins/HTMLDump.php for it to work.
I cannot reach this - it comes up empty. Is it being interpreted instead?
Sorry - found it in the mediawiki source tree.
Cheers, Andy!
wikitech-l@lists.wikimedia.org