As I understand it, the MediaWiki system works by interpreting the "source code" as stored in the database and rendering it into HTML. I am curious to know how tightly connected the code is to the database.
My ultimate goal is to make a WYSIWYG "offline" wiki editor. The user would browse the wiki as normal, but on pressing the Edit link (or a command in the app itself), it would download the source to a local cache for editing. I do this many times with my text editor, but I would like something much more capable, looking for missing closing refs and brackets, offering a separate window to keep cites and other clippings that are auto-included at the end, and offering a realtime preview
In order for this to work the interpreter would have to be able to be fed from a text file, or even better, a buffer. I would use CURL to manage downloading the storing the source, WebKit to display the rendered results, and various Cocoa objects to wrap it all together.
Any comments from the techs out there? Is this doable?
Maury
On 8/16/07, Maury Markowitz maury.markowitz@gmail.com wrote:
As I understand it, the MediaWiki system works by interpreting the "source code" as stored in the database and rendering it into HTML. I am curious to know how tightly connected the code is to the database.
The parser is, at present, very tightly connected to the rest of the code, and it cannot be easily extricated. A long-term goal is to write a better parser of some kind, ideally after making wikimarkup more regular and well-defined, but until then third parties either have to initialize the whole MediaWiki package or write their own parser and be incompatible in corner cases.
The Parser should not, however, be tied at all to the database. I'm fairly sure it would be easy enough to write an entry point that can parse arbitrary wikitext from stdin or wherever you like, you would just have to install and maintain a full local MediaWiki installation (including PHP but not necessarily MySQL or Apache, at least after you've generated LocalSettings) to do it.
Simetrical wrote:
On 8/16/07, Maury Markowitz maury.markowitz@gmail.com wrote:
As I understand it, the MediaWiki system works by interpreting the "source code" as stored in the database and rendering it into HTML. I am curious to know how tightly connected the code is to the database.
The parser is, at present, very tightly connected to the rest of the code, and it cannot be easily extricated.
Domas tells me he once did exactly that, as a short experimental project he replaced the calls to other parts of MediaWiki with stubs, thereby detaching the parser. Maury could probably reproduce that work.
-- Tim Starling
On 8/17/07, Tim Starling tstarling@wikimedia.org wrote:
Domas tells me he once did exactly that, as a short experimental project he replaced the calls to other parts of MediaWiki with stubs, thereby detaching the parser. Maury could probably reproduce that work.
Hmm. No, there's not *that* much interaction between it and the rest of the software. Now that you point it out, I grant there's no reason it shouldn't work to yank it out, necessarily. Maybe that could be done in core code, allowing the parser to stand alone.
On 8/17/07, Tim Starling tstarling@wikimedia.org wrote:
Domas tells me he once did exactly that, as a short experimental project he replaced the calls to other parts of MediaWiki with stubs, thereby detaching the parser. Maury could probably reproduce that work.
Perhaps I should drop him an e-mail? Anyone have an address?
Maury
On 8/17/07, Simetrical Simetrical+wikilist@gmail.com wrote:
On 8/16/07, Maury Markowitz maury.markowitz@gmail.com wrote:
As I understand it, the MediaWiki system works by interpreting the "source code" as stored in the database and rendering it into HTML. I am curious to know how tightly connected the code is to the database.
The Parser should not, however, be tied at all to the database.
The only caveat I can think of is that you've gotta make sure you have all the included files (mostly templates) downloaded before you go offline (and don't forget that templates can include templates, etc). Coloring of the links obviously won't work unless you somehow can pre-download that information (I'm not sure how they'll be handled). Images would have to be prefetched if you want them to show up, and at the least you probably want to note the sizes of the images for proper layout. I think the rest should be OK, in theory.
I'm assuming Maury doesn't mean offline in the sense of grab a page, disconnect from the net, edit it on a plane. In which case, he can go back to the db for the templates and images as needed. Getting extensions to render their content will be interesting...
Jim
On Aug 17, 2007, at 6:27 AM, Anthony wrote:
On 8/17/07, Simetrical Simetrical+wikilist@gmail.com wrote:
On 8/16/07, Maury Markowitz maury.markowitz@gmail.com wrote:
As I understand it, the MediaWiki system works by interpreting the "source code" as stored in the database and rendering it into HTML. I am curious to know how tightly connected the code is to the database.
The Parser should not, however, be tied at all to the database.
The only caveat I can think of is that you've gotta make sure you have all the included files (mostly templates) downloaded before you go offline (and don't forget that templates can include templates, etc). Coloring of the links obviously won't work unless you somehow can pre-download that information (I'm not sure how they'll be handled). Images would have to be prefetched if you want them to show up, and at the least you probably want to note the sizes of the images for proper layout. I think the rest should be OK, in theory.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
===================================== Jim Hu Associate Professor Dept. of Biochemistry and Biophysics 2128 TAMU Texas A&M Univ. College Station, TX 77843-2128 979-862-4054
Thanks for all the comments folks, they've really made me think about what the aim of the project should be.
I'm assuming Maury doesn't mean offline in the sense of grab a page, disconnect from the net, edit it on a plane.
MAYBE, but if this is a technical challenge, then no.
My main problem is losing edits because of a browser problem or timeout. Firefox and Safari has greatly reduced this problem, so the need is not so pressing.
Another is having real editing. Firefox can Find in the editors, but Safari cannot. I prefer Safari, but editing a long article can be a real chore. Neither is ideal, however, for instance understand the tag language, so they find things inside tags, which isn't appropriate.
And I really want something to make CITEs and REFs WAY easier. Right now I often don't bother with CITE because it's just too much work. I want something where I can just drag or paste an url into a box and add anything I want to the CITE, and have the system maintain the tags in the article body.
Finally, I want to clean up the screen. In general terms the editor takes up 25% of my real estate, and all the editor help and other fields takes up the rest. The editor should be it's own window, everything else should be put "elsewhere".
as needed. Getting extensions to render their content will be interesting...
What extensions do you refer to?
The only caveat I can think of is that you've gotta make sure you have all the included files (mostly templates) downloaded before you go offline (and don't forget that templates can include templates, etc).
If these have links in the resulting HTML, CURL will get them all. If not...
Maury
On Aug 20, 2007, at 8:51 AM, Maury Markowitz wrote:
Thanks for all the comments folks, they've really made me think about what the aim of the project should be.
I'm assuming Maury doesn't mean offline in the sense of grab a page, disconnect from the net, edit it on a plane.
MAYBE, but if this is a technical challenge, then no.
My main problem is losing edits because of a browser problem or timeout. Firefox and Safari has greatly reduced this problem, so the need is not so pressing.
Another is having real editing. Firefox can Find in the editors, but Safari cannot. I prefer Safari, but editing a long article can be a real chore. Neither is ideal, however, for instance understand the tag language, so they find things inside tags, which isn't appropriate.
And I really want something to make CITEs and REFs WAY easier. Right now I often don't bother with CITE because it's just too much work. I want something where I can just drag or paste an url into a box and add anything I want to the CITE, and have the system maintain the tags in the article body.
Not quite what you want, but I've been able to put the ref markup in the InsertChar stuff that appears under the edit box. Had to tweak it to deal with the space in <ref name=''/>; I just had it str_replace from <ref_name='+'/>
Finally, I want to clean up the screen. In general terms the editor takes up 25% of my real estate, and all the editor help and other fields takes up the rest. The editor should be it's own window, everything else should be put "elsewhere".
I haven't played with it much, but if you're on a Mac (which I'm guessing based on your references to Safari), I believe Omniweb does something similar to this. And of course, the Safari 3 beta lets you drag the edit box to make it a different size.
as needed. Getting extensions to render their content will be interesting...
What extensions do you refer to?
Extensions that have parser hooks render content based on what's inside the tags. You already know about CIte. There are lots of others, some of which can be pretty complex. The RSS feed extension, for example. There are extensions that do iFrames. The geshi extension recolors code. It may not matter for you, but if you're going to make this public, I think I'd like it to work with my custom extensions that embed an image or fill a box with the results of a database query, or reformat a DNA sequence to break it up into lines with spaces every 10 bases.
In other words, I think you'd need to pull the parser code and all the extension code into your local rendering engine if you really want to be able to see what the page will look like.
Perhaps I misunderstand the goal.
The only caveat I can think of is that you've gotta make sure you have all the included files (mostly templates) downloaded before you go offline (and don't forget that templates can include templates, etc).
If these have links in the resulting HTML, CURL will get them all. If not...
Maury
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
===================================== Jim Hu Associate Professor Dept. of Biochemistry and Biophysics 2128 TAMU Texas A&M Univ. College Station, TX 77843-2128 979-862-4054
On 8/20/07, Jim Hu jimhu@tamu.edu wrote:
And of course, the Safari 3 beta lets you drag the edit box to make it a different size.
As does Firefox, using (needless to say) an extension: Resizeable Form Fields.
Cool! That's good to know.
On Aug 20, 2007, at 12:31 PM, Simetrical wrote:
On 8/20/07, Jim Hu jimhu@tamu.edu wrote:
And of course, the Safari 3 beta lets you drag the edit box to make it a different size.
As does Firefox, using (needless to say) an extension: Resizeable Form Fields.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
===================================== Jim Hu Associate Professor Dept. of Biochemistry and Biophysics 2128 TAMU Texas A&M Univ. College Station, TX 77843-2128 979-862-4054
On 8/20/07, Simetrical Simetrical+wikilist@gmail.com wrote:
As does Firefox, using (needless to say) an extension: Resizeable Form Fields.
This is only semi-useful though; if you resize to use up the majority of the screen, the click-ons disappear and have to be scrolled too. Additionally, scrolling behavior, especially the scroll wheel, is "weird". But that's just a minor issue in the overall problem.
I'd really like a word-processor-like enviornment. Text in one window, optionally WYSIWYG, tools in palettes. A spell checker that remembers new works on a per-document basis would be great (ie, "zPhone" is spelled correctly in articles on the zPhone, it's likely spelled wrong in other ones). A grammar checker (yes, there really are some good ones) would be great. A notepad that lets you collect pages and text you find in other sources and keeps them handy would be amazing. A tool that converts this into CITEs would be great. Something that re-arranges your named REFs so they don't break would be EXTREMELY useful. A tool that looks up "broken" wikilinks in Google to try to find possible hits would make that whole part of my job so much easier. Etc.
Maury
wikitech-l@lists.wikimedia.org