I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either: * I'm not making the upload requests correctly * There is a server side security model that forbids this * I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Thanks, Jim
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
On Mon, Mar 21, 2005 at 05:14:08PM +0000, Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
Yes, but why in mediawiki-1.3.11, I have the same result when I'm using the canonical edit form? Tokens just fail to match. As well as some users get logged out right when they try to edit anything (cookies are ok on both server and clients), but I cannot reproduce this.
Jaroslaw Fedevych (UALUG wrote:
Yes, but why in mediawiki-1.3.11, I have the same result when I'm using the canonical edit form? Tokens just fail to match. As well as some users get logged out right when they try to edit anything (cookies are ok on both server and clients), but I cannot reproduce this.
Sounds like a failure of PHP's session data. Make sure this is set up correctly; if you're on a server farm (such as SourceForge's project web space) PHP may be incorrectly configured to save session data onto the local hard drive where it won't be picked up by subsequent hits on other servers.
-- brion vibber (brion @ pobox.com)
Oh, how about http://sourceforge.net/projects/phpmediawikibot/ ? That's in PHP! :) I wrote it myself.
On Mon, Mar 21, 2005 at 01:27:29PM -0800, Brion Vibber wrote:
Jaroslaw Fedevych (UALUG wrote:
Yes, but why in mediawiki-1.3.11, I have the same result when I'm using the canonical edit form? Tokens just fail to match. As well as some users get logged out right when they try to edit anything (cookies are ok on both server and clients), but I cannot reproduce this.
Sounds like a failure of PHP's session data. Make sure this is set up correctly; if you're on a server farm (such as SourceForge's project web space) PHP may be incorrectly configured to save session data onto the local hard drive where it won't be picked up by subsequent hits on other servers.
Well, well... It seems to pass wsEditToken for some articles, while for others it just fails to (I wrote a quick'n'dirty script which dumps the session).
I just don't know what can be reason for that. When you're calling editToken() for the first time it generates the token and saves it in the session, while upon subsequent calls it doesn't touch it. Truly, it is a mystery.
By the way, the script dumping the session does not use $_SESSION. It goes directly where PHP puts session files and reads the one for current session.
Could it be so that session data is silently destroyed and some values restored from cookies (I mean user login/password hash which stay there)?
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
Jim
-----BEGIN PGP SIGNED MESSAGE-----
Moin,
On Monday 21 March 2005 18:36, Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
I hacked together something similiar. A lot of people new to wiki's stuggle with just creating a new article. It really isn't obvious. So I added a "new article" entry to the toolbox menu.
This leads to a page (on another server, currently) with a form that lets you select a category (list currently hardcoded, and doesnt include the "no category" selection) and an input field for the article name, plus a preview button.
When you press "preview" it will bounce you back to the wiki with:
* "{{stub}}", some dummy headline/text and the category already included into the input box, * the preview already above it
The user can then edit the new article and finally submit it.
Currently it is a simple get form, but for the category inclusion I wanted to change it to an simple cgi script. I also haven't figured out how to prefill the "summary" entry field - new users are very likely to forget to fill it in (hopefully one day it will be mandatory to tack all your changes so I don't have to explain this over and over that this is important...)
Of course, my hack probably wont work with v1.4, but I have to test/adapt it :)
And in another day or to we talk about users who are not able to find the "New article" entry in the menu and complain about this all being "too complicated..." :-D
Best wishes,
Tels
- -- Signed on Mon Mar 21 19:02:15 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email.
"Let's say there are a thousand. But there are 284 million people in this country. You can't have public policy that is aimed at 100,000 people when the other multi-multi-millions are also involved. You can't do it that way." - Jack Valenty in http://tinyurl.com/2y65n
Tels wrote:
I hacked together something similiar. A lot of people new to wiki's stuggle with just creating a new article. It really isn't obvious. So I added a "new article" entry to the toolbox menu. When you press "preview" it will bounce you back to the wiki with:
- "{{stub}}", some dummy headline/text and the category already included
into the input box,
- the preview already above it
The user can then edit the new article and finally submit it.
Heh. I think Wikipedia could do with this.
'''Article name''' is ...
Further detail on Article name ...
==References==
*List your sources here.
==External links==
*List relevant external links that are not already references
- d.
Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
Having thought it some more, I'm pretty sure the only way this can be solved is to add recognition for &action=getedittoken to mediawiki. This would return a tiny http response with just the token as text/plain. When the server gets the token back, it would send back an "ok" or "collision"- this is editing equivalent of action=raw.
This would be useful for anyone who is trying to write a thick interface to mediawiki, or any other kind of alternate edit tool.
Does this sound like an acceptable addition?
Jim
On Mon, 21 Mar 2005 21:23:28 +0000, Jim Higson jh@333.org wrote:
Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
Having thought it some more, I'm pretty sure the only way this can be solved is to add recognition for &action=getedittoken to mediawiki. This would return a tiny http response with just the token as text/plain. When the server gets the token back, it would send back an "ok" or "collision"- this is editing equivalent of action=raw.
This would be useful for anyone who is trying to write a thick interface to mediawiki, or any other kind of alternate edit tool.
Does this sound like an acceptable addition?
Maybe I'm missing something (wouldn't be the first time) but couldn't you just parse the page for the edittoken? It's pretty easy, for example this Perl code which takes the scalar $editpage containing page retrevied with action=edit would return the token as a variable:
(my $wpEditToken) = $editpage =~ m#(?<=<input).*([\w\d]{32}).*(?=wpEditToken)#g;
Jim
Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
On Mon, 21 Mar 2005 22:55:50 +0000, Ævar Arnfjörð Bjarmason avarab@gmail.com wrote:
On Mon, 21 Mar 2005 21:23:28 +0000, Jim Higson jh@333.org wrote:
Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
Having thought it some more, I'm pretty sure the only way this can be solved is to add recognition for &action=getedittoken to mediawiki. This would return a tiny http response with just the token as text/plain. When the server gets the token back, it would send back an "ok" or "collision"- this is editing equivalent of action=raw.
This would be useful for anyone who is trying to write a thick interface to mediawiki, or any other kind of alternate edit tool.
Does this sound like an acceptable addition?
Maybe I'm missing something (wouldn't be the first time) but couldn't you just parse the page for the edittoken? It's pretty easy, for example this Perl code which takes the scalar $editpage containing page retrevied with action=edit would return the token as a variable:
(my $wpEditToken) = $editpage =~ m#(?<=<input).*([\w\d]{32}).*(?=wpEditToken)#g;
P.s.: not that I wouldn't mind something like a general robot interface, in fact, I voulenteer to (help) write one (if thats something the people with CVS access would be willing to accept that is).
Jim
Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
On Mon, 21 Mar 2005 23:00:25 +0000, Ævar Arnfjörð Bjarmason avarab@gmail.com wrote:
P.s.: not that I wouldn't mind something like a general robot interface, in fact, I voulenteer to (help) write one (if thats something the people with CVS access would be willing to accept that is).
Traditionally, code for already-made robots has been difficult to find - you actually have to email the bot author to recieve it. However, bot frameworks are at http://sourceforge.net/projects/pywikipediabot in Python and at http://sourceforge.net/projects/phpwikipediabot in PHP. (The PHP thing is my own; I'm planning to release the cron library soon, which has all the neat files I used to run en:User:Grammarbot)
Please would people stop quoting whole messages including backlogs. When I click on "Show quoted text" in gmail I have to scroll down to see what people actually meant to quote.
Traditionally, code for already-made robots has been difficult to find
- you actually have to email the bot author to recieve it. However,
bot frameworks are at http://sourceforge.net/projects/pywikipediabot in Python and at http://sourceforge.net/projects/phpwikipediabot in PHP. (The PHP thing is my own; I'm planning to release the cron library soon, which has all the neat files I used to run en:User:Grammarbot)
What I meant is an interface from the MediaWiki standpoint which would upon request easy-to-parse data for the bot, rather than having to scrape the html (which is 95%+ irrelivant to what one would want to do with a bot)
Ævar Arnfjörð Bjarmason wrote:
Maybe I'm missing something (wouldn't be the first time) but couldn't you just parse the page for the edittoken? It's pretty easy, for example this Perl code which takes the scalar $editpage containing page retrevied with action=edit would return the token as a variable:
(my $wpEditToken) = $editpage =~ m#(?<=<input).*([\w\d]{32}).*(?=wpEditToken)#g;
P.s.: not that I wouldn't mind something like a general robot interface, in fact, I voulenteer to (help) write one (if thats something the people with CVS access would be willing to accept that is).
This would be a huge help to me, since I have very little experience with PHP. I'm thinking some kind of webservices-like API that follows REST principles (ie not SOAP!)
I can already do most things on the client, but a few things are needed to allow alternative interfaces to fly:
* action=getedittoken - returns just an edit token * Special:Random&action=raw gives the raw content of a random page * Special:Export?title=RecentChanges gives an XML list of recent changes
None of these things need any major architectural changes.
Jim
P.s.: not that I wouldn't mind something like a general robot interface, in fact, I voulenteer to (help) write one (if thats something the people with CVS access would be willing to accept that is).
This would be a huge help to me, since I have very little experience with PHP. I'm thinking some kind of webservices-like API that follows REST principles (ie not SOAP!)
I can already do most things on the client, but a few things are needed to allow alternative interfaces to fly:
- action=getedittoken - returns just an edit token
- Special:Random&action=raw gives the raw content of a random page
- Special:Export?title=RecentChanges gives an XML list of recent changes
None of these things need any major architectural changes.
Jim
Rather than having some quick hack made just for you it would probably be better to consider well how to make a standard interface for something like this, something like action=robot which would return something in an easy-to-parse format, is there a bug open for this already where something like this could be brainstormed?
In the meantime, as I understand it what you need *now* is to just implement this on your own wiki, would it not be an option to just modify the code you use to not require the token?
Ævar Arnfjörð Bjarmason wrote:
P.s.: not that I wouldn't mind something like a general robot interface, in fact, I voulenteer to (help) write one (if thats something the people with CVS access would be willing to accept that is).
This would be a huge help to me, since I have very little experience with PHP. I'm thinking some kind of webservices-like API that follows REST principles (ie not SOAP!)
I can already do most things on the client, but a few things are needed to allow alternative interfaces to fly:
- action=getedittoken - returns just an edit token
- Special:Random&action=raw gives the raw content of a random page
- Special:Export?title=RecentChanges gives an XML list of recent changes
None of these things need any major architectural changes.
Jim
Rather than having some quick hack made just for you it would probably be better to consider well how to make a standard interface for something like this, something like action=robot which would return something in an easy-to-parse format, is there a bug open for this already where something like this could be brainstormed?
Yes, I agree. However, it should not be assumed that the agent is a robot, it may be a thick client written (like mine) in AJAX, GTK or some other thing.
I'm not sure what you mean by an easy to parse format? Just sending the token as 32 plaintext chars seems as easy as it could possibly be.
In the meantime, as I understand it what you need *now* is to just implement this on your own wiki, would it not be an option to just modify the code you use to not require the token?
Well, yes, but for my project, not for my wiki. My intention is to provide a mediawiki interface that can be used on any interface. My wiki is really only for the purpose of testing and demonstrating that interface.
What I am really trying to do is bring down the number of Apache machines needed to serve a big wiki to maybe a quarter what it is now. Serving static content is really, really cheap.
-- Jim
Yes, I agree. However, it should not be assumed that the agent is a robot, it may be a thick client written (like mine) in AJAX, GTK or some other thing.
A robot in this context is widely defined as any program that parses server responses and acts based on those responses, this would include your script
I'm not sure what you mean by an easy to parse format? Just sending the token as 32 plaintext chars seems as easy as it could possibly be.
As I said I'm thinking of a *general* interface for robots, not just something to get one single attribute on edit pages, although getting the token would be included it would only be a small part of it.
For example, I wrote a robot to upload files two days ago, what has to be done to upload a file to wikipedia is:
1. Send a POST request with the following: (so far so good)
wpUpload => 'Upload file', # actually it seems like this only has to be sent, not with these specific contents, the code that parses it is: } else if ( 'submit' == $this->mAction || $this->mUpload ) { where mUpload is this string. wpUploadFile => [ $file ], wpUploadDescription => encode($encoding, $description), wpUploadAffirm => 1,
2. Process whatever SpecialUpload spits out which can be * Error: ** The image page is protected, wfMsg( 'protectedpage' ) ** The filetype is not accepted, wfMsg( 'badfiletype' ) ** The name is not accepted, wfMsg( 'illegalfilename' ) * Warning: ** The filesize (too big) ** The name of the file (s/ /_/) ** The file is empty ** The file already exists * [.. probably something missing here]
in the case of an error you'll find <h4 class='error'> on the page, and <ul class='warning'> if you'll get a warning, however nothing tells you *what kind of warning* so you'll have to resort to actually looking up the warnings in question in Language.php (this is assuming you're using english and not one of the ~80 other languages) to tell what kind of warning you just recived, this of course ignores all the excess html you just recived which you have no use for at all.
So, what I'd like for Special:Upload is to be able to submit a POST request to http://commons.wikimedia.org/wiki/Special:Upload?action=robot and have it return something like:
largefile,fileexists
It could even return all possible strings as boolean values:
largefile=0 fileexists=1 emptyfile=1 [...]
or, in the case of ?action=robot on an article: token=1231asdfasdt....
The the format is something which can be agreed on, but the actualy point is that there's alot more that needs a robot interface than just your 32 character edit token.
Ævar Arnfjörð Bjarmason wrote:
Yes, I agree. However, it should not be assumed that the agent is a robot, it may be a thick client written (like mine) in AJAX, GTK or some other thing.
A robot in this context is widely defined as any program that parses server responses and acts based on those responses, this would include your script
I'm not sure what you mean by an easy to parse format? Just sending the token as 32 plaintext chars seems as easy as it could possibly be.
As I said I'm thinking of a *general* interface for robots, not just something to get one single attribute on edit pages, although getting the token would be included it would only be a small part of it.
For example, I wrote a robot to upload files two days ago, what has to be done to upload a file to wikipedia is:
- Send a POST request with the following: (so far so good)
wpUpload => 'Upload file', # actually it seems like this only has to be sent, not with these specific contents, the code that parses it is: } else if ( 'submit' == $this->mAction || $this->mUpload ) { where mUpload is this string. wpUploadFile => [ $file ], wpUploadDescription => encode($encoding, $description), wpUploadAffirm => 1,
- Process whatever SpecialUpload spits out which can be
- Error:
** The image page is protected, wfMsg( 'protectedpage' ) ** The filetype is not accepted, wfMsg( 'badfiletype' ) ** The name is not accepted, wfMsg( 'illegalfilename' )
- Warning:
** The filesize (too big) ** The name of the file (s/ /_/) ** The file is empty ** The file already exists
- [.. probably something missing here]
in the case of an error you'll find <h4 class='error'> on the page, and <ul class='warning'> if you'll get a warning, however nothing tells you *what kind of warning* so you'll have to resort to actually looking up the warnings in question in Language.php (this is assuming you're using english and not one of the ~80 other languages) to tell what kind of warning you just recived, this of course ignores all the excess html you just recived which you have no use for at all.
So, what I'd like for Special:Upload is to be able to submit a POST request to http://commons.wikimedia.org/wiki/Special:Upload?action=robot and have it return something like:
largefile,fileexists
It could even return all possible strings as boolean values:
largefile=0 fileexists=1 emptyfile=1 [...]
or, in the case of ?action=robot on an article: token=1231asdfasdt....
The the format is something which can be agreed on, but the actualy point is that there's alot more that needs a robot interface than just your 32 character edit token.
Ok, I agree here. Some kind of simple XML format might be a nice way to do it because we'd get the parser for 'free'. XML'd also fit in well with the XML from Special:Export
Even with the present mediawiki is it advisable for a robot to upload article text using index.php?title=foo&action=submit or Special:Upload? I've been trying the former so far.
-- Jim
On Tue, 22 Mar 2005 20:14:59 +0000, Jim Higson jh@333.org wrote:
Ævar Arnfjörð Bjarmason wrote:
Yes, I agree. However, it should not be assumed that the agent is a robot, it may be a thick client written (like mine) in AJAX, GTK or some other thing.
A robot in this context is widely defined as any program that parses server responses and acts based on those responses, this would include your script
I'm not sure what you mean by an easy to parse format? Just sending the token as 32 plaintext chars seems as easy as it could possibly be.
As I said I'm thinking of a *general* interface for robots, not just something to get one single attribute on edit pages, although getting the token would be included it would only be a small part of it.
For example, I wrote a robot to upload files two days ago, what has to be done to upload a file to wikipedia is:
- Send a POST request with the following: (so far so good)
wpUpload => 'Upload file', # actually it seems like this only has to be sent, not with these specific contents, the code that parses it is: } else if ( 'submit' == $this->mAction || $this->mUpload ) { where mUpload is this string. wpUploadFile => [ $file ], wpUploadDescription => encode($encoding, $description), wpUploadAffirm => 1,
- Process whatever SpecialUpload spits out which can be
- Error:
** The image page is protected, wfMsg( 'protectedpage' ) ** The filetype is not accepted, wfMsg( 'badfiletype' ) ** The name is not accepted, wfMsg( 'illegalfilename' )
- Warning:
** The filesize (too big) ** The name of the file (s/ /_/) ** The file is empty ** The file already exists
- [.. probably something missing here]
in the case of an error you'll find <h4 class='error'> on the page, and <ul class='warning'> if you'll get a warning, however nothing tells you *what kind of warning* so you'll have to resort to actually looking up the warnings in question in Language.php (this is assuming you're using english and not one of the ~80 other languages) to tell what kind of warning you just recived, this of course ignores all the excess html you just recived which you have no use for at all.
So, what I'd like for Special:Upload is to be able to submit a POST request to http://commons.wikimedia.org/wiki/Special:Upload?action=robot and have it return something like:
largefile,fileexists
It could even return all possible strings as boolean values:
largefile=0 fileexists=1 emptyfile=1 [...]
or, in the case of ?action=robot on an article: token=1231asdfasdt....
The the format is something which can be agreed on, but the actualy point is that there's alot more that needs a robot interface than just your 32 character edit token.
Ok, I agree here. Some kind of simple XML format might be a nice way to do it because we'd get the parser for 'free'. XML'd also fit in well with the XML from Special:Export
Even with the present mediawiki is it advisable for a robot to upload article text using index.php?title=foo&action=submit or Special:Upload? I've been trying the former so far.
What do you mean "advisable"?, it's possible, but as I demonstrated you somtimes run into parsing problems.
Last week's theme was: KISS = keep it short and simple, please you re-apply it ?
Ævar Arnfjörð Bjarmason schrieb: [..many Kilobytes deleted ]
-----BEGIN PGP SIGNED MESSAGE-----
Moin,
On Tuesday 22 March 2005 21:14, Jim Higson wrote:
Ævar Arnfjörð Bjarmason wrote:
Yes, I agree. However, it should not be assumed that the agent is a robot, it may be a thick client written (like mine) in AJAX, GTK or some other thing.
The the format is something which can be agreed on, but the actualy point is that there's alot more that needs a robot interface than just your 32 character edit token.
Ok, I agree here. Some kind of simple XML format might be a nice way to do it because we'd get the parser for 'free'. XML'd also fit in well with the XML from Special:Export
XML and simple is an oxymoron :) While I agree on the robot-api, I think XML is overkill. It would, for instance, make necc. that every robot includes the whole XML libs just to parse the response, which throws simple libwwwget Perl robots out of the window.
KISS :)
Best wishes,
Tels
- -- Signed on Wed Mar 23 17:37:20 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email.
http://www.informatik.hu-berlin.de/~hstamm/beweis.html - Nützliche Beweismethoden für jeden Tag
Ævar Arnfjörð Bjarmason wrote:
On Mon, 21 Mar 2005 21:23:28 +0000, Jim Higson jh@333.org wrote:
Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
I wasn't aware of such a thing - this is obviously the problem.
Is is possible to request a token from the server without it generating a page? (since the page will never be displayed) Or any another way that a thick(er) client can submit edits.
I could just request an edit page in the background to get the token, but doing so would defeat the object of a low-bandwidth, low-cpu wiki.
As an aside, what do people here think of this project? Personally I see a lot of potential in this kind of 'half-thick' interface - the very quick feedback when editing should make contributing a lot easier for those starting to learn the wiki syntax.
Having thought it some more, I'm pretty sure the only way this can be solved is to add recognition for &action=getedittoken to mediawiki. This would return a tiny http response with just the token as text/plain. When the server gets the token back, it would send back an "ok" or "collision"- this is editing equivalent of action=raw.
This would be useful for anyone who is trying to write a thick interface to mediawiki, or any other kind of alternate edit tool.
Does this sound like an acceptable addition?
Maybe I'm missing something (wouldn't be the first time) but couldn't you just parse the page for the edittoken? It's pretty easy, for example this Perl code which takes the scalar $editpage containing page retrevied with action=edit would return the token as a variable:
(my $wpEditToken) = $editpage =~ m#(?<=<input).*([\w\d]{32}).*(?=wpEditToken)#g;
The problem here is I'm trying to build a low bandwidth interface. If the edit token is 32 bytes it seems a terrible waste to have to download 40k or more of HTML to get at it!
If you goto http://81.5.150.113/wysi - it looks pretty much like the normal wikimedia, but NONE of the html you see was generated on the server. It's difficult to get this across sometimes because to the user it looks just the same.
The other issue is creating pages that are never shown is a huge waste of server-side CPU time. My interface uses only static content on the presentation tier. Potentially an Apache machine could serve an order of magnitude more pages using this project.
Jim
-----BEGIN PGP SIGNED MESSAGE-----
Moin,
On Tuesday 22 March 2005 11:39, Jim Higson wrote:
Ævar Arnfjörð Bjarmason wrote:
On Mon, 21 Mar 2005 21:23:28 +0000, Jim Higson jh@333.org wrote:
Jim Higson wrote:
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
[snip]
(my $wpEditToken) = $editpage =~ m#(?<=<input).*([\w\d]{32}).*(?=wpEditToken)#g;
The problem here is I'm trying to build a low bandwidth interface. If the edit token is 32 bytes it seems a terrible waste to have to download 40k or more of HTML to get at it!
If you goto http://81.5.150.113/wysi - it looks pretty much like the normal wikimedia, but NONE of the html you see was generated on the server. It's difficult to get this across sometimes because to the user it looks just the same.
Well, it may be because it doesnt work (for me :)
I can do some things at your site, but of the buttons are nonfunctional, or I get wierd renderings etc. Firefox 1.01 (no privoxy this time) on linux.
I DO have some problems with the new kernel, so maybe it is just me :)
Anyway, the life preview doesnt work, the "save" gives me an "error uploading" msg box, and the "preview" button cannot be clicked. And then there are 4 blue rectangles without any function I can see :)
The other issue is creating pages that are never shown is a huge waste of server-side CPU time. My interface uses only static content on the presentation tier. Potentially an Apache machine could serve an order of magnitude more pages using this project.
I find this project faszinating, since it moves quite a few things (rendering, upload of text/download of output) to the client - at least while editing with multiple previews.
I like the idea that only the final version is uploaded to the DB server. This his huge potential!
Best wishes,
Tels
- -- Signed on Tue Mar 22 17:53:33 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email.
"...pornographic images stay in the brain forever." -- Mary Anne Layden; "That's a feature, not a bug." -- God
If you goto http://81.5.150.113/wysi - it looks pretty much like the normal wikimedia, but NONE of the html you see was generated on the server. It's difficult to get this across sometimes because to the user it looks just the same.
Well, it may be because it doesnt work (for me :)
I can do some things at your site, but of the buttons are nonfunctional, or I get wierd renderings etc. Firefox 1.01 (no privoxy this time) on linux.
I DO have some problems with the new kernel, so maybe it is just me :)
This is very strange - not least because as far as I can remember the string "error uploading" is'nt in there!
Let's see... I'm tesing Firefox 1.0 preview (not the latest version I know). It sounds as if XMLHTTP is failing for some reason. Do services such as Google complete work on your Firefox.
Could you visit the site with ?debug=yes added to the URL and report what (if anything) is printed on the right of the window? And also what (again, if anything) is sent to the console. Remember to clear the cache first to make sure you get the latest version (shift+click refresh).
There are still a few interface things that aren't implemented, but they should be rendered struck-through.
{snip}
--Jim
-----BEGIN PGP SIGNED MESSAGE-----
Moin,
On Tuesday 22 March 2005 18:19, Jim Higson wrote:
If you goto http://81.5.150.113/wysi - it looks pretty much like the normal wikimedia, but NONE of the html you see was generated on the server. It's difficult to get this across sometimes because to the user it looks just the same.
Well, it may be because it doesnt work (for me :)
I can do some things at your site, but of the buttons are nonfunctional, or I get wierd renderings etc. Firefox 1.01 (no privoxy this time) on linux.
I DO have some problems with the new kernel, so maybe it is just me :)
This is very strange - not least because as far as I can remember the string "error uploading" is'nt in there!
Let's see... I'm tesing Firefox 1.0 preview (not the latest version I know). It sounds as if XMLHTTP is failing for some reason. Do services such as Google complete work on your Firefox.
I don't know. What is Google complete? (google so far works, AFAIK)
Could you visit the site with ?debug=yes added to the URL and report what (if anything) is printed on the right of the window? And also what (again, if anything) is sent to the console. Remember to clear the cache first to make sure you get the latest version (shift+click refresh).
Ah, ok. I *thought* I already did shift+click, but apparently not. The page looks _much_ more complete now.
There are still a few interface things that aren't implemented, but they should be rendered struck-through.
Yeah, the life preview works now - very slick! Sorry for the noise!
The only thing that is a bit of a problem is the "pause" that you experience when you type - e.g. when it renders the preview (I like how it does this only when I stop typing :) it pauses shortly or freezes.
There is probably not much you can do, but these pauses interrupt my typing too much - so I think I would disable the life preview - which is a shame because this is real cool.
Best wishes,
Tels
- -- Signed on Tue Mar 22 18:18:12 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email.
"A Thaum is the basic unit of magical strength. It has been universally established as the amount of magic needed to create one small white pigeon or three normal-sized billiard balls." -- Terry Pratchett
Tels wrote:
Let's see... I'm tesing Firefox 1.0 preview (not the latest version I know). It sounds as if XMLHTTP is failing for some reason. Do services such as Google complete work on your Firefox.
I don't know. What is Google complete? (google so far works, AFAIK)
A little beta they run that uses some of the same technology as my project. Very, very cool: http://www.google.com/webhp?complete=1&hl=en
Could you visit the site with ?debug=yes added to the URL and report what (if anything) is printed on the right of the window? And also what (again, if anything) is sent to the console. Remember to clear the cache first to make sure you get the latest version (shift+click refresh).
Ah, ok. I *thought* I already did shift+click, but apparently not. The page looks _much_ more complete now.
There are still a few interface things that aren't implemented, but they should be rendered struck-through.
Yeah, the life preview works now - very slick! Sorry for the noise!
The only thing that is a bit of a problem is the "pause" that you experience when you type - e.g. when it renders the preview (I like how it does this only when I stop typing :) it pauses shortly or freezes.
There is probably not much you can do, but these pauses interrupt my typing too much - so I think I would disable the life preview - which is a shame because this is real cool.
I think it is mostly a matter now of optimising the parser and getting the timings right - there's a variable called BUG_ME_NOT which is the minimum time between the last keystroke and the preview being updated. Too low and it interupts the typing, too high and the preview takes a long time to appear.
At the moment it's 400ms, but I'll experiment with longer timings. I suppose I could even make it something like 1500 - a second and a half isn't too long to wait for the preview.
-- Jim
Tomer Chachamu wrote:
On Mon, 21 Mar 2005 15:07:50 +0000, Jim Higson jh@333.org wrote:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Are you loading the edit page first and taking the wpEditToken? You need it. (It's used for various things, including detection of edit conflicts.)
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
-- Jim
On Tue, Mar 22, 2005 at 03:58:30PM +0000, Jim Higson wrote:
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
No, the field is called precisely "wpEditToken" and contains a MD5 sum. Good time to tell us which version of MediaWiki you are using, maybe there is a bug?..
Jaroslaw Fedevych (UALUG wrote:
On Tue, Mar 22, 2005 at 03:58:30PM +0000, Jim Higson wrote:
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
No, the field is called precisely "wpEditToken" and contains a MD5 sum. Good time to tell us which version of MediaWiki you are using, maybe there is a bug?..
Thanks for the help, I'm using 1.4beta6. There is definitely not any "wpEditToken" in the HTML. Maybe this is a known issue with beta6? When I get time I'll upgrade to 1.4rc1 and see if it persists.
-- Jim
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
No, the field is called precisely "wpEditToken" and contains a MD5 sum. Good time to tell us which version of MediaWiki you are using, maybe there is a bug?..
Thanks for the help, I'm using 1.4beta6. There is definitely not any "wpEditToken" in the HTML. Maybe this is a known issue with beta6? When I get time I'll upgrade to 1.4rc1 and see if it persists.
I've just looked at meta.wikipedia (which runs 1.4rc1) and their edit pages have no wpEditToken field either.
lynx -source "http://meta.wikimedia.org/w/index.php?title=Wikipedia&action=edit" | grep -i token gives no output
Maybe edit tokens were removed in 1.4? If so there must be some other reason my script's edits aren't being accepted.
-- Jim
lynx -source "http://meta.wikimedia.org/w/index.php?title=Wikipedia&action=edit" | grep -i token gives no output
The token isn't there unless you're logged in, it seems.
On Tue, Mar 22, 2005 at 10:15:58PM +0000, John Fader wrote:
lynx -source "http://meta.wikimedia.org/w/index.php?title=Wikipedia&action=edit" | grep -i token gives no output
The token isn't there unless you're logged in, it seems.
Well, exactly -- there is no point in verifying your login credentials if you are not logged in.
But, it should not then bounce you to preview, as no token is required.
Strange...
Jaroslaw Fedevych (UALUG wrote:
On Tue, Mar 22, 2005 at 10:15:58PM +0000, John Fader wrote:
lynx -source "http://meta.wikimedia.org/w/index.php?title=Wikipedia&action=edit" | grep -i token gives no output
The token isn't there unless you're logged in, it seems.
Well, exactly -- there is no point in verifying your login credentials if you are not logged in.
But, it should not then bounce you to preview, as no token is required.
Strange...
Ok, so since I'm not supporting logging in yet, it should work without considering edit tokens.
Monitoring the http traffic, I notice the cookie PHPSESSID - would this being set incorrectly cause an edit to fail?
I've pasted a captured session below. Hopefully someone might see from this why the edit isn't checked in:
POST /wiki/index.php?title=Very_small_test&action=submit HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 (X11; U; Linux i686; rv:1.7.3) Gecko/20041003 Firefox/0.10 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Content-Type: multipart/form-data; boundary=------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Length: 884 Cookie: PHPSESSID=c55689cac6afedce4dfa9f3f290cf89c Pragma: no-cache Cache-Control: no-cache ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Disposition: form-data; name=wpTextbox1
this is a very small test.
[[オーバーク]]
aaa ''' bbb ccc ''' ddd eee aaa ''' bbb '' ccc ''' ddd '' eee ''' ffff ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Disposition: form-data; name=wpSummary
type a summary of your edit here ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Disposition: form-data; name=wpSave
Save page ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Disposition: form-data; name=wpSection
sections! ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd Content-Disposition: form-data; name=wpEdittime
20050322224700 ------------t4X4PnvyQFePUtyUn1guOoRIREp5XVvPZTTxEMvdcZ06RyaEzF8L1Xd--
HTTP/1.1 200 OK Date: Tue, 22 Mar 2005 23:01:37 GMT Server: Apache/2.0.51 (Fedora) X-Powered-By: PHP/4.3.10 Content-language: en Vary: Accept-Encoding,Cookie Expires: -1 Cache-Control: private, must-revalidate, max-age=0 Content-Encoding: gzip Content-Length: 3145 Connection: close Content-Type: text/html; charset=utf-8
(html page with preview etc)
On Tue, 22 Mar 2005 15:58:30 +0000, Jim Higson jh@333.org wrote:
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
I believe wpEditToken was added in 1.4. wpEdittime is easily calculable if you know the server time. (Well, it *is* the server time.) Perhaps you could load a page with the server time on it when the client opens, then keep track. One second out won't hurt.
Tomer Chachamu wrote:
On Tue, 22 Mar 2005 15:58:30 +0000, Jim Higson jh@333.org wrote:
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
According to the DOM inspector form#editform fields are:
"wpTextbox1" "wpSummary" "wpSave" "wpSection" "wpEdittime"
Is the edit time used as the token?
I believe wpEditToken was added in 1.4. wpEdittime is easily calculable if you know the server time. (Well, it *is* the server time.) Perhaps you could load a page with the server time on it when the client opens, then keep track. One second out won't hurt.
I'm pretty sure I can get server time from the http headers when I check the raw text out. I'll just calculate the offset from that with the local machine (a second eitherway won't hurt)
-- Jim
Jim Higson wrote:
Actually, where is the token on the edit page? This sounds like a silly question, but I've just saved the HTML of an edit page and greped for 'token' without any hits!
The edit token is presently only used for logged-in users.
a) It's purpose is to prevent offsite form submissions from hijacking users' login credentials to perform actions on the wiki; an anonymous visitor has the same rights as the attacker would
b) It relies on having an open session to store the master token in, which will fail for an anonymous editor not accepting cookies.
-- brion vibber (brion @ pobox.com)
I have a script for this, call editpage.php Please put your name to the mailinglist on http://bugzilla.wikimedia.org/show_bug.cgi?id=831
Jim Higson schrieb:
I'm trying to programatically submit edits to my wiki via a http post, using XMLHTTP. However, I am unable to 'persuade' the server to accept them. The server always responds as if I'm asking for a preview.
Either:
- I'm not making the upload requests correctly
- There is a server side security model that forbids this
- I'm going mad :)
You can see the form-data I'm sending by going here:
http://81.5.150.113/wysi/Very_small_test?debug=yes&action=edit then change the wikitext and click save, the request and response body will be shown on the right (which I use as a debugging area)
Any help would be really appreciated, this is a real show-stopper for my project.
Thanks, Jim
Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org