Re: [Mediawiki-api] Editing a page from Bash: copy-pasted token works, parsed token value from API doesnt

Hi everyone, I'm trying to login using the MediaWiki API. This works if I copy the token manually in the code every time again. Using the token from the variable DOESNT work, but copy pasting the same token into the TOKEN= assignment (into the source code) DOES work! After that, I'm trying to edit a page. I have the same problem with the edit token. It cannot be read from the variable, unless I copied the token manually to the variable in the source code :( I know I'm almost there, but the tokens are getting in the way. Why doesn't it work when I'm parsing the token from the API? When I copy-paste the same token into the source code directly it works.. The token looks the same to me! Is curl getting in the way here? I tried several things, like dropping the last \ from the token and then passing it to curl, but to no avail. Output of the script when it doesn't work (reading the token from the TOKEN variable when parsing from the API): $ ./clientcode.sh UTF8 check: ☠ Logging into https://nl.wikipedia.org/w/api.php as Smile4ever... Get login token... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 112 100 112 0 0 649 0 --:--:-- --:--:-- --:--:-- 666 { "batchcomplete": "", "query": { "tokens": { "logintoken": "9b9a82c7ac27cf9573e1c7cb7419cf53589041ec+\\" } } } Login token is "9b9a82c7ac27cf9573e1c7cb7419cf53589041ec+\\" ----- Logging in... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 302 100 142 100 160 829 935 --:--:-- --:--:-- --:--:-- 958 { "error": { "code": "badtoken", "info": "Invalid CSRF token.", "*": "See https://nl.wikipedia.org/w/api.php for API usage." }, "servedby": "mw1288" } Unable to login, is logintoken "9b9a82c7ac27cf9573e1c7cb7419cf53589041ec+\\" correct? ========================================== Output of the script when it does work (reading the token from the TOKEN variable when using a fixed value for the login token): $ ./clientcode.sh UTF8 check: ☠ Logging into https://nl.wikipedia.org/w/api.php as Smile4ever... Get login token... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 111 100 111 0 0 642 0 --:--:-- --:--:-- --:--:-- 660 { "batchcomplete": "", "query": { "tokens": { "logintoken": "7bfac90f99869aeb77fee2646d0022f1589040d3+\\" } } } Login token is 3e929f9275d81f12a1f7c93e1beec99f589040b4+\ ----- Logging in... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 224 100 76 100 148 36 71 0:00:02 0:00:02 --:--:-- 72 { "clientlogin": { "status": "PASS", "username": "Smile4ever" } } Successfully logged in as Smile4ever, STATUS is "PASS" Script: #!/usr/bin/env bash #Needs curl USERNAME="Smile4ever" USERPASS="******" WIKIAPI="https://nl.wikipedia.org/w/api.php" cookie_jar="wikicj" #Will store file in wikifile echo "UTF8 check: ☠" #################login echo "Logging into $WIKIAPI as $USERNAME..." ############### #Login part 1 #printf "%s" "Logging in (1/2)..." echo "Get login token..." CR=$(curl -S \ --location \ --retry 2 \ --retry-delay 5\ --cookie $cookie_jar \ --cookie-jar $cookie_jar \ --user-agent "Curl Shell Script" \ --keepalive-time 60 \ --header "Accept-Language: en-us" \ --header "Connection: keep-alive" \ --compressed \ --request "GET" "${WIKIAPI}?action=query&meta= tokens&type=login&format=json") echo "$CR" | jq . rm login.json echo "$CR" > login.json TOKEN=$(jq '.query.tokens.logintoken' login.json) #TOKEN="${TOKEN//\"/}" #replace double quote by nothing #Remove carriage return! #printf "%s" "$TOKEN" > token.txt #TOKEN=$(cat token.txt | sed 's/\r$//') #TOKEN="$(<token.txt)" #TOKEN="${TOKEN//\+\\/\+}" #replace +\ by + #echo -n $TOKEN > token.txt #TOKEN="${TOKEN//\+\\/%2b%5c}" #replace +\ by %2b%5c #TOKEN="33bf0c310b22a836be6b28905470d3a1589031da+\\" #TOKEN="24949330db743d9af74e7e56f2b523725890407b+\\" if [ "$TOKEN" == "null" ]; then echo "Getting a login token failed." exit else echo "Login token is $TOKEN" echo "-----" fi ############### #Login part 2 echo "Logging in..." CR=$(curl -S \ --location \ --cookie $cookie_jar \ --cookie-jar $cookie_jar \ --user-agent "Curl Shell Script" \ --keepalive-time 60 \ --header "Accept-Language: en-us" \ --header "Connection: keep-alive" \ --compressed \ --data-urlencode "username=${USERNAME}" \ --data-urlencode "password=${USERPASS}" \ --data-urlencode "rememberMe=1" \ --data-urlencode "logintoken=${TOKEN}" \ --data-urlencode "loginreturnurl=http://google.be" \ --request "POST" "${WIKIAPI}?action=clientlogin&format=json") echo "$CR" | jq . STATUS=$(echo $CR | jq '.clientlogin.status') if [[ $STATUS == *"PASS"* ]]; then echo "Successfully logged in as $USERNAME, STATUS is $STATUS." echo "-----" else echo "Unable to login, is logintoken ${TOKEN} correct?" exit fi ############### #Get edit token echo "Fetching edit token..." CR=$(curl -S \ --location \ --cookie $cookie_jar \ --cookie-jar $cookie_jar \ --user-agent "Curl Shell Script" \ --keepalive-time 60 \ --header "Accept-Language: en-us" \ --header "Connection: keep-alive" \ --compressed \ --request "POST" "${WIKIAPI}?action=query&meta= tokens&format=json") echo "$CR" | jq . echo "$CR" > edittoken.json EDITTOKEN=$(jq '.query.tokens.csrftoken' edittoken.json) rm edittoken.json EDITTOKEN="${EDITTOKEN//\"/}" #replace double quote by nothing #EDITTOKEN="${EDITTOKEN//\+\\/\+}" #replace +\ by + if [[ $EDITTOKEN == *"+\\"* ]]; then echo "Edit token is: $EDITTOKEN" else echo "Edit token not set." exit fi ############### #Make a test edit #EDITTOKEN="d55014d69f1a8c821073bb6724aced7658904018+\\" CR=$(curl -S \ --location \ --cookie $cookie_jar \ --cookie-jar $cookie_jar \ --user-agent "Curl Shell Script" \ --keepalive-time 60 \ --header "Accept-Language: en-us" \ --header "Connection: keep-alive" \ --compressed \ --data-urlencode "title=Gebruiker:Smile4ever/test4" \ --data-urlencode "appendtext={{nocat|2017|01|31}}" \ --data-urlencode "token=${EDITTOKEN}" \ --request "POST" "${WIKIAPI}?action=edit&format=json") echo "$CR" | jq . Kind regards, Geoffrey De Belie _______________________________________________ Mediawiki-api mailing list Mediawiki-api(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api