I'm Btech(2nd year) from India. I code in python/c/c++/javascript.
I have selected a mentorship project from Project
list<https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects>
named Google Books > Internet Archive > Commons upload cycle
I have a doubt about downloading ebooks from Google-Books.
Is it even legal(not against google's t&c).
If it is please guide me on how to get the downloads.
My ideas:
The Bot will surf Google-books and search the required book.
Then it will click on each link corresponding to each page number
(for the book), and then download the image(jpeg) for each page.
Note: The project is in python
We need community's valuable suggestions to improve the proposal.
Thanks a Lot.
--
Rohit Dua
JIIT, Noida
username: 8ohit.dua
IRC: rohit-dua
Le 02/03/2014 01:33, Samuel Klein a écrit :
> Brilliant. Congrats to everyone who is working on this!
> What is needed to scrape categories?
0 - For all dumped pages (so at least NS_MAIN and NS_CATEGORY pages),
download the list of categories they belong to (with the MW API).
1 - For each dumped page, implement the HTML rendering of the category
list at the bottom.
2 - For each category page, get the content HTML rendering from Parsoid
and compute and render sorted lists of articles and sub-categories in a
similar fashion like the online version (with multiple pages if necessary).
All the stuff must be integrated in the nodejs script and category graph
must be stored in redis.
Emmanuel
--
Kiwix - Wikipedia Offline & more
* Web: http://www.kiwix.org
* Twitter: https://twitter.com/KiwixOffline
* more: http://www.kiwix.org/wiki/Communication
Hello,
>From last response on proosal for Multilingual, usable and effective captchas; i figured out following few solutions to the points raised by the mailing list members:
1)Captcha on the basis of selection of particular object:In this type of captcha the questions will be shown as shown here: https://commons.wikimedia.org/wiki/File:Proposal6correction.png ;other possible questions can be:"Select the images in which man is wearing sunglasses",now problem i encountered while making such captcha request is that a bot could easily use Google images or third eye to find the look for images of man wearing sunglasses from the Wikimedia Commons .....so to encounter this problem we can use a random numbering over the images and then ask user to to select the images in order of numbering for example the only answer to the sample question i provided above is :https://commons.wikimedia.org/wiki/File:Proposal7correction.png ;one of the wrong answer to the given question is:https://commons.wikimedia.org/wiki/File:Proposal8correction.png ; because it is not arranged in correct order which is 1,4,12 (in increasing numeric order).....ps:i did forgot to specify "increasing" numeric order in the question i provided but the plan is to provide the question as "Select human among the given photograph's and select it in increasing numeric order?".
Providing increasing numeric order might make bot guess wrong answer 90% of time,which is quite comparable to the current captcha system that uses OCR as provided in the given algorithm: http://www.gizmag.com/captcha-beating-ai/29559/ .
2)Ask User to click on the same image as provided:The question for this type of captcha looks like this:https://commons.wikimedia.org/wiki/File:Proposal5correction.png we can ask user to click on the options showing equivalent image to that provided in the question.For the question provided the answer is :https://commons.wikimedia.org/wiki/File:Proposal9correction.png .Ps:we ca use more than four options probably eight options since it would be harder of bot to make a guess.
3)For blind and visually impaired users:We can use and audio captcha system which ask user to select the number as it is asked in the audio .For example the visual equivalent of the audio asked by the user will be : https://commons.wikimedia.org/wiki/File:Proposal11correction.png .Now when audio asked user to select number "0" our user will use arrow key to move across different blocks like a slide show....with different voice speaking out the options and pressing enter will select the word spoken....hence verifying that the user is human.....although the above shown image is visual equivalent the actual image visible will be: https://commons.wikimedia.org/wiki/File%3AProposal4correction.png ;....to make the captcha reload we can user to by pressing key "r" which will be instructed in the audio while the captcha starts playing.
Please give your response to idea provided and also the idea is listed here:https://www.mediawiki.org/wiki/Talk:CAPTCHA
Thank You
Aalekh Nigam
aalekhN
https://www.mediawiki.org/wiki/User:AalekhN
--
Hey,
As you are probably aware of, it has been possible for some time now to
install Composer compatible MediaWiki extensions via Composer.
Markus Glaser recently wrote an RFC titled "Extension management with
Composer" [0]. This RFC mentioned that it is not possible for extensions to
specify which version of MediaWiki they are compatible with. After
discussing the problem with some people from the Composer community, I
created a commit that addresses this pain point [1]. It's been sitting on
gerrit getting stale, so some input there is appreciated.
[0]
https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_wi…
[1] https://gerrit.wikimedia.org/r/#/c/105092/
For your convenience, a copy of the commit message:
~~
Make it possible for extensions to specify which version of MediaWiki
they support via Composer.
This change allows extensions to specify they depend on a specific
version or version range of MediaWiki. This is done by adding the
package mediawiki/mediawiki in their composer.json require section.
As MediaWiki itself is not a Composer package and is quite far away
from becoming one, a workaround was needed, which is provided by
this commit.
It works as follows. When "composer install" or "composer update"
is run, a Composer hook is invoked. This hook programmatically
indicates the root package provides MediaWiki, as it indeed does
when extensions are installed into MediaWiki. The package link
of type "provides" includes the MediaWiki version, which is read
from DefaultSettings.php.
This functionality has been tested and confirmed to work. One needs
a recent Composer version for it to have an effect. The upcoming
Composer alpha8 release will suffice. See
https://github.com/composer/composer/issues/2520
Tests are included. Composer independent tests will run always,
while the Composer specific ones are skipped when Composer is
not installed.
People that already have a composer.json file in their MediaWiki
root directory will need to make the same additions there as this
commit makes to composer-json.example. If this is not done, the
new behaviour will not work for them (though no existing behaviour
will break). The change to the json file has been made in such a
way to minimize the likelihood that any future modifications there
will be needed.
Thanks go to @beausimensen (Sculpin) and @seldaek (Composer) for
their support.
~~
I also wrote up a little blog post on the topic:
http://www.bn2vs.com/blog/2014/02/15/mediawiki-extensions-to-define-their-m…
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
Starting on Tuesday, March 4th, the new Labs install in the eqiad data
center will be open for business. Two dramatic things will happen on
that day: Wikitech will gain the ability to create instances in eqiad,
and wikitech will lose the ability to create new instances in pmtpa.
About a month from Tuesday, the pmtpa labs install will be shut down.
If you want your project to still be up and running in April, you must
take action!
We are committed to not destroying any instances or data during the
shutdown, but projects that remain untouched by human hands during the
next few weeks will be mothballed by staff: the data will be preserved
but most likely compressed and archived, and instances will be left in a
shutdown state.
(Note: Toollabs users can sit tight for a bit; Coren will provide
specific migration instructions for you shortly.)
I've written a migration guide, here:
https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration_Howto It's a
work in progress, so check back frequently. Please don't hesitate to
ask questions on IRC, make suggestions as to guide improvements, or
otherwise question this process. Quite a few of the suggested steps in
that guide require action on the part of a Labs op -- for that purpose
we've created a bugzilla tracking bug, 62042. To add a migration bug
that links to the tracker, use this link:
https://bugzilla.wikimedia.org/enter_bug.cgi?product=Wikimedia%20Labs&compo…
At the very least, please visit this page and edit it with your project
migration plans:
https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration_Progress
Projects that have no activity on that page will be early candidates for
mothballing. If you want me to delete your project, please note that as
well -- that will allow us to free up resources for future projects.
I am cautiously optimistic about this migration. Most of our testing
has gone fairly well, so a lot of you should find the process smooth and
easy. That said, we're all going to be early adopters of this tech, so
I appreciate your patience and understanding when inevitable bugs shake
out. I look forward to hearing about them on IRC!
-Andrew
Wikitech admin peoples!
I was doing bad things to my phone last night (reflashing it) and I lost
the 2 factor auth metadata for my authentication app. Because of this I can
no longer log in to wikitech.
I wasn't able to find any documentation on wikitech about how to reset it
-- so I need your help to do that I think? I still know my password; so I'm
not looking to reset that -- maybe just temporarily disable two factor auth
on my account (Mwalker) and I'll re-enroll myself?
~Matt Walker
Wikimedia Foundation
Fundraising Technology Team
Hello and welcome to the latest edition of the WMF Roadmap and
Deployments update.
See the full roadmap for next week and beyond here:
https://wikitech.wikimedia.org/wiki/Deployments#Week_of_March_3rd
Some important call outs:
== Monday ==
The migration of WMF Labs from pmtpa to eqiad begins
* new instance creation disabled in pmtpa, only available in eqiad
* See the emails from Andrew and Marc for more details:
** http://lists.wikimedia.org/pipermail/labs-l/2014-February/002152.html
** http://lists.wikimedia.org/pipermail/labs-l/2014-February/002153.html
We will be disabling ArticleFeedBack on all wikis.
* https://bugzilla.wikimedia.org/show_bug.cgi?id=61163
== Tuesday ==
MediaWiki upgrades
* group1 to 1.23wmf16: All non-Wikipedia sites (Wiktionary, Wikisource,
Wikinews, Wikibooks, Wikiquote, Wikiversity, and a few other sites)
* see also:
** https://www.mediawiki.org/wiki/MediaWiki_1.23/Roadmap#Schedule_for_the_depl…
** https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf16
== Wednesday ==
The new search cluster will be upgraded (to ElasticSearch 1.0.1).
* This will begin at 0:00 UTC March 6th/4pm Pacific March 5th and will
take a few hours to complete.
* All wikis currently using the new search (CirrusSearch) will be
temporarily switched back to the old serach (lsearchd)
* You shouldn't see much of a change in search behavior (CirrusSearch is
mostly feature parity to lsearchd) if your wiki is on new search, but
to see a list of wikis that currently have CirrusSearch enabled (and
in what way: Beta Feature or Primary), see:
** https://www.mediawiki.org/wiki/Search#Wikis
== Thursday ==
MediaWiki upgrades
* group2 to 1.23wmf16 (all Wikipedias)
* group0 to 1.23wmf17 (test/test2/testwikidata/mediawiki)
As always, questions welcome,
Greg
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
