jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860982 )

Change subject: [docs] Improve basic content structure and navigation ......................................................................

[docs] Improve basic content structure and navigation

* add code block language indicators * unify section and page naming, add sections where appropriate, align heading syntax with Python style guide * remove horizontal line on framework.png * improve style in some sections * rename installation to introduction (keep old page as redirect) * remove unnecessary indentation * remove duplicated navigation entries * fix README tables

Patch detached from tests branch because file outside docs folder are affected.

Bug: T323812 Change-Id: I810d27449a43aba1e361bab64aab3a42618efe30 --- M docs/getting_help.rst M ROADMAP.rst M README.rst M docs/recipes.rst M scripts/README.rst M docs/index.rst M docs/scripts_ref/scripts.maintenance.rst M docs/api_ref/index.rst M docs/global_options.rst M docs/library_usage.rst M docs/utilities/scripts_ref.rst M HISTORY.rst M docs/scripts_ref/index.rst M docs/scripts_ref/scripts.i18n.rst M docs/scripts_ref/scripts.rst M CODE_OF_CONDUCT.rst M docs/mwapi.rst M docs/utilities/scripts.rst M docs/tests_ref/index.rst M docs/changelog.rst M docs/installation.rst M docs/scripts/index.rst M docs/framework.png A docs/introduction.rst M docs/utilities/install.rst M docs/faq.rst M docs/scripts_ref/scripts.userscripts.rst 27 files changed, 391 insertions(+), 351 deletions(-)

Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified

diff --git a/CODE_OF_CONDUCT.rst b/CODE_OF_CONDUCT.rst index cb0641f..71156e2 100644 --- a/CODE_OF_CONDUCT.rst +++ b/CODE_OF_CONDUCT.rst @@ -1,5 +1,5 @@ Code of Conduct ---------------- +===============

The development of this software is covered by a `Code of Conduct https://www.mediawiki.org/wiki/Special:MyLanguage/Code_of_Conduct`_. diff --git a/HISTORY.rst b/HISTORY.rst index de1ace6..f1eeda2 100644 --- a/HISTORY.rst +++ b/HISTORY.rst @@ -1,5 +1,5 @@ Release history -^^^^^^^^^^^^^^^ +===============

7.7.1 ----- @@ -197,7 +197,7 @@ *26 February 2022*

Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^

* i18n updates for date.py * Add number transliteration of 'lo', 'ml', 'pa', 'te' to NON_LATIN_DIGITS @@ -240,7 +240,7 @@ * New "maxlimit" property was added to APISite (:phab:`T291770`)

Bugfixes -~~~~~~~~ +^^^^^^^^

* Don't raise an exception if BlockEntry initializer found a hidden title (:phab:`T78152`) * Fix KeyError in create_warnings_list (:phab:`T301610`) @@ -261,14 +261,14 @@ * Raise a generic ServerError if the http status code is unofficial (:phab:`T293208`)

Breaking changes -~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^

* Support of Python 3.5.0 - 3.5.2 has been dropped (:phab:`T286867`) * generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be used with pwb wrapper script * *See also Code cleanups below*

Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^

* Deprecated http.get_fake_user_agent() function was removed * FilePage.fileIsShared() was removed in favour of FilePage.file_is_shared() @@ -436,7 +436,7 @@ *28 May 2021*

Improvements and Bugfixes -~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^

* Use different logfiles for multiple processes of the same script (:phab:`T56685`) * throttle.pip will be reused as soon as possbile @@ -452,7 +452,7 @@ * Handle abusefilter-{disallow,warning} codes (:phab:`T85656`)

Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^

* MultipleSitesBot.site attribute was removed (:phab:`T283209`) * Deprecated BaseSite.category_namespaces() method was removed @@ -481,7 +481,7 @@ *17 April 2021*

Improvements and Bugfixes -~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^

* interwiki_graph module was restored (:phab:`T223826`) * proofreadpage: search for "new" class after purge (:phab:`T280357`) @@ -496,7 +496,7 @@ * Family files can be collected from a zip folder (:phab:`T278076`)

Dependencies -~~~~~~~~~~~~ +^^^^^^^^^^^^

* **mwparserfromhell** or **wikitextparser** are strictly recommended (:phab:`T106763`) * Require **Pillow**>=8.1.1 due to vulnerability found (:phab:`T278743`) @@ -505,7 +505,7 @@ * Add support for **PyMySQL** 1.0.0+

Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^

* APISite.resolvemagicwords(), BaseSite.ns_index() and remove BaseSite.getNamespaceIndex() were removed * Deprecated MoveEntry.new_ns() and new_title() methods were removed @@ -528,7 +528,7 @@ *16 March 2021*

Breaking changes -~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^

* interwiki_graph module was removed (:phab:`T223826`) * Require setuptools >= 20.2 due to :pep:`440` @@ -538,7 +538,7 @@ * config.db_hostname has been renamed to db_hostname_format

Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^

* tools.PY2 was removed (:phab:`T213287`) * Site.language() method was removed in favour of Site.lang property @@ -555,7 +555,7 @@ * Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response (:phab:`T265206`)

Other changes -~~~~~~~~~~~~~ +^^^^^^^^^^^^^

* Raise a SiteDefinitionError if api request response is Non-JSON and site is AutoFamily (:phab:`T272911`) * Support deleting and undeleting specific file versions (:phab:`T276725`) @@ -1331,7 +1331,7 @@ * CODE_OF_CONDUCT included

Bugfixes -~~~~~~~~ +^^^^^^^^ * Manage temporary readonly error (:phab:`T154011`) * Unbreak wbGeoShape and WbTabularData (:phab:`T166362`) * Clean up issue with _WbDataPage (:phab:`T166362`) @@ -1343,7 +1343,7 @@ * Fix TestLazyLoginNotExistUsername test for Stewardwiki (:phab:`T169458`)

Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Introduce the new WbUnknown data type for Wikibase (:phab:`T165961`) * djvu.py: add replace_page() and delete_page() * Build GeoShape and TabularData from shared base class @@ -1367,7 +1367,7 @@ * Make WbRepresentations hashable (:phab:`T167827`)

Updates -~~~~~~~ +^^^^^^^ * Update linktails * Update languages_by_size * Update cross_allowed (global bot wikis group) @@ -1389,7 +1389,7 @@ * Localisation updates

Bugfixes -~~~~~~~~ +^^^^^^^^ * Increase the default socket_timeout to 75 seconds (:phab:`T163635`) * use repr() of exceptions to prevent UnicodeDecodeErrors (:phab:`T120222`) * Handle offset mismatches during chunked upload (:phab:`T156402`) @@ -1402,7 +1402,7 @@ * Remove wikidataquery from pywikibot (:phab:`T162585`)

Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Introduce user_add_claim and allow asynchronous ItemPage.addClaim (:phab:`T87493`) * Enable private edit summary in specialbots (:phab:`T162527`) * Make a decorator for asynchronous methods @@ -1429,7 +1429,7 @@ * Get thumburl information in FilePage() (:phab:`T137011`)

Updates -~~~~~~~ +^^^^^^^ * Update languages_by_size in family files * wikisource_family.py: Add "pa" to languages_by_size * Config2: limit the number of retries to 15 (:phab:`T165898`) @@ -1444,7 +1444,7 @@ * Deprecate previous 2.0 branches and tags

Bugfixes -~~~~~~~~ +^^^^^^^^ * Use default summary when summary value does not contain a string (:phab:`T160823`) * Enable specialbots.py for PY3 (:phab:`T161457`) * Change tw(n)translate from Site.code to Site.lang dependency (:phab:`T140624`) @@ -1466,7 +1466,7 @@ * Fix broken LDAP based login (:phab:`T90149`)

Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Simplify User class * Renamed isImage and isCategory * Add -property option to pagegenerators.py @@ -1485,7 +1485,7 @@ * Fix and improve default regexes

Updates -~~~~~~~ +^^^^^^^ * Update linktrails * Update languages_by_size * Updating global bot wikis, closed wikis and deleted wikis @@ -1507,7 +1507,7 @@ * Last stable 2.0 branch

Bugfixes -~~~~~~~~ +^^^^^^^^ * Establish the project's name, once and for all * setup.py: Add Python 3.4 and 3.5 to pypi classifiers * Remove item count output in page generators @@ -1543,7 +1543,7 @@ *15 December 2015*

Bugfixes -~~~~~~~~ +^^^^^^^^ * Remove dependency on pYsearch * Require google>=0.7 * Desupport Python 2.6 for Pywikibot 2.0 release branch @@ -1577,7 +1577,7 @@ *30 September 2015*

Bugfixes -~~~~~~~~ +^^^^^^^^ * New Wikipedia site: azb * Indexes in str.format * MediaWikiVersion: Accept new wmf style @@ -1593,18 +1593,18 @@ *9 July 2015*

Configuration updates -~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^ * Changing the sandbox content template on Fa WP

Family file updates -~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^ * Remove broken wikis from battlestarwiki family * Adding euskara and sicilianu languages to Vikidia family * WOW Wiki subdomains hr, ro & sr deleted * Add new Wikipedia languages gom and lrc

Bugfixes -~~~~~~~~ +^^^^^^^^ * fix UnicodeDecodeError on api error * pwb.py now correctly passes arguments to generate_family_file * Fix Win32 config.editor detection diff --git a/README.rst b/README.rst index 007e564..876bb38 100644 --- a/README.rst +++ b/README.rst @@ -35,9 +35,9 @@ :alt: Last commit :target: https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/

- +********* Pywikibot -========= +*********

The Pywikibot framework is a Python library that interfaces with the `MediaWiki API https://www.mediawiki.org/wiki/API:Main_page`_ @@ -50,9 +50,9 @@ the full `code documentation https://doc.wikimedia.org/pywikibot/`_.

Quick start ------------ +===========

-:: +.. code:: text

pip install requests git clone https://gerrit.wikimedia.org/r/pywikibot/core.git @@ -62,7 +62,7 @@

Or to install using PyPI (excluding scripts)

-:: +.. code:: text

pip install -U setuptools pip install pywikibot @@ -70,13 +70,13 @@

In addition a MediaWiki markup parser is required. Please install one of them:

-:: +.. code:: text

pip install mwparserfromhell

or

-:: +.. code:: text

pip install wikitextparser

@@ -85,11 +85,11 @@ has more details for advanced usage.

Basic Usage ------------ +===========

If you wish to write your own script it's very easy to get started:

-:: +.. code:: python

import pywikibot site = pywikibot.Site('en', 'wikipedia') # The site we want to run our bot on @@ -98,12 +98,12 @@ page.save('Replacing "foo" with "bar"') # Saves the page

Wikibase Usage --------------- +==============

Wikibase is a flexible knowledge base software that drives Wikidata. A sample pywikibot script for getting data from Wikibase:

-:: +.. code:: python

import pywikibot site = pywikibot.Site('wikipedia:en') @@ -113,11 +113,11 @@ data = item.get() # get all item data from repository for this item

Script example --------------- +==============

Pywikibot provides bot classes to develop your own script easily:

-:: +.. code:: python

import pywikibot from pywikibot import pagegenerators @@ -157,17 +157,17 @@

Roadmap -------- +=======

.. include:: ROADMAP.rst

Release history ---------------- +===============

See https://github.com/wikimedia/pywikibot/blob/stable/HISTORY.rst

Contributing ------------- +============

Our code is maintained on Wikimedia's `Gerrit installation https://gerrit.wikimedia.org/`_, `learn https://www.mediawiki.org/wiki/Developer_account`_ how to get diff --git a/ROADMAP.rst b/ROADMAP.rst index 41032d0..d478166 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -1,5 +1,5 @@ Current release 8.0.0 -^^^^^^^^^^^^^^^^^^^^^ +---------------------

* All parameters of :meth:`Category.members()<page.Category.members>`, :meth:`Category.subcategories()<page.Category.subcategories>` and @@ -29,7 +29,7 @@ * MediaWiki API cross reference was added to the documentation

Deprecations -^^^^^^^^^^^^ +------------

* 8.0.0: :meth:`APISite.messages()<pywikibot.site._apisite.APISite.messages>` method is deprecated in favour of :attr:`userinfo['messages']<pywikibot.site._apisite.APISite.userinfo>` @@ -67,6 +67,6 @@

Will be removed in Pywikibot 8 ------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* 5.5.0: APISite.redirectRegex() will be removed in favour of APISite.redirect_regex() diff --git a/docs/api_ref/index.rst b/docs/api_ref/index.rst index c6bb0df..bb402e6 100644 --- a/docs/api_ref/index.rst +++ b/docs/api_ref/index.rst @@ -13,7 +13,7 @@ functions provided by :mod:`data.api`. This layer then uses :func:`comms.http.request` to do the actual HTTP request.

-Table of Contents +Table of contents =================

.. toctree:: diff --git a/docs/changelog.rst b/docs/changelog.rst index 8e6a551..b12b4ac 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -1,10 +1,12 @@ -:tocdepth: 3 - +********** Change log -========== +**********

What is new with Pywikibot |release|? What are the main changes of older version?

+Roadmap +======= + .. include:: ../ROADMAP.rst

.. include:: ../HISTORY.rst diff --git a/docs/faq.rst b/docs/faq.rst index f0530b7..f707136 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -1,5 +1,6 @@ -Frequent Asked Questions +Frequent asked questions ======================== + **How to speed up Pywikibot?** 1. The first time you are using Pywikibot for multiple Wikimedia sites you can run :py:mod:`preload_sites <pywikibot.scripts.preload_sites>` script @@ -35,4 +36,3 @@ **pywikibot.i18n.TranslationError: No English translation has been defined** It can happen due to lack of i18n submodule or files. Update i18n submodule or download these files first. See also: :manpage:`i18n` manual. - diff --git a/docs/framework.png b/docs/framework.png index ec4cbc2..1956ec8 100644 --- a/docs/framework.png +++ b/docs/framework.png Binary files differ diff --git a/docs/getting_help.rst b/docs/getting_help.rst index 208af04..52c3f16 100644 --- a/docs/getting_help.rst +++ b/docs/getting_help.rst @@ -2,8 +2,8 @@ ------------

.. note:: - Please see :manpage:`Communication`. + For information on how to connect with Pywikibot developers and users, see :manpage:`Communication`.

.. tip:: - Please report bugs at `Phabricator https://phabricator.wikimedia.org/`_. - You may use `this report form https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=pywikibot-core`_. + Please report bugs in `Phabricator https://phabricator.wikimedia.org/`_. + You can use `this report form https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=pywikibot-core`_. diff --git a/docs/global_options.rst b/docs/global_options.rst index 57af62a..59f082d 100644 --- a/docs/global_options.rst +++ b/docs/global_options.rst @@ -1,4 +1,4 @@ -Global Options +Global options --------------

Global arguments available for all bots: @@ -6,4 +6,4 @@ .. literalinclude:: ../pywikibot/bot.py :language: console :start-after: (Global arguments available for all bots) - :end-before: """ \ No newline at end of file + :end-before: """ diff --git a/docs/index.rst b/docs/index.rst index 4384161..4dd75b9 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -36,13 +36,14 @@ :alt: Framework modules dependency network

-For bot users: --------------- +Contents +--------

.. toctree:: :maxdepth: 1 + :caption: User guide

- installation + introduction utilities/scripts scripts/index global_options @@ -50,34 +51,27 @@ getting_help

-For bot developers: -------------------- - .. toctree:: :maxdepth: 1 + :caption: Bot developer guide

library_usage recipes api_ref/index mwapi

-For framework developers: -------------------------- - .. toctree:: :maxdepth: 1 + :caption: Pywikibot contributor guide

- Utility Scripts Reference<utilities/scripts_ref> + utilities/scripts_ref scripts_ref/index - tests_ref/index - Maintenance Scripts<scripts_ref/scripts.maintenance> utilities/install + tests_ref/index

- -Miscellaneous -------------- .. toctree:: :maxdepth: 1 + :caption: About

glossary changelog @@ -86,4 +80,4 @@

.. _CC-BY-SA 3.0: https://creativecommons.org/licenses/by-sa/3.0/ -.. _Pywikibot Manual: https://www.mediawiki.org/wiki/Manual:Pywikibot \ No newline at end of file +.. _Pywikibot Manual: https://www.mediawiki.org/wiki/Manual:Pywikibot diff --git a/docs/installation.rst b/docs/installation.rst index cc555a4..b39bd99 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -1,43 +1,6 @@ +:orphan: + Installation ============

-.. include:: ../README.rst - :end-before: For more documentation on Pywikibot - -Settings --------- - -It is recommended to create a `user-config.py` file if Pywikibot is used as a -site package. If Pywikibot is used in directory mode e.g. from a repository -the configuration file is mandatory. A minimal sample is shown below. - -.. literalinclude:: ../user-config.py.sample - -This sample is shipped with the repository but is not available with -the site-package. For more settings use -:mod:`generate_user_files<pywikibot.scripts.generate_user_files>` script -or refer :py:mod:`pywikibot.config` module. - -.. seealso:: :manpage:`Installation` Manual - - -Internationalisation (i18n) ---------------------------- - -Some of the framework input interaction is translated. The user interface -language to be used can be set as follows: - -#. set the `userinterface_lang` in your :ref:`user-config.py<User Interface Settings>` to your preferred language -#. set environment variable `PYWIKIBOT_USERINTERFACE_LANG` to your preferred language -#. default is obtained from `locale.getdefaultlocale` -#. fallback is `'en'` for english if all other options fails - -.. note:: The preferred language code must follow ISO 639. -.. versionadded:: 7.0 - Added to site-package distribution -.. seealso:: - * :manpage:`i18n` Manual - * `MediaWiki Language Codes https://www.mediawiki.org/wiki/Manual:Language#Language_code`_ - * :ref:`User Interface Settings` - * :py:mod:`pywikibot.i18n` - +Moved to :doc:`introduction`. diff --git a/docs/introduction.rst b/docs/introduction.rst new file mode 100644 index 0000000..922fd22 --- /dev/null +++ b/docs/introduction.rst @@ -0,0 +1,44 @@ +************ +Introduction +************ + +.. include:: ../README.rst + :end-before: For more documentation on Pywikibot + :start-line: 41 + +Settings +======== + +It is recommended to create a `user-config.py` file if Pywikibot is used as a +site package. If Pywikibot is used in directory mode e.g. from a repository +the configuration file is mandatory. A minimal sample is shown below. + +.. literalinclude:: ../user-config.py.sample + +This sample is shipped with the repository but is not available with +the site-package. For more settings use +:mod:`generate_user_files<pywikibot.scripts.generate_user_files>` script +or refer :py:mod:`pywikibot.config` module. + +.. seealso:: :manpage:`Installation` Manual + + +Internationalisation (i18n) +=========================== + +Some of the framework input interaction is translated. The user interface +language to be used can be set as follows: + +#. set the `userinterface_lang` in your :ref:`user-config.py<User Interface Settings>` to your preferred language +#. set environment variable `PYWIKIBOT_USERINTERFACE_LANG` to your preferred language +#. default is obtained from `locale.getdefaultlocale` +#. fallback is `'en'` for English if all other options fails + +.. note:: The preferred language code must follow ISO 639. +.. versionadded:: 7.0 + Added to site-package distribution +.. seealso:: + * :manpage:`i18n` Manual + * `MediaWiki Language Codes https://www.mediawiki.org/wiki/Manual:Language#Language_code`_ + * :ref:`User Interface Settings` + * :py:mod:`pywikibot.i18n` diff --git a/docs/library_usage.rst b/docs/library_usage.rst index c75d413..6afb89e 100644 --- a/docs/library_usage.rst +++ b/docs/library_usage.rst @@ -1,8 +1,13 @@ +************************** Using pywikibot as library --------------------------- +**************************

-Pywikibot provides bot classes to develop your own script easily. Here -is a minimal example script which shows their usage: +Pywikibot provides bot classes to develop your own script easily. + +Minimal example +=============== + +Here is a minimal example script which shows their usage:

.. code-block:: python :linenos: @@ -45,7 +50,8 @@

python mybot -site:wikipedia:test -page:Sandbox -text:"A text added to the sandbox"

-**Explanations:** +Explanation +-----------

:1-3: Import necessary framework code parts: :mod:`pywikibot`, :mod:`pywikibot.pagegenerators`, @@ -89,7 +95,7 @@ :29: Create the bot passing keyword only parameters and run it.

Basic script -~~~~~~~~~~~~ +============

:py:obj:`scripts.basic` is a more advanced sample script and shipped with the scripts folder. Here is the content: @@ -98,4 +104,4 @@ :language: python

.. note:: - Please also see the documentation at :manpage:`Create your own script` + Also see the documentation at :manpage:`Create your own script` diff --git a/docs/mwapi.rst b/docs/mwapi.rst index 321aaaa..fed2829 100644 --- a/docs/mwapi.rst +++ b/docs/mwapi.rst @@ -3,9 +3,11 @@

<br />

-MediaWiki API Cross Reference +MediaWiki API cross reference =============================

+See the table below for a cross reference between MediaWiki's API and Pywikibot's API. + .. list-table:: :header-rows: 1 :align: left diff --git a/docs/recipes.rst b/docs/recipes.rst index 5e335ed..c0c3983 100644 --- a/docs/recipes.rst +++ b/docs/recipes.rst @@ -1,24 +1,25 @@ Recipes =======

-Page content ------------- +How to modify a page +--------------------

-**How to modify a page** - >>> page = pywikibot.Page(pywikibot.Site(), 'Sample page') - >>> new_content = page.text.replace('this', 'that') - >>> page.put(new_content, summary='Bot: Test edit') +>>> page = pywikibot.Page(pywikibot.Site(), 'Sample page') +>>> new_content = page.text.replace('this', 'that') +>>> page.put(new_content, summary='Bot: Test edit')

- See :doc:`library_usage` for more advanced samples. +See :doc:`library_usage` for more advanced samples.

-**How to get links from summary section of page** - >>> import pywikibot - >>> from pwikibot import textlib - >>> site = pywikibot.Site('wikipedia:en') # create a Site object - >>> page = pywikibot.Page(site, 'Deep learning') # create a Page object - >>> sect = textlib.extract_sections(page.text, site) # divide content into sections - >>> links = sorted(link['title'] for link in pywikibot.link_regex.finditer(sect.header)) - >>> pages = [pywikibot.Page(site, title) for title in links] +How to get links from summary section of page +---------------------------------------------

- ``links`` is a list containing all link titles in alphabethical order. - ``pages`` is a sorted list containing all ``Page`` objects \ No newline at end of file +>>> import pywikibot +>>> from pwikibot import textlib +>>> site = pywikibot.Site('wikipedia:en') # create a Site object +>>> page = pywikibot.Page(site, 'Deep learning') # create a Page object +>>> sect = textlib.extract_sections(page.text, site) # divide content into sections +>>> links = sorted(link['title'] for link in pywikibot.link_regex.finditer(sect.header)) +>>> pages = [pywikibot.Page(site, title) for title in links] + +* ``links`` is a list containing all link titles in alphabethical order +* ``pages`` is a sorted list containing all ``Page`` objects diff --git a/docs/scripts/index.rst b/docs/scripts/index.rst index 95d07b9..dec2fe6 100644 --- a/docs/scripts/index.rst +++ b/docs/scripts/index.rst @@ -1,5 +1,6 @@ -Scripts Package Description ---------------------------- +=============== +Scripts package +===============

.. automodule:: scripts :no-members: @@ -9,8 +10,8 @@ :start-after: included. :end-before: More precise

-Scripts Descriptions --------------------- +Script descriptions +===================

.. toctree:: :maxdepth: 1 @@ -29,4 +30,4 @@ unsorted maintenance archive - outdated \ No newline at end of file + outdated diff --git a/docs/scripts_ref/index.rst b/docs/scripts_ref/index.rst index 0e32244..04b5165 100644 --- a/docs/scripts_ref/index.rst +++ b/docs/scripts_ref/index.rst @@ -1,9 +1,6 @@ -Scripts Reference +Scripts reference -----------------

-Provided scripts by Pywikibot -============================= - Pywikibot framework has a lot of ready-to-use scripts for several tasks. In general the usage and options of all scripts may be shown by using the command ``<scriptname> -help`` @@ -15,4 +12,7 @@ .. toctree:: :glob:

- * + scripts + scripts.i18n + scripts.maintenance + scripts.userscripts diff --git a/docs/scripts_ref/scripts.i18n.rst b/docs/scripts_ref/scripts.i18n.rst index 20aa2a0..a47108e 100644 --- a/docs/scripts_ref/scripts.i18n.rst +++ b/docs/scripts_ref/scripts.i18n.rst @@ -1,4 +1,4 @@ -scripts.i18n translation files -============================== +Translation files - scripts.i18n +================================

-.. automodule:: scripts.i18n \ No newline at end of file +.. automodule:: scripts.i18n diff --git a/docs/scripts_ref/scripts.maintenance.rst b/docs/scripts_ref/scripts.maintenance.rst index 112695e..4f8df36 100644 --- a/docs/scripts_ref/scripts.maintenance.rst +++ b/docs/scripts_ref/scripts.maintenance.rst @@ -1,34 +1,35 @@ -scripts.maintenance utility scripts -=================================== +Maintenance scripts +=================== + +This page contains documentation of the `scripts.maintenance` module:

.. automodule:: scripts.maintenance

-Maintenance Submodules +Maintenance submodules ----------------------

-scripts.maintenance.cache script +scripts.maintenance.cache --------------------------------

.. automodule:: scripts.maintenance.cache :ignore-module-all:

-scripts.maintenance.colors script +scripts.maintenance.colors ---------------------------------

.. automodule:: scripts.maintenance.colors

-scripts.maintenance.make_i18n_dict script +scripts.maintenance.make_i18n_dict -------------------------------------------

.. automodule:: scripts.maintenance.make_i18n_dict

-scripts.maintenance.sorting_order script +scripts.maintenance.sorting_order -----------------------------------------

.. automodule:: scripts.maintenance.sorting_order

-scripts.maintenance.wikimedia_sites script +scripts.maintenance.wikimedia_sites -------------------------------------------

.. automodule:: scripts.maintenance.wikimedia_sites - diff --git a/docs/scripts_ref/scripts.rst b/docs/scripts_ref/scripts.rst index 2254477..da12f0a 100644 --- a/docs/scripts_ref/scripts.rst +++ b/docs/scripts_ref/scripts.rst @@ -1,17 +1,8 @@ -scripts package +Scripts package ===============

.. automodule:: scripts

-Scripts Subpackages -------------------- - -.. toctree:: - - scripts.i18n - scripts.maintenance - scripts.userscripts - add_text script ----------------

@@ -281,4 +272,11 @@

.. automodule:: scripts.welcome

+Script subpackages +------------------

+For information on contents of subpackages, see + +* :doc:`./scripts.i18n` +* :doc:`./scripts.maintenance` +* :doc:`./scripts.userscripts` diff --git a/docs/scripts_ref/scripts.userscripts.rst b/docs/scripts_ref/scripts.userscripts.rst index 60ac1d4..11f5130 100644 --- a/docs/scripts_ref/scripts.userscripts.rst +++ b/docs/scripts_ref/scripts.userscripts.rst @@ -1,5 +1,4 @@ -scripts.userscripts user scripts -================================ +User scripts - scripts.userscripts +==================================

.. automodule:: scripts.userscripts - diff --git a/docs/tests_ref/index.rst b/docs/tests_ref/index.rst index c1eac79..b0d289c 100644 --- a/docs/tests_ref/index.rst +++ b/docs/tests_ref/index.rst @@ -1,5 +1,5 @@ =============== -Tests Reference +Tests reference ===============

.. include:: ../../tests/README.rst @@ -17,7 +17,7 @@ Library tests *************

- Tests in this project: +Tests in this project:

.. toctree:: :maxdepth: 1 @@ -90,7 +90,7 @@ Script tests ************

- Tests in this project: +Tests in this project:

.. toctree:: :maxdepth: 1 diff --git a/docs/utilities/install.rst b/docs/utilities/install.rst index dbf4134..794c9b9 100644 --- a/docs/utilities/install.rst +++ b/docs/utilities/install.rst @@ -1,4 +1,4 @@ -Installer Scripts +Installer scripts =================

make_dist script diff --git a/docs/utilities/scripts.rst b/docs/utilities/scripts.rst index 9a957e6..5c179f6 100644 --- a/docs/utilities/scripts.rst +++ b/docs/utilities/scripts.rst @@ -1,53 +1,53 @@ -Framework Utility Scripts Description +Framework utility scripts =====================================

.. automodule:: pywikibot.scripts :no-members: :noindex:

-pwb wrapper script description +pwb wrapper script ------------------------------

.. automodule:: pywikibot.scripts.wrapper :no-members: :noindex:

-generate_family_file script description +generate_family_file script -----------------------------------------

.. automodule:: pywikibot.scripts.generate_family_file :no-members: :noindex:

-generate_user_files script description +generate_user_files script ----------------------------------------

.. automodule:: pywikibot.scripts.generate_user_files :no-members: :noindex:

-login script description +login script ------------------------

.. automodule:: pywikibot.scripts.login :no-members: :noindex:

-preload_sites script description +preload_sites script --------------------------------

.. automodule:: pywikibot.scripts.preload_sites :no-members: :noindex:

-shell script description +shell script ------------------------

.. automodule:: pywikibot.scripts.shell :no-members: :noindex:

-version script description +version script --------------------------

.. automodule:: pywikibot.scripts.version diff --git a/docs/utilities/scripts_ref.rst b/docs/utilities/scripts_ref.rst index 7f1346b..1b25394 100644 --- a/docs/utilities/scripts_ref.rst +++ b/docs/utilities/scripts_ref.rst @@ -1,5 +1,5 @@ -Framework Utility Scripts -========================= +Framework utility scripts reference +===================================

.. automodule:: pywikibot.scripts :synopsis: Folder which holds framework scripts diff --git a/scripts/README.rst b/scripts/README.rst index 2d65034..62b413e 100644 --- a/scripts/README.rst +++ b/scripts/README.rst @@ -3,180 +3,185 @@ ===========================================================================

Some example robots are included. ---------------------------------- +=================================

These programs can actually modify the live wiki on the net, and proper wiki-etiquette should be followed before running it on any wiki.

-To get started on proper usage of the bot framework, please refer to: - - `Manual:Pywikibot https://www.mediawiki.org/wiki/Manual:Pywikibot`_ +To get started on proper usage of the bot framework, refer to `Manual:Pywikibot https://www.mediawiki.org/wiki/Manual:Pywikibot`_.

The contents of the package ---------------------------- +===========================

- +----------------------------------------------------------------------------------+ - | Bots and Scripts | - +========================+=========================================================+ - | add_text.py | Adds text at the top or end of pages. | - +------------------------+---------------------------------------------------------+ - | archivebot.py | Archives discussion threads. | - +------------------------+---------------------------------------------------------+ - | basic.py | Is a template from which simple bots can be made. | - +------------------------+---------------------------------------------------------+ - | blockpagechecker.py | Deletes any protection templates that are on pages | - | | which aren't actually protected. | - +------------------------+---------------------------------------------------------+ - | category.py | Add a category link to all pages mentioned on a page, | - | | change or remove category tags. | - +------------------------+---------------------------------------------------------+ - | category_redirect.py | Maintain category redirects and replace links to | - | | redirected categories. | - +------------------------+---------------------------------------------------------+ - | change_pagelang.py | Changes the content language of pages. | - +------------------------+---------------------------------------------------------+ - | checkimages.py | Check recently uploaded files. Checks if a file | - | | description is present and if there are other problems | - | | in the image's description. | - +------------------------+---------------------------------------------------------+ - | claimit.py | Adds claims to Wikidata items based on categories. | - +------------------------+---------------------------------------------------------+ - | clean_sandbox.py | This bot resets a sandbox with predefined text. | - +------------------------+---------------------------------------------------------+ - | commonscat.py | Adds {{commonscat}} to Wikipedia categories (or | - | | articles), if other language Wikipedia already has such | - | | a template. | - +------------------------+---------------------------------------------------------+ - | commons_information.py | Insert a language template into the description field. | - +------------------------+---------------------------------------------------------+ - | coordinate_import.py | Coordinate importing script. | - +------------------------+---------------------------------------------------------+ - | cosmetic_changes.py | Can do slight modifications to a wiki page source code | - | | such that the code looks cleaner. | - +------------------------+---------------------------------------------------------+ - | data_ingestion.py | A generic bot to do batch uploading to Commons. | - +------------------------+---------------------------------------------------------+ - | delete.py | This script can be used to delete pages en masse. | - +------------------------+---------------------------------------------------------+ - | delinker.py | Delink file references of deleted images. | - +------------------------+---------------------------------------------------------+ - | djvutext.py | Extracts OCR text from djvu files and uploads onto | - | | pages in the "Page" namespace on Wikisource. | - +------------------------+---------------------------------------------------------+ - | download_dump.py | Downloads dumps from dumps.wikimedia.org | - +------------------------+---------------------------------------------------------+ - | fixing_redirects.py | Correct all redirect links of processed pages. | - +------------------------+---------------------------------------------------------+ - | harvest_template.py | Template harvesting script. | - +------------------------+---------------------------------------------------------+ - | illustrate_wikidata.py | Bot to add images to Wikidata items. | - +------------------------+---------------------------------------------------------+ - | image.py | Script to replace transclusions of files. | - +------------------------+---------------------------------------------------------+ - | imagetransfer.py | Given a wiki page, check the interwiki links for | - | | images, and let the user choose among them for | - | | images to upload. | - +------------------------+---------------------------------------------------------+ - | interwiki.py | A robot to check interwiki links on all pages (or | - | | a range of pages) of a wiki. | - +------------------------+---------------------------------------------------------+ - | interwikidata.py | Script to handle interwiki links based on Wikibase. | - +------------------------+---------------------------------------------------------+ - | listpages.py | Print a list of pages, defined by a page generator. | - +------------------------+---------------------------------------------------------+ - | misspelling.py | Similar to solve_disambiguation.py. It is supposed to | - | | fix links that contain common spelling mistakes. | - +------------------------+---------------------------------------------------------+ - | movepages.py | Bot that can move pages to another title. | - +------------------------+---------------------------------------------------------+ - | newitem.py | Script creates new items on Wikidata based on criteria. | - +------------------------+---------------------------------------------------------+ - | noreferences.py | Searches for pages where <references /> is missing | - | | although a <ref> tag is present, and in that case adds | - | | a new references section. | - +------------------------+---------------------------------------------------------+ - | nowcommons.py | This bot can delete images with NowCommons template. | - +------------------------+---------------------------------------------------------+ - | pagefromfile.py | This bot takes its input from a file that contains a | - | | number of pages to be put on the wiki. | - +------------------------+--+------------------------------------------------------+ - | parser_function_count.py | Find expensive templates that are subject to be | - | | converted to Lua. | - +------------------------+--+------------------------------------------------------+ - | patrol.py | Obtains a list pages and marks the edits as patrolled | - | | based on a whitelist. | - +------------------------+---------------------------------------------------------+ - | protect.py | Protect and unprotect pages en masse. | - +------------------------+---------------------------------------------------------+ - | redirect.py | Fix double redirects and broken redirects. Note: | - | | solve_disambiguation also has functions which treat | - | | redirects. | - +------------------------+---------------------------------------------------------+ - | reflinks.py | Search for references which are only made of a link | - | | without title and fetch the html title from the link to | - | | use it as the title of the wiki link in the reference. | - +------------------------+---------------------------------------------------------+ - | replace.py | Search articles for a text and replace it by another | - | | text. Both text are set in two configurable | - | | text files. The bot can either work on a set of given | - | | pages or crawl an SQL dump. | - +------------------------+---------------------------------------------------------+ - | replicate_wiki.py | Replicates pages in wiki to a second wiki within family | - +------------------------+---------------------------------------------------------+ - | revertbot.py | Script that can be used for reverting certain edits. | - +------------------------+---------------------------------------------------------+ - | solve_disambiguation.py| Interactive robot doing disambiguation. | - +------------------------+---------------------------------------------------------+ - | speedy_delete.py | Help sysops to quickly check and/or delete pages listed | - | | for speedy deletion. | - +------------------------+---------------------------------------------------------+ - | template.py | Change one template (that is {{...}}) into another. | - +------------------------+---------------------------------------------------------+ - | templatecount.py | Display the list of pages transcluding a given list | - | | of templates. | - +------------------------+---------------------------------------------------------+ - | touch.py | Bot goes over all pages of the home wiki, and edits | - | | them without changes. | - +------------------------+---------------------------------------------------------+ - | transferbot.py | Transfers pages from a source wiki to a target wiki. | - +------------------------+---------------------------------------------------------+ - | unusedfiles.py | Bot appends some text to all unused images and other | - | | text to the respective uploaders. | - +------------------------+---------------------------------------------------------+ - | upload.py | Upload an image to a wiki. | - +------------------------+---------------------------------------------------------+ - | watchlists.py | Allows access to the account's watchlist. | - +------------------------+---------------------------------------------------------+ - | weblinkchecker.py | Check if external links are still working. | - +------------------------+---------------------------------------------------------+ - | welcome.py | Script to welcome new users. | - +------------------------+---------------------------------------------------------+ +Bots and scripts +----------------

++------------------------------------------------------------------------------------+ +| Bots and Scripts | ++==========================+=========================================================+ +| add_text.py | Adds text at the top or end of pages. | ++--------------------------+---------------------------------------------------------+ +| archivebot.py | Archives discussion threads. | ++--------------------------+---------------------------------------------------------+ +| basic.py | Is a template from which simple bots can be made. | ++--------------------------+---------------------------------------------------------+ +| blockpagechecker.py | Deletes any protection templates that are on pages | +| | which aren't actually protected. | ++--------------------------+---------------------------------------------------------+ +| category.py | Add a category link to all pages mentioned on a page, | +| | change or remove category tags. | ++--------------------------+---------------------------------------------------------+ +| category_redirect.py | Maintain category redirects and replace links to | +| | redirected categories. | ++--------------------------+---------------------------------------------------------+ +| change_pagelang.py | Changes the content language of pages. | ++--------------------------+---------------------------------------------------------+ +| checkimages.py | Check recently uploaded files. Checks if a file | +| | description is present and if there are other problems | +| | in the image's description. | ++--------------------------+---------------------------------------------------------+ +| claimit.py | Adds claims to Wikidata items based on categories. | ++--------------------------+---------------------------------------------------------+ +| clean_sandbox.py | This bot resets a sandbox with predefined text. | ++--------------------------+---------------------------------------------------------+ +| commonscat.py | Adds {{commonscat}} to Wikipedia categories (or | +| | articles), if other language Wikipedia already has such | +| | a template. | ++--------------------------+---------------------------------------------------------+ +| commons_information.py | Insert a language template into the description field. | ++--------------------------+---------------------------------------------------------+ +| coordinate_import.py | Coordinate importing script. | ++--------------------------+---------------------------------------------------------+ +| cosmetic_changes.py | Can do slight modifications to a wiki page source code | +| | such that the code looks cleaner. | ++--------------------------+---------------------------------------------------------+ +| data_ingestion.py | A generic bot to do batch uploading to Commons. | ++--------------------------+---------------------------------------------------------+ +| delete.py | This script can be used to delete pages en masse. | ++--------------------------+---------------------------------------------------------+ +| delinker.py | Delink file references of deleted images. | ++--------------------------+---------------------------------------------------------+ +| djvutext.py | Extracts OCR text from djvu files and uploads onto | +| | pages in the "Page" namespace on Wikisource. | ++--------------------------+---------------------------------------------------------+ +| download_dump.py | Downloads dumps from dumps.wikimedia.org | ++--------------------------+---------------------------------------------------------+ +| fixing_redirects.py | Correct all redirect links of processed pages. | ++--------------------------+---------------------------------------------------------+ +| harvest_template.py | Template harvesting script. | ++--------------------------+---------------------------------------------------------+ +| illustrate_wikidata.py | Bot to add images to Wikidata items. | ++--------------------------+---------------------------------------------------------+ +| image.py | Script to replace transclusions of files. | ++--------------------------+---------------------------------------------------------+ +| imagetransfer.py | Given a wiki page, check the interwiki links for | +| | images, and let the user choose among them for | +| | images to upload. | ++--------------------------+---------------------------------------------------------+ +| interwiki.py | A robot to check interwiki links on all pages (or | +| | a range of pages) of a wiki. | ++--------------------------+---------------------------------------------------------+ +| interwikidata.py | Script to handle interwiki links based on Wikibase. | ++--------------------------+---------------------------------------------------------+ +| listpages.py | Print a list of pages, defined by a page generator. | ++--------------------------+---------------------------------------------------------+ +| misspelling.py | Similar to solve_disambiguation.py. It is supposed to | +| | fix links that contain common spelling mistakes. | ++--------------------------+---------------------------------------------------------+ +| movepages.py | Bot that can move pages to another title. | ++--------------------------+---------------------------------------------------------+ +| newitem.py | Script creates new items on Wikidata based on criteria. | ++--------------------------+---------------------------------------------------------+ +| noreferences.py | Searches for pages where <references /> is missing | +| | although a <ref> tag is present, and in that case adds | +| | a new references section. | ++--------------------------+---------------------------------------------------------+ +| nowcommons.py | This bot can delete images with NowCommons template. | ++--------------------------+---------------------------------------------------------+ +| pagefromfile.py | This bot takes its input from a file that contains a | +| | number of pages to be put on the wiki. | ++--------------------------+---------------------------------------------------------+ +| parser_function_count.py | Find expensive templates that are subject to be | +| | converted to Lua. | ++--------------------------+---------------------------------------------------------+ +| patrol.py | Obtains a list pages and marks the edits as patrolled | +| | based on a whitelist. | ++--------------------------+---------------------------------------------------------+ +| protect.py | Protect and unprotect pages en masse. | ++--------------------------+---------------------------------------------------------+ +| redirect.py | Fix double redirects and broken redirects. Note: | +| | solve_disambiguation also has functions which treat | +| | redirects. | ++--------------------------+---------------------------------------------------------+ +| reflinks.py | Search for references which are only made of a link | +| | without title and fetch the html title from the link to | +| | use it as the title of the wiki link in the reference. | ++--------------------------+---------------------------------------------------------+ +| replace.py | Search articles for a text and replace it by another | +| | text. Both text are set in two configurable | +| | text files. The bot can either work on a set of given | +| | pages or crawl an SQL dump. | ++--------------------------+---------------------------------------------------------+ +| replicate_wiki.py | Replicates pages in wiki to a second wiki within family | ++--------------------------+---------------------------------------------------------+ +| revertbot.py | Script that can be used for reverting certain edits. | ++--------------------------+---------------------------------------------------------+ +| solve_disambiguation.py | Interactive robot doing disambiguation. | ++--------------------------+---------------------------------------------------------+ +| speedy_delete.py | Help sysops to quickly check and/or delete pages listed | +| | for speedy deletion. | ++--------------------------+---------------------------------------------------------+ +| template.py | Change one template (that is {{...}}) into another. | ++--------------------------+---------------------------------------------------------+ +| templatecount.py | Display the list of pages transcluding a given list | +| | of templates. | ++--------------------------+---------------------------------------------------------+ +| touch.py | Bot goes over all pages of the home wiki, and edits | +| | them without changes. | ++--------------------------+---------------------------------------------------------+ +| transferbot.py | Transfers pages from a source wiki to a target wiki. | ++--------------------------+---------------------------------------------------------+ +| unusedfiles.py | Bot appends some text to all unused images and other | +| | text to the respective uploaders. | ++--------------------------+---------------------------------------------------------+ +| upload.py | Upload an image to a wiki. | ++--------------------------+---------------------------------------------------------+ +| watchlists.py | Allows access to the account's watchlist. | ++--------------------------+---------------------------------------------------------+ +| weblinkchecker.py | Check if external links are still working. | ++--------------------------+---------------------------------------------------------+ +| welcome.py | Script to welcome new users. | ++--------------------------+---------------------------------------------------------+

- +------------------------+---------------------------------------------------------+ - | maintenance | Framework helper scripts. . | - +========================+=========================================================+ - | cache.py | Script for showing and deleting API cache. | - +------------------------+---------------------------------------------------------+ - | colors.py | Utility to show pywikibot colors. | - +------------------------+---------------------------------------------------------+ - | make_i18n_dict.py | Generate an i18n file from a given script. | - +------------------------+---------------------------------------------------------+ - | sorting_order.py | Updates interwiki sorting order in family.py file. | - +------------------------+---------------------------------------------------------+ - | wikimedia_sites.py | Updates the language lists in Wikimedia family files. | - +------------------------+---------------------------------------------------------+ +Maintenance +-----------

++------------------------+---------------------------------------------------------+ +| maintenance | Framework helper scripts | ++========================+=========================================================+ +| cache.py | Script for showing and deleting API cache. | ++------------------------+---------------------------------------------------------+ +| colors.py | Utility to show pywikibot colors. | ++------------------------+---------------------------------------------------------+ +| make_i18n_dict.py | Generate an i18n file from a given script. | ++------------------------+---------------------------------------------------------+ +| sorting_order.py | Updates interwiki sorting order in family.py file. | ++------------------------+---------------------------------------------------------+ +| wikimedia_sites.py | Updates the language lists in Wikimedia family files. | ++------------------------+---------------------------------------------------------+

- +------------------------+---------------------------------------------------------+ - | Others | | - +========================+=========================================================+ - | i18n (folder) | Contains i18n translations for bot edit summaries. | - +------------------------+---------------------------------------------------------+ - | userscripts (folder) | Empty folder for user scripts. | - +------------------------+---------------------------------------------------------+ - | README.rst | This file (Short info of all scripts). | - +------------------------+---------------------------------------------------------+ +Others +------ + ++------------------------+---------------------------------------------------------+ +| Others | | ++========================+=========================================================+ +| i18n (folder) | Contains i18n translations for bot edit summaries. | ++------------------------+---------------------------------------------------------+ +| userscripts (folder) | Empty folder for user scripts. | ++------------------------+---------------------------------------------------------+ +| README.rst | This file (Short info of all scripts). | ++------------------------+---------------------------------------------------------+

**External packages could be required with Pywikibot:**