Pywikibot-commits

pywikibot-commits@lists.wikimedia.org

1 participants
13500 discussions

[Gerrit] ...core[tests]: [docs] Improve basic content structure and navigation
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860879 ) Change subject: [docs] Improve basic content structure and navigation ...................................................................... [docs] Improve basic content structure and navigation * add code block language indicators * unify section and page naming, add sections where appropriate, align heading syntax with Python style guide * remove horizontal line on framework.png * improve style in some sections * rename installation to introduction (keep old page as redirect) * remove unnecessary indentation * remove duplicated navigation entries * fix README tables Bug: T323812 Change-Id: I810d27449a43aba1e361bab64aab3a42618efe30 --- M docs/getting_help.rst M ROADMAP.rst M README.rst M docs/recipes.rst M scripts/README.rst M docs/index.rst M docs/scripts_ref/scripts.maintenance.rst M docs/api_ref/index.rst M docs/global_options.rst M docs/library_usage.rst M docs/utilities/scripts_ref.rst M HISTORY.rst M docs/scripts_ref/index.rst M docs/scripts_ref/scripts.i18n.rst M docs/scripts_ref/scripts.rst M CODE_OF_CONDUCT.rst M docs/mwapi.rst M docs/utilities/scripts.rst M docs/tests_ref/index.rst M docs/changelog.rst M docs/installation.rst M docs/scripts/index.rst M docs/framework.png A docs/introduction.rst M docs/utilities/install.rst M docs/faq.rst M docs/scripts_ref/scripts.userscripts.rst 27 files changed, 388 insertions(+), 351 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/CODE_OF_CONDUCT.rst b/CODE_OF_CONDUCT.rst index cb0641f..71156e2 100644 --- a/CODE_OF_CONDUCT.rst +++ b/CODE_OF_CONDUCT.rst @@ -1,5 +1,5 @@ Code of Conduct ---------------- +=============== The development of this software is covered by a `Code of Conduct <https://www.mediawiki.org/wiki/Special:MyLanguage/Code_of_Conduct>`_. diff --git a/HISTORY.rst b/HISTORY.rst index de1ace6..f1eeda2 100644 --- a/HISTORY.rst +++ b/HISTORY.rst @@ -1,5 +1,5 @@ Release history -^^^^^^^^^^^^^^^ +=============== 7.7.1 ----- @@ -197,7 +197,7 @@ *26 February 2022* Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * i18n updates for date.py * Add number transliteration of 'lo', 'ml', 'pa', 'te' to NON_LATIN_DIGITS @@ -240,7 +240,7 @@ * New "maxlimit" property was added to APISite (:phab:`T291770`) Bugfixes -~~~~~~~~ +^^^^^^^^ * Don't raise an exception if BlockEntry initializer found a hidden title (:phab:`T78152`) * Fix KeyError in create_warnings_list (:phab:`T301610`) @@ -261,14 +261,14 @@ * Raise a generic ServerError if the http status code is unofficial (:phab:`T293208`) Breaking changes -~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^ * Support of Python 3.5.0 - 3.5.2 has been dropped (:phab:`T286867`) * generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be used with pwb wrapper script * *See also Code cleanups below* Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^ * Deprecated http.get_fake_user_agent() function was removed * FilePage.fileIsShared() was removed in favour of FilePage.file_is_shared() @@ -436,7 +436,7 @@ *28 May 2021* Improvements and Bugfixes -~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^ * Use different logfiles for multiple processes of the same script (:phab:`T56685`) * throttle.pip will be reused as soon as possbile @@ -452,7 +452,7 @@ * Handle abusefilter-{disallow,warning} codes (:phab:`T85656`) Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^ * MultipleSitesBot.site attribute was removed (:phab:`T283209`) * Deprecated BaseSite.category_namespaces() method was removed @@ -481,7 +481,7 @@ *17 April 2021* Improvements and Bugfixes -~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^ * interwiki_graph module was restored (:phab:`T223826`) * proofreadpage: search for "new" class after purge (:phab:`T280357`) @@ -496,7 +496,7 @@ * Family files can be collected from a zip folder (:phab:`T278076`) Dependencies -~~~~~~~~~~~~ +^^^^^^^^^^^^ * **mwparserfromhell** or **wikitextparser** are strictly recommended (:phab:`T106763`) * Require **Pillow**>=8.1.1 due to vulnerability found (:phab:`T278743`) @@ -505,7 +505,7 @@ * Add support for **PyMySQL** 1.0.0+ Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^ * APISite.resolvemagicwords(), BaseSite.ns_index() and remove BaseSite.getNamespaceIndex() were removed * Deprecated MoveEntry.new_ns() and new_title() methods were removed @@ -528,7 +528,7 @@ *16 March 2021* Breaking changes -~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^ * interwiki_graph module was removed (:phab:`T223826`) * Require setuptools >= 20.2 due to :pep:`440` @@ -538,7 +538,7 @@ * config.db_hostname has been renamed to db_hostname_format Code cleanups -~~~~~~~~~~~~~ +^^^^^^^^^^^^^ * tools.PY2 was removed (:phab:`T213287`) * Site.language() method was removed in favour of Site.lang property @@ -555,7 +555,7 @@ * Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response (:phab:`T265206`) Other changes -~~~~~~~~~~~~~ +^^^^^^^^^^^^^ * Raise a SiteDefinitionError if api request response is Non-JSON and site is AutoFamily (:phab:`T272911`) * Support deleting and undeleting specific file versions (:phab:`T276725`) @@ -1331,7 +1331,7 @@ * CODE_OF_CONDUCT included Bugfixes -~~~~~~~~ +^^^^^^^^ * Manage temporary readonly error (:phab:`T154011`) * Unbreak wbGeoShape and WbTabularData (:phab:`T166362`) * Clean up issue with _WbDataPage (:phab:`T166362`) @@ -1343,7 +1343,7 @@ * Fix TestLazyLoginNotExistUsername test for Stewardwiki (:phab:`T169458`) Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Introduce the new WbUnknown data type for Wikibase (:phab:`T165961`) * djvu.py: add replace_page() and delete_page() * Build GeoShape and TabularData from shared base class @@ -1367,7 +1367,7 @@ * Make WbRepresentations hashable (:phab:`T167827`) Updates -~~~~~~~ +^^^^^^^ * Update linktails * Update languages_by_size * Update cross_allowed (global bot wikis group) @@ -1389,7 +1389,7 @@ * Localisation updates Bugfixes -~~~~~~~~ +^^^^^^^^ * Increase the default socket_timeout to 75 seconds (:phab:`T163635`) * use repr() of exceptions to prevent UnicodeDecodeErrors (:phab:`T120222`) * Handle offset mismatches during chunked upload (:phab:`T156402`) @@ -1402,7 +1402,7 @@ * Remove wikidataquery from pywikibot (:phab:`T162585`) Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Introduce user_add_claim and allow asynchronous ItemPage.addClaim (:phab:`T87493`) * Enable private edit summary in specialbots (:phab:`T162527`) * Make a decorator for asynchronous methods @@ -1429,7 +1429,7 @@ * Get thumburl information in FilePage() (:phab:`T137011`) Updates -~~~~~~~ +^^^^^^^ * Update languages_by_size in family files * wikisource_family.py: Add "pa" to languages_by_size * Config2: limit the number of retries to 15 (:phab:`T165898`) @@ -1444,7 +1444,7 @@ * Deprecate previous 2.0 branches and tags Bugfixes -~~~~~~~~ +^^^^^^^^ * Use default summary when summary value does not contain a string (:phab:`T160823`) * Enable specialbots.py for PY3 (:phab:`T161457`) * Change tw(n)translate from Site.code to Site.lang dependency (:phab:`T140624`) @@ -1466,7 +1466,7 @@ * Fix broken LDAP based login (:phab:`T90149`) Improvements -~~~~~~~~~~~~ +^^^^^^^^^^^^ * Simplify User class * Renamed isImage and isCategory * Add -property option to pagegenerators.py @@ -1485,7 +1485,7 @@ * Fix and improve default regexes Updates -~~~~~~~ +^^^^^^^ * Update linktrails * Update languages_by_size * Updating global bot wikis, closed wikis and deleted wikis @@ -1507,7 +1507,7 @@ * Last stable 2.0 branch Bugfixes -~~~~~~~~ +^^^^^^^^ * Establish the project's name, once and for all * setup.py: Add Python 3.4 and 3.5 to pypi classifiers * Remove item count output in page generators @@ -1543,7 +1543,7 @@ *15 December 2015* Bugfixes -~~~~~~~~ +^^^^^^^^ * Remove dependency on pYsearch * Require google>=0.7 * Desupport Python 2.6 for Pywikibot 2.0 release branch @@ -1577,7 +1577,7 @@ *30 September 2015* Bugfixes -~~~~~~~~ +^^^^^^^^ * New Wikipedia site: azb * Indexes in str.format * MediaWikiVersion: Accept new wmf style @@ -1593,18 +1593,18 @@ *9 July 2015* Configuration updates -~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^ * Changing the sandbox content template on Fa WP Family file updates -~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^ * Remove broken wikis from battlestarwiki family * Adding euskara and sicilianu languages to Vikidia family * WOW Wiki subdomains hr, ro & sr deleted * Add new Wikipedia languages gom and lrc Bugfixes -~~~~~~~~ +^^^^^^^^ * fix UnicodeDecodeError on api error * pwb.py now correctly passes arguments to generate_family_file * Fix Win32 config.editor detection diff --git a/README.rst b/README.rst index e0d9850..3508a9c 100644 --- a/README.rst +++ b/README.rst @@ -35,9 +35,9 @@ :alt: Last commit :target: https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/ - +********* Pywikibot -========= +********* The Pywikibot framework is a Python library that interfaces with the `MediaWiki API <https://www.mediawiki.org/wiki/API:Main_page>`_ @@ -50,9 +50,9 @@ the full `code documentation <https://doc.wikimedia.org/pywikibot/>`_. Quick start ------------ +=========== -:: +.. code:: text pip install requests git clone https://gerrit.wikimedia.org/r/pywikibot/core.git @@ -62,7 +62,7 @@ Or to install using PyPI (excluding scripts) -:: +.. code:: text pip install -U setuptools pip install pywikibot @@ -70,13 +70,13 @@ In addition a MediaWiki markup parser is required. Please install one of them: -:: +.. code:: text pip install mwparserfromhell or -:: +.. code:: text pip install wikitextparser @@ -85,11 +85,11 @@ has more details for advanced usage. Basic Usage ------------ +=========== If you wish to write your own script it's very easy to get started: -:: +.. code:: python import pywikibot site = pywikibot.Site('en', 'wikipedia') # The site we want to run our bot on @@ -98,12 +98,12 @@ page.save('Replacing "foo" with "bar"') # Saves the page Wikibase Usage --------------- +============== Wikibase is a flexible knowledge base software that drives Wikidata. A sample pywikibot script for getting data from Wikibase: -:: +.. code:: python import pywikibot site = pywikibot.Site('wikipedia:en') @@ -113,11 +113,11 @@ data = item.get() # get all item data from repository for this item Script example --------------- +============== Pywikibot provides bot classes to develop your own script easily: -:: +.. code:: python import pywikibot from pywikibot import pagegenerators @@ -157,17 +157,17 @@ Roadmap -------- +======= .. include:: ROADMAP.rst Release history ---------------- +=============== See https://github.com/wikimedia/pywikibot/blob/stable/HISTORY.rst Contributing ------------- +============ Our code is maintained on Wikimedia's `Gerrit installation <https://gerrit.wikimedia.org/>`_, `learn <https://www.mediawiki.org/wiki/Developer_account>`_ how to get diff --git a/ROADMAP.rst b/ROADMAP.rst index 41032d0..d478166 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -1,5 +1,5 @@ Current release 8.0.0 -^^^^^^^^^^^^^^^^^^^^^ +--------------------- * All parameters of :meth:`Category.members()<page.Category.members>`, :meth:`Category.subcategories()<page.Category.subcategories>` and @@ -29,7 +29,7 @@ * MediaWiki API cross reference was added to the documentation Deprecations -^^^^^^^^^^^^ +------------ * 8.0.0: :meth:`APISite.messages()<pywikibot.site._apisite.APISite.messages>` method is deprecated in favour of :attr:`userinfo['messages']<pywikibot.site._apisite.APISite.userinfo>` @@ -67,6 +67,6 @@ Will be removed in Pywikibot 8 ------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * 5.5.0: APISite.redirectRegex() will be removed in favour of APISite.redirect_regex() diff --git a/docs/api_ref/index.rst b/docs/api_ref/index.rst index c6bb0df..bb402e6 100644 --- a/docs/api_ref/index.rst +++ b/docs/api_ref/index.rst @@ -13,7 +13,7 @@ functions provided by :mod:`data.api`. This layer then uses :func:`comms.http.request` to do the actual HTTP request. -Table of Contents +Table of contents ================= .. toctree:: diff --git a/docs/changelog.rst b/docs/changelog.rst index 8e6a551..b12b4ac 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -1,10 +1,12 @@ -:tocdepth: 3 - +********** Change log -========== +********** What is new with Pywikibot |release|? What are the main changes of older version? +Roadmap +======= + .. include:: ../ROADMAP.rst .. include:: ../HISTORY.rst diff --git a/docs/faq.rst b/docs/faq.rst index f0530b7..f707136 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -1,5 +1,6 @@ -Frequent Asked Questions +Frequent asked questions ======================== + **How to speed up Pywikibot?** 1. The first time you are using Pywikibot for multiple Wikimedia sites you can run :py:mod:`preload_sites <pywikibot.scripts.preload_sites>` script @@ -35,4 +36,3 @@ **pywikibot.i18n.TranslationError: No English translation has been defined** It can happen due to lack of i18n submodule or files. Update i18n submodule or download these files first. See also: :manpage:`i18n` manual. - diff --git a/docs/framework.png b/docs/framework.png index ec4cbc2..1956ec8 100644 --- a/docs/framework.png +++ b/docs/framework.png Binary files differ diff --git a/docs/getting_help.rst b/docs/getting_help.rst index 208af04..52c3f16 100644 --- a/docs/getting_help.rst +++ b/docs/getting_help.rst @@ -2,8 +2,8 @@ ------------ .. note:: - Please see :manpage:`Communication`. + For information on how to connect with Pywikibot developers and users, see :manpage:`Communication`. .. tip:: - Please report bugs at `Phabricator <https://phabricator.wikimedia.org/>`_. - You may use `this report form <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=pywikibo…>`_. + Please report bugs in `Phabricator <https://phabricator.wikimedia.org/>`_. + You can use `this report form <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=pywikibo…>`_. diff --git a/docs/global_options.rst b/docs/global_options.rst index 57af62a..59f082d 100644 --- a/docs/global_options.rst +++ b/docs/global_options.rst @@ -1,4 +1,4 @@ -Global Options +Global options -------------- Global arguments available for all bots: @@ -6,4 +6,4 @@ .. literalinclude:: ../pywikibot/bot.py :language: console :start-after: (Global arguments available for all bots) - :end-before: """ \ No newline at end of file + :end-before: """ diff --git a/docs/index.rst b/docs/index.rst index 4384161..4dd75b9 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -36,13 +36,14 @@ :alt: Framework modules dependency network -For bot users: --------------- +Contents +-------- .. toctree:: :maxdepth: 1 + :caption: User guide - installation + introduction utilities/scripts scripts/index global_options @@ -50,34 +51,27 @@ getting_help -For bot developers: -------------------- - .. toctree:: :maxdepth: 1 + :caption: Bot developer guide library_usage recipes api_ref/index mwapi -For framework developers: -------------------------- - .. toctree:: :maxdepth: 1 + :caption: Pywikibot contributor guide - Utility Scripts Reference<utilities/scripts_ref> + utilities/scripts_ref scripts_ref/index - tests_ref/index - Maintenance Scripts<scripts_ref/scripts.maintenance> utilities/install + tests_ref/index - -Miscellaneous -------------- .. toctree:: :maxdepth: 1 + :caption: About glossary changelog @@ -86,4 +80,4 @@ .. _CC-BY-SA 3.0: https://creativecommons.org/licenses/by-sa/3.0/ -.. _Pywikibot Manual: https://www.mediawiki.org/wiki/Manual:Pywikibot \ No newline at end of file +.. _Pywikibot Manual: https://www.mediawiki.org/wiki/Manual:Pywikibot diff --git a/docs/installation.rst b/docs/installation.rst index cc555a4..b39bd99 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -1,43 +1,6 @@ +:orphan: + Installation ============ -.. include:: ../README.rst - :end-before: For more documentation on Pywikibot - -Settings --------- - -It is recommended to create a `user-config.py` file if Pywikibot is used as a -site package. If Pywikibot is used in directory mode e.g. from a repository -the configuration file is mandatory. A minimal sample is shown below. - -.. literalinclude:: ../user-config.py.sample - -This sample is shipped with the repository but is not available with -the site-package. For more settings use -:mod:`generate_user_files<pywikibot.scripts.generate_user_files>` script -or refer :py:mod:`pywikibot.config` module. - -.. seealso:: :manpage:`Installation` Manual - - -Internationalisation (i18n) ---------------------------- - -Some of the framework input interaction is translated. The user interface -language to be used can be set as follows: - -#. set the `userinterface_lang` in your :ref:`user-config.py<User Interface Settings>` to your preferred language -#. set environment variable `PYWIKIBOT_USERINTERFACE_LANG` to your preferred language -#. default is obtained from `locale.getdefaultlocale` -#. fallback is `'en'` for english if all other options fails - -.. note:: The preferred language code must follow ISO 639. -.. versionadded:: 7.0 - Added to site-package distribution -.. seealso:: - * :manpage:`i18n` Manual - * `MediaWiki Language Codes <https://www.mediawiki.org/wiki/Manual:Language#Language_code>`_ - * :ref:`User Interface Settings` - * :py:mod:`pywikibot.i18n` - +Moved to :doc:`introduction`. diff --git a/docs/introduction.rst b/docs/introduction.rst new file mode 100644 index 0000000..922fd22 --- /dev/null +++ b/docs/introduction.rst @@ -0,0 +1,44 @@ +************ +Introduction +************ + +.. include:: ../README.rst + :end-before: For more documentation on Pywikibot + :start-line: 41 + +Settings +======== + +It is recommended to create a `user-config.py` file if Pywikibot is used as a +site package. If Pywikibot is used in directory mode e.g. from a repository +the configuration file is mandatory. A minimal sample is shown below. + +.. literalinclude:: ../user-config.py.sample + +This sample is shipped with the repository but is not available with +the site-package. For more settings use +:mod:`generate_user_files<pywikibot.scripts.generate_user_files>` script +or refer :py:mod:`pywikibot.config` module. + +.. seealso:: :manpage:`Installation` Manual + + +Internationalisation (i18n) +=========================== + +Some of the framework input interaction is translated. The user interface +language to be used can be set as follows: + +#. set the `userinterface_lang` in your :ref:`user-config.py<User Interface Settings>` to your preferred language +#. set environment variable `PYWIKIBOT_USERINTERFACE_LANG` to your preferred language +#. default is obtained from `locale.getdefaultlocale` +#. fallback is `'en'` for English if all other options fails + +.. note:: The preferred language code must follow ISO 639. +.. versionadded:: 7.0 + Added to site-package distribution +.. seealso:: + * :manpage:`i18n` Manual + * `MediaWiki Language Codes <https://www.mediawiki.org/wiki/Manual:Language#Language_code>`_ + * :ref:`User Interface Settings` + * :py:mod:`pywikibot.i18n` diff --git a/docs/library_usage.rst b/docs/library_usage.rst index c75d413..6afb89e 100644 --- a/docs/library_usage.rst +++ b/docs/library_usage.rst @@ -1,8 +1,13 @@ +************************** Using pywikibot as library --------------------------- +************************** -Pywikibot provides bot classes to develop your own script easily. Here -is a minimal example script which shows their usage: +Pywikibot provides bot classes to develop your own script easily. + +Minimal example +=============== + +Here is a minimal example script which shows their usage: .. code-block:: python :linenos: @@ -45,7 +50,8 @@ python mybot -site:wikipedia:test -page:Sandbox -text:"A text added to the sandbox" -**Explanations:** +Explanation +----------- :1-3: Import necessary framework code parts: :mod:`pywikibot`, :mod:`pywikibot.pagegenerators`, @@ -89,7 +95,7 @@ :29: Create the bot passing keyword only parameters and run it. Basic script -~~~~~~~~~~~~ +============ :py:obj:`scripts.basic` is a more advanced sample script and shipped with the scripts folder. Here is the content: @@ -98,4 +104,4 @@ :language: python .. note:: - Please also see the documentation at :manpage:`Create your own script` + Also see the documentation at :manpage:`Create your own script` diff --git a/docs/mwapi.rst b/docs/mwapi.rst index 321aaaa..fed2829 100644 --- a/docs/mwapi.rst +++ b/docs/mwapi.rst @@ -3,9 +3,11 @@ <br /> -MediaWiki API Cross Reference +MediaWiki API cross reference ============================= +See the table below for a cross reference between MediaWiki's API and Pywikibot's API. + .. list-table:: :header-rows: 1 :align: left diff --git a/docs/recipes.rst b/docs/recipes.rst index 5e335ed..c0c3983 100644 --- a/docs/recipes.rst +++ b/docs/recipes.rst @@ -1,24 +1,25 @@ Recipes ======= -Page content ------------- +How to modify a page +-------------------- -**How to modify a page** - >>> page = pywikibot.Page(pywikibot.Site(), 'Sample page') - >>> new_content = page.text.replace('this', 'that') - >>> page.put(new_content, summary='Bot: Test edit') +>>> page = pywikibot.Page(pywikibot.Site(), 'Sample page') +>>> new_content = page.text.replace('this', 'that') +>>> page.put(new_content, summary='Bot: Test edit') - See :doc:`library_usage` for more advanced samples. +See :doc:`library_usage` for more advanced samples. -**How to get links from summary section of page** - >>> import pywikibot - >>> from pwikibot import textlib - >>> site = pywikibot.Site('wikipedia:en') # create a Site object - >>> page = pywikibot.Page(site, 'Deep learning') # create a Page object - >>> sect = textlib.extract_sections(page.text, site) # divide content into sections - >>> links = sorted(link['title'] for link in pywikibot.link_regex.finditer(sect.header)) - >>> pages = [pywikibot.Page(site, title) for title in links] +How to get links from summary section of page +--------------------------------------------- - ``links`` is a list containing all link titles in alphabethical order. - ``pages`` is a sorted list containing all ``Page`` objects \ No newline at end of file +>>> import pywikibot +>>> from pwikibot import textlib +>>> site = pywikibot.Site('wikipedia:en') # create a Site object +>>> page = pywikibot.Page(site, 'Deep learning') # create a Page object +>>> sect = textlib.extract_sections(page.text, site) # divide content into sections +>>> links = sorted(link['title'] for link in pywikibot.link_regex.finditer(sect.header)) +>>> pages = [pywikibot.Page(site, title) for title in links] + +* ``links`` is a list containing all link titles in alphabethical order +* ``pages`` is a sorted list containing all ``Page`` objects diff --git a/docs/scripts/index.rst b/docs/scripts/index.rst index 95d07b9..dec2fe6 100644 --- a/docs/scripts/index.rst +++ b/docs/scripts/index.rst @@ -1,5 +1,6 @@ -Scripts Package Description ---------------------------- +=============== +Scripts package +=============== .. automodule:: scripts :no-members: @@ -9,8 +10,8 @@ :start-after: included. :end-before: More precise -Scripts Descriptions --------------------- +Script descriptions +=================== .. toctree:: :maxdepth: 1 @@ -29,4 +30,4 @@ unsorted maintenance archive - outdated \ No newline at end of file + outdated diff --git a/docs/scripts_ref/index.rst b/docs/scripts_ref/index.rst index 0e32244..04b5165 100644 --- a/docs/scripts_ref/index.rst +++ b/docs/scripts_ref/index.rst @@ -1,9 +1,6 @@ -Scripts Reference +Scripts reference ----------------- -Provided scripts by Pywikibot -============================= - Pywikibot framework has a lot of ready-to-use scripts for several tasks. In general the usage and options of all scripts may be shown by using the command ``<scriptname> -help`` @@ -15,4 +12,7 @@ .. toctree:: :glob: - * + scripts + scripts.i18n + scripts.maintenance + scripts.userscripts diff --git a/docs/scripts_ref/scripts.i18n.rst b/docs/scripts_ref/scripts.i18n.rst index 20aa2a0..a47108e 100644 --- a/docs/scripts_ref/scripts.i18n.rst +++ b/docs/scripts_ref/scripts.i18n.rst @@ -1,4 +1,4 @@ -scripts.i18n translation files -============================== +Translation files - scripts.i18n +================================ -.. automodule:: scripts.i18n \ No newline at end of file +.. automodule:: scripts.i18n diff --git a/docs/scripts_ref/scripts.maintenance.rst b/docs/scripts_ref/scripts.maintenance.rst index 112695e..4f8df36 100644 --- a/docs/scripts_ref/scripts.maintenance.rst +++ b/docs/scripts_ref/scripts.maintenance.rst @@ -1,34 +1,35 @@ -scripts.maintenance utility scripts -=================================== +Maintenance scripts +=================== + +This page contains documentation of the `scripts.maintenance` module: .. automodule:: scripts.maintenance -Maintenance Submodules +Maintenance submodules ---------------------- -scripts.maintenance.cache script +scripts.maintenance.cache -------------------------------- .. automodule:: scripts.maintenance.cache :ignore-module-all: -scripts.maintenance.colors script +scripts.maintenance.colors --------------------------------- .. automodule:: scripts.maintenance.colors -scripts.maintenance.make\_i18n\_dict script +scripts.maintenance.make\_i18n\_dict ------------------------------------------- .. automodule:: scripts.maintenance.make_i18n_dict -scripts.maintenance.sorting\_order script +scripts.maintenance.sorting\_order ----------------------------------------- .. automodule:: scripts.maintenance.sorting_order -scripts.maintenance.wikimedia\_sites script +scripts.maintenance.wikimedia\_sites ------------------------------------------- .. automodule:: scripts.maintenance.wikimedia_sites - diff --git a/docs/scripts_ref/scripts.rst b/docs/scripts_ref/scripts.rst index 2254477..da12f0a 100644 --- a/docs/scripts_ref/scripts.rst +++ b/docs/scripts_ref/scripts.rst @@ -1,17 +1,8 @@ -scripts package +Scripts package =============== .. automodule:: scripts -Scripts Subpackages -------------------- - -.. toctree:: - - scripts.i18n - scripts.maintenance - scripts.userscripts - add\_text script ---------------- @@ -281,4 +272,11 @@ .. automodule:: scripts.welcome +Script subpackages +------------------ +For information on contents of subpackages, see + +* :doc:`./scripts.i18n` +* :doc:`./scripts.maintenance` +* :doc:`./scripts.userscripts` diff --git a/docs/scripts_ref/scripts.userscripts.rst b/docs/scripts_ref/scripts.userscripts.rst index 60ac1d4..11f5130 100644 --- a/docs/scripts_ref/scripts.userscripts.rst +++ b/docs/scripts_ref/scripts.userscripts.rst @@ -1,5 +1,4 @@ -scripts.userscripts user scripts -================================ +User scripts - scripts.userscripts +================================== .. automodule:: scripts.userscripts - diff --git a/docs/tests_ref/index.rst b/docs/tests_ref/index.rst index c1eac79..b0d289c 100644 --- a/docs/tests_ref/index.rst +++ b/docs/tests_ref/index.rst @@ -1,5 +1,5 @@ =============== -Tests Reference +Tests reference =============== .. include:: ../../tests/README.rst @@ -17,7 +17,7 @@ Library tests ************* - Tests in this project: +Tests in this project: .. toctree:: :maxdepth: 1 @@ -90,7 +90,7 @@ Script tests ************ - Tests in this project: +Tests in this project: .. toctree:: :maxdepth: 1 diff --git a/docs/utilities/install.rst b/docs/utilities/install.rst index dbf4134..794c9b9 100644 --- a/docs/utilities/install.rst +++ b/docs/utilities/install.rst @@ -1,4 +1,4 @@ -Installer Scripts +Installer scripts ================= make\_dist script diff --git a/docs/utilities/scripts.rst b/docs/utilities/scripts.rst index 9a957e6..5c179f6 100644 --- a/docs/utilities/scripts.rst +++ b/docs/utilities/scripts.rst @@ -1,53 +1,53 @@ -Framework Utility Scripts Description +Framework utility scripts ===================================== .. automodule:: pywikibot.scripts :no-members: :noindex: -pwb wrapper script description +pwb wrapper script ------------------------------ .. automodule:: pywikibot.scripts.wrapper :no-members: :noindex: -generate\_family\_file script description +generate\_family\_file script ----------------------------------------- .. automodule:: pywikibot.scripts.generate_family_file :no-members: :noindex: -generate\_user\_files script description +generate\_user\_files script ---------------------------------------- .. automodule:: pywikibot.scripts.generate_user_files :no-members: :noindex: -login script description +login script ------------------------ .. automodule:: pywikibot.scripts.login :no-members: :noindex: -preload_sites script description +preload_sites script -------------------------------- .. automodule:: pywikibot.scripts.preload_sites :no-members: :noindex: -shell script description +shell script ------------------------ .. automodule:: pywikibot.scripts.shell :no-members: :noindex: -version script description +version script -------------------------- .. automodule:: pywikibot.scripts.version diff --git a/docs/utilities/scripts_ref.rst b/docs/utilities/scripts_ref.rst index 7f1346b..1b25394 100644 --- a/docs/utilities/scripts_ref.rst +++ b/docs/utilities/scripts_ref.rst @@ -1,5 +1,5 @@ -Framework Utility Scripts -========================= +Framework utility scripts reference +=================================== .. automodule:: pywikibot.scripts :synopsis: Folder which holds framework scripts diff --git a/scripts/README.rst b/scripts/README.rst index 2d65034..62b413e 100644 --- a/scripts/README.rst +++ b/scripts/README.rst @@ -3,180 +3,185 @@ =========================================================================== Some example robots are included. ---------------------------------- +================================= These programs can actually modify the live wiki on the net, and proper wiki-etiquette should be followed before running it on any wiki. -To get started on proper usage of the bot framework, please refer to: - - `Manual:Pywikibot <https://www.mediawiki.org/wiki/Manual:Pywikibot>`_ +To get started on proper usage of the bot framework, refer to `Manual:Pywikibot <https://www.mediawiki.org/wiki/Manual:Pywikibot>`_. The contents of the package ---------------------------- +=========================== - +----------------------------------------------------------------------------------+ - | Bots and Scripts | - +========================+=========================================================+ - | add_text.py | Adds text at the top or end of pages. | - +------------------------+---------------------------------------------------------+ - | archivebot.py | Archives discussion threads. | - +------------------------+---------------------------------------------------------+ - | basic.py | Is a template from which simple bots can be made. | - +------------------------+---------------------------------------------------------+ - | blockpagechecker.py | Deletes any protection templates that are on pages | - | | which aren't actually protected. | - +------------------------+---------------------------------------------------------+ - | category.py | Add a category link to all pages mentioned on a page, | - | | change or remove category tags. | - +------------------------+---------------------------------------------------------+ - | category_redirect.py | Maintain category redirects and replace links to | - | | redirected categories. | - +------------------------+---------------------------------------------------------+ - | change_pagelang.py | Changes the content language of pages. | - +------------------------+---------------------------------------------------------+ - | checkimages.py | Check recently uploaded files. Checks if a file | - | | description is present and if there are other problems | - | | in the image's description. | - +------------------------+---------------------------------------------------------+ - | claimit.py | Adds claims to Wikidata items based on categories. | - +------------------------+---------------------------------------------------------+ - | clean_sandbox.py | This bot resets a sandbox with predefined text. | - +------------------------+---------------------------------------------------------+ - | commonscat.py | Adds {{commonscat}} to Wikipedia categories (or | - | | articles), if other language Wikipedia already has such | - | | a template. | - +------------------------+---------------------------------------------------------+ - | commons_information.py | Insert a language template into the description field. | - +------------------------+---------------------------------------------------------+ - | coordinate_import.py | Coordinate importing script. | - +------------------------+---------------------------------------------------------+ - | cosmetic_changes.py | Can do slight modifications to a wiki page source code | - | | such that the code looks cleaner. | - +------------------------+---------------------------------------------------------+ - | data_ingestion.py | A generic bot to do batch uploading to Commons. | - +------------------------+---------------------------------------------------------+ - | delete.py | This script can be used to delete pages en masse. | - +------------------------+---------------------------------------------------------+ - | delinker.py | Delink file references of deleted images. | - +------------------------+---------------------------------------------------------+ - | djvutext.py | Extracts OCR text from djvu files and uploads onto | - | | pages in the "Page" namespace on Wikisource. | - +------------------------+---------------------------------------------------------+ - | download_dump.py | Downloads dumps from dumps.wikimedia.org | - +------------------------+---------------------------------------------------------+ - | fixing_redirects.py | Correct all redirect links of processed pages. | - +------------------------+---------------------------------------------------------+ - | harvest_template.py | Template harvesting script. | - +------------------------+---------------------------------------------------------+ - | illustrate_wikidata.py | Bot to add images to Wikidata items. | - +------------------------+---------------------------------------------------------+ - | image.py | Script to replace transclusions of files. | - +------------------------+---------------------------------------------------------+ - | imagetransfer.py | Given a wiki page, check the interwiki links for | - | | images, and let the user choose among them for | - | | images to upload. | - +------------------------+---------------------------------------------------------+ - | interwiki.py | A robot to check interwiki links on all pages (or | - | | a range of pages) of a wiki. | - +------------------------+---------------------------------------------------------+ - | interwikidata.py | Script to handle interwiki links based on Wikibase. | - +------------------------+---------------------------------------------------------+ - | listpages.py | Print a list of pages, defined by a page generator. | - +------------------------+---------------------------------------------------------+ - | misspelling.py | Similar to solve_disambiguation.py. It is supposed to | - | | fix links that contain common spelling mistakes. | - +------------------------+---------------------------------------------------------+ - | movepages.py | Bot that can move pages to another title. | - +------------------------+---------------------------------------------------------+ - | newitem.py | Script creates new items on Wikidata based on criteria. | - +------------------------+---------------------------------------------------------+ - | noreferences.py | Searches for pages where <references /> is missing | - | | although a <ref> tag is present, and in that case adds | - | | a new references section. | - +------------------------+---------------------------------------------------------+ - | nowcommons.py | This bot can delete images with NowCommons template. | - +------------------------+---------------------------------------------------------+ - | pagefromfile.py | This bot takes its input from a file that contains a | - | | number of pages to be put on the wiki. | - +------------------------+--+------------------------------------------------------+ - | parser_function_count.py | Find expensive templates that are subject to be | - | | converted to Lua. | - +------------------------+--+------------------------------------------------------+ - | patrol.py | Obtains a list pages and marks the edits as patrolled | - | | based on a whitelist. | - +------------------------+---------------------------------------------------------+ - | protect.py | Protect and unprotect pages en masse. | - +------------------------+---------------------------------------------------------+ - | redirect.py | Fix double redirects and broken redirects. Note: | - | | solve_disambiguation also has functions which treat | - | | redirects. | - +------------------------+---------------------------------------------------------+ - | reflinks.py | Search for references which are only made of a link | - | | without title and fetch the html title from the link to | - | | use it as the title of the wiki link in the reference. | - +------------------------+---------------------------------------------------------+ - | replace.py | Search articles for a text and replace it by another | - | | text. Both text are set in two configurable | - | | text files. The bot can either work on a set of given | - | | pages or crawl an SQL dump. | - +------------------------+---------------------------------------------------------+ - | replicate_wiki.py | Replicates pages in wiki to a second wiki within family | - +------------------------+---------------------------------------------------------+ - | revertbot.py | Script that can be used for reverting certain edits. | - +------------------------+---------------------------------------------------------+ - | solve_disambiguation.py| Interactive robot doing disambiguation. | - +------------------------+---------------------------------------------------------+ - | speedy_delete.py | Help sysops to quickly check and/or delete pages listed | - | | for speedy deletion. | - +------------------------+---------------------------------------------------------+ - | template.py | Change one template (that is {{...}}) into another. | - +------------------------+---------------------------------------------------------+ - | templatecount.py | Display the list of pages transcluding a given list | - | | of templates. | - +------------------------+---------------------------------------------------------+ - | touch.py | Bot goes over all pages of the home wiki, and edits | - | | them without changes. | - +------------------------+---------------------------------------------------------+ - | transferbot.py | Transfers pages from a source wiki to a target wiki. | - +------------------------+---------------------------------------------------------+ - | unusedfiles.py | Bot appends some text to all unused images and other | - | | text to the respective uploaders. | - +------------------------+---------------------------------------------------------+ - | upload.py | Upload an image to a wiki. | - +------------------------+---------------------------------------------------------+ - | watchlists.py | Allows access to the account's watchlist. | - +------------------------+---------------------------------------------------------+ - | weblinkchecker.py | Check if external links are still working. | - +------------------------+---------------------------------------------------------+ - | welcome.py | Script to welcome new users. | - +------------------------+---------------------------------------------------------+ +Bots and scripts +---------------- ++------------------------------------------------------------------------------------+ +| Bots and Scripts | ++==========================+=========================================================+ +| add_text.py | Adds text at the top or end of pages. | ++--------------------------+---------------------------------------------------------+ +| archivebot.py | Archives discussion threads. | ++--------------------------+---------------------------------------------------------+ +| basic.py | Is a template from which simple bots can be made. | ++--------------------------+---------------------------------------------------------+ +| blockpagechecker.py | Deletes any protection templates that are on pages | +| | which aren't actually protected. | ++--------------------------+---------------------------------------------------------+ +| category.py | Add a category link to all pages mentioned on a page, | +| | change or remove category tags. | ++--------------------------+---------------------------------------------------------+ +| category_redirect.py | Maintain category redirects and replace links to | +| | redirected categories. | ++--------------------------+---------------------------------------------------------+ +| change_pagelang.py | Changes the content language of pages. | ++--------------------------+---------------------------------------------------------+ +| checkimages.py | Check recently uploaded files. Checks if a file | +| | description is present and if there are other problems | +| | in the image's description. | ++--------------------------+---------------------------------------------------------+ +| claimit.py | Adds claims to Wikidata items based on categories. | ++--------------------------+---------------------------------------------------------+ +| clean_sandbox.py | This bot resets a sandbox with predefined text. | ++--------------------------+---------------------------------------------------------+ +| commonscat.py | Adds {{commonscat}} to Wikipedia categories (or | +| | articles), if other language Wikipedia already has such | +| | a template. | ++--------------------------+---------------------------------------------------------+ +| commons_information.py | Insert a language template into the description field. | ++--------------------------+---------------------------------------------------------+ +| coordinate_import.py | Coordinate importing script. | ++--------------------------+---------------------------------------------------------+ +| cosmetic_changes.py | Can do slight modifications to a wiki page source code | +| | such that the code looks cleaner. | ++--------------------------+---------------------------------------------------------+ +| data_ingestion.py | A generic bot to do batch uploading to Commons. | ++--------------------------+---------------------------------------------------------+ +| delete.py | This script can be used to delete pages en masse. | ++--------------------------+---------------------------------------------------------+ +| delinker.py | Delink file references of deleted images. | ++--------------------------+---------------------------------------------------------+ +| djvutext.py | Extracts OCR text from djvu files and uploads onto | +| | pages in the "Page" namespace on Wikisource. | ++--------------------------+---------------------------------------------------------+ +| download_dump.py | Downloads dumps from dumps.wikimedia.org | ++--------------------------+---------------------------------------------------------+ +| fixing_redirects.py | Correct all redirect links of processed pages. | ++--------------------------+---------------------------------------------------------+ +| harvest_template.py | Template harvesting script. | ++--------------------------+---------------------------------------------------------+ +| illustrate_wikidata.py | Bot to add images to Wikidata items. | ++--------------------------+---------------------------------------------------------+ +| image.py | Script to replace transclusions of files. | ++--------------------------+---------------------------------------------------------+ +| imagetransfer.py | Given a wiki page, check the interwiki links for | +| | images, and let the user choose among them for | +| | images to upload. | ++--------------------------+---------------------------------------------------------+ +| interwiki.py | A robot to check interwiki links on all pages (or | +| | a range of pages) of a wiki. | ++--------------------------+---------------------------------------------------------+ +| interwikidata.py | Script to handle interwiki links based on Wikibase. | ++--------------------------+---------------------------------------------------------+ +| listpages.py | Print a list of pages, defined by a page generator. | ++--------------------------+---------------------------------------------------------+ +| misspelling.py | Similar to solve_disambiguation.py. It is supposed to | +| | fix links that contain common spelling mistakes. | ++--------------------------+---------------------------------------------------------+ +| movepages.py | Bot that can move pages to another title. | ++--------------------------+---------------------------------------------------------+ +| newitem.py | Script creates new items on Wikidata based on criteria. | ++--------------------------+---------------------------------------------------------+ +| noreferences.py | Searches for pages where <references /> is missing | +| | although a <ref> tag is present, and in that case adds | +| | a new references section. | ++--------------------------+---------------------------------------------------------+ +| nowcommons.py | This bot can delete images with NowCommons template. | ++--------------------------+---------------------------------------------------------+ +| pagefromfile.py | This bot takes its input from a file that contains a | +| | number of pages to be put on the wiki. | ++--------------------------+---------------------------------------------------------+ +| parser_function_count.py | Find expensive templates that are subject to be | +| | converted to Lua. | ++--------------------------+---------------------------------------------------------+ +| patrol.py | Obtains a list pages and marks the edits as patrolled | +| | based on a whitelist. | ++--------------------------+---------------------------------------------------------+ +| protect.py | Protect and unprotect pages en masse. | ++--------------------------+---------------------------------------------------------+ +| redirect.py | Fix double redirects and broken redirects. Note: | +| | solve_disambiguation also has functions which treat | +| | redirects. | ++--------------------------+---------------------------------------------------------+ +| reflinks.py | Search for references which are only made of a link | +| | without title and fetch the html title from the link to | +| | use it as the title of the wiki link in the reference. | ++--------------------------+---------------------------------------------------------+ +| replace.py | Search articles for a text and replace it by another | +| | text. Both text are set in two configurable | +| | text files. The bot can either work on a set of given | +| | pages or crawl an SQL dump. | ++--------------------------+---------------------------------------------------------+ +| replicate_wiki.py | Replicates pages in wiki to a second wiki within family | ++--------------------------+---------------------------------------------------------+ +| revertbot.py | Script that can be used for reverting certain edits. | ++--------------------------+---------------------------------------------------------+ +| solve_disambiguation.py | Interactive robot doing disambiguation. | ++--------------------------+---------------------------------------------------------+ +| speedy_delete.py | Help sysops to quickly check and/or delete pages listed | +| | for speedy deletion. | ++--------------------------+---------------------------------------------------------+ +| template.py | Change one template (that is {{...}}) into another. | ++--------------------------+---------------------------------------------------------+ +| templatecount.py | Display the list of pages transcluding a given list | +| | of templates. | ++--------------------------+---------------------------------------------------------+ +| touch.py | Bot goes over all pages of the home wiki, and edits | +| | them without changes. | ++--------------------------+---------------------------------------------------------+ +| transferbot.py | Transfers pages from a source wiki to a target wiki. | ++--------------------------+---------------------------------------------------------+ +| unusedfiles.py | Bot appends some text to all unused images and other | +| | text to the respective uploaders. | ++--------------------------+---------------------------------------------------------+ +| upload.py | Upload an image to a wiki. | ++--------------------------+---------------------------------------------------------+ +| watchlists.py | Allows access to the account's watchlist. | ++--------------------------+---------------------------------------------------------+ +| weblinkchecker.py | Check if external links are still working. | ++--------------------------+---------------------------------------------------------+ +| welcome.py | Script to welcome new users. | ++--------------------------+---------------------------------------------------------+ - +------------------------+---------------------------------------------------------+ - | maintenance | Framework helper scripts. . | - +========================+=========================================================+ - | cache.py | Script for showing and deleting API cache. | - +------------------------+---------------------------------------------------------+ - | colors.py | Utility to show pywikibot colors. | - +------------------------+---------------------------------------------------------+ - | make_i18n_dict.py | Generate an i18n file from a given script. | - +------------------------+---------------------------------------------------------+ - | sorting_order.py | Updates interwiki sorting order in family.py file. | - +------------------------+---------------------------------------------------------+ - | wikimedia_sites.py | Updates the language lists in Wikimedia family files. | - +------------------------+---------------------------------------------------------+ +Maintenance +----------- ++------------------------+---------------------------------------------------------+ +| maintenance | Framework helper scripts | ++========================+=========================================================+ +| cache.py | Script for showing and deleting API cache. | ++------------------------+---------------------------------------------------------+ +| colors.py | Utility to show pywikibot colors. | ++------------------------+---------------------------------------------------------+ +| make_i18n_dict.py | Generate an i18n file from a given script. | ++------------------------+---------------------------------------------------------+ +| sorting_order.py | Updates interwiki sorting order in family.py file. | ++------------------------+---------------------------------------------------------+ +| wikimedia_sites.py | Updates the language lists in Wikimedia family files. | ++------------------------+---------------------------------------------------------+ - +------------------------+---------------------------------------------------------+ - | Others | | - +========================+=========================================================+ - | i18n (folder) | Contains i18n translations for bot edit summaries. | - +------------------------+---------------------------------------------------------+ - | userscripts (folder) | Empty folder for user scripts. | - +------------------------+---------------------------------------------------------+ - | README.rst | This file (Short info of all scripts). | - +------------------------+---------------------------------------------------------+ +Others +------ + ++------------------------+---------------------------------------------------------+ +| Others | | ++========================+=========================================================+ +| i18n (folder) | Contains i18n translations for bot edit summaries. | ++------------------------+---------------------------------------------------------+ +| userscripts (folder) | Empty folder for user scripts. | ++------------------------+---------------------------------------------------------+ +| README.rst | This file (Short info of all scripts). | ++------------------------+---------------------------------------------------------+ **External packages could be required with Pywikibot:** -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860879 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: tests Gerrit-Change-Id: I810d27449a43aba1e361bab64aab3a42618efe30 Gerrit-Change-Number: 860879 Gerrit-PatchSet: 2 Gerrit-Owner: KBach <kbach(a)wikimedia.org> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [tests] Solve unexpected success with script_tests
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860877 ) Change subject: [tests] Solve unexpected success with script_tests ...................................................................... [tests] Solve unexpected success with script_tests Change-Id: I50dd4f93e811306c7ca84221f5aaea2eb92be31e --- M tests/script_tests.py 1 file changed, 9 insertions(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/script_tests.py b/tests/script_tests.py index 70a0d3a..c239f8f 100755 --- a/tests/script_tests.py +++ b/tests/script_tests.py @@ -405,7 +405,6 @@ 'claimit', 'clean_sandbox', 'commonscat', - 'coordinate_import', 'create_isbn_edition', 'dataextend', 'data_ingestion', @@ -413,7 +412,6 @@ 'delinker', 'djvutext', 'download_dump', - 'fixing_redirects', 'harvest_template', 'imagetransfer', 'interwiki', @@ -421,14 +419,12 @@ 'login', 'misspelling', 'movepages', - 'newitem', 'pagefromfile', 'parser_function_count', 'patrol', 'protect', 'redirect', 'reflinks', # 404-links.txt is required - 'replace', 'replicate_wiki', 'revertbot', 'shell', -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860877 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I50dd4f93e811306c7ca84221f5aaea2eb92be31e Gerrit-Change-Number: 860877 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[tests]: [doc] Improve documentation look and feel
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/859448 ) Change subject: [doc] Improve documentation look and feel ...................................................................... [doc] Improve documentation look and feel * add the sphinx-opengraph extension * customize edit link to allow readers to access source RST files in gerrit easily * allow users to navigate between pages using arrow keys (left for previous, right for next) * set pygments light style explicitly * add more prominent highlighting on a:hover * ensure the logo does not take too much space * add make mini to Makefile as a way to quickly rebuild docs after minor changes Bug: T322212 Change-Id: Ifde05319db3591ff3423b21c66d92cbb1f06e8b5 --- M .gitignore M docs/make.bat M docs/conf.py M docs/requirements.txt M docs/_static/css/pywikibot.css M docs/Makefile 6 files changed, 67 insertions(+), 39 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/.gitignore b/.gitignore index 9d20600..965b100 100644 --- a/.gitignore +++ b/.gitignore @@ -23,4 +23,5 @@ docs/_build/ /scripts/userscripts/* !/scripts/userscripts/__init__.py -.vscode/ \ No newline at end of file +.vscode/ +**/.DS_Store diff --git a/docs/Makefile b/docs/Makefile index 6f63c8c..1a49959 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -2,7 +2,8 @@ # # You can set these variables from the command line. -SPHINXOPTS = -a +SPHINXOPTS = -a -j auto +SPHINXOPTSMINI= -j auto SPHINXBUILD = sphinx-build SPHINXPROJ = Pywikibot SOURCEDIR = . @@ -12,6 +13,11 @@ help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) +# make mini only rebuilds what has changed +# make html and others rebuild all content (because of the -a flag) +mini: + @$(SPHINXBUILD) -M html "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTSMINI) $(O) + .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new diff --git a/docs/_static/css/pywikibot.css b/docs/_static/css/pywikibot.css index 56e8d93..6721511 100644 --- a/docs/_static/css/pywikibot.css +++ b/docs/_static/css/pywikibot.css @@ -1,43 +1,31 @@ /** css/pywikibot.css **/ -/* This line is theme specific - it includes the base theme CSS */ -@import '../nature.css'; /* for Nature */ - - -div.related { - background-color: #396; - line-height: 32px; - color: #fff; - text-shadow: 0px 1px 0 #444; - font-size: 0.9em; +/** logo slightly smaller than default to preserve vertical space **/ +.sidebar-logo { + max-width: 75%; } -a:hover { - color: #900; - text-decoration: underline; +/** brand text in the sidebar smaller than default **/ +.sidebar-brand-text { + font-size: 1rem; } -a.headerlink { - color: #069; - font-size: 0.8em; - padding: 0 4px 0 4px; +/** border highlight for links in article content **/ +article a:hover { + box-shadow: 0 0 0 .125rem var(--color-link); + border-radius: .125rem; text-decoration: none; + padding-bottom: .110rem; } -a.headerlink:hover { - background-color: #069; - color: white; +/** better readability of toctree captions **/ +.toctree-wrapper .caption { + font-weight: bolder; + margin-bottom: .5rem !important; } -code.xref { - color: #069; - font-size: 1.2em + +/** disable centering of tables by default in regular content **/ +table.align-default { + margin-left: unset; + margin-right: unset; } -code.xref:hover { - color: #900; -} -div.sphinxsidebar { - width: 250px -} -div.bodywrapper { - margin: 0 0 0 250px; -} \ No newline at end of file diff --git a/docs/conf.py b/docs/conf.py index b28e251..b5a4fe3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -57,6 +57,7 @@ 'sphinx.ext.napoleon', 'sphinx.ext.todo', 'sphinx.ext.viewcode', + 'sphinxext.opengraph', ] @@ -131,7 +132,8 @@ # show_authors = False # The name of the Pygments (syntax highlighting) style to use. -pygments_style = 'sphinx' +pygments_style = 'default' + # A list of ignored prefixes for module index sorting. # modindex_common_prefix = [] @@ -154,7 +156,7 @@ # further. For a list of options available for each theme, see the # documentation. # -# html_theme_options = {} + # Add any paths that contain custom themes here, relative to this directory. # html_theme_path = [] @@ -222,6 +224,12 @@ # html_split_index = True +docs_url = 'https://gerrit.wikimedia.org/g/pywikibot/core/%2B/HEAD/docs/' +html_theme_options = { + 'source_edit_link': docs_url + '{filename}', + 'navigation_with_keys': True, +} + # If true, links to the reST sources are added to the pages. # # html_show_sourcelink = True @@ -377,7 +385,9 @@ # Pywikibot theme style html_permalinks_icon = '#' -# html_style = 'css/pywikibot.css' +html_css_files = [ + 'css/pywikibot.css', +] extlinks = { diff --git a/docs/make.bat b/docs/make.bat index 8bacfe3..4ea5f91 100644 --- a/docs/make.bat +++ b/docs/make.bat @@ -10,6 +10,7 @@ set SOURCEDIR=. set BUILDDIR=_build set SPHINXPROJ=Pywikibot +set SPHINXOPTS=-j auto if "%1" == "" goto help diff --git a/docs/requirements.txt b/docs/requirements.txt index e287d85..61d54ac 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,5 +1,6 @@ # This is a PIP requirements file for building Sphinx documentation of pywikibot # requirements.txt is also needed -furo sphinx >= 5.2.3 -rstcheck >=6.1.0 \ No newline at end of file +rstcheck >=6.1.0 +sphinxext-opengraph >= 0.7.2 +furo >= 2022.9.29 -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/859448 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: tests Gerrit-Change-Id: Ifde05319db3591ff3423b21c66d92cbb1f06e8b5 Gerrit-Change-Number: 859448 Gerrit-PatchSet: 11 Gerrit-Owner: KBach <kbach(a)wikimedia.org> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [IMPR] Improvements for editor
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/857001 ) Change subject: [IMPR] Improvements for editor ...................................................................... [IMPR] Improvements for editor - Redefine config.editor setting: No longer call os.environment here. config.editor may be str, bool or None. str: use this as editor command True: use Tkinter False: use 'break' or 'true' as command which does nothing (mostly used for tests purpose) None: Use EDITOR environment value if present or on Windows detect editor from registry or fially ue Tkinter - Move editor detection functions from config.py to editor.py - copy EXTERNAL EDITOR SETTINGS section in generate_user_files.py - use pathlib.Path in TextEditor.edit() Change-Id: I84b097c09ee2a4a582582878e08afb3cbb086264 --- M pywikibot/editor.py M pywikibot/scripts/generate_user_files.py M pywikibot/config.py 3 files changed, 151 insertions(+), 104 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/config.py b/pywikibot/config.py index 64cc696..1746a02 100644 --- a/pywikibot/config.py +++ b/pywikibot/config.py @@ -29,6 +29,9 @@ .. versionchanged:: 6.2 config2 was renamed to config +.. versionchanged:: 8.0 + Editor settings has been revised. *editor* variable is None by + default. Editor detection functions were moved to :mod:`editor`. """ # # (C) Pywikibot team, 2003-2022 @@ -71,8 +74,6 @@ OSWIN32 = (sys.platform == 'win32') -if OSWIN32: - import winreg # This frozen set should contain all imported modules/variables, so it must # occur directly after the imports. At that point globals() only contains the @@ -568,9 +569,14 @@ tkvertsize = 800 # ############# EXTERNAL EDITOR SETTINGS ############## -# The command for the editor you want to use. If set to None, a simple Tkinter -# editor will be used. -editor = os.environ.get('EDITOR') +# The command for the editor you want to use. If set to True, Tkinter +# editor will be used. If set to False, no editor will be used. In +# script tests to be a noop (like /bin/true) so the script continues. +# If set to None, the EDITOR environment variable will be used as +# command. If EDITOR is not set, on windows plattforms it tries to +# determine the default text editor from registry. Finally, Tkinter is +# used as fallback. +editor: Union[bool, str, None] = None # Warning: DO NOT use an editor which doesn't support Unicode to edit pages! # You will BREAK non-ASCII symbols! @@ -925,51 +931,6 @@ return path -def _win32_extension_command(extension: str) -> Optional[str]: - """Get the command from the Win32 registry for an extension.""" - fileexts_key = \ - r'Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts' - key_name = fileexts_key + r'\.' + extension + r'\OpenWithProgids' - try: - key1 = winreg.OpenKey(winreg.HKEY_CURRENT_USER, key_name) - _prog_id = winreg.EnumValue(key1, 0)[0] - _key2 = winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, - fr'{_prog_id}\shell\open\command') - _cmd = winreg.QueryValueEx(_key2, '')[0] - # See T102465 for issues relating to using this value. - cmd = _cmd - if cmd.find('%1'): - cmd = cmd[:cmd.find('%1')] - # Remove any trailing character, which should be a quote or space - # and then remove all whitespace. - return cmd[:-1].strip() - except OSError as e: - # Catch any key lookup errors - output('Unable to detect program for file extension "{}": {!r}' - .format(extension, e)) - return None - - -def _detect_win32_editor() -> Optional[str]: - """Detect the best Win32 editor.""" - # Notepad is even worse than our Tkinter editor. - unusable_exes = ['notepad.exe', - 'py.exe', - 'pyw.exe', - 'python.exe', - 'pythonw.exe'] - - for ext in ['py', 'txt']: - editor = _win32_extension_command(ext) - if editor: - for unusable in unusable_exes: - if unusable in editor.lower(): - break - else: - return editor - return None - - # System-level and User-level changes. # Store current variables and their types. _public_globals = { @@ -1107,18 +1068,6 @@ if console_encoding is None: console_encoding = 'utf-8' -if OSWIN32 and editor is None: - editor = _detect_win32_editor() - -# single character string literals from -# https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-… -# encode('unicode-escape') also changes Unicode characters -if OSWIN32 and editor and set(editor) & set('\a\b\f\n\r\t\v'): - warning( - 'The editor path contains probably invalid escaped characters. Make ' - 'sure to use a raw-string (r"..." or r\'...\'), forward slashes as a ' - 'path delimiter or to escape the normal path delimiter.') - if userinterface_lang is None: userinterface_lang = os.getenv('PYWIKIBOT_USERINTERFACE_LANG') \ or getlocale()[0] diff --git a/pywikibot/editor.py b/pywikibot/editor.py index e29b2de..29e4829 100644 --- a/pywikibot/editor.py +++ b/pywikibot/editor.py @@ -4,10 +4,10 @@ # # Distributed under the terms of the MIT license. # -import codecs import os import subprocess import tempfile +from pathlib import Path from sys import platform from textwrap import fill from typing import Optional @@ -24,14 +24,36 @@ GUI_ERROR = e +OSWIN32 = platform == 'win32' +if OSWIN32: + import winreg + + class TextEditor: - """Text editor.""" + """Text editor. - @staticmethod - def _command(file_name: str, text: str, + .. versionchanged:: 8.0 + Editor detection functions were moved from :mod:`config`. + """ + + def __init__(self): + """Setup external Editor.""" + self.editor: str + if config.editor is True: + self.editor = '' + elif config.editor is False: + self.editor = 'break' if OSWIN32 else 'true' + elif config.editor is None: + self.editor = os.environ.get('EDITOR', '') + if OSWIN32 and not self.editor: + self.editor = self._detect_win32_editor() + else: + self.editor = config.editor + + def _command(self, file_name: str, text: str, jump_index: Optional[int] = None) -> List[str]: - """Return editor selected in user config file (user-config.py).""" + """Return command of editor selected in user config file.""" if jump_index: # Some editors make it possible to mark occurrences of substrings, # or to jump to the line of the first occurrence. @@ -41,28 +63,28 @@ column = jump_index - (text[:jump_index].rfind('\n') + 1) else: line = column = 0 + # Linux editors. We use startswith() because some users might use # parameters. - assert config.editor is not None - if config.editor.startswith('kate'): + if self.editor.startswith('kate'): command = ['-l', str(line + 1), '-c', str(column + 1)] - elif config.editor.startswith(('gedit', 'emacs')): + elif self.editor.startswith(('gedit', 'emacs')): command = [f'+{line + 1}'] # columns seem unsupported - elif config.editor.startswith('jedit'): + elif self.editor.startswith('jedit'): command = [f'+line:{line + 1}'] # columns seem unsupported - elif config.editor.startswith('vim'): + elif self.editor.startswith('vim'): command = [f'+{line + 1}'] # columns seem unsupported - elif config.editor.startswith('nano'): + elif self.editor.startswith('nano'): command = [f'+{line + 1},{column + 1}'] # Windows editors - elif config.editor.lower().endswith('notepad++.exe'): + elif self.editor.lower().endswith('notepad++.exe'): command = [f'-n{line + 1}'] # seems not to support columns else: command = [] - # See T102465 for problems relating to using config.editor unparsed. - command = [config.editor] + command + [file_name] - pywikibot.log(f'Running editor: {TextEditor._concat(command)}') + # See T102465 for problems relating to using self.editor unparsed. + command = [self.editor] + command + [file_name] + pywikibot.log(f'Running editor: {self._concat(command)}') return command @staticmethod @@ -83,39 +105,93 @@ :return: the modified text, or None if the user didn't save the text file in his text editor """ - if config.editor: - handle, tempFilename = tempfile.mkstemp() - tempFilename = '{}.{}'.format(tempFilename, - config.editor_filename_extension) - try: - with codecs.open(tempFilename, 'w', - encoding=config.editor_encoding) as tempFile: - tempFile.write(text) - creationDate = os.stat(tempFilename).st_mtime - cmd = self._command(tempFilename, text, jumpIndex) - subprocess.run(cmd, shell=platform == 'win32', check=True) - lastChangeDate = os.stat(tempFilename).st_mtime - if lastChangeDate == creationDate: - # Nothing changed - return None + if self.editor: + handle, filename = tempfile.mkstemp( + suffix=f'.{config.editor_filename_extension}', text=True) + path = Path(filename) - with codecs.open(tempFilename, 'r', - encoding=config.editor_encoding) as temp_file: - newcontent = temp_file.read() - return newcontent + try: + encoding = config.editor_encoding + path.write_text(text, encoding=encoding) + + creation_date = path.stat().st_mtime + cmd = self._command(filename, text, jumpIndex) + subprocess.run(cmd, shell=platform == 'win32', check=True) + last_change_date = path.stat().st_mtime + + if last_change_date == creation_date: + return None # Nothing changed + + return path.read_text(encoding=encoding) + finally: os.close(handle) - os.unlink(tempFilename) + os.unlink(path) if GUI_ERROR: raise ImportError(fill( - 'Could not load GUI modules: {}. No editor available. ' - 'Set your favourite editor in user-config.py "editor", ' - 'or install python packages tkinter and idlelib, which ' - 'are typically part of Python but may be packaged separately ' - 'on your platform.'.format(GUI_ERROR)) + '\n') + f'Could not load GUI modules: {GUI_ERROR}. No editor' + ' available. Set your favourite editor in user-config.py' + ' "editor", or install python packages tkinter and idlelib,' + ' which are typically part of Python but may be packaged' + ' separately on your platform.') + '\n') assert pywikibot.ui is not None - return pywikibot.ui.editText(text, jumpIndex=jumpIndex, highlight=highlight) + + @staticmethod + def _win32_extension_command(extension: str) -> Optional[str]: + """Get the command from the Win32 registry for an extension.""" + fileexts_key = \ + r'Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts' + key_name = fr'{fileexts_key}\.{extension}\OpenWithProgids' + try: + key1 = winreg.OpenKey(winreg.HKEY_CURRENT_USER, key_name) + _prog_id = winreg.EnumValue(key1, 0)[0] + _key2 = winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, + fr'{_prog_id}\shell\open\command') + _cmd = winreg.QueryValueEx(_key2, '')[0] + # See T102465 for issues relating to using this value. + cmd = _cmd + if cmd.find('%1'): + cmd = cmd[:cmd.find('%1')] + # Remove any trailing character, which should be a quote or + # space and then remove all whitespace. + return cmd[:-1].strip() + except OSError as e: + # Catch any key lookup errors + pywikibot.info(f'Unable to detect program for file extension ' + f'{extension!r}: {e!r}') + return None + + @staticmethod + def _detect_win32_editor() -> str: + """Detect the best Win32 editor.""" + # Notepad is even worse than our Tkinter editor. + unusable_exes = ['notepad.exe', + 'py.exe', + 'pyw.exe', + 'python.exe', + 'pythonw.exe'] + + for ext in ['py', 'txt']: + editor = TextEditor._win32_extension_command(ext) + if editor: + for unusable in unusable_exes: + if unusable in editor.lower(): + break + else: + if set(editor) & set('\a\b\f\n\r\t\v'): + # single character string literals from + # https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-… + # encode('unicode-escape') also changes Unicode + # characters + pywikibot.warning(fill( + 'The editor path contains probably invalid ' + 'escaped characters. Make sure to use a ' + 'raw-string (r"..." or r\'...\'), forward slashes ' + 'as a path delimiter or to escape the normal path ' + 'delimiter.')) + return editor + return '' diff --git a/pywikibot/scripts/generate_user_files.py b/pywikibot/scripts/generate_user_files.py index 101a066..ab2005c 100755 --- a/pywikibot/scripts/generate_user_files.py +++ b/pywikibot/scripts/generate_user_files.py @@ -5,6 +5,7 @@ moved to pywikibot.scripts folder. .. versionchanged:: 8.0 let user the choice which section to be copied. +...Also EXTERNAL EDITOR SETTINGS section can be copied. """ # # (C) Pywikibot team, 2010-2022 @@ -30,7 +31,6 @@ # DISABLED_SECTIONS cannot be copied; variables must be set manually DISABLED_SECTIONS = { 'USER INTERFACE SETTINGS', # uses sys - 'EXTERNAL EDITOR SETTINGS', # uses os } OBSOLETE_SECTIONS = { 'ACCOUNT SETTINGS', # already set -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/857001 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I84b097c09ee2a4a582582878e08afb3cbb086264 Gerrit-Change-Number: 857001 Gerrit-PatchSet: 8 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [IMPR] Let user the choice which section to be copied
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/309965 ) Change subject: [IMPR] Let user the choice which section to be copied ...................................................................... [IMPR] Let user the choice which section to be copied This patch enables to select all wanted sections step by step or in two bulks for the section types related to the framework and related to the scripts. A help option is provided to get more information about a section or a section type. generate_user_files.py: - New SCRIPT_SECTION to differentiate between script and framework settings - Let user the choice which which sections to include. copy_sections() now calls input_sections to get the wanted sections - input_sections() asks which section to be included. Framework sections and scripts section could be included or excluded in bulk or each section step by step. Help options are provided to show the section of the given section type and a short help message for each section to be processed. input_sections also add the selected section to a list; this mutable is hold by copy_sections. - Default response for bulk inclusion is [n]one, default answer of section selection is [n]o. - default parameters of input_sections and copy_sections are for tests purposes only - tests suite doesn't copy the whole sections of user-config.py any longer but uses the SMALL_CONFIG part with its modifications by the batch script. The force parameter is used by the test suite. generate_user_files_tests.py - tests modified and added Bug: T145372 Change-Id: I68a04e9198dc179f06c6245f1c1aec518603bd34 --- M tests/generate_user_files_tests.py M pywikibot/scripts/generate_user_files.py 2 files changed, 134 insertions(+), 21 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/scripts/generate_user_files.py b/pywikibot/scripts/generate_user_files.py index 6d4018e..5cfaa09 100755 --- a/pywikibot/scripts/generate_user_files.py +++ b/pywikibot/scripts/generate_user_files.py @@ -2,7 +2,9 @@ """Script to create user-config.py. Other file names are not supported. .. versionchanged:: 7.0 - moved to pywikibot.scripts folder + moved to pywikibot.scripts folder. +.. versionchanged:: 8.0 + let user the choice which section to be copied. """ # # (C) Pywikibot team, 2010-2022 @@ -18,7 +20,7 @@ from textwrap import fill from typing import Optional -from pywikibot.backports import Tuple +from pywikibot.backports import Callable, List, Tuple from pywikibot.scripts import _import_with_no_user_config @@ -33,6 +35,12 @@ OBSOLETE_SECTIONS = { 'ACCOUNT SETTINGS', # already set } +SCRIPT_SECTIONS = { + 'INTERWIKI SETTINGS', + 'SOLVE_DISAMBIGUATION SETTINGS', + 'WEBLINK CHECKER SETTINGS', + 'REPLICATION BOT SETTINGS', +} # Disable user-config usage as we are creating it here pywikibot = _import_with_no_user_config('pywikibot') @@ -221,6 +229,9 @@ {botpasswords}""" +ConfigSection = namedtuple('ConfigSection', 'head, info, section') + + def parse_sections() -> list: """Parse sections from config.py file. @@ -230,7 +241,6 @@ :return: a list of ConfigSection named tuples. """ data = [] - ConfigSection = namedtuple('ConfigSection', 'head, info, section') config_path = Path(__file__).resolve().parents[1].joinpath('config.py') with codecs.open(config_path, 'r', 'utf-8') as config_f: @@ -250,19 +260,84 @@ return data -def copy_sections() -> str: +def copy_sections(force: bool = False, default: str = 'n') -> str: """Take config sections and copy them to user-config.py. + .. versionchanged:: 8.0 + *force* and *default* options were added. + + :param force: Copy all sections if force is True + :param default: Default answer for input_sections. + Should be 'a' for all or 'n' for none to copy. :return: config text of all selected sections. """ - result = [] sections = parse_sections() - # copy settings - for section in filter(lambda x: x.head not in (DISABLED_SECTIONS - | OBSOLETE_SECTIONS), - sections): - result.append(section.section) - return ''.join(result) + if not sections: # Something is wrong with the regex + return None + + # copy framework settings + copies = input_sections( + 'framework', sections, force=force, default=default, + skip=lambda x: x.head not in (DISABLED_SECTIONS + | OBSOLETE_SECTIONS + | SCRIPT_SECTIONS)) + + # copy scripts settings + copies += input_sections( + 'scripts', sections, force=force, default=default, + skip=lambda x: x.head in SCRIPT_SECTIONS) + + return ''.join(copies) + + +def input_sections(variant: str, + sections: List['ConfigSection'], + skip: Optional[Callable] = None, + force: bool = False, + default: str = 'n') -> None: + """Ask for settings to copy. + + .. versionadded:: 8.0 + + :param variant: Variant of the setting section. Either 'framework' + or 'scripts' + :param sections: A sections list previously read from the config file + :param skip: a filter function + :param force: Force input if True + """ + # First ask what to do which the whole section type variant + select = 'h' + answers = [('Yes', 'y'), ('All', 'a'), ('None', 'n'), ('Help', 'h')] + while select == 'h': + select = pywikibot.input_choice( + f'Do you want to select {variant} setting sections?', + answers, default=default, force=force, automatic_quit=False) + if select == 'h': + answers.pop(-1) + pywikibot.info( + f'The following {variant} setting sections are provided:') + for item in filter(skip, sections): + pywikibot.output(item.head) + + copies = [] + # Now ask for a single section of the given variant + # or use all or none of them + choice = {'a': 'all', 'n': 'none', 'y': 'h'}[select] # mapping + for item in filter(skip, sections): + answers = [('Yes', 'y'), ('No', 'n'), ('Help', 'h')] + while choice == 'h': + choice = pywikibot.input_choice( + f'Do you want to add {item.head} section?', + answers, default='n', force=force, automatic_quit=False) + if choice == 'h': + answers.pop(-1) + pywikibot.info(fill(item.info)) + if choice in ('all', 'y'): + copies.append(item.section) # Fill the mutable directly + if choice not in ('all', 'none'): + choice = 'h' + + return copies def create_user_config( @@ -331,7 +406,7 @@ "('{}', BotPassword('{}', {!r}))".format(*botpassword) for botpassword in botpasswords) - config_text = copy_sections() + config_text = copy_sections(force=force) if config_text: config_content = EXTENDED_CONFIG else: diff --git a/tests/generate_user_files_tests.py b/tests/generate_user_files_tests.py index f5e20bd..770ace4 100755 --- a/tests/generate_user_files_tests.py +++ b/tests/generate_user_files_tests.py @@ -84,25 +84,24 @@ def test_copy_sections_not_found(self): """Test copy_sections function for sections not in config text.""" - config_text = guf.copy_sections() + config_text = guf.copy_sections(force=True, default='a') for section in guf.DISABLED_SECTIONS | guf.OBSOLETE_SECTIONS: self.assertNotIn(section, config_text) def test_copy_sections_found(self): """Test copy_sections function for sections found in config text.""" - config_text = guf.copy_sections() + config_text = guf.copy_sections(force=True, default='a') self.assertIsNotNone(config_text) - for section in ('LOGFILE SETTINGS', - 'EXTERNAL SCRIPT PATH SETTINGS', - 'INTERWIKI SETTINGS', - 'FURTHER SETTINGS', - 'HTTP SETTINGS', - 'REPLICATION BOT SETTINGS', - ): + for section in guf.SCRIPT_SECTIONS: self.assertIn(section, config_text) lines = config_text.splitlines() self.assertGreater(len(lines), 200) + def test_copy_sections_none(self): + """Test read_sections function.""" + config_text = guf.copy_sections(force=True) + self.assertEqual(config_text, '') + if __name__ == '__main__': # pragma: no cover with suppress(SystemExit): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/309965 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I68a04e9198dc179f06c6245f1c1aec518603bd34 Gerrit-Change-Number: 309965 Gerrit-PatchSet: 29 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz> Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [IMPR] use roundrobin_generators to combine generators when limit is ...
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/432772 ) Change subject: [IMPR] use roundrobin_generators to combine generators when limit is given ...................................................................... [IMPR] use roundrobin_generators to combine generators when limit is given Change-Id: I11957c120c911b701581e60d6c3b965980478999 --- M pywikibot/pagegenerators/__init__.py M pywikibot/pagegenerators/_factory.py 2 files changed, 24 insertions(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/pagegenerators/__init__.py b/pywikibot/pagegenerators/__init__.py index fa30b7d..4608b6c 100644 --- a/pywikibot/pagegenerators/__init__.py +++ b/pywikibot/pagegenerators/__init__.py @@ -432,8 +432,10 @@ -intersect Work on the intersection of all the provided generators. --limit When used with any other argument -limit:n specifies a set - of pages, work on no more than n pages in total. +-limit When used with any other argument ``-limit:n`` + specifies a set of pages, work on no more than n + pages in total. If used with multiple generators, + pages are yielded in a roundrobin way. -namespaces Filter the page generator to only yield pages in the -namespace specified namespaces. Separate multiple namespace diff --git a/pywikibot/pagegenerators/_factory.py b/pywikibot/pagegenerators/_factory.py index 0b42c73..f32c8e3 100644 --- a/pywikibot/pagegenerators/_factory.py +++ b/pywikibot/pagegenerators/_factory.py @@ -54,7 +54,11 @@ WikidataSPARQLPageGenerator, ) from pywikibot.tools.collections import DequeGenerator -from pywikibot.tools.itertools import filter_unique, intersect_generators +from pywikibot.tools.itertools import ( + filter_unique, + intersect_generators, + roundrobin_generators, +) HANDLER_RETURN_TYPE = Union[None, bool, Iterable['pywikibot.page.BasePage']] @@ -200,6 +204,10 @@ .. versionchanged:: 7.3 set the instance variable :attr:`is_preloading` to True or False. + .. versionchanged:: 8.0 + if ``limit`` option is set and multiple generators are given, + pages are yieded in a :func:`roundrobin + <tools.itertools.roundrobin_generators>` way. :param gen: Another generator to be combined with :param preload: preload pages using PreloadingGenerator @@ -247,7 +255,8 @@ # By definition no duplicates are possible. dupfiltergen = intersect_generators(*self.gens) else: - dupfiltergen = _filter_unique_pages(itertools.chain(*self.gens)) + combine = roundrobin_generators if self.limit else itertools.chain + dupfiltergen = _filter_unique_pages(combine(*self.gens)) # Add on subpage filter generator if self.subpage_max_depth is not None: -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/432772 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I11957c120c911b701581e60d6c3b965980478999 Gerrit-Change-Number: 432772 Gerrit-PatchSet: 3 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [IMPR] remove version hints when printing module documentation
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/858590 ) Change subject: [IMPR] remove version hints when printing module documentation ...................................................................... [IMPR] remove version hints when printing module documentation Also use ModuleNotFoundError exception instead of generic Exception and only keep import_module() function inside try statement. Change-Id: I469ff08a24957b78577fd402ea0cdb595b38e39f --- M pywikibot/bot.py 1 file changed, 22 insertions(+), 6 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/bot.py b/pywikibot/bot.py index 4a85b17..3ccc9e9 100644 --- a/pywikibot/bot.py +++ b/pywikibot/bot.py @@ -93,6 +93,7 @@ import logging import logging.handlers import os +import re import sys import time import warnings @@ -1026,6 +1027,8 @@ .. versionchanged:: 4.0 Renamed from showHelp() to show_help(). + .. versionchanged:: 8.0 + Do not show version changes. """ if not module_name: module_name = calledModuleName() @@ -1039,16 +1042,17 @@ try: module = import_module(module_name) - help_text: str = module.__doc__ # type: ignore[assignment] - if hasattr(module, 'docuReplacements'): - for key, value in module.docuReplacements.items(): - help_text = help_text.replace(key, value.strip('\n\r')) - except Exception: + except ModuleNotFoundError: if module_name: pywikibot.stdout('Sorry, no help available for ' + module_name) pywikibot.log('show_help:', exc_info=True) else: - pywikibot.stdout(help_text) # output to STDOUT + help_text = re.sub(r'^\.\. version(added|changed)::.+', '', + module.__doc__, flags=re.MULTILINE | re.DOTALL) + if hasattr(module, 'docuReplacements'): + for key, value in module.docuReplacements.items(): + help_text = help_text.replace(key, value.strip()) + pywikibot.stdout(help_text) if show_global or module_name == 'pwb': pywikibot.stdout(_GLOBAL_HELP.format(module_name)) -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/858590 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I469ff08a24957b78577fd402ea0cdb595b38e39f Gerrit-Change-Number: 858590 Gerrit-PatchSet: 2 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[tests]: [tests] Update flake tests on jenkins CI
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860845 ) Change subject: [tests] Update flake tests on jenkins CI ...................................................................... [tests] Update flake tests on jenkins CI Backport change from master to tests. several flake8 modules fails with new flake8 6.0.0 which needs Python 3.8.1. Jenkins CI does not reflect the right Python version. Therefore only add quotes check for Python < 3.8 which uses flake8 5.0.4. - Use flake8-quotes with Python < 3.8 only - Adjust flake8-quotes to >=3.3.0 - Adjust importlib-mtadata for Python < 3.8 - Test flake8 and hacking for the lowest and highest available Python Bug: T323799 Bug: T323752 Change-Id: Ie00f8d0c8ec19b1174b880b1e7024b2bd0342520 --- M setup.py M tox.ini 2 files changed, 31 insertions(+), 5 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/setup.py b/setup.py index f6ecef0..aac66f2 100755 --- a/setup.py +++ b/setup.py @@ -62,7 +62,9 @@ 'flake8-docstrings>=1.3.1', 'verve-flake8-mock>=0.4', 'flake8-print>=2.0.1', - 'flake8-quotes>=2.0.1', + # flake8-quotes is incompatible with flake8 6 (T323752), + # jenkins CI ignores "3.8.1" + 'flake8-quotes>=3.3.0; python_version < "3.8"', 'flake8-string-format', 'flake8-tuple>=0.2.8', 'flake8-no-u-prefixed-strings>=0.2', @@ -72,8 +74,8 @@ 'hacking': [ 'hacking', # importlib-metadata module already installed with hacking 4.1.0 - # but importlib-metadata 5.0.0 fails, so adjust it - 'importlib-metadata!=5.0.0', + # but importlib-metadata 5 fails, so adjust it + 'importlib-metadata<5.0.0; python_version < "3.8"', ], } diff --git a/tox.ini b/tox.ini index 69ebd75..4f85c2b 100644 --- a/tox.ini +++ b/tox.ini @@ -5,8 +5,8 @@ skip_missing_interpreters = True envlist = commit-message - flake8-{py39,py38,py36} - hacking-py37 + flake8-{py36,py39} + hacking-{py36,py39} [params] # gui needs tkinter @@ -28,6 +28,7 @@ PYWIKIBOT_TEST_RUNNING=1 usedevelop = True commands = + python --version flake8: flake8 --version flake8-py36: flake8 --doctests {posargs} {[params]exclude} flake8-py38: flake8 --doctests {posargs} -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860845 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: tests Gerrit-Change-Id: Ie00f8d0c8ec19b1174b880b1e7024b2bd0342520 Gerrit-Change-Number: 860845 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: Fix usage hint in a regex comment
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860100 ) Change subject: Fix usage hint in a regex comment ...................................................................... Fix usage hint in a regex comment Change-Id: If6c748a3e67dc96d5a19c414c5ba592c496dce1a --- M pywikibot/textlib.py 1 file changed, 11 insertions(+), 1 deletion(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py index e8e0af3..d7e2366 100644 --- a/pywikibot/textlib.py +++ b/pywikibot/textlib.py @@ -86,7 +86,8 @@ # and correctly matches the end of the file link if the wikilink contains # [[ or ]]. # The namespace names must be substituted into this regex. -# e.g. FILE_LINK_REGEX % 'File' or FILE_LINK_REGEX % '|'.join(site.namespaces) +# e.g. FILE_LINK_REGEX % 'File' +# or FILE_LINK_REGEX % '|'.join(site.namespaces[6]) FILE_LINK_REGEX = r""" \[\[\s* (?:%s) # namespace aliases -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860100 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: If6c748a3e67dc96d5a19c414c5ba592c496dce1a Gerrit-Change-Number: 860100 Gerrit-PatchSet: 2 Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [tests] Update flake tests on jenkins CI
by jenkins-bot (Code Review) 25 Nov '22

25 Nov '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860479 ) Change subject: [tests] Update flake tests on jenkins CI ...................................................................... [tests] Update flake tests on jenkins CI several flake8 modules fails with new flake8 6.0.0 which needs Python 3.8.1. Jenkins CI does not reflect the right Python version. Therefore only add quotes check for Python < 3.8 which uses flake8 5.0.4. - Use flake8-quotes with Python < 3.8 only - Adjust flake8-quotes to >=3.3.0 - Adjust importlib-mtadata for Python < 3.8 - Test flake8 and hacking for the lowest and highest available Python Bug: T323799 Bug: T323752 Change-Id: Ia97370f5f0ca98261e04a0e6a8315feaf61400ca --- M setup.py M tox.ini 2 files changed, 29 insertions(+), 5 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/setup.py b/setup.py index f6ecef0..aac66f2 100755 --- a/setup.py +++ b/setup.py @@ -62,7 +62,9 @@ 'flake8-docstrings>=1.3.1', 'verve-flake8-mock>=0.4', 'flake8-print>=2.0.1', - 'flake8-quotes>=2.0.1', + # flake8-quotes is incompatible with flake8 6 (T323752), + # jenkins CI ignores "3.8.1" + 'flake8-quotes>=3.3.0; python_version < "3.8"', 'flake8-string-format', 'flake8-tuple>=0.2.8', 'flake8-no-u-prefixed-strings>=0.2', @@ -72,8 +74,8 @@ 'hacking': [ 'hacking', # importlib-metadata module already installed with hacking 4.1.0 - # but importlib-metadata 5.0.0 fails, so adjust it - 'importlib-metadata!=5.0.0', + # but importlib-metadata 5 fails, so adjust it + 'importlib-metadata<5.0.0; python_version < "3.8"', ], } diff --git a/tox.ini b/tox.ini index 69ebd75..4f85c2b 100644 --- a/tox.ini +++ b/tox.ini @@ -5,8 +5,8 @@ skip_missing_interpreters = True envlist = commit-message - flake8-{py39,py38,py36} - hacking-py37 + flake8-{py36,py39} + hacking-{py36,py39} [params] # gui needs tkinter @@ -28,6 +28,7 @@ PYWIKIBOT_TEST_RUNNING=1 usedevelop = True commands = + python --version flake8: flake8 --version flake8-py36: flake8 --doctests {posargs} {[params]exclude} flake8-py38: flake8 --doctests {posargs} -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/860479 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ia97370f5f0ca98261e04a0e6a8315feaf61400ca Gerrit-Change-Number: 860479 Gerrit-PatchSet: 11 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-CC: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-MessageType: merged

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits