jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703030 )
Change subject: [doc] Start with a recipes page
......................................................................
[doc] Start with a recipes page
Change-Id: Ic31e45ffb20df94bb412f09cb11ba3695fa8deef
---
M docs/index.rst
A docs/recipes.rst
2 files changed, 25 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/index.rst b/docs/index.rst
index a428078..7bc1f1b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -57,6 +57,7 @@
installation
library_usage
+ recipes
getting_help
api_ref/index
diff --git a/docs/recipes.rst b/docs/recipes.rst
new file mode 100644
index 0000000..2ee6ce9
--- /dev/null
+++ b/docs/recipes.rst
@@ -0,0 +1,24 @@
+Recipes
+=======
+
+Page content
+------------
+
+**How to modify a page**
+ >>> page = pywikibot.Page(pywikibot.Site(), 'Sample page')
+ >>> new_content = page.text.replace('this', 'that')
+ >>> page.put(new_content, summary='Bot: Test edit')
+
+ See :doc:`library_usage` for more advanced samples.
+
+**How to get links from summary section of page**
+ >>> import pywikibot
+ >>> from pwikibot import textlib
+ >>> site = pywikibot.Site('wikipedia:en') # create a Site object
+ >>> page = pywikibot.Page(site, 'Deep learning') # create a Page object
+ >>> sect = textlib.extract_sections(page.text, site) # divide content into sections
+ >>> links = sorted(link.group('title') for link in pywikibot.link_regex.finditer(sect.head))
+ >>> pages = [pywikibot.Page(site, title) for title in links]
+
+ ``links`` is a list containing all link titles in alphabethical order.
+ ``pages`` is a sorted list containing all ``Page`` objects
\ No newline at end of file
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703030
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ic31e45ffb20df94bb412f09cb11ba3695fa8deef
Gerrit-Change-Number: 703030
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703029 )
Change subject: [doc] start a faq page with sphinx
......................................................................
[doc] start a faq page with sphinx
Change-Id: I9830e195e4642cd8b6ae41219141ba25e4fa62eb
---
A docs/faq.rst
M docs/index.rst
2 files changed, 43 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/faq.rst b/docs/faq.rst
new file mode 100644
index 0000000..04f0311
--- /dev/null
+++ b/docs/faq.rst
@@ -0,0 +1,42 @@
+Frequent Asked Questions
+========================
+**How to speed up Pywikibot?**
+ 1. The first time you are using Pywikibot for multiple Wikimedia sites you
+ can run :py:mod:`preload_sites <scripts.maintenance.preload_sites>` script
+ to preload site info quickly.
+ 2. If you need the content, use :py:mod:`PreloadingGenerator
+ <pywikibot.pagegenerators.PreloadingGenerator>` with page generators,
+ :py:mod:`EntityGenerator <pywikibot.pagegenerators.EntityGenerator>`
+ for wikibase entities and :py:mod:`DequePreloadingGenerator
+ <pywikibot.pagegenerators.DequePreloadingGenerator>` for a
+ :py:mod:`DequeGenerator <pywikibot.tools.DequeGenerator>`.
+ 3. If you use :py:mod:`GeneratorFactory
+ <pywikibot.pagegenerators.GeneratorFactory>` with your bot and use its
+ :py:mod:`getCombinedGenerator
+ <pywikibot.pagegenerators.GeneratorFactory.getCombinedGenerator>` method
+ you can set ``preload=True`` to preload page content. This is an alternate
+ to the ``PreloadingGenerator`` function mentioned above.
+ 4. Use :py:mod:`MySQLPageGenerator
+ <pywikibot.pagegenerators.MySQLPageGenerator >` if direct DB access is
+ available and appropriate. See also: `MySQL`_
+
+**The bot cannot delete pages**
+ Your account needs delete rights on your wiki. If you have setup another
+ account in your user_config use ``-user``
+ :ref:`global option <global options>` to change it.
+ Maybe you have to login first.
+
+**ERROR: Unable to execute script because no *generator* was defined.**
+ Using ``-help`` option is a good way to find all generators which can be
+ used with that script. You can also find all generator options and filter
+ options at :py:mod:`pywikibot.pagegenerators` module.
+ See also `Page Generators`_ MediaWiki page for additional information.
+
+**pywikibot.i18n.TranslationError: No English translation has been defined**
+ It can happen due to lack of i18n submodule or files. Update i18n submodule
+ or download these files first. See also: `i18n submodule`_
+
+
+.. _i18n submodule: https://www.mediawiki.org/wiki/Manual:Pywikibot/i18n
+.. _MySQL: https://www.mediawiki.org/wiki/Manual:Pywikibot/MySQL
+.. _Page Generators: https://www.mediawiki.org/wiki/Manual:Pywikibot/Page_Generators
diff --git a/docs/index.rst b/docs/index.rst
index a428078..581e23f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -46,6 +46,7 @@
utilities/index
scripts/index
global_options
+ faq
getting_help
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703029
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I9830e195e4642cd8b6ae41219141ba25e4fa62eb
Gerrit-Change-Number: 703029
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703883 )
Change subject: [doc] Add a note about T286348 to pywikibot.bot module
......................................................................
[doc] Add a note about T286348 to pywikibot.bot module
Bug: T286348
Change-Id: I06cb69b60f90d35b1084cba009daa89eb9643b76
---
M pywikibot/bot.py
1 file changed, 6 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 06b2574..b261d09 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -2261,3 +2261,9 @@
set_interface(config.userinterface)
+
+# NOTE: (T286348)
+# Do not use ModuleDeprecationWrapper with this module.
+# pywikibot.bot.ui would be wrapped through the ModuleDeprecationWrapper
+# and a cannot be changed later. Use another depecation method instead
+# (until T286348 has been solved somehow different).
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703883
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I06cb69b60f90d35b1084cba009daa89eb9643b76
Gerrit-Change-Number: 703883
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703837 )
Change subject: [doc] Add manpage reference
......................................................................
[doc] Add manpage reference
Change-Id: I10282ab172b63f11d4f0d254af9a4d34eea38c19
---
M docs/conf.py
M docs/getting_help.rst
M docs/glossary.rst
M docs/installation.rst
M docs/library_usage.rst
5 files changed, 5 insertions(+), 6 deletions(-)
Approvals:
JJMC89: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/conf.py b/docs/conf.py
index 6a10812..d4430c2 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -318,6 +318,7 @@
['Pywikibot team'], 1)
]
+manpages_url = 'https://www.mediawiki.org/wiki/Manual:Pywikibot/{path}'
# If true, show URL addresses after external links.
#
# man_show_urls = False
diff --git a/docs/getting_help.rst b/docs/getting_help.rst
index 5a062b1..208af04 100644
--- a/docs/getting_help.rst
+++ b/docs/getting_help.rst
@@ -2,7 +2,7 @@
------------
.. note::
- Please see `Manual:Pywikibot/Communication <https://www.mediawiki.org/wiki/Manual:Pywikibot/Communication>`_.
+ Please see :manpage:`Communication`.
.. tip::
Please report bugs at `Phabricator <https://phabricator.wikimedia.org/>`_.
diff --git a/docs/glossary.rst b/docs/glossary.rst
index 2de688d..b6290d1 100644
--- a/docs/glossary.rst
+++ b/docs/glossary.rst
@@ -67,8 +67,8 @@
is a Jupyter notebooks deployment hosted by Wikimedia. It has
preinstalled the :term:`stable` release of Pywikibot. Refer:
+ - :manpage:`PAWS`
- https://wikitech.wikimedia.org/wiki/PAWS
- - https://www.mediawiki.org/wiki/Manual:Pywikibot/PAWS
PyPI
The Python Package Index (PyPI), a repository of software for the
diff --git a/docs/installation.rst b/docs/installation.rst
index 74ba2f5..0f1f34e 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -18,5 +18,4 @@
(not available with site-package) or refer :mod:`pywikibot.config`.
.. note::
- Please also see the documentation at
- `Manual:Pywikibot/Installation <https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation>`_
+ Please also see the documentation at :manpage:`Installation`
diff --git a/docs/library_usage.rst b/docs/library_usage.rst
index 95da419..79d5c25 100644
--- a/docs/library_usage.rst
+++ b/docs/library_usage.rst
@@ -84,5 +84,4 @@
:language: python
.. note::
- Please also see the documentation at
- `Manual:Pywikibot/Create your own script <https://www.mediawiki.org/wiki/Manual:Pywikibot/Create_your_own_script>`_
+ Please also see the documentation at :manpage:`Create your own script`
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703837
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I10282ab172b63f11d4f0d254af9a4d34eea38c19
Gerrit-Change-Number: 703837
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703859 )
Change subject: [bugfix] Don't ignore identical references with newline in ref content
......................................................................
[bugfix] Don't ignore identical references with newline in ref content
If a line feed was inside the ref content the processing for this
reference was skipped because the regex wasn't able to detect this
reference. Fix this issue with following steps:
- use DOTALL flag with self.REFS regex
- remove catchall start and endpattern of self.NAMES and self.GROUPS
regex and use re.search instead of re.match later
- use a set for found_ref_names instead a dict. The only value assigned
to the dict was 1.
- don't use None as found_refs key but an empty list
- introduce an IntEnum for named index constants for the reference
values; this improves readability
- remove debugging stuff of 'population' name
- rename variables for better readability
- add some comments
Bug: T286369
Change-Id: I47f81dcc9537591efea7bbb4628c55af0789cc14
---
M scripts/reflinks.py
1 file changed, 48 insertions(+), 38 deletions(-)
Approvals:
Rubin: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/reflinks.py b/scripts/reflinks.py
index 0fd5da7..c753dc0 100755
--- a/scripts/reflinks.py
+++ b/scripts/reflinks.py
@@ -49,6 +49,7 @@
import tempfile
from contextlib import suppress
+from enum import IntEnum
from functools import partial
from http import HTTPStatus
from textwrap import shorten
@@ -274,6 +275,16 @@
self.title = self.title.title()
+class IX(IntEnum):
+
+ """Index class for references data."""
+
+ name = 0
+ reflist = 1
+ quoted = 2
+ change_needed = 3
+
+
class DuplicateReferences:
"""Helper to de-duplicate references in text.
@@ -289,8 +300,8 @@
# Match references
self.REFS = re.compile(
- r'(?i)<ref(?P<params>[^>/]*)>(?P<content>.*?)</ref>')
- fmt = r'(?i).*{0}\s*=\s*(?P<quote>["\']?)\s*(?P<{0}>.+)\s*(?P=quote).*'
+ r'(?is)<ref(?P<params>[^>/]*)>(?P<content>.*?)</ref>')
+ fmt = r'(?i){0}\s*=\s*(?P<quote>["\']?)\s*(?P<{0}>.+)\s*(?P=quote)'
self.NAMES = re.compile(fmt.format('name'))
self.GROUPS = re.compile(fmt.format('group'))
self.autogen = i18n.twtranslate(site, 'reflinks-autogen')
@@ -303,101 +314,100 @@
# values are [name, [list of full ref matches],
# quoted, need_to_change]
found_refs = {}
- found_ref_names = {}
+ found_ref_names = set()
# Replace key by [value, quoted]
named_repl = {}
+ # Parse references
for match in self.REFS.finditer(text):
content = match.group('content')
if not content.strip():
continue
params = match.group('params')
- group = self.GROUPS.match(params)
+ group = self.GROUPS.search(params) or ''
if group not in found_refs:
found_refs[group] = {}
groupdict = found_refs[group]
if content in groupdict:
v = groupdict[content]
- v[1].append(match.group())
+ v[IX.reflist].append(match.group())
else:
v = [None, [match.group()], False, False]
- found = self.NAMES.match(params)
+ found = self.NAMES.search(params)
if found:
quoted = found.group('quote') in ['"', "'"]
name = found.group('name')
- if v[0]:
- if v[0] != name:
- named_repl[name] = [v[0], v[2]]
- else:
+
+ if not v[IX.name]:
# First name associated with this content
- if name == 'population':
- pywikibot.output(content)
if name not in found_ref_names:
# first time ever we meet this name
- if name == 'population':
- pywikibot.output('in')
- v[2] = quoted
- v[0] = name
+ v[IX.quoted] = quoted
+ v[IX.name] = name
else:
# if has_key, means that this name is used
# with another content. We'll need to change it
- v[3] = True
- found_ref_names[name] = 1
+ v[IX.change_needed] = True
+ elif v[IX.name] != name:
+ named_repl[name] = [v[IX.name], v[IX.quoted]]
+
+ found_ref_names.add(name)
groupdict[content] = v
+ # Find used autogenerated numbers
used_numbers = set()
for name in found_ref_names:
number = removeprefix(name, self.autogen)
with suppress(ValueError):
used_numbers.add(int(number))
- # generator to give the next free number
+ # generator to give the next free number for autogenerating names
free_number = (str(i) for i in itertools.count(start=1)
if i not in used_numbers)
- for (g, d) in found_refs.items():
- group = ''
- if g:
- group = 'group="{}" '.format(group)
+ # Fix references
+ for groupname, references in found_refs.items():
+ group = 'group="{}" '.format(group) if groupname else ''
- for (k, v) in d.items():
- if len(v[1]) == 1 and not v[3]:
+ for ref, v in references.items():
+ if len(v[IX.reflist]) == 1 and not v[IX.change_needed]:
continue
- name = v[0]
+ name = v[IX.name]
if not name:
name = '"{}{}"'.format(self.autogen, next(free_number))
- elif v[2]:
+ elif v[IX.quoted]:
name = '"{}"'.format(name)
- named = '<ref {}name={}>{}</ref>'.format(group, name, k)
- text = text.replace(v[1][0], named, 1)
+ named = '<ref {}name={}>{}</ref>'.format(group, name, ref)
+ text = text.replace(v[IX.reflist][0], named, 1)
- # make sure that the first (named ref) is not
- # removed later :
+ # make sure that the first (named ref) is not removed later
pos = text.index(named) + len(named)
header = text[:pos]
end = text[pos:]
- unnamed = '<ref {}name={} />'.format(group, name)
- for ref in v[1][1:]:
+ # replace multiple identical references with repeated ref
+ repeated_ref = '<ref {}name={} />'.format(group, name)
+ for ref in v[IX.reflist][1:]:
# Don't replace inside templates (T266411)
- end = replaceExcept(end, re.escape(ref), unnamed,
+ end = replaceExcept(end, re.escape(ref), repeated_ref,
exceptions=['template'])
text = header + end
- for (k, v) in named_repl.items():
+ # Fix references with different names
+ for ref, v in named_repl.items():
# TODO : Support ref groups
- name = v[0]
- if v[1]:
+ name = v[IX.name]
+ if v[IX.reflist]:
name = '"{}"'.format(name)
text = re.sub(
r'<ref name\s*=\s*(?P<quote>["\']?)\s*{}\s*(?P=quote)\s*/>'
- .format(k),
+ .format(ref),
'<ref name={} />'.format(name), text)
return text
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703859
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I47f81dcc9537591efea7bbb4628c55af0789cc14
Gerrit-Change-Number: 703859
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Rubin <rubin.happy(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703655 )
Change subject: [IMPR] improving work with deadlinks for ru.wp
......................................................................
[IMPR] improving work with deadlinks for ru.wp
Current template "Dead" doesn't support working with substitution,
I have created a different template that would work and changed reference here.
Change-Id: Ife4bfe8cdd5aa094147bce3e9276ec3dc29529a9
---
M scripts/reflinks.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/reflinks.py b/scripts/reflinks.py
index fd599a4..0fd5da7 100755
--- a/scripts/reflinks.py
+++ b/scripts/reflinks.py
@@ -111,7 +111,7 @@
'it': '{{Collegamento interrotto|%s}}',
'en': '[%s] {{dead link}}',
'pl': '[%s] {{Martwy link}}',
- 'ru': '[%s] {{subst:dead}}',
+ 'ru': '[%s] {{subst:deadlink2}}',
'sr': '[%s] {{dead link}}',
'ur': '[%s] {{مردہ ربط}}',
}
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/703655
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ife4bfe8cdd5aa094147bce3e9276ec3dc29529a9
Gerrit-Change-Number: 703655
Gerrit-PatchSet: 2
Gerrit-Owner: Rubin <rubin.happy(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged