Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article has been deleted from Wikipedia. Specifically, I am getting a list of articles from the cxtranslation list and I would like to know which articles are no longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API or some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to get articles and the API returns missing for some articles, but this does not seem to be fully accurate for determining if an article has been deleted because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition and article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not, "missing" should be accurate. Can you share some example URLs for which the page exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article has been deleted from Wikipedia. Specifically, I am getting a list of articles from the cxtranslation list and I would like to know which articles are no longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API or some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to get articles and the API returns missing for some articles, but this does not seem to be fully accurate for determining if an article has been deleted because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition and article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Hi Doris,
If you look at the links for some examples you may find that much of the difference is articles that are redirected. Sometimes redirected and merged.
Non notable fictional characters are usually redirected to the article on the film or book they come from. If you look at the history of the redirect you may well find an article.
WSC
On Wed, 27 Oct 2021 at 15:50, Adam Wight adam.wight@wikimedia.de wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not, "missing" should be accurate. Can you share some example URLs for which the page exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API
or
some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to
get
articles and the API returns missing for some articles, but this does not seem to be fully accurate for determining if an article has been deleted because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition and article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article but the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n is here.
Would you know if there is a way to input article revision ID or pageid instead of source title for the logevents API? The strings seem to be problematic at times. For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the cxtranslation list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and perhaps I am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
------------------------------ endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not, "missing" should be accurate. Can you share some example URLs for which the page exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API
or
some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to
get
articles and the API returns missing for some articles, but this does not seem to be fully accurate for determining if an article has been deleted because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition and article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article but the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n is here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe something in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or pageid instead of source title for the logevents API? The strings seem to be problematic at times.
This was prescient :-). But I don't see any record of the article being deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire log history, then process it locally to filter by page ID.
Help page: https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the cxtranslation list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and perhaps I am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not, "missing" should be accurate. Can you share some example URLs for which the page exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API
or
some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to
get
articles and the API returns missing for some articles, but this does not seem to be fully accurate for determining if an article has been deleted because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition and article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Hi Adam,
It seems that the issue is the apostrophe after "L", in the wikidata
query it is "´" and the wikipedia link above uses "'".
I see, a lof of the issues were caused by my string mishandling.
One approach you might consider is to download the entire log
history, then process it locally to filter by page ID.
I am still unclear on how to know definitely for sure that an article was deleted. It seems like the only way is to tell through the comments. For example, this call: https://en.wikipedia.org/w/api.php?action=query&list=logevents&leact... shows the comment "[[Wikipedia:Articles for deletion/Louis Tomlinson]]" which I have noticed to exist for other articles that were successfully deleted, but the article "Zayn Malik" exists. The most recent event has the comment "[[WP:CSD#G6|G6]]: Deleted to make way for move" which would imply the other deletions weren't successful but the article still exists.
Thanks,
Doris
On Thu, Nov 4, 2021 at 3:20 AM Adam Wight adam.wight@wikimedia.de wrote:
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article but the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n is here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe something in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or pageid instead of source title for the logevents API? The strings seem to be problematic at times.
This was prescient :-). But I don't see any record of the article being deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F... https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+Flores+%C3%81lvarez&wpdate=&tagfilter=
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire log history, then process it locally to filter by page ID.
Help page: https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the
cxtranslation
list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and perhaps
I
am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de
wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not, "missing" should be accurate. Can you share some example URLs for which the page exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an API
or
some way to access something like this form so I could check if a mass amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php to
get
articles and the API returns missing for some articles, but this does
not
seem to be fully accurate for determining if an article has been
deleted
because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition
and
article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Hi Doris,
as you can see here https://en.wikipedia.org/w/index.php?title=Special:Log&page=Zayn+Malik, the article was actually deleted multiple times. But it was also restored and recreated.
It was definitely deleted, but: * Deletion is not always permanent. Articles can be restored (e.g. after revisiting a decision). * Deletion does not necessarily prohibit recreation. The reasons for deletion may not apply to future version (for example, notability for living persons may change) * Deletion does not necessarily mean it was a 'bad article': sometimes an article was created twice and needs to be merged, or a redirect is being deleted in order to make place for the 'real article'. * As WSC mentioned, a removal of content is also possible through other means than a 'deletion' (e.g. replace the article with a redirect).
It depends a bit on your desired definition of 'deleted' how to approach this.
Best, Lodewijk
On Fri, Nov 5, 2021 at 11:11 AM D Z doriszhou1224@gmail.com wrote:
Hi Adam,
It seems that the issue is the apostrophe after "L", in the wikidata
query it is "´" and the wikipedia link above uses "'".
I see, a lof of the issues were caused by my string mishandling.
One approach you might consider is to download the entire log
history, then process it locally to filter by page ID.
I am still unclear on how to know definitely for sure that an article was deleted. It seems like the only way is to tell through the comments. For example, this call:
https://en.wikipedia.org/w/api.php?action=query&list=logevents&leact... shows the comment "[[Wikipedia:Articles for deletion/Louis Tomlinson]]" which I have noticed to exist for other articles that were successfully deleted, but the article "Zayn Malik" exists. The most recent event has the comment "[[WP:CSD#G6|G6]]: Deleted to make way for move" which would imply the other deletions weren't successful but the article still exists.
Thanks,
Doris
On Thu, Nov 4, 2021 at 3:20 AM Adam Wight adam.wight@wikimedia.de wrote:
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article
but
the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n
is
here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe something in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or pageid instead of source title for the logevents API? The strings seem to be problematic at times.
This was prescient :-). But I don't see any record of the article being deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F... https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+Flores+%C3%81lvarez&wpdate=&tagfilter=
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire log history, then process it locally to filter by page ID.
Help page:
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the
cxtranslation
list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and
perhaps
I
am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de
wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not,
"missing"
should be accurate. Can you share some example URLs for which the
page
exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an
article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles
are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an
API
or
some way to access something like this form so I could check if a
mass
amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php
to
get
articles and the API returns missing for some articles, but this does
not
seem to be fully accurate for determining if an article has been
deleted
because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition
and
article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Going back to your original question,
which articles are no longer on Wikipedia
This is easy enough to query in bulk:
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvpro...
The first page exists, but the other two never existed and were deleted, respectively. Both missing articles have a "missing" key in the response data, which you can rely on for determining if the articles exist.
It sounds like this is what you needed, and maybe the inconsistencies were due to non-Latin character encoding issues? Let me know if I misunderstood, and you also need to know whether the page used to exist but was deleted.
Regards, Adam W.
On Fri, Nov 5, 2021 at 7:12 PM D Z doriszhou1224@gmail.com wrote:
I am still unclear on how to know definitely for sure that an article was deleted. It seems like the only way is to tell through the comments. For example, this call:
https://en.wikipedia.org/w/api.php?action=query&list=logevents&leact... shows the comment "[[Wikipedia:Articles for deletion/Louis Tomlinson]]" which I have noticed to exist for other articles that were successfully deleted, but the article "Zayn Malik" exists. The most recent event has the comment "[[WP:CSD#G6|G6]]: Deleted to make way for move" which would imply the other deletions weren't successful but the article still exists.
Thanks,
Doris
On Thu, Nov 4, 2021 at 3:20 AM Adam Wight adam.wight@wikimedia.de wrote:
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article
but
the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n
is
here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe something in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or pageid instead of source title for the logevents API? The strings seem to be problematic at times.
This was prescient :-). But I don't see any record of the article being deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F... https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+Flores+%C3%81lvarez&wpdate=&tagfilter=
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire log history, then process it locally to filter by page ID.
Help page:
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the
cxtranslation
list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and
perhaps
I
am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de
wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not,
"missing"
should be accurate. Can you share some example URLs for which the
page
exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation in Wikipedia articles. I am having trouble with how to know if an
article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles
are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an
API
or
some way to access something like this form so I could check if a
mass
amount of articles have been deleted?
I have used the Media Wiki API https://en.wikipedia.org/w/api.php
to
get
articles and the API returns missing for some articles, but this does
not
seem to be fully accurate for determining if an article has been
deleted
because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language edition
and
article title, or an article pageid, is there an API to check if the article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Just to add a little further complexity.
Lots of articles and deleted articles are about people. Names are often not unique, and just because one person with a particular name has had an article on them deleted it does not mean that there won't be a notable person of the same name.
For example I was once asked to restore a particular deleted article so that someone could look at the deleted version before creating an article on a professor who they assured me was very notable and they had plenty of sources for. I had a look at the deleted article, and told them I doubted there was anything there worth restoring, and to go ahead with the article on the professor. I also added that I didn't know if the deleted article was about the same person or a different person of the same name, but if they found that their professor had been a pro skateboarder in his teens, I suggested they give that its own section, and not make that his main claim to notability or have it dominate the lede. In another instance I resolved an edit war over whether an article should be about either of two people of the same name by deleting the article, restoring all the versions that were about person A and moving them to a new clearer name, then restoring the other revisions and moving them to a page with a name that made it clear they were about person B, then I turned the original battleground article into a disambiguation page that listed both people. That would be a rare situation compared to redirects, but I hope it gives you an idea of the complexity of Wikipedia article names over time.
When there are multiple topics with the same name the default should be that the primary one gets the name with the secondary topics getting longer names and a mention in a disambiguation page. There are people who get very concerned as to which if any article should be primary, and while sometimes that is as obvious as Dallas, Scotland v Dallas, Texas, other times that can be contentious and even be changed over time. I can remember heated arguments about Perth Scotland v Perth Australia and I dread to think how the Mercury, Atlas and Apollo decisions were made.
TLDR Names of articles don't just go through a process of deletion
WSC
On Fri, 5 Nov 2021 at 18:30, Adam Wight adam.wight@wikimedia.de wrote:
Going back to your original question,
which articles are no longer on Wikipedia
This is easy enough to query in bulk:
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvpro... https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=ids&titles=Zayn%20Malik%7CNonexistentPage%7CDraft:Kajl%C3%A2ngvoj
The first page exists, but the other two never existed and were deleted, respectively. Both missing articles have a "missing" key in the response data, which you can rely on for determining if the articles exist.
It sounds like this is what you needed, and maybe the inconsistencies were due to non-Latin character encoding issues? Let me know if I misunderstood, and you also need to know whether the page used to exist but was deleted.
Regards, Adam W.
On Fri, Nov 5, 2021 at 7:12 PM D Z doriszhou1224@gmail.com wrote:
I am still unclear on how to know definitely for sure that an article was deleted. It seems like the only way is to tell through the comments. For example, this call:
https://en.wikipedia.org/w/api.php?action=query&list=logevents&leact...
shows the comment "[[Wikipedia:Articles for deletion/Louis Tomlinson]]" which I have noticed to exist for other articles that were successfully deleted, but the article "Zayn Malik" exists. The most recent event has the comment "[[WP:CSD#G6|G6]]: Deleted to make way for move" which would imply the other deletions weren't successful but the article still exists.
Thanks,
Doris
On Thu, Nov 4, 2021 at 3:20 AM Adam Wight adam.wight@wikimedia.de
wrote:
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this article
but
the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n
is
here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe something in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or
pageid
instead of source title for the logevents API? The strings seem to be problematic at times.
This was prescient :-). But I don't see any record of the article
being
deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F... https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+Flores+%C3%81lvarez&wpdate=&tagfilter=
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire log history, then process it locally to filter by page ID.
Help page:
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the
cxtranslation
list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and
perhaps
I
am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight adam.wight@wikimedia.de
wrote:
The "logevents" API should return the same data as Special:Log. For example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not,
"missing"
should be accurate. Can you share some example URLs for which the
page
exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote:
Hello All,
I am doing research investigating the role of machine translation
in
Wikipedia articles. I am having trouble with how to know if an
article
has
been deleted from Wikipedia. Specifically, I am getting a list of
articles
from the cxtranslation list and I would like to know which articles
are
no
longer on Wikipedia. I see that there is the deletion log form https://en.wikipedia.org/wiki/Special:Log/delete but is there an
API
or
some way to access something like this form so I could check if a
mass
amount of articles have been deleted?
I have used the Media Wiki API <https://en.wikipedia.org/w/api.php
to
get
articles and the API returns missing for some articles, but this
does
not
seem to be fully accurate for determining if an article has been
deleted
because the API has returned 'missing' for articles that do exist.
To summarize, my main question is: given an article language
edition
and
article title, or an article pageid, is there an API to check if
the
article has been deleted?
Any help would be greatly appreciated!
Thanks,
Doris Zhou _______________________________________________ Wiki-research-l mailing list --
wiki-research-l@lists.wikimedia.org
To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
-- Adam Wight - Developer - Wikimedia Deutschland e.V. - https://wikimedia.de _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
Hi Everyone,
Thank you very much for all the replies and insight. Thanks WSC for sharing your experience. I've been using the call https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvpro... https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=ids&titles=Zayn%20Malik%7CNonexistentPage%7CDraft:Kajl%C3%A2ngvoj for a while now to get articles and always wondered what 'missing' meant exactly. Great to have confirmation it means deleted/gone.
Thanks again and have a great weekend!
Sincerely, Doris Zhou
On Mon, Nov 8, 2021 at 8:00 AM WereSpielChequers < werespielchequers@gmail.com> wrote:
Just to add a little further complexity.
Lots of articles and deleted articles are about people. Names are often not unique, and just because one person with a particular name has had an article on them deleted it does not mean that there won't be a notable person of the same name.
For example I was once asked to restore a particular deleted article so that someone could look at the deleted version before creating an article on a professor who they assured me was very notable and they had plenty of sources for. I had a look at the deleted article, and told them I doubted there was anything there worth restoring, and to go ahead with the article on the professor. I also added that I didn't know if the deleted article was about the same person or a different person of the same name, but if they found that their professor had been a pro skateboarder in his teens, I suggested they give that its own section, and not make that his main claim to notability or have it dominate the lede. In another instance I resolved an edit war over whether an article should be about either of two people of the same name by deleting the article, restoring all the versions that were about person A and moving them to a new clearer name, then restoring the other revisions and moving them to a page with a name that made it clear they were about person B, then I turned the original battleground article into a disambiguation page that listed both people. That would be a rare situation compared to redirects, but I hope it gives you an idea of the complexity of Wikipedia article names over time.
When there are multiple topics with the same name the default should be that the primary one gets the name with the secondary topics getting longer names and a mention in a disambiguation page. There are people who get very concerned as to which if any article should be primary, and while sometimes that is as obvious as Dallas, Scotland v Dallas, Texas, other times that can be contentious and even be changed over time. I can remember heated arguments about Perth Scotland v Perth Australia and I dread to think how the Mercury, Atlas and Apollo decisions were made.
TLDR Names of articles don't just go through a process of deletion
WSC
On Fri, 5 Nov 2021 at 18:30, Adam Wight adam.wight@wikimedia.de wrote:
Going back to your original question,
which articles are no longer on Wikipedia
This is easy enough to query in bulk:
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvpro... https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=ids&titles=Zayn%20Malik%7CNonexistentPage%7CDraft:Kajl%C3%A2ngvoj
<
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvpro...
The first page exists, but the other two never existed and were deleted, respectively. Both missing articles have a "missing" key in the response data, which you can rely on for determining if the articles exist.
It sounds like this is what you needed, and maybe the inconsistencies
were
due to non-Latin character encoding issues? Let me know if I misunderstood, and you also need to know whether the page used to exist
but
was deleted.
Regards, Adam W.
On Fri, Nov 5, 2021 at 7:12 PM D Z doriszhou1224@gmail.com wrote:
I am still unclear on how to know definitely for sure that an article
was
deleted. It seems like the only way is to tell through the comments.
For
example, this call:
https://en.wikipedia.org/w/api.php?action=query&list=logevents&leact...
shows the comment "[[Wikipedia:Articles for deletion/Louis Tomlinson]]" which I have noticed to exist for other articles that were successfully deleted, but the article "Zayn Malik" exists. The most recent event
has
the comment "[[WP:CSD#G6|G6]]: Deleted to make way for move" which would imply the other deletions weren't successful but the article still exists.
Thanks,
Doris
On Thu, Nov 4, 2021 at 3:20 AM Adam Wight adam.wight@wikimedia.de
wrote:
On 11/4/21 8:09 AM, D Z wrote:
Hi Adam,
Thanks for your reply. The qitem api returns missing for this
article
but
the article exists:
https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&...
The Wikipedia page link <
https://es.wikipedia.org/wiki/Playas_de_L%27Atalaya_y_Focar%C3%B3n%3E
is
here.
It seems that the issue is the apostrophe after "L", in the wikidata query it is "´" and the wikipedia link above uses "'". Maybe
something
in your query script is normalizing the fancy apostrophe to a simple one? I would check for proper UTF-8 handling.
Would you know if there is a way to input article revision ID or
pageid
instead of source title for the logevents API? The strings seem to
be
problematic at times.
This was prescient :-). But I don't see any record of the article
being
deleted, so perhaps the API is correct in this case?
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F... https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+Flores+%C3%81lvarez&wpdate=&tagfilter=
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
<
https://pt.wikipedia.org/wiki/Special:Log?type=&user=&page=Rodrigo+F...
Unfortunately, the API help page doesn't mention filtering the log by page ID. One approach you might consider is to download the entire
log
history, then process it locally to filter by page ID.
Help page:
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Blogevent...
Regards, Adam W. [[mw:User:Adamw]
For example, the article 'Rodrigo Flores Álvarez' of 'pt' Wikipedia gives me trouble (I got this article from the
cxtranslation
list). This page seems to be missing https://pt.wikipedia.org/wiki/Rodrigo_Flores_%C3%81lvarez and
perhaps
I
am not using the logevents API correctly, but it returns empty.
{'batchcomplete': '', 'query': {'logevents': []}}
endpoint = str('pt') + '.wikipedia.org/w/api.php' query_url = "https://%7B0%7D%22.format(endpoint) params = {} params['action'] = 'query' params['list'] = 'logevents' params['format'] = 'json' params['leaction'] = 'delete/delete' params['letitle'] = 'Rodrigo Flores Álvarez' json_response = requests.get(url=query_url, params=params).json()
Thanks again and cheers,
Doris Zhou
On Wed, Oct 27, 2021 at 9:51 AM Adam Wight <
adam.wight@wikimedia.de>
wrote:
The "logevents" API should return the same data as Special:Log.
For
example,
https://en.wikipedia.org/w/api.php?action=query&list=logevents&letit...
This can be filtered further to just delete events, and so on.
But if you only want to know whether an article exists or not,
"missing"
should be accurate. Can you share some example URLs for which the
page
exists, but the API returns "missing"?
Kind regards, Adam W.
On 10/27/21 3:40 AM, D Z wrote: > Hello All, > > I am doing research investigating the role of machine translation
in
> Wikipedia articles. I am having trouble with how to know if an
article
has > been deleted from Wikipedia. Specifically, I am getting a list of articles > from the cxtranslation list and I would like to know which
articles
are
no > longer on Wikipedia. I see that there is the deletion log form > https://en.wikipedia.org/wiki/Special:Log/delete but is there
an
API
or > some way to access something like this form so I could check if a
mass
> amount of articles have been deleted? > > I have used the Media Wiki API <
https://en.wikipedia.org/w/api.php
to
get > articles and the API returns missing for some articles, but this
does
not
> seem to be fully accurate for determining if an article has been
deleted
> because the API has returned 'missing' for articles that do
exist.
> > To summarize, my main question is: given an article language
edition
and
> article title, or an article pageid, is there an API to check if
the
> article has been deleted? > > Any help would be greatly appreciated! > > Thanks, > > Doris Zhou > _______________________________________________ > Wiki-research-l mailing list --
wiki-research-l@lists.wikimedia.org
> To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list --
wiki-research-l@lists.wikimedia.org
To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list --
wiki-research-l@lists.wikimedia.org
To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
-- Adam Wight - Developer - Wikimedia Deutschland e.V. -
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to
wiki-research-l-leave@lists.wikimedia.org
Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
wiki-research-l@lists.wikimedia.org