On 11/13/2013 05:16 AM, Philippe Beaudette wrote: Yes, an intelligent solution would take into account when the mirror was first indexed (or ideally first published), and when the Wikipedia article was edited, to reduce false positives requiring manual intervention. Matt Flaschen

On Wed, Nov 13, 2013 at 11:44 AM, Gerard Meijssen <gerard.meijssen(a)gmail.com Actually, we consider these as copyvios, we delete the text straight away, and we tell the editor "if you're the author write to OTRS". Of course, if the text is already somewhere else under a compatible free-license, we don't need this. Until you can't be sure that User:MrX is actually the physical person MrX, we need to protect the author's right. Marco

On Wed, Nov 13, 2013 at 12:36 PM, Chris McKenna <cmckenna(a)sucs.org> wrote: something saying: Part of the text comes from website X, ticket 1234567890. And if the author wants to use his work for many articles, we tell him/her to put the template in all his/her articles' talk page. Marco

On Wed, Nov 13, 2013 at 12:39 PM, Chris McKenna <cmckenna(a)sucs.org> wrote: To avoid false positives from mirrors, the best option is to compare a text as soon as it is saved. Also, you exclude certain websites from the comparison because you know they're the mirrors, you exclude rollbacks, ... Then, it is better to have a human checking that it is really a copyvio (it could well be a public domain text, or another Wikipedia article). Marco

Hmmmm Rupert, The case you mention is unrelated to any copyright infringement (the book is explicitely published under cc by sa. So there is no copyvio). Its mention here is like hair falling in soup. Now, I think there is a developing personal feud between you and Iolenda. It sincerely saddens me to see two people I appreciate come to such a situation. Would you both consider talking to each other on Skype or something like this ? Alternatively, find someone neutral and nice to help fix things so that you can come to a mutual understanding ? I understand that you both see things differently, but ultimately, you both are here to make things move on. Flo On 11/14/13 2:36 PM, rupert THURNER wrote:

The problem of false positives from mirrors doesn't exist if we scan edits as they are made. Maggie says here<https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboa… copyright bots populate WP:SCV <https://en.wikipedia.org/wiki/Wikipedia:SCV> So a similarly-configured bot could scan recent changes and tag suspected copyvios in watchlists and page histories like suspected vandalism is currently tagged. Ideally the edit summary would contain a url to the suspected source. Maggie points out that those copyright bots were blocked for a while (from scanning Google I presume) due to a negative impact on Google, and this problem was solved by someone writing a cheque. False positives won't be a problem, unless they're more than, say, 50%. If a recent changes patroller can't confirm the copyvio, they can let it go. But an editor whose "contributions" list is peppered with such warnings would stand out like a sore thumb. <https://en.wikipedia.org/wiki/Wikipedia:SCV> Anthony Cole <http://en.wikipedia.org/wiki/User_talk:Anthonyhcole> Memberships secretary Wiki Project Med Foundation<http://meta.wikimedia.org/wiki/Wiki_Project_Med> On Sat, Nov 16, 2013 at 3:36 AM, rupert THURNER &lt;rupert.thurner(a)gmail.com&gt;wrote;wrote:

It could use abuse-filter tags, just not in an entirely standard way: * Bot scans edit X * Script flags it as a problem * Bot makes edit X+1 to page (perhaps adding copyvio template?) which triggers an abusefilter rule for (if this bot and does such-and-such an edit) and tags it. The offending edit itself won't be tagged, but the page history will and it can probably be spotted quite easily from there. A. On 19 November 2013 01:07, Matthew Flaschen &lt;mflaschen(a)wikimedia.org&gt; wrote: -- - Andrew Gray andrew.gray(a)dunelm.org.uk

On Thu, Nov 14, 2013 at 4:47 PM, Andrew Lih &lt;andrew.lih(a)gmail.com&gt; wrote: I've done a study of student contributions on English Wikinews. We've found that only about 15% of student submissions have a copyright issue. The level of plagiarism and copyright problems is about the same for regular contributors, new contributors and student contributors on English Wikinews with that range of 10 to 15%. This is an issue we have to be on top of because nothing gets published on the project without being reviewed for this issue. Sincerely, Laura Hale -- twitter: purplepopple blog: ozziesport.com

Marco: I agree, we had also issues on the Dutch Wikipedia - these have been around for ages, the English Wikipedia is just less aware of them. Often, copypasting in the same language is caught easily - between different languages is much harder and persistent. There are many people, including experienced editors, that think translating from random sources is OK. It is no new problem, and chapters have indeed been working on getting this understanding of what free licenses really mean more widely accepted in the general audience. Not something that is easily measured of course. Technical solutions sound great, but are only catching a small amount inside the same language. Steven: I understand this research was limited to the English Wikipedia (where most of the plagiarism will be in the same language). It would not strike me out of the realm of realism to assume this might be very different for other languages than English. It also says little about the problem in general of course. For those who don't want to click on links to get information, it basically says (simplification alert) that they don't have any indication that the US & Canada education program makes the plagiarism problem on the English Wikipedia any worse than it already is. Anyway: I think this problem is more prominently there in non-English communities, and that technical solutions are not going to be the answer there. An educational answer is more likely to be successful, focusing on explaining people how Wikipedia works and doesn't work, and what are do's and don'ts. This doesn't have to be an education program like executed in the US, but basically all outreach programs as executed by chapters, user groups, thematic organizations or groups of volunteers can contribute to this. This is already happening in most countries. In some countries (like Germany ;-) ) politicians are doing the work for us, explaining how evil plagiarism is and how it works by firing government ministers over it :) Best, Lodewijk 2013/11/13 Marco Chiesa &lt;chiesa.marco(a)gmail.com&gt;

On 11/13/2013 10:39 AM, Nathan wrote: I think he meant that the English Wikipedia community is less aware of the fact that we face these sorts of large-scale challenges in many other languages as well. In other words, the antecedent to "them" is "issues on the Dutch/Italian/etc. Wikipedia", rather than "copyright issues" generally. Most people participating in other languages are reasonably aware when major concerns surface from the English Wikipedia; people participating only in English often haven't a clue about the concerns being dealt with in other languages. --Michael Snow

Marco Chiesa, 13/11/2013 10:21: Rectius: there *used* to be a bot (RevertBot, Lusumbot). The program <https://www.mediawiki.org/wiki/Manual:Pywikibot/copyright.py> has been stopped when search engines changed their limits and Lusum has been waiting for the WMF's Yahoo! BOSS key, needed to run the bot, for a while. Nemo

Marc A. Pelletier, 16/11/2013 16:34: It would be awesome! Thank you. Nemo

Matthew Flaschen, 20/11/2013 06:05: That bot links a code (Coren's) that asks a key. So either the user is paying one himself, or he got the WMF's one some time ago: in both cases, he can't give it to more people. Nemo

On Wed, Nov 13, 2013 at 3:48 AM, Fæ &lt;faewik(a)gmail.com&gt; wrote: We are not attempting to perform a complete and unassailable verification; imagining that we can is folly. The point is, we need someone who credibly is the author or rightsholder, and with whom we have an audit trail of their claims and identity (email address we corresponded with, etc). When it comes down to it, we have no idea if an email is associated with the given person, that the alleged sender of a certified letter really is that person, or that the "John Doe" that came in to the office and showed valid government issued ID with a claim of copyright violation is the same John Doe who wrote the original material. There's no way for us to confirm in any reasonable manner. If there is an attempt at identity theft that is discovered, that audit trail is available to investigators with proper legal authorization etc. -- -george william herbert george.herbert(a)gmail.com

Samuel Klein, 19/11/2013 21:44: How to make them include the whole Commons dataset into their own, to start with? Nemo

Yes, let's keep on pushing for policies that drive away editors! On Nov 20, 2013 2:10 AM, "Fæ" &lt;faewik(a)gmail.com&gt; wrote:

On 11/20/2013 07:13 AM, The Cunctator wrote: Let's be clear here: contributions that are copyright violations are not desirable to begin with. If someone is driven away because they cannot cut and paste from random websites anymore, I'm not sure that this could reasonably be taken to be a bad thing. -- Marc

On 11/20/2013 11:59 AM, Michael Snow wrote: That's actually an interesting question that has been lurking beneath all the "editing is going down" nervousness. How much of that 'editing' was, in fact, busy work made immaterial by technical advantage (bots, extensions, abusefilter)? The number of antivandalism edits a /human/ has to do in a day has most certainly come down a *lot* since c. 2006; this no doubt contributed to a large - now diminishing - fraction of total edits. It's not clear to me that the number of *productive* edits has been going down all that much (if at all) in the past several years; the proportion of edits that were tedious and repetitive clearly has. Are you arguing that there is *value* in volunteers spending time on work that could be automated? Except for artificially driving up edit counts, that is time (and effort) that would be better spent pretty much anywhere else! -- Marc

On 11/20/2013 01:06 PM, Richard Symonds wrote: I'm pretty sure that typo correction fills pretty much the same niche, though. -- Marc

On 11/21/2013 02:23 PM, Benjamin Lees wrote: Re the "If a drafts namespace is created, special case the act of moving a draft to mainspace. Make a publish button or something. [...]" paragraph in that essay, that's the kind of thing we're looking at on the Growth team right now. See https://www.mediawiki.org/wiki/Wikipedia_article_creation Matt Flaschen

On 11/20/2013 01:13 PM, Michael Snow wrote: That's an interesting approach, but I'm not sure how constructive it is in the long run. I suppose it depends greatly on whether one considers our mission to be 'building an encyclopedia to share in the sum[...]' or 'having an encyclopedia to share in the sum[...]' (I'm not sure if I make the subtle distinction here clear). Perhaps another way of putting it is to ask whether the encyclopedia-building community is the means or the ends. To my eyes, having "more contributors" is not valuable unless it has "better encyclopedia" as a direct consequence. -- Marc

On 11/20/2013 10:52 AM, Marc A. Pelletier wrote: I believe the mission is sufficiently large in scope that having more people involved is fundamentally desirable in general. Although to circle back to an earlier point in the discussion, that doesn't require that we accept involvement that is counterproductive. Maintaining our standards is a way of acknowledging that the number of people involved is not itself the end goal. --Michael Snow

On 11/13/2013 04:41 PM, Tobias wrote: I think that's the best way of summing it up. "Sufficient" is a vague metric, and leaves room for improvement, but the nutshell is that the community /does/ take copyright violations seriously and deploys very good efforts to curtail it. Do some slip through? Yes, without doubt. Are they eliminated with prejudice the second they are noticed? Yes. The Wikimedia projects are no worse than any other collected works when it comes to copyright infringement and indeed tends to handle it with more vigilance than the other sites in the top 10 (proactively, rather than reactively). Could we do better? No doubt. Is improvement so desperately critical that we should drop everything else to concentrate on that? Not a chance. And I speak as the author and (for a long time) maintainer of one of the most visible and used copyright violation detection tool used on our project (CorenSearchBot, now handled by MadmanBot and - last I heard - used on around a dozen projects). -- Marc