[posted to foundation-l and wikitech-l, thread fork of a discussion elsewhere]
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
This is the single point of failure problem. The reasons for it having happened are obvious, but it's still a problem. Blog posts (please excuse me linking these yet again):
* http://davidgerard.co.uk/notes/2007/04/10/disaster-recovery-planning/ * http://davidgerard.co.uk/notes/2011/01/19/single-point-of-failure/
I dream of the encyclopedia being meaningfully backed up. This will require technical attention specifically to making the projects - particularly that huge encyclopedia in English - meaningfully forkable.
Yes, we should be making ourselves forkable. That way people don't *have* to trust us.
We're digital natives - we know the most effective way to keep something safe is to make sure there's lots of copies around.
How easy is it to set up a copy of English Wikipedia - all text, all pictures, all software, all extensions and customisations to the software? What bits are hard? If a sizable chunk of the community wanted to fork, how can we make it *easy* for them to do so?
And I ask all this knowing that we don't have the paid tech resources to look into it - tech is a huge chunk of the WMF budget and we're still flat-out just keeping the lights on. But I do think it needs serious consideration for long-term preservation of all this work.
- d.
On Fri, 12 Aug 2011 11:55:47 +0100, David Gerard dgerard@gmail.com wrote:
[posted to foundation-l and wikitech-l, thread fork of a discussion elsewhere]
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
This is the single point of failure problem. The reasons for it having happened are obvious, but it's still a problem. Blog posts (please excuse me linking these yet again):
- http://davidgerard.co.uk/notes/2007/04/10/disaster-recovery-planning/
- http://davidgerard.co.uk/notes/2011/01/19/single-point-of-failure/
I dream of the encyclopedia being meaningfully backed up. This will require technical attention specifically to making the projects - particularly that huge encyclopedia in English - meaningfully forkable.
I do agree that the monopoly, at least in this case, is a bad thing, but I do not see why stimulating creation of the forks would be the best way to create competition. As far as I am concerned, the only real competition to us comes from Chinese projects like Baidu, and not from many Wikipedia-like forks or not even from Google Knol.
Cheers Yaroslav
On 12 August 2011 13:07, Yaroslav M. Blanter putevod@mccme.ru wrote:
I do agree that the monopoly, at least in this case, is a bad thing, but I do not see why stimulating creation of the forks would be the best way to create competition. As far as I am concerned, the only real competition to us comes from Chinese projects like Baidu, and not from many Wikipedia-like forks or not even from Google Knol.
Making it easy to fork keeps us honest. I think we really need good competitors, and we don't have any.
- d.
On Fri, 12 Aug 2011 13:32:43 +0100, David Gerard dgerard@gmail.com wrote:
On 12 August 2011 13:07, Yaroslav M. Blanter putevod@mccme.ru wrote:
I do agree that the monopoly, at least in this case, is a bad thing,
but
I do not see why stimulating creation of the forks would be the best way
to
create competition. As far as I am concerned, the only real competition to us comes from Chinese projects like Baidu, and not from many Wikipedia-like forks or not even from Google Knol.
Making it easy to fork keeps us honest. I think we really need good competitors, and we don't have any.
My point is that making it easy to fork does not create good competitors. Good competitors come from elsewhere. And they will come, if we do not deploy WISIWIG, not lower the entrance barrier for novices, not make it harder to troll out respectable users, and not find a way to make connections to academia or otherwise considerably improve the quality.
Cheers Yaroslav
On 12 August 2011 13:37, Yaroslav M. Blanter putevod@mccme.ru wrote:
My point is that making it easy to fork does not create good competitors. Good competitors come from elsewhere. And they will come, if we do not deploy WISIWIG, not lower the entrance barrier for novices, not make it harder to troll out respectable users, and not find a way to make connections to academia or otherwise considerably improve the quality.
Oh, absolutely. The other thing they'd need is an actual sizable editing community, big enough to take on the task. Citizendium failed to achieve this, for example, and ended up deleting most of the articles they'd forked from Wikipedia.
I'm pointing out that the technical ability is also a prerequisite. Even if you have the other stuff, the ability to do it at all needs to be present. Technical forkability is explicitly acknowledged by the tech team as obviously the Right Thing, and it's why WMF is so gung-ho about open source everything; the trouble is actually putting it into practice in a resource-restricted environment. It's a variety of technical debt [*].
WYSIWYG is in progress. Moon shot ahoy!
Academics appear to be coming to us, despite our inability to keep idiots out of experts' faces. Or out of respectable users' faces. Or out of anyone's face. A dissolution of the "expert problem" I hadn't been expecting.
- d.
[*] http://en.wikipedia.org/wiki/Technical_debt - the shortcuts you take to get something working, knowing you need to fix them later if not now. Numerical measure and accounting is tricky, but the analogy to financial debt is surprisingly useful.
On 12 August 2011 13:47, David Gerard dgerard@gmail.com wrote:
On 12 August 2011 13:37, Yaroslav M. Blanter putevod@mccme.ru wrote:
My point is that making it easy to fork does not create good competitors. Good competitors come from elsewhere. And they will come, if we do not deploy WISIWIG, not lower the entrance barrier for novices, not make it harder to troll out respectable users, and not find a way to make connections to academia or otherwise considerably improve the quality.
Oh, absolutely. The other thing they'd need is an actual sizable editing community, big enough to take on the task. Citizendium failed to achieve this, for example, and ended up deleting most of the articles they'd forked from Wikipedia.
That assumes it's actually worth editing wikipedia on any scale at this point. For most normal applications of encyclopedias it probably isn't.
On Fri, Aug 12, 2011 at 12:16 PM, geni geniice@gmail.com wrote:
On 12 August 2011 13:47, David Gerard dgerard@gmail.com wrote:
On 12 August 2011 13:37, Yaroslav M. Blanter putevod@mccme.ru wrote:
My point is that making it easy to fork does not create good competitors. Good competitors come from elsewhere. And they will come, if we do not deploy WISIWIG, not lower the entrance barrier for novices, not make it harder to troll out respectable users, and not find a way to make connections to academia or otherwise considerably improve the quality.
Oh, absolutely. The other thing they'd need is an actual sizable editing community, big enough to take on the task. Citizendium failed to achieve this, for example, and ended up deleting most of the articles they'd forked from Wikipedia.
That assumes it's actually worth editing wikipedia on any scale at this point. For most normal applications of encyclopedias it probably isn't.
We still have wide gaps in knowledge coverage. Not in the most common areas, but in many specialized areas, where they're not heavily geek-populated.
On 12 August 2011 20:24, George Herbert george.herbert@gmail.com wrote:
We still have wide gaps in knowledge coverage. Not in the most common areas, but in many specialized areas, where they're not heavily geek-populated.
Yes but those don't have much to do with normal applications of encyclopedias.
On Fri, Aug 12, 2011 at 12:53 PM, geni geniice@gmail.com wrote:
On 12 August 2011 20:24, George Herbert george.herbert@gmail.com wrote:
We still have wide gaps in knowledge coverage. Not in the most common areas, but in many specialized areas, where they're not heavily geek-populated.
Yes but those don't have much to do with normal applications of encyclopedias.
Sure they do. The question is what coverage you want in the encyclopedia.
You may not be a construction guy, but wouldn't it be useful if you could say "Hmm, what are those standardized 1.5 inch square open metal channels used everywhere in construction?" and find [[Strut channel]] on Wikipedia.
And a few thousand other construction things I haven't had time to add, yet.
And engineering.
All these specialized things are encyclopedic, and matter in the world, even if they're not geek-significant. There's no reason not to define encyclopedic as inclusive of topics such as these.
On 12 August 2011 20:59, George Herbert george.herbert@gmail.com wrote:
On Fri, Aug 12, 2011 at 12:53 PM, geni geniice@gmail.com wrote:
On 12 August 2011 20:24, George Herbert george.herbert@gmail.com wrote:
We still have wide gaps in knowledge coverage. Not in the most common areas, but in many specialized areas, where they're not heavily geek-populated.
Yes but those don't have much to do with normal applications of encyclopedias.
Sure they do. The question is what coverage you want in the encyclopedia.
You may not be a construction guy, but wouldn't it be useful if you could say "Hmm, what are those standardized 1.5 inch square open metal channels used everywhere in construction?" and find [[Strut channel]] on Wikipedia.
And a few thousand other construction things I haven't had time to add, yet.
And engineering.
All these specialized things are encyclopedic, and matter in the world, even if they're not geek-significant. There's no reason not to define encyclopedic as inclusive of topics such as these.
You appear to be confusing "articles needed for normal applications of encyclopedias." and encyclopedic. [[Nabu-apla-iddina]] is encyclopedic, a Babylonian king no less, however history shows that encyclopedias can function just fine without having an article on him.
On 12 August 2011 20:53, geni geniice@gmail.com wrote:
On 12 August 2011 20:24, George Herbert george.herbert@gmail.com wrote:
We still have wide gaps in knowledge coverage. Not in the most common areas, but in many specialized areas, where they're not heavily geek-populated.
Yes but those don't have much to do with normal applications of encyclopedias.
Neither does Wikipedia. An encyclopedia is now "something like Wikipedia." We are in indeterminate territory. The question we're trying to answer in this subthread is "what would we use Wikipedia for if it were there?"
- d.
+1 for the need to make it easy to fork (I suggested this back in 2010 in Gdansk during the barcamp session.)
"Yaroslav M. Blanter" putevod@mccme.ru writes:
I do agree that the monopoly, at least in this case, is a bad thing, but I do not see why stimulating creation of the forks would be the best way to create competition.
Forks are not only a matter of _competition_ between branches, but also a matter of _freedom_.
The whole point of using CC-by-sa in WM project is to allow people to reuse and to improve the content, either within the projects or outside the projects.
No doubt that the content is being massively reused outside the WM project.
But I doubt the content is improved outside the project -- which is what matters most to me. Various communities disagree on what "improve" means, so it would be great if WM could let those communities to fork the projects' content and start new ones.
The easiest way I can think of is a Mediawiki plugin that allows people to grab content from WM projects and create new pages with this content.
Man, Gerard is thinking about new methods to fork (in an easy way) single articles, sets of articles or complete wikipedias, and people reply about setting up servers/mediawiki/importing_databases and other geeky weekend parties. That is why there is no successful forks. Forking Wikipedia is _hard_.
People need a button to create a branch of an article or sets of articles, and be allowed to re-write and work in the way they want. Of course, the resulting articles can't be saved/showed close to the Wikipedia articles, but in a new plataform. It would be an interesting experiment.
2011/8/12 David Gerard dgerard@gmail.com
[posted to foundation-l and wikitech-l, thread fork of a discussion elsewhere]
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
This is the single point of failure problem. The reasons for it having happened are obvious, but it's still a problem. Blog posts (please excuse me linking these yet again):
- http://davidgerard.co.uk/notes/2007/04/10/disaster-recovery-planning/
- http://davidgerard.co.uk/notes/2011/01/19/single-point-of-failure/
I dream of the encyclopedia being meaningfully backed up. This will require technical attention specifically to making the projects - particularly that huge encyclopedia in English - meaningfully forkable.
Yes, we should be making ourselves forkable. That way people don't *have* to trust us.
We're digital natives - we know the most effective way to keep something safe is to make sure there's lots of copies around.
How easy is it to set up a copy of English Wikipedia - all text, all pictures, all software, all extensions and customisations to the software? What bits are hard? If a sizable chunk of the community wanted to fork, how can we make it *easy* for them to do so?
And I ask all this knowing that we don't have the paid tech resources to look into it - tech is a huge chunk of the WMF budget and we're still flat-out just keeping the lights on. But I do think it needs serious consideration for long-term preservation of all this work.
- d.
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
On Sat, Aug 13, 2011 at 4:53 AM, emijrp emijrp@gmail.com wrote:
Man, Gerard is thinking about new methods to fork (in an easy way) single articles, sets of articles or complete wikipedias, and people reply about setting up servers/mediawiki/importing_databases and other geeky weekend parties. That is why there is no successful forks. Forking Wikipedia is _hard_.
People need a button to create a branch of an article or sets of articles, and be allowed to re-write and work in the way they want. Of course, the resulting articles can't be saved/showed close to the Wikipedia articles, but in a new plataform. It would be an interesting experiment.
Something like this.. ?
http://wikimedia.org.au/wiki/Proposal:PersonalWikiTool
Yes, that tool looks similar to the idea I wrote. Other approaches may be possible too.
2011/8/13 John Vandenberg jayvdb@gmail.com
On Sat, Aug 13, 2011 at 4:53 AM, emijrp emijrp@gmail.com wrote:
Man, Gerard is thinking about new methods to fork (in an easy way) single articles, sets of articles or complete wikipedias, and people reply about setting up servers/mediawiki/importing_databases and other geeky weekend parties. That is why there is no successful forks. Forking Wikipedia is _hard_.
People need a button to create a branch of an article or sets of
articles,
and be allowed to re-write and work in the way they want. Of course, the resulting articles can't be saved/showed close to the Wikipedia articles, but in a new plataform. It would be an interesting experiment.
Something like this.. ?
http://wikimedia.org.au/wiki/Proposal:PersonalWikiTool
-- John Vandenberg
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
On 12/08/11 20:55, David Gerard wrote:
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
I must have missed the place where you actually made this case. I tried reading your blog posts but I didn't see it there.
In 2005 you said that the point is to insure the data against the financial collapse of the Foundation. But the chance of that appears to be vanishingly small, and shrinking as the Foundation gets larger. If there was some financial problem, then we would have plenty of warning and plenty of time to plan an exit strategy. The technical risks (meteorite strike etc.) are also receding as we grow larger.
Also, you seem to be conflating forking with mirroring. If the Foundation did get into trouble in say 2030, then presumably the community would want a copy of the whole site as it is in 2030, not a content fork from 2011.
-- Tim Starling
On Mon, Aug 15, 2011 at 6:04 AM, Tim Starling tstarling@wikimedia.org wrote:
On 12/08/11 20:55, David Gerard wrote:
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
I must have missed the place where you actually made this case. I tried reading your blog posts but I didn't see it there.
In 2005 you said that the point is to insure the data against the financial collapse of the Foundation.
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community.
Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
2011/8/15 David Richfield davidrichfield@gmail.com:
On Mon, Aug 15, 2011 at 6:04 AM, Tim Starling tstarling@wikimedia.org wrote:
On 12/08/11 20:55, David Gerard wrote:
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
I must have missed the place where you actually made this case. I tried reading your blog posts but I didn't see it there. In 2005 you said that the point is to insure the data against the financial collapse of the Foundation.
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community. Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
Pretty much. It's not urgent - I do understand we're chronically underresourced - but I think it's fairly obvious it's a Right Thing, and at the very least something to keep in the back of one's mind.
- d.
On 15/08/11 16:30, David Gerard wrote:
2011/8/15 David Richfield davidrichfield@gmail.com:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community. Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
Pretty much. It's not urgent - I do understand we're chronically underresourced - but I think it's fairly obvious it's a Right Thing, and at the very least something to keep in the back of one's mind.
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity?
I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
-- Tim Starling
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity?
Adding ads (for instance, Google ads) to the Wikipedia pages? (I do not mean WMF is planning to do this, but as a remote possibility - why not?)
Cheers Yaroslav
On 08/15/11 12:10 AM, Yaroslav M. Blanter wrote:
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity?
Adding ads (for instance, Google ads) to the Wikipedia pages? (I do not mean WMF is planning to do this, but as a remote possibility - why not?)
A comprehensive fork would probably need ad revenue more than the WMF unless it has deep pockets to get it going.
Ray
On Mon, Aug 15, 2011 at 09:38, Ray Saintonge saintonge@telus.net wrote:
A comprehensive fork would probably need ad revenue more than the WMF unless it has deep pockets to get it going.
I don't think this is a requirement. Wikipedia have to support enormous amount of traffic while a fork don't expect such. I'm sure I could easily fork enwp with just one machine, and handle a few hundred visitors a day, or even in an hour. I believe Fred Bauder have made a fork of a kind (yes I know it used a different method) and I guess he do see the traffic stats and resource requirements to do that. ;-)
By the way someone asked about reasons to fork. A very possible reason could be when an established ("senior") editor gets mass attacked and/or banned and/or having to work in a very hostile environment. (Not just a few editors avoid commons for the very reason that they feel images tend to disappear without warning. I try to handle these cases, nevertheless, but I know it happens. It happens on enwp often that religious/nationalist pressure drives editors away, and if someone is involved enough s/he may feel the need to fork and continue the work with a different ruleset.)
I'm sure I could easily fork enwp with just one machine, and handle a few hundred visitors a day, or even in an hour. I believe Fred Bauder have made a fork of a kind (yes I know it used a different method) and I guess he do see the traffic stats and resource requirements to do that. ;-)
byte-byte, grin
Some of Wikinfo's technical problems at ibiblio result from being on a shared server, less than one server. I'm pretty sure one small server would handle a full fledged setup. If there is high traffic there is high potential for donations.
Fred
On 15 August 2011 07:51, Tim Starling tstarling@wikimedia.org wrote:
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity? I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
I don't have a particular risk in mind, no, which is why I've been consistently saying this is not urgent. You seem to be assuming I'm saying something I'm not.
- d.
On 15/08/11 16:30, David Gerard wrote:
2011/8/15 David Richfield davidrichfield@gmail.com:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community. Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
Pretty much. It's not urgent - I do understand we're chronically underresourced - but I think it's fairly obvious it's a Right Thing, and at the very least something to keep in the back of one's mind.
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity?
I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
-- Tim Starling
That technical staff have effective power to decide whether a fork is justified is reason enough.
Fred Bauder
Hi!
That technical staff have effective power to decide whether a fork is justified is reason enough.
If you tried reading more of the message than just From: header, you wouldn't write this bullshit.
The whole topic is about ease of forking, and:
a) Member of technical staff did not allege that he is making any kind of decision, he just explains that there're tradeoffs to be made b) It was not being discussed whether "resources to allow forking" are to be spent or not. The message was more about "how much" c) He also questioned if organization bylaws/format/etc is a high risk, and he solicited feedback. d) He also wanted to know what is seen as a bad foundation decision worth forking. That is more of a "how not to need a fork" rather than "how not to allow a fork".
I hope you will fix your attitude.
As for resources spent on ease of forking, it can go many ways.
WMF could be extremely supportive of forks/mirrors/whatever. That is not just about providing a dump, that is also about allowing to query an API for page-loads of remote forks. Essentially, it could become data engine for whatever gets built on top. That is expensive, fundraising becomes inefficient, but here you are, lots of resources and a commitment to spend them just to support forks. WMF could also give initial grants and actually help on fork engineering problems ;-)
WMF could also impose rules on number of articles/data size, to put editors into britannica-like editing more, where each word added is a word removed from elsewhere :-) This would keep projects way more forkable, albeit this would be a catalyst to a fork without such a restriction ;-D
WMF could provide reliable/robust change feeds/distribution mechanisms, including media. Depending on a requested feature set this may be relatively expensive operation (albeit it may be more expensive to be on a receiving end at that time - it is just multitude of polling people that may be costly for WMF ;)
As you see, each of these things may require lesser or bigger MTS participation at the tactical level, k'thx.
Domas
Just a point: WMF projects have spilt out before, for example was the September 11 remembrance wiki (sep11.wikipedia) although the fork is now offline, also one of the other "plain" language specific projects (Spanish Wikipedia comes to mind but I can't confirm) but as far as I'm aware never really took off so it basically became idle.
On 15/08/11 18:14, Fred Bauder wrote:
I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
-- Tim Starling
That technical staff have effective power to decide whether a fork is justified is reason enough.
I'm not in a position to actually allocate resources to this or to decide whether it's justified. I'm asking these questions mostly for my own curiosity, and in case someone seeks my opinion on it in the future.
When you launched Internet-Encyclopedia, I was very positive about its utility. Brion and I gave it all the support it needed. The "green link" feature in particular required Wikinfo to be whitelisted in the server configuration so that it didn't get blocked for its high request rate. I reviewed Proteus's fork of MediaWiki to see if there were any changes that we could reincorporate. So it's not like I'm staunchly anti-fork.
-- Tim Starling
On 15/08/11 18:14, Fred Bauder wrote:
I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
-- Tim Starling
That technical staff have effective power to decide whether a fork is justified is reason enough.
I'm not in a position to actually allocate resources to this or to decide whether it's justified. I'm asking these questions mostly for my own curiosity, and in case someone seeks my opinion on it in the future.
When you launched Internet-Encyclopedia, I was very positive about its utility. Brion and I gave it all the support it needed. The "green link" feature in particular required Wikinfo to be whitelisted in the server configuration so that it didn't get blocked for its high request rate. I reviewed Proteus's fork of MediaWiki to see if there were any changes that we could reincorporate. So it's not like I'm staunchly anti-fork.
-- Tim Starling
Tim,
Yes, your help was greatly appreciated.
Here's the conclusion I've come to though. We need to get the software good enough, and simple enough, that it is firmly in the background. Mediawiki is like an old DOS computer that constantly drags you into programing mode, particularly if you fork. We need the equivalent of a Macintosh that almost anyone can use effortlessly. The emphasis needs to be on content, not on trying to figure out extensions and templates.
Fred
Hi!
Here's the conclusion I've come to though. We need to get the software good enough, and simple enough, that it is firmly in the background.
OK!
Mediawiki is like an old DOS computer that constantly drags you into programing mode, particularly if you fork.
Yes, especially if you actually are running it, all you do is sit in some black and white text screens, that definitely sucks.
We need the equivalent of a Macintosh that almost anyone can use effortlessly. The emphasis needs to be on content, not on trying to figure out extensions and templates.
Yup, we need drag&drop forking support. With clouds nowadays that should be easy - you enter cloud account information (it may auto-detect password), and drag the website you want to fork onto a "drag here" target. We should definitely work on this kind of functionality. Then you click on it, and it runs, in a cloud!
Emphasis needs to be on content and DRM, so that people don't copy articles without leaving 30% of their revenue to your fork. ArticleStore is going to be core essence of all content distribution, after it has been previewed on the website, of course, seamlessly integrated with reading devices, like computers.
It is easy to resolve templates and extensions iOS-development way, charge community for being able to write them, that will make the remaining ones truly useful, because someone was motivated to do that. Of course, you need an approval process, but it is nothing technical, you can approve that stuff solely on moon phase or peyote effects :)
Anyway, we should definitely build something like that, just don't pay attention to suicide rate.
Cheers, Domas
On 16 August 2011 09:06, Domas Mituzas midom.lists@gmail.com wrote:
Anyway, we should definitely build something like that, just don't pay attention to suicide rate.
:-) I am quite cognisant that the likely number of people wanting to build a full fork of Wikipedia may well be *zero*. I apologise if I have given any of this the sound of urgency. I am saying, however, that forkability is an important right thing, a guard against disasters and a good way to keep ourselves honest. And a lot (if not all) of what it requires is stuff we really should be doing anyway.
(BTW - we *do* have someone making sure the Internet Archive - or a similar organisation, if there are any similar organisations - has a full collection of all our backups, so if Florida was hit by a meteor tomorrow people would have something to start from?)
- d.
On 16 August 2011 09:18, David Gerard dgerard@gmail.com wrote:
(BTW - we *do* have someone making sure the Internet Archive - or a similar organisation, if there are any similar organisations - has a full collection of all our backups, so if Florida was hit by a meteor tomorrow people would have something to start from?)
argh. That's a question, not a statement. Do we have some third party with copies of everything? I suggest the IA as they have the disk space and, as a library, rabidly archive everything they can get their hands on.
(Would anyone from IA happen to be on the list?)
- d.
On 08/16/11 1:20 AM, David Gerard wrote:
On 16 August 2011 09:18, David Gerarddgerard@gmail.com wrote
(BTW - we *do* have someone making sure the Internet Archive - or a similar organisation, if there are any similar organisations - has a full collection of all our backups, so if Florida was hit by a meteor tomorrow people would have something to start from?)
argh. That's a question, not a statement. Do we have some third party with copies of everything? I suggest the IA as they have the disk space and, as a library, rabidly archive everything they can get their hands on.
One suggestion for archiving would be to have a complete set of projects filed with the copyright office and other key depositories quarterly.
This could also address a potential long-term copyright problem. This has less to do with Wikipedia infringing on the copyrights of others than with the reverse. It already happens that others use Wikipedia material without credit in works on which they claim copyright. Re-use of that material on-wiki at a later date will inevitably result in a copyvio squabble, especially if the originally plundered version is no longer recognizable. This could be many years hence. What other means are available to protect the viral nature of freely licensed material?
Forks could also be helpful in this regard. They would need to respect free licences, and, as a by-product, add evidence favouring the freeness of the material. A person creating a fork based on some topic area is unlikely to significantly alter all the articles imported, preferring to draw different conclusions from the same underlying facts. This is bound to leave an identifiable residue that will protect the licence.
Ray
2011/8/16 David Gerard dgerard@gmail.com
On 16 August 2011 09:06, Domas Mituzas midom.lists@gmail.com wrote:
Anyway, we should definitely build something like that, just don't pay
attention to suicide rate.
:-) I am quite cognisant that the likely number of people wanting to build a full fork of Wikipedia may well be *zero*. I apologise if I have given any of this the sound of urgency. I am saying, however, that forkability is an important right thing, a guard against disasters and a good way to keep ourselves honest. And a lot (if not all) of what it requires is stuff we really should be doing anyway.
(BTW - we *do* have someone making sure the Internet Archive - or a similar organisation,
I heard Internet Archive downloads dumps every 3 months (but no images).
Also, some time ago I heard about a contact with Library of Congress to host dumps duplicates. No more news about that.
if there are any similar organisations - has a full collection of all our backups, so if Florida was hit by a meteor tomorrow people would have something to start from?)
Instead of a meteor, maybe a hurricane. Instead of a hurricane, maybe a faulty RAID. Did you hear about the RAID problem some months ago?
Regards, emijrp
- d.
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
On 08/14/11 11:51 PM, Tim Starling wrote:
On 15/08/11 16:30, David Gerard wrote:
2011/8/15 David Richfielddavidrichfield@gmail.com:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community. Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
Pretty much. It's not urgent - I do understand we're chronically underresourced - but I think it's fairly obvious it's a Right Thing, and at the very least something to keep in the back of one's mind.
So you're worried about a policy change? What sort of policy change specifically would necessitate forking the project? Is there any such policy change which could plausibly be implemented by the Foundation while it remains a charity?
I'm just trying to evaluate the scale of the risk here. The amount of resources that we need to spend on this should be proportional to the risk.
The primary value of a fork(s) is not financial or technical, but epistemological. We are the big kid in the playground, and that has a significant effect on the nature of the content. When we work so hard to build an aura of reliability readers begin to depend on us. Paradoxically, that's not always good. If we are so reliable, the reader is not motivated to look elsewhere for alternatives. Natural human laziness is bad enough by itself. We too easily fall into the trap of treating Group POV as Neutral POV. Forks, would develop their own versions of NPOV, and end up with very different results that are as easily reliable as ours, but still different. It becomes up to the reader to compare corresponding pages, and draw his own conclusions on the matter at hand.
We should not be viewing forks as inherent evils to be resisted at all costs. We should be encouraging them, and helping them.
Ray
On 15/08/11 08:16, David Richfield wrote:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community.
I'm fairly confident it would be much easier to fork Wikipedia than OpenOffice.
On Mon, Aug 15, 2011 at 08:26, Nikola Smolenski smolensk@eunet.rs wrote:
On 15/08/11 08:16, David Richfield wrote:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community.
I'm fairly confident it would be much easier to fork Wikipedia than OpenOffice.
Technically, it's much easier to fork code than it is to fork wikis especially now in an era of distributed version control systems (Git, hg, bzr) where everyone who checks the code out of a repository has a full copy of the repository. The only technical infrastructure you need is some hosting space for the repo and the other common bits you need for software dev (mailing list, bug tracker etc.)
One thing I've been thinking about from the failure of Citizendium is how an expert community could set up their own external version of pending changes: basically a simple database of stable versions, so any individual or group could set up a server with stable versions of articles, then you could subscribe to a set of stable version sets - so, say, the International Astronomical Union mark a bunch of revisions of astronomy articles as stable, and if you've got the browser plugin installed with their dataset installed, when you visit one of those pages, it'd show you the stable version they chose. And the flipside is that if you are (in my humble opinion) a cold fusion nut or a homeopathy nut, you could find some crazy person who believes in those things to come up with his or her own set of crank stable versions.
And the stable version could be marked as checked by a particular person from a particular institution with their real name if that is the practice in that community: perhaps in physics or philosophy or psychology or some other academic subject, having a real name person sign off on a particular stable version is fine and dandy, but in, say, the Pokémon fan community, they don't really have the same assumptions. (Again, one of the failures of Citizendium: you don't need a guy with a Ph.D to approve the articles on Pokémon in the way you might want a credentialed expert to sign off on, say, an article on cancer treatment.)
The essential thing is to separate out the things that people want: some people want "distributed Wikipedia", but why? Well, one good reason seems to be so you can have stable versions with expert oversight (like Citizendium) - well you can get most of the desiderata that led to Citizendium by having a third-party distributed approval layer and browser plugins etc. A little bit of hacking provides a lot of opportunity for different communities to take Wikipedia and run with it in the ways they want to. This kind of proposal would provide a lot of what Citizendium was shooting for but without the coordination problem of trying to get disparate communities of people to work together in a way the CZ community kind of failed to do. Consider for instance the ethnic studies/women's studies people who didn't find Citizendium a welcoming environment.[1] Under this kind of proposal, if there is a community of people involved in ethnic studies who want to participate in Citizendium-style expert approval, they can set up some very lightweight software and organise their approvals in whatever way fits best with their academic community norms.
Essentially, in software terms, this would be like a 'packager', someone who takes Wikipedia's output on a certain topic and marks specific revisions or whatever as good or bad. They'd still be welcome (and indeed encouraged) to participate in editing on Wikipedia in the traditional way, and ideally the community wouldn't take participation in such an enterprise against them as an editor (just as they currently don't or shouldn't take participating in Wikinfo or Citizendium or even Conservapedia against someone), and any comments that come up in the 'packaging' process could be taken as feedback in the normal way just as if packager at Debian finds a bug with a piece of software, he or she can point that out the upstream maintainer.
Feedback?
[1] see http://cryptome.info/citizendium.htm and http://rationalwiki.org/wiki/Citizendium
Feedback: Approval based systems only work on a tiny subset of articles as they disenfranchise the vast majority of contributors who don't have a multi-tiered content approach at all.
-----Original Message----- From: Tom Morris tom@tommorris.org To: Wikimedia Foundation Mailing List foundation-l@lists.wikimedia.org Sent: Mon, Aug 15, 2011 2:04 am Subject: Re: [Foundation-l] We need to make it easy to fork and leave
On Mon, Aug 15, 2011 at 08:26, Nikola Smolenski smolensk@eunet.rs wrote: On 15/08/11 08:16, David Richfield wrote:
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community.
I'm fairly confident it would be much easier to fork Wikipedia than OpenOffice.
Technically, it's much easier to fork code than it is to fork wikis specially now in an era of distributed version control systems (Git, g, bzr) where everyone who checks the code out of a repository has a ull copy of the repository. The only technical infrastructure you eed is some hosting space for the repo and the other common bits you eed for software dev (mailing list, bug tracker etc.) One thing I've been thinking about from the failure of Citizendium is ow an expert community could set up their own external version of ending changes: basically a simple database of stable versions, so ny individual or group could set up a server with stable versions of rticles, then you could subscribe to a set of stable version sets - o, say, the International Astronomical Union mark a bunch of evisions of astronomy articles as stable, and if you've got the rowser plugin installed with their dataset installed, when you visit ne of those pages, it'd show you the stable version they chose. And he flipside is that if you are (in my humble opinion) a cold fusion ut or a homeopathy nut, you could find some crazy person who believes n those things to come up with his or her own set of crank stable ersions. And the stable version could be marked as checked by a particular erson from a particular institution with their real name if that is he practice in that community: perhaps in physics or philosophy or sychology or some other academic subject, having a real name person ign off on a particular stable version is fine and dandy, but in, ay, the Pokémon fan community, they don't really have the same ssumptions. (Again, one of the failures of Citizendium: you don't eed a guy with a Ph.D to approve the articles on Pokémon in the way ou might want a credentialed expert to sign off on, say, an article n cancer treatment.) The essential thing is to separate out the things that people want: ome people want "distributed Wikipedia", but why? Well, one good eason seems to be so you can have stable versions with expert versight (like Citizendium) - well you can get most of the desiderata hat led to Citizendium by having a third-party distributed approval ayer and browser plugins etc. A little bit of hacking provides a lot f opportunity for different communities to take Wikipedia and run ith it in the ways they want to. This kind of proposal would provide lot of what Citizendium was shooting for but without the oordination problem of trying to get disparate communities of people o work together in a way the CZ community kind of failed to do. onsider for instance the ethnic studies/women's studies people who idn't find Citizendium a welcoming environment.[1] Under this kind of roposal, if there is a community of people involved in ethnic studies ho want to participate in Citizendium-style expert approval, they can et up some very lightweight software and organise their approvals in hatever way fits best with their academic community norms. Essentially, in software terms, this would be like a 'packager', omeone who takes Wikipedia's output on a certain topic and marks pecific revisions or whatever as good or bad. They'd still be welcome and indeed encouraged) to participate in editing on Wikipedia in the raditional way, and ideally the community wouldn't take participation n such an enterprise against them as an editor (just as they urrently don't or shouldn't take participating in Wikinfo or itizendium or even Conservapedia against someone), and any comments hat come up in the 'packaging' process could be taken as feedback in he normal way just as if packager at Debian finds a bug with a piece f software, he or she can point that out the upstream maintainer. Feedback? [1] see http://cryptome.info/citizendium.htm and ttp://rationalwiki.org/wiki/Citizendium
Yes, it's not about the "end of the world is neigh" type scenario. It's just a simple matter of, 'If I wanted a complete copy of Wikipedia, how do I get it?'
There answer is that there are several ways.
First off, there are the DB dumps ( http://en.wikipedia.org/wiki/Wikipedia:Database_download). However, these are so phenomenally large that they cannot be considered for everyday use. These are what I assume professional forks of Wikipedia content are based off today.
For everyday users, the simplest way to fork a project is to use the Special:Export/Special:Import tools (e.g. http://en.wikipedia.org/wiki/Special:Export). Using these, anyone can simply choose the articles they want to fork, export them to a file, and then import that file them into their own MediaWiki installation. Templates are included and if images are from the Commons then these will be picked up by a MediaWiki installation anyway.
There is also the API (http://en.wikipedia.org/w/api.php), which is a very useful tool. If someone had a particular requirement for forking (e.g. they wanted to create a compendium of articles that they or their friends had contributed to on Wikipedia), the API would be a very able tool to do so. Yes, you need technical knowledge to do so, but no more than you would need to properly maintain a fork in any case. For someone with medium programmings skills, it's not hard, and there are tools out there to make it even easier (http://www.mediawiki.org/wiki/API:Client_Code).
In all, it is quite easy to fork any Wikimedia project. There may be a question of distributed sharing of the DB dumps (e.g. share via a torrent so that if the Wikimedia turned evil there may still be seeders of the DB dumps out there). But, all things being considered, there are plenty of Plan C's out there - and they are being used in practice already.
Regards, Oliver
2011/8/15 David Richfield davidrichfield@gmail.com
On Mon, Aug 15, 2011 at 6:04 AM, Tim Starling tstarling@wikimedia.org wrote:
On 12/08/11 20:55, David Gerard wrote:
THESIS: Our inadvertent monopoly is *bad*. We need to make it easy to fork the projects, so as to preserve them.
I must have missed the place where you actually made this case. I tried reading your blog posts but I didn't see it there.
In 2005 you said that the point is to insure the data against the financial collapse of the Foundation.
It's not just financial collapse. When Sun was acquired by Oracle and they started messing about with OpenOffice, it was not hard to fork the project - take the codebase and run with it. It's not that easy for Wikipedia, and we want to make sure that it remains doable, or else the Foundation has too much power over the content community.
Let me make it clear that I currently am happy with the Foundation, and don't see a fork as necessary. If the community has a problem with the board at any point, we can elect a new one. If things change, however, and it becomes clear that the project is being jeopardised by the management, we need a plan C.
-- David Richfield e^(πi)+1=0
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
wikimedia-l@lists.wikimedia.org