95% is pretty extreme.

I have always questioned the balance being struck here, and would welcome an adjustment of the minimum requirements to run MediaWiki. In many cases, if we can just require shell access we can automate away the complexity for the typical use cases.

95% is pretty extreme.

On Jan 16, 2015 2:49 AM, "Max Semenik" <maxsem.wiki(a)gmail.com> wrote: wrote: a especially question by Im quite concerned by this. Well shared hosting might be on the decline, I still feel like most third part users dont have the competence to do much more than ftp some files to their server and run the install script.

On 16 January 2015 at 07:38, Chad <innocentkiller(a)gmail.com> wrote: In this case the problem is leaving users and their wiki content unsupported. Because they won't move while it "works", even as it becomes a Swiss cheese of security holes. Because their content is important to them. This is the part of the mission that involves everyone else producing the sum of human knowledge. They used our software, if we're abandoning them then don't pretend there's a viable alternative for them. You know there isn't.

This sounds like a problem we need to fix, rather than making it worse. I'd most wikis are not up to date then we should work on making it easier to keep up to date, not making it harder. Any SOA approach is sadly DOA for any shared host; even if the user has shell access or other means of launching processes (so excluding almost every free host in existence here), there is no guarantee that a particular environment such as node.js or Python is installed or at the correct version (and similarly no guarantee for a host to install them for you). Such a move, while possibly ideal for the WMF, would indeed make running on shared hosting nearly if not entirely impossible. The net effect is that people will keep their existing MW installs at their current version or even install old versions of the software so they can look like Wikipedia or whatnot, rife with unpatched security vulnerabilities. I cannot fathom that the WMF would be ok with leaving third-party users in such a situation.

Moving advanced things into services may make sense, but it should not come at the expense of making the core worthless for third parties; sensible refactors could still be done on core while leaving it as pure PHP. Services, if implemented, should be entirely optional (as an example, Parsoid is not required to have a base working wiki. I would like to stick to that model for any future service as well).

I challenge the very foundation of arguments based on this 95% statistic. The 95% statistic is bogus, not because it's inaccurate, but because it's misleading. The number of MediaWiki instances out there is meaningless. The value we get from 3rd party use correlates much more strongly with the number of users, not the number of instances. Consider the experience of using a MediaWiki instance running on a shared host. MediaWiki doesn't actually perform well enough in this environment to support a significant amount of usage. It's simply not possible that shared host installs can achieve enough usage to be relevant.

On Fri, Jan 16, 2015 at 12:09 PM, Isarra Yos <zhorishna(a)gmail.com> wrote: The 95% statistic may not be meaningful, but neither is this. The number of I believe the claim is being made that many of those people with tiny one-off self-hosted projects are *not* being well-served right now by a MediaWiki instance that's out of date with security holes and no new features. Here's where I tend to make the comparison to WordPress: WordPress is open source and available for self-hosting (including on shared hosts with PHP and MySQL), but a lot of people find it's a pain to maintain -- even with the HUGE usability and upgrade improvements that WordPress folks have made in the last few years. I personally maintain my own WordPress blog, and pretty much just occasionally make a security update and otherwise hardly touch it. Plenty of other people do this too, with varying degrees of success staying up to date and fixing occasional operational problems. But lots of people also use WordPress's hosted service as well. No need to manually install security upgrades, maintain your own vhost, etc. In our comparison, we have MediaWiki for self-hosters, and we have hosted projects for Wikimedia, but if you want a hosted wiki that's not covered in ads your choices are vague and poorly documented. I would argue that many small projects would be better served by good wiki farm hosting than trying to maintain their own sites. Sure, that's not for everyone -- but I think the people who are best served by running their own servers have a lot of overlap with the people who would be capable of maintaining multiple services on a virtual machine. -- brion

wrote:

wrote:

I think the confusion between "third party support" and "an open source project" is unhelpful. We're obviously an open source project with lots of contributors who aren't paid (and many of them are motivated by Wikimedia's mission), it's just that the project puts primary emphasis on the Wikimedia mission, and only secondary emphasis on third party needs. I don't think it's a dirty secret that we moved to a model of contracting out support to third parties -- there was even an RFP for it. ;-)

On Fri, Jan 16, 2015 at 1:05 PM, Brad Jorsch (Anomie) < bjorsch(a)wikimedia.org Source There was a WMF engineering meeting where it was announced internally. I was the only one that spoke against it. I can't give a citation to it because it was never announced outside of WMF, but soon after that third party support was moved to a grant funded org, which is the current status quo.

On Fri, Jan 16, 2015 at 1:05 PM, Brad Jorsch (Anomie) <

> On Fri, Jan 16, 2015 at 12:27 PM, Ryan Lane &lt;rlane32(a)gmail.com&gt; wrote: > > > What you're forgetting is that WMF abandoned MediaWiki as an Open

There was a WMF engineering meeting where it was announced internally. I was the only one that spoke against it. I can't give a citation to it because it was never announced outside of WMF, but soon after that third party support was moved to a grant funded org, which is the current status quo. Don't want to air dirty laundry, but you asked ;). - Ryan

I don't think adding a couple more services is going to drastically alter the current situation.

On 16 January 2015 at 07:38, Chad &lt;innocentkiller(a)gmail.com&gt; wrote: In this case the problem is leaving users and their wiki content unsupported. Because they won't move while it "works", even as it becomes a Swiss cheese of security holes. Because their content is important to them. This is the part of the mission that involves everyone else producing the sum of human knowledge. They used our software, if we're abandoning them then don't pretend there's a viable alternative for them. You know there isn't.

However, with clear API architecture we could maybe have alternatives - i.e. be able to have the same service performed by a superpowered cluster or by a PHP implementation on the URL on the same host.

The problem there is that the PHP implementation is likely to code-rot, unless we create really good unit tests that actually run for it.

It would be less bad if the focus were on librarization and daemonization (and improvement) of the existing PHP code so both methods could share a majority of the code base, rather than rewriting things from scratch in an entirely different language.

That becomes a hazard when other stuff starts to depend on the non-reduced feature set.

So what we might end up with: - Wikimedia using the SOA MediaWiki with split components maintained by staff and the Wikimedia volunteers devs. Code which is of no use for the cluster is dropped which would surely ease maintainability. We can then reduce MediaWiki to a very thin middleware and eventually rewrite whatever few code is left. - The old MediaWiki PHP based is forked and given to the community to maintain. WMF is no more involved in it and the PHP-only project live it's on life. That project could be made to remove a lot of the rather complicated code that suits mostly Wikimedia, making MediaWiki simpler and easier to adjust for small installations. So, is it time to fork both intent? I think so.

<quote name="David Gerard" date="2015-01-16" time="16:27:22 +0000">

> This is not a great idea because it makes WMF wikis unforkable in > practical terms. The data is worthless without being able to run an > instance of the software. This will functionally proprietise all WMF > wikis, whatever the licence statement.

"Unforkable" as a binary is not true.

"Not easy to fork without implementing all of the pieces carefully" is more accurate. But it's still possible, especially since all of the config/puppet code that is used to do it is open and Freely licensed.

It ain't easy hosting a site this large with this amount of traffic (thanks Ops!) so it's not surprising that "forking" it wouldn't be easy either.

Note that feature parity with Wikipedia has not been possible in pure PHP since 2003, when texvc was introduced. And now that we have Scribunto, you can't even copy an infobox template from Wikipedia to a pure-PHP hosted MediaWiki instance. The shared hosting environment has never been preferred, and I'm not particularly attached to it. Support for it is an accidental consequence of MediaWiki's simplicity and flexibility, and those qualities should be valued for their own reasons.

HTML storage would be a pretty simple feature, and would allow third-party users to use VE without Parsoid.

On Tue, Jan 20, 2015 at 10:54 AM, Mark A. Hershberger &lt;mah(a)nichework.com&gt; wrote: Funny, we were just talking about that in the office :-) There are two things that are needed: 1. ContentHandler support for storing HTML. As Tim points out, this should be reasonably straightforward. I imagine there is a long tail of small implementation gaps, but a prototype-quality implementation of this would probably be easy. 2. HTML validation - our current security model relies on the HTML being generated server-side by a wikitext parser. If we cut wikitext out of the loop, we'll need some other way of ensuring that people can't inject arbitrary Javascript/Flash embedding/Java applet/ActionScript/iframe or any other security horrors. #2 is a lot harder, and something we need to decide how we want to implement (e.g. is this something we do in PHP or Node?).

2. HTML validation - our current security model relies on the HTML being generated server-side by a wikitext parser. If we cut wikitext out of the loop, we'll need some other way of ensuring that people can't inject arbitrary Javascript/Flash embedding/Java applet/ActionScript/iframe or any other security horrors.

But there's plenty of room for other initiatives (some could even make

Hello everyone, On 19/01/15 06:47, Tim Starling wrote: functionality, I kind of agree. Instead of seeing things in black and white, aka shared hosting or not, we should take a look at the needs of users who run their MW on a shared hosting. What exactly do they consider "core functionality"? I don't think we actually know the answer yet. To me, it seems very likely that MWs on shared hosts are a starting base into the MW world. Probably, their admins are mostly not technologically experienced. In the near future, most of them will want to see VE on their instances for improved user experience. But do they really care about wikicode? Or do they care about some functionality that solves their problems. I could imagine, templating is one of the main reasons to ask for wikicode. Can we, then, support templating in pure HTML versions of parsoid? Are there other demands and solutions? What I mean is: there are many shades of [any color you like], in order to make users outside the WMF happy. I see shared hosts somewhere in a middle position in terms of skills needed to run and in terms of dedication to the site. On the "lower" ground, there are test instances run on local computers. These can be supported, for example, with vagrant setups, in order to make it very easy to get started. On the "upper" level, there are instances that run on servers with root access, vms, in clouds, etc. They can be supported, for instance, with modular setup instructions, packages, predefined machines, puppet and other install scripts in order to get a proper setup. So shared hosting is a special case, then, but it seems to have a significant base of users and supporters. While the current SOA approach makes things more complex in terms of technologies one needs to support in order to run a setup that matches one of the top 5 websites, it also makes things easier in terms of modularity, if we do it right. So, for example, we (tm) could provide a lightweight PHP implementation of parsoid. Or someone (tm) runs a parsoid service somewhere in the net. The question is, then, who should be "someone". Naturally, WMF seems to be predestined to lay the ground, e.g. by publishing setup instructions, interfaces, puppet rules, etc. But there's plenty of room for other initiatives (some could even make money out of this :)). An ecosystem around MediaWiki can help do the trick. But here's the crucial bit: We will only get a healthy ecosystem around MediaWiki, if things are reliable in a way. If the developer community and the WMF commits to support such an environment. In the current situation, there's so much insecurity I doubt anyone will seriously consider putting a lot of effort in, say, a PHP parsoid port (I'd be happy if someone proves me wrong). So, to make a long story short: Let's look forward and try to find solutions. MediaWiki is an amazing piece of software and we should never stop to salutate and support the hundreds of thousands of people that are using it as a basis of furthering the cause of free knowledge. Best, Markus -----Ursprüngliche Nachricht----- Von: wikitech-l-bounces(a)lists.wikimedia.org [mailto: wikitech-l-bounces(a)lists.wikimedia.org] Im Auftrag von Tim Starling Gesendet: Montag, 19. Januar 2015 06:47 An: wikitech-l(a)lists.wikimedia.org Betreff: Re: [Wikitech-l] The future of shared hosting On 16/01/15 17:38, Bryan Davis wrote: Parsoid depends on the MediaWiki parser, it calls it via api.php. It's not a complete, standalone implementation of wikitext to HTML transformation. HTML storage would be a pretty simple feature, and would allow third-party users to use VE without Parsoid. It's not so simple to use Parsoid without the MediaWiki parser, especially if you want to support all existing extensions. So, as currently proposed, HTML storage is actually a way to reduce the dependency on services for non-WMF wikis, not to increase it. Based on recent comments from Gabriel and Subbu, my understanding is that there are no plans to drop the MediaWiki parser at the moment. There is a proposal to move revision storage to Cassandra, possibly with node.js middleware. I don't think that project requires dropping support for revision storage in MySQL. I think MediaWiki should be a client for multiple revision storage backends, like what we are already doing for file storage. There's no reason to think Cassandra is the best storage system that will ever be conceived; the end of history. There will be new technologies in the future, and an abstract backend API for revision storage will help us to utilise them when they become available. As long as there are no actual reasons for dropping pure-PHP core functionality, the idea of WMF versus shared hosting is a false dichotomy. Note that feature parity with Wikipedia has not been possible in pure PHP since 2003, when texvc was introduced. And now that we have Scribunto, you can't even copy an infobox template from Wikipedia to a pure-PHP hosted MediaWiki instance. The shared hosting environment has never been preferred, and I'm not particularly attached to it. Support for it is an accidental consequence of MediaWiki's simplicity and flexibility, and those qualities should be valued for their own reasons. -- Tim Starling _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l