I see little effort on finding solutions potentially able to scale to all

It's inappropriate an unsustainable to hardcode such wiki-specific

Is it considered acceptable now to produce a service or API that hardcodes wiki-specific parsing of certain wikitext or HTML patterns in certain wiki pages (such as the "On this day" section of the main page of one wiki)? I'm confused by the status of things and after my comment https://phabricator.wikimedia.org/T143408#2919000 I see little effort on finding solutions potentially able to scale to all our projects and languages (which I assume to be the mission, see "globally" in https://wikimediafoundation.org/wiki/Mission_statement ; please point it out if this assumption is incorrect). It might be that wiki-specific parsing hardcoded in MediaWiki/Wikimedia code is actually able to scale, if written correctly; a comment on the association patch seemed to imply so. This would be a very surprising finding, and one which goes against 15 years of experience, so if we have some examples or evidence of this it would be very worthwhile to point them out. Nemo _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l

What is also in my mind: the app wasn't easy to fork and use on other (third-party) projects before that, too, however with the track in this direction (specific Parsons of contents, which apply to Wikimedia projects only) makes it, in reality, impossible to fork (and maintain) the app projects to other, MediaWiki based, third-party projects. I understand, that this might not be the goal of the WMF, which in my personal opinion isn't quite correct, however it's very very sad. I tried to maintain an up-to-date version of the Wikipedia app some time ago, but it takes so much time, as it is so Wikimedia, and not MediaWiki specific, that I ended up mostly stopping the effort in this direction. This is probably not a response which should be in this thread, and I apologize for that, however I wanted to say that somewhere. Best, Florian _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l

Hey all, just want to drop in some thoughts on his thread… I would say the premise of this email is absolutely right: Parsing out this data by hand coding against specific HTML structures is untenable. On Wikipedia we have a wealth of community curated information: featured articles, in the news, on this day, etc… Over the past year, the Reading team has been working with Services to setup some basic infrastructure for ingesting some of this data and making it available as structured data via an API that any client can ingest. This includes our own WMF maintained projects (apps, mobile web, desktop) Because of the nature of our projects, it is difficult to extract this information in a uniform way across all wikis. Though this is clearly the target (and in line with our mission), before we invest significant time in developing such a standardized method for each service, we need to first deploy an API to test whether these services are actually a good direction for our products, services, and mission. To that end, we develop each new service on en.wikipedia first in a method that we do not intend to scale. Now that some of these services have proven useful, we have begun efforts to develop a way for all project maintainers to opt in to making their curated content available for consumption in these APIs. You can see some tickets focused on this effort here: https://phabricator.wikimedia.org/T150806 https://phabricator.wikimedia.org/T152284 https://phabricator.wikimedia.org/T148680 We have also created some draft documentation that we are currently gathering feedback on to see if it is viable for all projects here: https://www.mediawiki.org/wiki/User:CFloyd_(WMF)/Feed_Markup_Documentation Additionally, we have also added additional resources to our Reading Infrastructure team (which maintains our services) in part to help with this effort. All this is to say, is that creating and scaling these services to multiple wikis is a continuing effort. While we would love to deploy a solution to all projects at once, in order to make the problem tractable, we are tackling it in steps and re-evaluating our assumptions along the way. Hopefully this explains our thinking and the projects in a way that make sense. Because this is a large project, we are looking for solutions and help to spread these services across all wikis - If you have or anyone time and would like to help, the tickets and documentation above are great place you to contribute to the process. Thanks for any help Corey On Thu, Jan 12, 2017 at 10:45 AM Monte Hurd &lt;mhurd(a)wikimedia.org&gt; wrote: Thanks Florian. I see your point about ease of forking. Would you have time later, perhaps off thread, to detail the challenges you faced? Regarding the data endpoint topic of this thread, it isn't app-specific despite being part of 'mediawiki/services/mobileapps'. We'll probably want a more generic name with only truly app specific endpoints named accordingly. On Thu, Jan 12, 2017 at 10:05 AM, Florian Schmidt < florian.schmidt.welzow(a)t-online.de&gt; wrote: What is also in my mind: the app wasn't easy to fork and use on other (third-party) projects before that, too, however with the track in this direction (specific Parsons of contents, which apply to Wikimedia projects only) makes it, in reality, impossible to fork (and maintain) the app projects to other, MediaWiki based, third-party projects. I understand, that this might not be the goal of the WMF, which in my personal opinion isn't quite correct, however it's very very sad. I tried to maintain an up-to-date version of the Wikipedia app some time ago, but it takes so much time, as it is so Wikimedia, and not MediaWiki specific, that I ended up mostly stopping the effort in this direction. This is probably not a response which should be in this thread, and I apologize for that, however I wanted to say that somewhere. Best, Florian _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l