<tl;dr>: Your feedback is wanted on
https://www.mediawiki.org/wiki/Developer_Advocacy/Developer_Portal/Content_…
Hi everyone,
last year[1] the Developer Advocacy team started to work on a single,
central entry point for developers and tech-minded people for
Wikimedia's technical documentation[2].
A central entry point ("developer portal") would cover common technical
use cases to allow existing and future technical contributors and
developers to find the information they need.
Each use case links to its most relevant documentation (i.e. to pages
on wikitech or mediawiki.org).
This is part of a larger initiative to implement an organization
strategy for key technical documents: Understand challenges about
finding and maintaining docs, identify key docs, and investigate ways
to improve our workflows around documentation.
So far we:
* Researched and reviewed existing documentation venues and pages
* Reviewed developer/documentation portals in the broader industry
* Interviewed several engineering teams around technical documentation
workflows, audiences, and key technical docs (the key themes from
these conversations are available, see the link above)
* Created an initial draft for the structure and content of the single
entry point
Now we would like to improve this initial content draft with your help.
1) Please take a look at the initial draft at
https://www.mediawiki.org/wiki/Developer_Advocacy/Developer_Portal/Content_…
2) Then, please help improve it by sharing your thoughts and feedback
until *May 25th* at
https://www.mediawiki.org/wiki/Developer_Advocacy/Developer_Portal/Content_…
Note that this is only a draft how to structure content.
It is not a design or layout proposal and it is not an implementation.
For additional future work, see also the Phabricator workboard[3].
Next steps include:
* A session at the remote Hackathon (May 22-23; [4])
* Incorporate content improvements, based on your feedback
* Check the documents linked from the single entry point proposal for
accuracy
* Investigate requirements for the technical implementation
* Investigate improvements of processes around technical documentation
(structure, locations, navigation, stewardship, etc).
If you want to learn more about the project, please see
https://www.mediawiki.org/wiki/Developer_Advocacy/Developer_Portal
Thanks to everybody who has provided their valuable input to get to
this stage, and thanks in advance to everyone who will!
Cheers,
andre
[1] https://lists.wikimedia.org/pipermail/wikitech-l/2020-August/093773.html
[2] https://www.mediawiki.org/wiki/Developer_Advocacy/Developer_Portal
[3] https://phabricator.wikimedia.org/tag/wikimedia-developer-portal/
[4] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2021
--
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
Hello,
The committee has finished selecting new members and the new committee
candidates are (In alphabetical order):
- Amir Sarabadani
- Kunal Mehta
- Martin Urbanec
- MusikAnimal
- Tony Thomas
And auxiliary members will be (Also in alphabetical order):
- Ariel Glenn
- Effie Mouzeli
- Huji
- Jayprakash12345
- Nuria Ruiz
You can read more about the members in the mediawiki.org page
<https://www.mediawiki.org/wiki/Code_of_Conduct/Committee/Members/Candidates>
.
The changes compared to last term are:
- Kunal Mehta is joining the main CoC committee
- Ariel Glenn is coming back to auxiliary committee
- Effie Mouzeli is joining auxiliary committee for the first time
- Tonina Zhelyazkova is leaving the committee
- Matanya and Tpt are leaving the auxiliary committee
- I won't run for the chair this year.
This is not the final structure. According to the CoC
<https://www.mediawiki.org/wiki/Code_of_Conduct/Committee#Selection_of_new_m…>,
the current committee publishes the new members and calls for public
feedback for *six weeks* and after that, the current committee might apply
changes to the structure based on public feedback.
Please let the committee know if you have any concern regarding the members
and its structure until *26 May 2021* and after that, the new committee
will be in effect and will serve for a year.
Amir, On behalf of the Code of Conduct committee
Best
Hello all,
As the 2021 WMF Board elections have been announced, I have proposed to
reconsider the eligibility criteria for “developers” and technical
contributors, as the previous criteria do not make sense any longer and
exclude several groups of contributors.
A discussion has been started at https://phabricator.wikimedia.org/T281977
and
https://meta.wikimedia.org/wiki/Talk:Wikimedia_Foundation_elections/2021#Re…
.
Please share your thoughts and feedback.
Regards
Jay Prakash (he/him)
Volunteer Developer, Wikimedia
I figure this might be of interest to some here: we could make
sponsorship of contributions more evident by adding a git commit
pseudo-header:
Sponsored-by: Name Of Sponsoring Entity
Obviously this would be optional, and only used by those who want to.
Spelled out in a little more detail:
https://blog.liw.fi/posts/2021/05/26/sponsored-by/
--
WMF release engineering team | he/him or they/them
"Imagine a world in which every single human being can freely share in
the sum of all knowledge."
Hello,
If you missed the previous updates, there's the first
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>and
the second
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>one.
This is the last one I'm sending but in a good way.
We now finished migrating all 57 core tables to abstract schema. Now tables.sql
is empty for MySQL
<https://gerrit.wikimedia.org/g/mediawiki/core/+/c60ccf4e6d4932dddc2efd72a8a…>.
We will soon remove the tables.sql files and links to them.
We have also cleaned more than hundreds of old schema change files.
<https://phabricator.wikimedia.org/T272199> That enabled us to actually
look for unused sql files and drop tens of unused ones that have not been
used since 2002 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/668552>,
2004 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/668761>, or 2005
<https://gerrit.wikimedia.org/r/c/mediawiki/core/+/670176> (and much more)
and they got lost in the sheer number of our sql patch files.
The next update is exciting for me. With abstraction in place, we can now
have a proper tracking of drifts between schema in paper and our production
(This is a follow up from a major incident in 2018). Now we have
https://drift-tracker.toolforge.org/ that keeps track of these drifts. Our
schema has been around for more than twenty years and we have hundreds of
database hosts, making sure everything is using the right database schema
(and stays correct) is impossible manually and we have been finding and
fixing these drifts since 2018, see this comment onwards
<https://phabricator.wikimedia.org/T104459#4314828>.). Of course more work
in improving the tracker is welcome (here's the list
<https://phabricator.wikimedia.org/project/board/5350/>)
If you want to enjoy the benefits of abstract schema [1] in extension(s)
you or your team maintains, Please abstract the schema of your extension.
There's a long list of WMF deployed extensions that are not using abstract
schema <https://phabricator.wikimedia.org/T261912> and some even already
have a patch that only needs reviewing. Once that's done, we can add that
to drift tracking and have a more comprehensive list of potential issues.
If you need help with the abstraction work, just ping me.
This also helped us resolve several long-standing tickets like T104459
<https://phabricator.wikimedia.org/T104459> (5.5 years old), T62962
<https://phabricator.wikimedia.org/T62962> (7 years old), and T42626
<https://phabricator.wikimedia.org/T42626> (soon reaching its ninth
birthday) and will help us to address even more tech debt in future.
There is more to be done, improving the abstract schema *change* system,
finding a home of schema documentation, improving the drift tracker and
making it more automated, so much more. But the biggest chunk of work is
now finally done.
I really would like to thank Ammarpad for great work on abstracting the
tables and handling all sorts of edge cases, James Forrester and Tgr for
their reviews which without them this wouldn't be possible and Sam Reed who
wrote a script to speed up migration
<https://github.com/Ladsgroup/db-analyzor-tools/blob/master/db_abstractor.py>.
This was a team work to its core.
[1] In more details, by abstracting you will have automated checks for
dirfits of the schema of extension(s) you maintain and production. You will
have Postgres support for free. Also, you can have automated documentation
generation, ability to test the schema itself, and have better consistency
of your data types (like one datatype for timestamps).
Until the next adventure.
--
Amir (he/him)
Oh, hi there
tl;dr: we’re still moving MediaWiki into Kubernetes in Wikimedia
Production; there are resources on Wikitech[0] for folks interested in
learning more about the specifics of our Kubernetes pipeline.
⏳ Progress
Kubernetes is the future deployment platform for most Wikimedia code.
The developer and deployment experience under Kubernetes will
(hopefully) be easier and safer and developers will have more
autonomy.
The first Wikimedia production service (Mathoid) moved to Kubernetes
in July, 2018. Today there are 25 services using our production
Kubernetes infrastructure serving 30,000 requests per second, about
67-ish% of our overall traffic. All services are deployed by their
service owners without much oversight.
Our Docker registry[1] contains 311 Docker images with 7,690 tags. Our
production images are built via Continuous Integration (CI), using our
deployment pipeline. Blubber[2] and PipelineLib[3] were introduced in
2018 to allow developers to specify how they wanted their CI to run
and when to publish images to our shared registry.
Now we’re in the process of applying all we’ve learned from our
service migration to the MediaWiki migration to Kubernetes.
🏆 Goal
* Open Container Initiative (OCI)-compatible MediaWiki images
* Built, tested, and promoted by the image pipeline
* Deployed to Kubernetes via Helm
* Serving production traffic
🧐 Why
* Development - Standard platform for development and production
* Deployment - Safer, simpler, industry standard
* Production - Increased capacity, redundancy, reliability
👏 Now
* We’re building Docker images and Helm charts
** Release Engineering crammed all 184 branched extensions, 4 skins,
vendor, and core Mediawiki code into a single image alongside our
massive localization of 435 languages
** ServiceOps is working on php-fpm, apache, nutcracker, mcrouter and
envoy images and their corresponding helm charts
* We’re making our Kubernetes better
** ServiceOps has upgraded our base images and Kubernetes cluster
🔜 Next
* Figuring out how deployments work
** Backports, security releases, and train
* Figuring out how image upgrades work
** Is a php-fpm upgrade a deployment now? What does that look like?
* Figuring out how our Wikimedia configuration changes
** Currently a change to configuration requires a code change, does
this code change kick off a whole image build?
🎓 Resources
Kubernetes will be very impactful in our production services and we
would like to encourage those interested in this change to become
familiar with its concepts.
Please have a look at our collected Wikitech Kubernetes education
resources, tutorials, and guides[0]. Add material if you have had a
good experience with a class or tutorial.
Otherwise stay tuned and watch that page for additional resources to
be added. You will hear from us again as we have additional things to
report.
– <3
Tyler Cipriani (he/him) (On behalf of all the fine folks working on
Kubernetes for MediaWiki)
Engineering Manager, Release Engineering
Wikimedia Foundation
[0]: <https://wikitech.wikimedia.org/wiki/Kubernetes/Kubernetes_Education>
[1]: <https://docker-registry.wikimedia.org>
[2]: <https://wikitech.wikimedia.org/wiki/Blubber>
[3]: <https://wikitech.wikimedia.org/wiki/PipelineLib>
*subject line reference: https://en.wikipedia.org/wiki/The_More_You_Know
Hello,
On Tuesday June 1st at 7:00 UTC we will switch Gerrit from Java 8 to
Java 11. The system will be briefly unavailable has it restarts with the
JVM.
The window would be roughly half an hour, the service interruption
should be just a few minutes. If anything is suspicious we will roll
back to Java 8 and restart which would lead another short interruption.
The task is: https://phabricator.wikimedia.org/T268225
--
Antoine "hashar" Musso
= 2021-05-26=
== Callouts ==
* The Structured Data team will be removing MediaSearch UI code from
WikibaseMediaInfo soon. If you want to look at or use our reusable Vue
components, you can find them in the new MediaSearch extension:
https://gerrit.wikimedia.org/r/admin/repos/mediawiki/extensions/MediaSearch
* No train week of 2021-05-31
== Gerrit patches or GitHub Pull Requests for reviews or feedback ==
*
=== No updates ===
CommTech, AHT, Editing, iOS, Android, Prod Infra, Parsing, Library,
Analytics, Cloud Services, Security, WMDE Technical Wishes
== SoS Meeting Bookkeeping ==
* Updates: today is the last instance of this meeting in its current form
== Product ==
=== Growth ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Continuing work on Add Link https://wikitech.wikimedia.org/wiki/Add_Link
*** planning to enable for 50% of new users on pilot wikis (arwiki, bnwiki,
cswiki, viwiki) this week
** Continuing work on on-wiki configuration
https://www.mediawiki.org/wiki/Growth/Community_configuration
*** planning to enable on pilot wikis this week
=== Web ===
* Blocked by:
* Blocking:
* Thank yous:
** Huge thanks to Trey Jones for their help with the highlighting of query
highlighting in the WVUI typeahead-search component:
https://phabricator.wikimedia.org/T281797
* Updates:
** Instrument QA for the language switcher treatment A/B test:
https://phabricator.wikimedia.org/T280770
** Continuing work on on the new user links (personal tools) treatment:
https://phabricator.wikimedia.org/T266536
=== Structured Data ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** New MediaSearch extension is on production, will disable WBMI version of
the UI this week
** We will soon remove our Vue code from WBMI, so if you want to look at or
use our reusable components, you can find them in the MediaSearch
extension:
https://gerrit.wikimedia.org/r/admin/repos/mediawiki/extensions/MediaSearch
=== Abstract Wikipedia ===
* Blocked by:
** None.
* Blocking:
** None known.
* Thank yous:
** Thanks to Jen Wright for help with the TDMP process alignment.
* Updates:
** Close to finishing Phase ε (epsilon):
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases
** Security & Performance reviews scheduled to end this Phase; our thanks
to them!
=== Language ===
* Blocked by:
* Blocking:
* Thank yous:
** Big thanks to the DBAs (Jaime and Manuel) for helping to find the cause
for https://phabricator.wikimedia.org/T282905
** To Antoine "hashar" Musso for quickly fixing:
https://phabricator.wikimedia.org/T283665 that was failing ULS patches from
being merged
* Updates:
** No major updates
=== Inuka ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Icon / Banner for Wikipedia Preview Wordpress Plugin:
https://phabricator.wikimedia.org/T283623
** High memory consume issue for KaiOS v1.5.0 on Jio phone:
https://phabricator.wikimedia.org/T283725
=== Vue.js ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Revewing feedback from the Technical Desicion Making forum on adding a
build step to mediawiki, working on an additional proof of concept
** Radio component is now available in WVUI:
https://doc.wikimedia.org/wvui/master/ui/?path=/story/components-radio--rad…
** Evaluating WVUI icon system: https://phabricator.wikimedia.org/T282625
** Evaluating the use of TypeScript in WVUI:
https://phabricator.wikimedia.org/T282835
** Preparing to cut a new release of WVUI
== Technology ==
=== Fundraising Tech ===
[From last week.]
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Working on new API integration for backup card processor
** Moving lots of CiviCRM code off drupal 7
=== Engineering Productivity ===
==== Quality and Test Engineering ====
[From last week.]
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Blog post by Željko Filipin: macOS vs Ubuntu Linux - `npm ci` & `npm run
selenium-test` with and without Fresh
https://phabricator.wikimedia.org/phame/post/view/241/macos_vs_ubuntu_linux/
==== Release Engineering ====
* Blocked by:
**
* Blocking:
**
* Thank yous:
** Thank you to Zabe for lots of quick fixes for train blockers.
** Thank you to James Forrester for code reviews.
* Updates:
** [All] Deployments/Covid-19
https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
** Train Health
*** Last week: 1.37.0-wmf.6 [[phab:T281147]] <!--
https://phabricator.wikimedia.org/T281147 -->
*** This week: 1.37.0-wmf.7 [[phab:T281148]] <!--
https://phabricator.wikimedia.org/T281148 -->
*** Next week: no train
https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-05-26
=== Search Platform ===
[From last week.]
* Blocked by:
* Blocking:
* Thank yous:
** Alex, Effie and Janis for helping navigate deploying Flink to k8s and
for being flexible and finding creating solutions!
* Updates:
** Service implementation for relforge100[34] -
https://phabricator.wikimedia.org/T262211
** Generate SSL certification for relforge1003.eqiad.wmnet and
relforge1004.eqiad.wmnet - https://phabricator.wikimedia.org/T275885
** Expose wdqs1009 to wdqs users and gather feedback -
https://phabricator.wikimedia.org/T266470
** Reboot elasticsearch* and relforge* to apply kernel security updates -
https://phabricator.wikimedia.org/T280563
** Unpack Spanish Elasticsearch Analyzer -
https://phabricator.wikimedia.org/T277699
** Cleanup duplicate indices in cloudelastic -
https://phabricator.wikimedia.org/T279009
** SonarQube should analyze all Search Platform projects -
https://phabricator.wikimedia.org/T264877
** Transfer one codfw wdqs-internal host over to codfw wdqs (public) -
https://phabricator.wikimedia.org/T281498
=== Site Reliability Engineering ===
[From last week.]
* Blocked by:
** None
* Blocking:
** None
* Thank yous:
* Updates:
** Shellbox deployment ongoing
** new caching PoP (point of presence - 2nd EMEA site) named drmrs buildout
to begin, puppet patches started trickling in
== Cross-cutting ==
* Blocked by:
** [long term] Search Platform: PHP 8.0 work is long-term blocked on the
migration to ElasticSearch 7.0 https://phabricator.wikimedia.org/T263142
(or at least 6.7).
* Blocking:
** None known.
* Thank yous:
** Special thanks to bd808 and legoktm for their work helping with the
freenode crisis.
* Updates:
** Lots of energy spent on the freenode -> Libera migration; disruption not
ideal. :(
** REL1_36 release process:
*** 1.36.0 should be released soon. Thanks to all who have helped.
** CI tools:
*** mediawiki-codesniffer v36 upgrade completed. Thanks to everyone who has
made this happen, especially Umherirrender.
*** CI tools' upgrade status:
https://libraryupgrader2.wmcloud.org/status?branch=master
** PHP 8.0:
*** No changes this week.
*** Our target remains REL1_35 (and thus also REL1_36), as well as master.
*** Upstream libraries: Elastica-related PHP code is theoretically the last
one.
*** Core: Some unit and integration tests still fail; thank you to everyone
working on fixing them.
[[Category:Scrum of scrums{{#translation:}}|*]]