Hello all!
*Bottom Line *Please review the Event Schema Audit
<https://docs.google.com/spreadsheets/d/1WXbGPyuu2S6TYvrb-DvWWmrEx_K7TJ5rYPkjhvgWjoI/edit?usp=sharing>[1]
and request changes via comment by October 16*.
**Any and all schemas that are not designated for migration will be
deprecated. Datasets who rely on deprecated schemas will no longer receive
data*
*Background:*
- The Modern Event Platform (MEP)
<https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC2:_Modern_Event_Platform>
will
be the new infrastructure for building services that produce and consume
event data for analytics and production. In order to take advantage of the
new platform data, schemas need to be migrated from the old format
(EventLogging) on meta to the new MEP specification. Given all teams have
schemas, instrumentation and process surrounding the previous way of
generating event data, teams will be required to shift in how they
instrument and produce these events moving forward on the new MEP.
- To do this, we must
- Make a copy of the original Event Logging schema into the new system
and
- Change the name of the original schema to legacy_<name>
- Any instrumentation using the original schema will automatically use
the updated schema. There is no change in application code. Events will
flow to the same database table. There should be no interruption in data,
and there is a QA process in place during this transition.
- Consequences for you and your team:
- Any future modifications to the schema will need to be made in the new
system
- A few additional fields will appear in your database table
- Switching the schema in this way will not make the features of the
Modern Event Platform project available immediately. To get these features,
we need to author a fresh schema and update the instrumentation code. This
is the final step of the migration and will be done in concert with Data
Scientists and Product Teams at a later date.
- You can find out more about our Migration Plan & Timeline
<https://docs.google.com/document/d/1LZ3ZijXePGqur3LAkH9LWSmkTBuTZuGJPlgHeiISwx4/edit#>[3]
and reach out to me if you have any further questions
[1]
https://docs.google.com/spreadsheets/d/1WXbGPyuu2S6TYvrb-DvWWmrEx_K7TJ5rYPk…
[2]
https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC…
[3]
https://docs.google.com/document/d/1LZ3ZijXePGqur3LAkH9LWSmkTBuTZuGJPlgHeiI…
Best,
--
Seve Kim *(he/him)*
Sr. Technical Product Manager
<https://wikimediafoundation.org/>
*"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."*