[Wikitech-l] Gather update

Hi People Interested in Gather, Given the reorg and the traffic being driven to beta, we need to revisit Gather's Q4 goals: TLDR: Continue working on gather to finish MVP features (end of the month, max), not pushing to stable unless we see 2x the number of logged-in edits on beta (or >10x current state). Before the serious stuff: Top 5 best collections since Monday: https://en.m.wikipedia.org/wiki/Special:Gather/by/Johnrenfro https://en.m.wikipedia.org/wiki/Special:Gather/id/3454/Philadelphia_watch https://en.m.wikipedia.org/wiki/Special:Gather/id/3401/engineering https://en.m.wikipedia.org/wiki/Special:Gather/id/3532/Mental_Health https://en.m.wikipedia.org/wiki/Special:Gather/id/3132/Libertarianismo Okay, now the goals. Please feel free to comment in email or in this google doc <https://docs.google.com/a/wikimedia.org/document/d/1DK1pa3PEpIbiON0Bj6BrhGAnePqZcKSmqTW1z6sTKvU/edit?usp=sharing> Thanks, Jon Given the reorg and an improvement made to beta, we need to revisit Gather's goals Pushing to Stable Given that we can now test Gather numbers in beta, the original goal of launching on stable to test adoption is no longer valid. It is expensive in terms of future maintenance and commitments to launch features to stable, so we only want to do so after we have proven success, if possible. Target numbers Originally, the benchmarks for success on stable that were agreed on were low (10k creators a month on stable, and 1k shares). Given the current beta numbers, it looks like we will blow the first number out of the water. Share has been deprioritized given current usage patterns. However, given that we now have 4 engineers in charge of the entire web, the standards for what we work on have to be more rigorous. We cannot allocate multiple engineers to an experimental product unless it shows promise of impacting greater numbers 1. Goal 1. New Goal: 1. Round out Gather hypothesis (criteria below) 2. By end of June know whether or not we want to push Gather to stable 1. Next Eng+PM Steps (to round out hypothesis) 1. Improve onboarding (a few tasks) 2. Surface collections publicly (this is a big missing feature) 3. Qualitative and quantitative research 2. Criteria for passing to stable: We don’t have a great way to measure success of Gather based on usage by a proportion of users or logged in users. However, we can compare to something similar like edits. In terms of pure value to WP, we can consider a collection to be like a low-value edit. - There are 2x ‘good’ collections made as there are total logged-in edits/month - in May there were 2,180 edits by logged in users (2,694 logged out). - At current rates this suggests ~4,500 collections per month is our target. - ‘good’ here, means >1 collection--it’s not a perfect definition, but it’s a strong proxy. - Our current rate of ‘good’ collections is roughly 250 a month, so we will need to increase the number by almost 20x. - It might be worth exploring % of those 2,180 edits that are reverted and adjusting down accordingly OR - Views of collections or where collection is the referrer > .5% of total pageviews - Currently, the views of collections are minimal. - If very few people create collections, but they drive a significant boost in page views, say .5% of total PVs (not an end goal, but also not bad for an MVP introducing a new use case), then the feature is a success. 1. What if we don’t pass to stable: Lets burn that bridge when we get to it :) . Seriously though, until we get some qualitative data back from our readers, we will not be able to make important calls on the feature as is. There are a few great alternatives I can think of right off the bat: - Keep code in beta and work on Gather opportunistically or as qualitative data dictates - One example might be to make collections private by default and launch as bookmarker for readers (primary current use-case) - Promote as beta feature on desktop - Use codebase as start of multiple watchlists (some good work started here by JRobson) - Codebase is fairly generic list table that has some basic and interesting features built in that could be used for a number of other ends 1. Validation and why we choose this criteria: Qualitative questions to answer: Why aren't more people using Gather? (correctly) Why aren't people returning to use Gather more Success metric questions to answer (that we can’t already): What % of logged in users use Gather What % of users who visit > 1 page use Gather What is our denominator Status Measure logins and signup funnel directed from Gather, what is % success rate? This is not instrumented Measure % of sessions with more than 1 pageview Working on this Measure % of sessions with logged in users Cannot get this What is our baseline? Logged in edits is probably the best thing to measure against.