Brandon, great to hear from you! I think you're working off of old
information--like Matt said, you can edit via the app now. It's cool that
you're inspired to bring up more questions, though, and I'm glad you're
focusing on the design phase of the next experiment. Are you busy for the
next ten years?
Another misconception or oversight I want to bring up is that Fundraising
is the team pioneering the 2/3-page or full-page banners. We're driving
readers from the website to completely closed and somewhat evil payments
platforms. If there's any relevant or even irrelevant research about
interstitials, please apply it to our work, cos we're about to have a huge
impact on the English-speaking community in December. Any complaints about
the Finnish mobile experiment and dogpiling on the awesome apps developers
seem incredibly misplaced while I'm walking around with this "Kick Me" sign
on my backside.
Thanks,
Adam
On Fri, Sep 4, 2015 at 12:12 PM, Brandon Harris <bharris(a)gaijin.com> wrote:
On Sep 2, 2015, at 3:53 PM, Toby Negrin
<tnegrin(a)wikimedia.org> wrote:
1. We're moving people from an open platform
to a closed platform: I
think
this is an oversimplification of the situation --
as has been noted
before,
the android app is 100% open source and while the
data is not, in my
opinion, comprehensive, it's inarguable that a large percentage of mobile
traffic on the internet is from apps. It's not possible to fulfill our
mission[4] if Wikipedia and sister project content is not available in
widely used channels.
I'm not sure this makes a lot of sense. The widest, most-open
content channel that the projects have is through the web interfaces: all
phones, all devices, all computers can access the same content in the same
manner. That is to say: 100% of our readers have the ability to use the
web versions (either desktop or mobile web) where as only a subset can use
the Android app, which is a different subset that can use iOS. (They also
end up having fragmented experiences, which is sub-optimal.)
So it seems to me that the apps are not required to fulfill the
mission. They feel like distractions, and - quite possibly - negatives to
the mission (in that we can't convert Readers into Editors through the app).
(Which, by the way, this whole "focus entirely on readers" shift
seems counter-intuitive to me. Having a billion readers doesn't mean
anything if there aren't any editors anymore. It's a complete failure at
that point.)
2. The campaign was not publicized before launch:
We notified the Finnish
community on their Village pump before the campaign began[5] and the
campaign is detailed on the central notice page[6]. We felt this was
appropriate considering the scope of the test.
Restricting the conversation to two very small, almost
impenetrable discussion areas seems unwise. It seems obvious to me that
this idea and action would cause friction with the community. I don't
think there's any bad-faith going on here, but this definitely feels like
an oversight.
3. Banners/Interstitials don't work/suck/etc:
There's a difference
between
a forced install and letting users know that an
app exists and our
designers have worked hard to make the banners effective without being
excessively intrusive. You can see the designs on the Phab ticket above.
I
don't generally place a great deal of faith
in blog posts or other
company's data -- the google study showing the ineffectiveness of
interstitials has already been challenged by other similarly reputable
sources [7,8]. For this and other reasons, I believe that we need to
gather
our own data.
Is "our own data" more important than the goodwill of our users or
developers? I think that's a big part of why people might be upset about
this: it's a step away from what had classically been the principles
underlying the movement's activities.
Even that said, though: this is the first anyone is saying "yes,
we did some research about interstitials". It seems to me that the Google
study was something that could have been discussed ahead of time. I also
don't understand why we can't do the whole Open Source thing and make use
of other people's research, unless this indicates a further shift into "not
invented here" territory.
4. We don't understand what success looks
like: We are planning a meeting
with our Research team[9] to assess the statistical validity of our
results, but the basic question is if users read more content using the
app
than the mobile web. This information will help
guide us on future
product
decisions and will be shared with the community.
An experiment without a box isn't an experiment.
"We would like to determine if people read more through
the apps than through the web interface" is a _great_ question (but also
one that could probably be answered just by looking at squid logs). I
don't know that it needs an advertising campaign to create app users to do
it (though I could be wrong, and often am, and would love to hear how if
so). It further seems that advertising the mobile apps would create a
biases in the research (if only that "newish" app users are likely to use
it more often earlier in their
"We would like to determine if people download the app
more often if they've been given an interstitial" is also an interesting
question but it's got a secondary question that no one seems to care about:
"How many readers have we put off from returning by showing them this
interstitial?" I know that I often immediately shut windows and tabs when
I'm told "download our app!"
If this were brought to the wider community in a different manner,
there may have been a completely different response:
"We do not believe that people are aware that there are
official Wikipedia apps. We would like to run an experiment to see how
likely people are to switch to the app experience if they know it exists.
Of those that switch, we would like to find out how many of them increase
their usage of the content, and, ideally, we'd like to know which features
of the app are the most popular and useful. Additionally, we'd like to
know the drop-off counts. We want to do it in a controlled environment
where we understand the patterns as they exist fairly well. We'll run this
experiment for X days, and we know that there will be biases on W, Y, and
Z."
I could easily get behind that set of questions.
I don't really expect a response to any of this, by the way.
---
Brandon Harris :: bharris(a)gaijin.com :: made of steel wool and whiskey
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l