In production we run CirrusSearch against Elasticsearch with a bunch of
plugins installed - wikimedia-extra and the experimental highlighter. But
Cirrus is capable of running without them. It'd be simpler for us if we
simply mandated that they be installed. I created
https://phabricator.wikimedia.org/T101029 to talk about it by my current
feeling is that it probably wouldn't be a huge hardship to make those
plugins required.
Opinions?
Nik
At two recent meetings, several topics around event logging came up.
Specifically, how/where our different systems log (or should log) their
events.
The topics I'm aware of are:
1. Should WDQS use our standard event logging system? I think the answer
was yes. Do we have or need a phab ticket to represent that?
2. Should Our new maps work use our standard event logging system?
3. Should Cirrus/Elastic use our standard event logging system?
4. Should Cirrus/Elastic log to Kibana? What are the privacy implications
there?
It's probably best to reply to only one topic at a time, so this thread
gets forked. Ideally, please change the subject line to reflect the
specific topic you are replying to.
Discuss!
Kevin Smith
Agile Coach
Wikimedia Foundation
*Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment. Help us make it a reality.*
As was mentioned in scrum-of-scrums on May 20, the analytics team is
running some Visual Editor A/B tests from now through June 11. As a result,
analytics has requested that everyone "reach out to us if you're planning
any changes in your Event Logging use" during that time.
Hopefully this won't mess up what we're trying to do for the next couple
weeks.
Kevin Smith
Agile Coach
Wikimedia Foundation
*Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment. Help us make it a reality.*
Answer: we don't know yet, because there are Infinity ways to search.
But for action=opensearch,
http://ironholds.org/misc/api_presentation.html might make an
interesting read.
We're trying to document all the various forms of API search so that
we can do more ad-hoc and regular analysis of this stuff. There's an
awesome start at
https://www.mediawiki.org/wiki/API:Search_and_discovery that looks
pretty complete - if anyone is aware of API methods /not/ listed
there, please put them in, or reply here and I will :)
--
Oliver Keyes
Research Analyst
Wikimedia Foundation
+search
On Tue, May 19, 2015 at 3:14 PM, Brian Gerstle <bgerstle(a)wikimedia.org>
wrote:
> The subject hints at a question that's been nagging me for a while, and
> now that I'm going to be hacking on testing in Lyon I wanted to ask:
>
> Do we have a list of articles we usually run tests against?
>
> If not, do we have any processes for curating such a list? Would anyone
> be interested in a brainstorming session at Lyon to discuss this further?
>
> Basically, as a developer, I would love to have more confidence that some
> code I wrote doesn't break on our most popular articles. Or, if we can get
> more sophisticated, that *certain properties of my code hold true for
> certain kinds of generated pages*.*
>
> Please respond with your thoughts and whether you think I should create a
> phab task for the hackathon about this. In either case, ping me anytime or
> grab me at Lyon to discuss further!
>
> Regards,
>
> Brian
>
> * Yes, I'm talking about using property-based testing generators to create
> random, shrinkable MW pages that we can run tests on. Not sure if it's
> practical, but could be an interesting experiment.
>
> --
> EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
> IRC: bgerstle
>
--
EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle
FYI, I just renamed the phabricator Search-Team project to
Search-and-Discovery-Department. It still has the #search-team hashtag, so
most searches for the old name should still work.
This has introduced an inconvenience, thanks to an odd Phabricator
behavior. To auto-complete a project name, Phabricator only shows the first
5 matches, AND for some reason it is showing the "Search-and-Discovery-xxx"
projects, and not the simpler "Search-and-Discovery". Argh! I have filed a
phab ticket for this.
I have updated the team page[1] to point to the new phab project. Let me
know if there is any other documentation that also needs to be updated.
Kevin Smith
Agile Coach
Wikimedia Foundation
*Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment. Help us make it a reality.*
I've removed the restriction that this list had on non-members posting to
it. Its a public list and it doesn't make a whole lot of sense to have that
restriction.
Nik
Our plan has been to have all Search&Discovery issues managed in the
Search-Team phabricator project, and then for each subteam to have a sprint
board for its current work.
However, Yuri pointed out that people wanting to create a new task related
to maps would expect to add it to a project with "maps" in the name. Folks
outside our team often would not realize that maps are part of the Search
vertical.
We really want to consolidate all of our product backlogs in one place, to
help keep Dan sane. So I don't think keeping an external high-level "Maps"
project is a viable option. And I don't think adding the word "maps" to the
main Search-Team project would make sense either. We might want to rename
our top-level project to "Search-and-Discovery-Team" for other reasons, but
that wouldn't help this specific case.
Even if we had a maps-related sprint board whose name would match a phab
search for "map", that wouldn't be what we would want. We don't want random
users dropping tasks directly into our sprint board. We really want new
tasks to land in the product backlog first.
Is this just a case where we have to educate folks that "maps" work goes in
"Search-Team"? Or are there other options?
Kevin Smith
Agile Coach
Wikimedia Foundation
*Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment. Help us make it a reality.*
(retrying mail aliases, sorry if you've already seen this)
---------- Forwarded message ----------
From: S Page <spage(a)wikimedia.org>
Date: Thu, May 14, 2015 at 1:04 PM
Subject: Search API documentation
To: search-tech <search-tech(a)wikimedia.org>
Cc: Quim Gil <qgil(a)wikimedia.org>, Brad Jorsch <bjorsch(a)wikimedia.org>
Hey, James Douglas talked to me about search APIs. I started an overview of
what's available at mw:API:Search_and_discovery
<https://www.mediawiki.org/wiki/API:Search_and_discovery> that links to
existing API pages and the generated MediaWiki API help. In particular the
API pages have a link to [try in ApiSandbox] so you can load them up and
play around with the MediaWiki API search modules. I added a link to this
overview on your team's top-level mw:Search and Discovery page
<https://www.mediawiki.org/wiki/Search_and_Discovery> page.
<https://www.mediawiki.org/wiki/Search_and_Discovery>
I'm also writing articles for a planned "Data and developer hub" to
encourage third-party developers to use Wikimedia data and APIs.
Fortuitously two of the early articles are about how desktop and mobile use
search, "Page info in search results" and "Showing nearby wiki
information", so API:Search and discovery links to these as well.
Though I'm on the Reading team I'll do what I can to document your stuff,
particularly what it can do for third-party developers.
Cheers,
--
=S Page WMF Tech writer
Hello list,
this is a test mail. If you see it it means you can now also reach
this list as the alias
search(a)wikimedia.org
as requested by Tomasz on https://phabricator.wikimedia.org/T98415
Best regards,
Danie
--
Daniel Zahn <dzahn(a)wikimedia.org>
Operations Engineer