Hey all Wikimania people,
I sent this mail to wikitech-l, but realized (with a prod from Sumana) that wikimania-l is a thoroughly relevant audience as well.
-- Asheesh.
---------- Forwarded message ---------- Date: Tue, 31 Jul 2012 17:00:25 From: Asheesh Laroia lists@asheesh.org To: wikitech-l@lists.wikimedia.org Subject: Lessons from the newcomer side of the Hackathon
Hi all wikitech-ers,
I wanted to summarize a few lessons learned from the Wikimania Hackathon and make myself available to discuss any of these issues at greater length on the list.
Also, if you (as an attendee) have other thoughts you want to share about the Hackathon, feel quite free to email the list, just myself, or just Sumana if for any reason you want your identity or your feedback to be more private.
(Note: This is probably going to get long! If you just want the "lessons", look at just the header names.)
= Preface =
My major interest in the Hackathon was to encourage newcomers to step up their abilities, in terms of editing the encyclopedia, programming bots, or whatever exciting tech-related thing they want to be better at.
= Thanks =
Major thanks for the Hackathon go to:
* Sumana Harihareswara for inviting OpenHatch to lead the newcomer portion of the Hackathon;
* Greg Varnum, for being a hugely awesome local co-organizer;
* James Hare, for managing the local logistics and generally helping make thigns go smoothly behind the scenes;
* Katie Filbert, for helping tremendously with pre-event planning; and
* Mike Linksvayer, who was my other OpenHatch co-organizer, for taking care of lots of essential tasks like emailing local logistics folks and signing attendees in.
= Thoughts =
== The combination of newcomers and experienced people seemed to work ==
We got two different notes in the exit survey about this -- one from a person who said they were happy working in the big open main room and didn't feel distracted by noise, and from another person who said they went off because they found the main room too noisy.
I did really like being able to send people to nearby experienced folks to have a chat. This was especially helpful as I walked around and asked people what they wanted help on, or what they wanted to work on.
I did notice some experienced people, by the second day, had wandered off to quieter rooms than the big main room. I'm glad we had those rooms. (I'd love to hear from those people if they felt "pushed out", or if they instead felt happy that the quieter rooms were available.)
== We made a good impression by just running the event ==
One prospective attendee who, sadly, couldn't make it, indicated to me that just by reading the survey we sent to prospective attendees, and skimming the list of tasks, it was something that they'd be interested in attending, and that it was great that such an event was going on. In particular, they suggested it felt more like a "play-with-stuff-a-thon" rather than a "hack-a-thon", and described that as making them feel welcome.
If anything, we should have capitalized on this more. I heard from other prospective attendees that they didn't know the Hackathon was intended to be newcomer-friendly this year.
Personally, I think the term "Hackathon" gives an exclusionary vibe, and that a newcomer-oriented event should have a different name. For example, Boston Ruby recently started using "project night" at our (OpenHatch's) suggestion and it seems to have gone well for them: https://openhatch.org/blog/2012/the-steps-boston-ruby-is-taking-to-become-fr...
== Many people did cool things for the first time ==
I heard from at least two people who made their first edit to Wikipedia during the Hackathon! I saw one person write a bot for what I believe was the first time, as well get a labs account and work on moving that bot there. In the exit survey, one attendee wrote, "I learned a lot about batch upload." Another wrote about making their first commits with Git and Gerrit.
I think this is the most exciting thing about a newcomer-friendly Hackathon -- it creates an environment where people can step up to the next level, and hopefully stay at that level through self-driven follow-up practice. This is how community growth happens.
That reminds me: Two people did indicate on the exit survey they'd be interested in follow-up mentorship. I'm going to see how we can best support them; I might just send them a periodic email to see what they're up to and see how I can personally help or direct them to help.
== Wi-fi was a serious obstacle ==
I personally had a lot of trouble with my laptop failing to stay on the wifi. I would estimate at least 10-25% of attendees had a serious problem with this.
It's especially tough because on one hand, the Hackathon is a very wifi-dependent event. On the other hand, the Hackathon is a pre-Wikimania event, which means that unless there's serious pre-pre-event testing of the wifi, Hackathon attendees are the ones that will run into any problems that occur.
(For those interested in minutiae, it seemed that roaming between access points triggered the problem. The problem as users experienced it was that randomly they would suddenly see 100% packet loss with no obvious way to fix it.)
A sample of things we saw from this:
* At least one person reports in the exit survey that it was impossible to get work done with the wifi the way it was.
* Mike and I couldn't sign people in using the wiki because our own wifi had failed, so we used a spreadsheet local to his computer. This meant that it was harder to use the wiki to locate like-minded people.
One organizer attempted to set up a separate wifi network, which was operational toward the end of the event. In the future, we need more testing of this, and more of a plan for what to do if it fails.
== Logistics concerns ==
The room that we had agreed we would use turned out not to have power strips lining the bottom of every table, so we switched rooms and had to update signs across the event.
Some exit survey respondents indicated they wanted the event to start later in the morning, and that they wanted the room to not close at 6pm.
I received more than a few emails from people who were, according to the wikimania2012 registration system, signed up, but who replied to me indicating that they couldn't in fact attend due to not receiving a visa as needed. It would have been nice if those people had not been in the registration system anymore marked as attendees.
I also sent out at least one email to the wrong address; the problem was that the user who did the registration isn't necessarily the person who's being registered. I think that in this case, one person in a company was responsible for registering another person. We could fix this by making personal email address a field that you enter at registration time (though I do realize data will basically never be entirely perfect).
(Similarly, the script I was given for extracting information from the registration system ended up suddenly failing, which slowed down the second pre-event email.)
== Exit survey suggests people overall liked the event ==
Of those people, all of them indicate they're either "likely" or "very likely" to recommend a hackathon organized by OpenHatch to a friend, so generally speaking people enjoyed themselves.
== Tutorials can use TAs ==
On the second day, I found volunteeres to be "TAs" for the tutorials. Their job was to wander around and help people with environment problems or people just having trouble following along because e.g. their web browser was different than the one being used by the presenter.
Another difficulty I found was that sometimes a tutorial speaker wasn't loud enough to be audible in the back of the room.
This is above the tasks I had labeled as needed for a "Talkmeister": http://wikimania2012.wikimedia.org/wiki/Hackathon/Volunteers#Talkmeister
Also, people who are having problem following along during a tutorial don't always speak up. I'm glad we added the TAs, although I think further work is required to find out how to non-intrusively convince people that asking questions is okay. The best way I've seen is to have a very small group, no bigger than 10, preferably about six people. I almost wonder if we'd be better-served to use pre-recorded video tutorials with a lot of TAs available, rather than live lecturers. Then we could easily have small group rooms, and could pause the video.
== Next time, I'll get firmer commitments from helpers ==
One thing I did was walk around the main room, asking if people needed anything. I did find some people willing to help out with this, but didn't have a good sense of when they could help. Next time, I'll do something like create an hourly sign-up sheet for this.
We did receive feedback at the end of the first day that at least some people were very happy with the helpers. One person indicated he wished there had been even more help. While I do think that everyone just about got the help they needed, it would have been nice to have had a more clear list of who's helping, partly to ensure we have more capacity, and partly for me to know when people are planning to help, and partly to encourage mentors to sign up for brief time slots and know they've helped the event.
== Favorites ==
People listed a wide variety of favorite activities, with all these being mentioned more than once in the exit survey:
* Tutorials
* Talking to people
* Coding/hacking
The laptop setup process showed up once, too, as did break-out rooms. Sadly for me, the list of "Tasks" I made wasn't a favorite for anyone who filled out the exit survey. (Surprisingly and pleasingly, the laptop setup was!)
== Diversity ==
I would estimate about 10-20% of our newcomer-oriented attendees were women.
One difficulty wasa that we experienced a lot of competition for attendees with AdaCamp DC, which took place on the same days and times. (I heard from more than a few attendees and prospective attendees that this was a conflict.)
(If you're not familiar with the event: "AdaCamp is a Ada Initiative event focused on increasing women’s participation in open technology and culture" http://dc.adacamp.org/about-adacamp-dc/)
Given that conflict, though, I think we did reasonably well.
In terms of other diversity, one attendee remarked to me that they were quite impressed at the diversity of ages in attendance. I was personally quite impressed by the diversity of experience levels in attendance.
I think we accommodate all that reasonably well.
== Exit survey: Qualtrics, etc. ==
The exit survey we ran received only 11 responses. (Our entrance survey received over 100, but I sent it to about 400 people. The exit survey was sent to about 45, so a 25% response rate is somewhat consistent. We had about 65 people sign in on our spreadsheet, and about 45 of those people gave us email addresses.)
I used Qualtrics to run the exit survey, since it complies with Wikimedia's guidelines on data privacy/storage and relationships with vendors.
I ran into an issue with Qualtrics where the sample email that it sent me to preview what the form would look like wasn't an accurate simulation -- in particular, it didn't pre-fill the email address in the form in the same way as the real email did.
Anyway, it took too long to get this sent out, partly due to time spent learning Qualtrics that I didn't expect to be spending, partly due to my own conference travel post-Wikimania. Given the wifi, we could have just created a paper exit survey for people to fill out, or otherwise generally been faster at this if we had prep'd the exit survey questions before-hand. We can aim to do that for future events.
== Other thoughts ==
* One exit survey respondent wanted it to be easier to find people interested in using MediaWiki and related software for non-Wikimedia tasks, to chat with and work on tasks with.
* I treated laptop setup as a good thing for all attendees to go through first, because it was a prerequisite for some of the tasks, but by no means all of them. In hindsight, I would prefer to indicate on each "Task" what setup steps are required. The people that ended up just mostly editing Wikipedia didn't need to do them. (The other side of the spectrum is that it was good for many people to go through them, to enable them to do tasks that they might not have thought they could do!)
= Conclusion =
That's "it". If you read this far, thank you!
Discuss, if you like! I'm here to chat (although might take a day to see all responses, if any, and respond in bulk).
-- Asheesh.