Wikimedia monthly activities meetings/Quarterly reviews/Community Engagement, October 2015

Notes from the Quarterly Review meeting with the Wikimedia Foundation's Community Engagement department, October 5, 2015, 8:30am - 10:00 am PDT.

Please keep in mind that these minutes are mostly a rough paraphrase of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Present (in the office): Lila Tretikov, Jake Orlowitz, Siko Bouterse, Rachel diCerbo, Rosemary Rein, Terry Gilbey, Luis Villa, Sarah Malik, Geoff Brigham, Floor Koudijs, Tilman Bayer, James Alexander, Toby Negrin, Kevin Leduc, Sylvia Ventura, participating remotely: Quim Gil, Wes Moran, Asaf Bartov, Maggie Dennis, Katy Love, Edward Galvez, Katherine Maher, Jaime Anstee, Trevor Parscal, Haitham Shammaa

Intro, overall department objectives

Luis: welcome

slide 2

[slide 2]

1.: (define overall strategy) Lila asked us to clarify this
Geoff: when will this be published/shared with rest of org?
Luis: mid-quarter, as planned earlier

slide 3

[slide 3]

2.: STRENGTHEN community health

slide 4

[slide 4]

3.: (STRENGTHEN support of other WMF departments)

slide 5

[slide 5]

4.: (EXPERIMENT with new community tools and approaches)

slide 6

[slide 6]

(EXPERIMENT with new community tools and approaches, continued)

slide 7

[slide 7]

5.: core workflows; hit these regularly

slide 8

[slide 8]

Developer Relations

Quim: still cannot measure our KPI, but Engineering is helping to work on it, since this data point is useful for more teams. Hopefully ready by the next quarterly review.
Terry: is it that we don't know how to do it, or we haven't done it?
Quim: there are different ways of doing this, and we are working on the implementation.
Toby: know how to do it, but need to write software, Readership and Discovery are working on it.
Lila: should it be on this team's quarterly goals then?

slide 9

[slide 9]

Quim: hackathon

slide 10

[slide 10]

Terry: lack of UX resources because of resources moved away during quarter, or because of planning shortcomings?
Quim: we knew the UX resources were not committed, the discussion happened when Engineering was recovering from the reorg. We still wanted to try, and in fact Reading Design went beyond call of duty.
Lila: current situation?
Quim: this quarter focus on improving content
not going to get more UX resources in the short term, but that's fine
Toby: need to sync on Blueprint, UX people see it as a reference design, not as an implementation.
--> Lila: let's talk [about Blueprint]; you should be able to get the right resources at the right time
Quim: OK, taking that as action item

slide 11

[slide 11]

Gerrit cleanup
Quim: in the Appendix we are including code review metrics that help seeing the impact of this Gerrit Cleanup Day, and also the dimension of the problem we have handling code reviews.
Quim: even if we didn't achieve the goal as initially defined, we are happy with the result and we are going to recommend the celebration of Cleanup Days every quarter until the queues are at reasonable levels -- now they are not.

slide 12

[slide 12]

GSoC and Outreachy
Lila: background on GSoC?
Quim: students, one intern, two mentors
Luis: summer of code casts a broad net, so typically the interns come from outside our community.
Lila: do we hire from that pool?
Quim: yes, many of today's employees came from there
Trevor: after this round, I just hired one from GSoC and one from Outreachy

slide 13

[slide 13]

Quim: Wiki Loves Open Data
the initial framework was created with the Wikidata team and community, but we failed on the objective of ongoing work with one open data org. Big orgs don't move very fast.
Terry: what dials will this move?
Quim: GLAM initiative (decentral) was big success
WMF should help, but if chapters do it, great
Luis: hope it feeds into Knowledge Engine (initative)
--> Lila: figure out a full framework
I asked to use one of the partnerships as model for rest
but this is cross-functional. sync with Sylvia, Wes
want data that's pushed/pulled to remain fresh
complex ecosystem to manage
in next(this) quarter, get much clearer on what goals are, and how this will be managed
Sylvia: CC0 (=Wikidata) is high bar, they usually release on CC BY-SA. Similar with OECD
--> Lila: need working group and a leader
iterate on this, give exec team more insight
Sylvia: OK. Wes is involved with that
Terry: need to know about success of experiment
Luis: yes, as experriment it was successful
this new slide format is triggering questions on every goal ;)
that's a good thing, but need to keep time
Quim: chapter like WMDE or WMBE might be ahead on such partnerships; we need to including in any working group hat it is formed.
For WMDE involving open data orgs is strategic, and they have the Wikidata team.

slide 14

[slide 14]

slide 15

[slide 15]

Community Resources

Siko: ....
Lila: does Wikimania count as Global South funding?
Siko: yes, in case of support for GS individuals attending
Luis: and since next q there is no Wikimania, GS ratio might drop
Siko: number of people supported is kind of the KPI taken from community-reported Global Metrics
did not track this previous q, so no QoQ
it's a lot of work to generate that number
Lila: these are your KPIs, if it's not helpful / takes too much time, then scrap it
Siko: looked at amount of hours team spent collecting data, and the work community members spend to gather Global Metrics - it's huge

slide 16

[slide 16]

Resources Consultation
Lila: so where was that private space?
Siko: besides 1. wiki: 2. survey, 3. smaller focus groups/conversations

slide 17

[slide 17]

survey was most successful
money is a touchy subject [i.e. difficult to discuss in public]
Luis: this is critical - gives us more honest and perhaps less adversarial feedback
Lila: changes the frame - very interesting, might learn from this for product
Luis: in Legal team's consultation some years ago, we thought about this. But this is the first time it's implemented that way.
James: harassment survey is also designed that way
Jake: dramatically different tone
Siko: see also appendix slide on resulting changes, and linked blog post etc.

slide 18

[slide 18]

Siko: interesting to see requests from new user groups
Geoff: quality of requests?
Siko: quite good. want to make it easier for [them]
--> Lila: can we start tracking amount of [grant/support] requests per category, see how this evolves?
my guess is that the slice going to smaller entities/GS goes up, ratio of big entities from GN goes
see how will this look 12 months or 3 years from now
Siko: OK, already track $ (GS/GN), but can do this too - can see your point that it might be interesting to see user groups vs chaptes

slide 19

[slide 19]

we are about 80% on this goal
FDC proposal on track
C-team still needs to discuss
Lila: I am aware, goes into Q2
Siko: our core workflows take almost 100% of prog officers time
need to find way to resource this
Lila: see as blueprint
Siko: yes, early experiment

slide 20

[slide 20]

slide 21

[slide 21]

Siko: these disccussions take time, decided to not press in Q1

slide 22

[slide 22]

how to bring in ideas
want to run idea campaigns throughout year
hired Chris
Lila: example of ideas coming out of Idealab that were really impactful?
Siko: Wikiproject X
Idealab is incubator, a large number of our grants come from it
---> Lila: find some IdeaLab examples that were really successful, so that people can learn from them
Siko: Inspire campaign ran entirely in idealab, but these will need 6-12 months to show success
but point taken
Reference : https://meta.wikimedia.org/wiki/Category:IdeaLab/Ideas/Proposal/Selected

slide 23

[slide 23]

Core workflows
this q, rather than like 2 affiliates seeking additional support/guidance in transitions, about 12
Lila: why do we think it rose that much?
Siko: coincidence rather than fundamental reasons. it's a reactive workflow
Luis: triggered by site visits perhaps?
Siko: had some visits, but many beyond that scope

Community Advocacy

slide 24

[slide 24]

Maggie: both KPIs are about response speed
--> Lila: make sure slides are screen-shared on Hangout

slide 25

[slide 25]

(Maggie)
Internal collaboration

slide 26

[slide 26]

Harassment strategy
Terry: why was this so successful (compared to other goal)?
Maggie: it's year-long and top importance, can't afford to fall off schedule or deprioritize
Luis: in the transition after Philippe left, talked about prioritization, made sure this got priority
[slide 27
(Maggie)
(Global ban enforcement)

slide 28

[slide 28]

put together service level agreement (SLA)
easy channel where everyone can get to us
Luis: here not deprioritized, but we researched, planned and realized it was wrong plan

slide 29

[slide 29]

slide 30

[slide 30]

(Maggie:)
lack of time, hope you will read full slide deck, but call out: smooth transition; team rose to occasion and James stepped up

Further appendix slides for Community Advocacy

slide 34

[slide 34]

Learning and Evaluation

Rosemary:
we love KPIs ;)
just as we at WMF don't create content, we also don't create learning - community does: Learning patterns are way up

slide 35

[slide 35]

Disney process: film does not get made unless there is a [concrete artefact]
brutal "shark tank" approach to deciding what goes into MPL (master project list) ;)
Lila: result?
Rosemary: e.g. not teach conferences unless we teach community members who can spread this ("train the trainers")
Geoff: example for priority research project?
Rose: use data more, e.g. Edward focusing on surveys now - if you continue just reponding on talk pages, continue to run into resistance
fix pain point of tracking data by ....(?)
example for train the trainers : Estonia
partner a lot with Comm Resources on their survey work

slide 36

[slide 36]

Edward focusing on avoiding survey fatigue, building survey support center
Terry: overview of survey tools, I guess we have like 5-7?
and hope data can be used cross-functional
Toby: should talk about Quicksurveys tool that Reading is building
Rose: Edward did assessment of all surveys done throughout the year, central repo
Geoff: ...
Luis: only building this now, so not surprising your team didn't yet know about this support option
Rose: e.g. 50 hours spent on Wikimania survey
Lila: how many surveys did we run over the year?
Rose: over 40

slide 37

[slide 37]

tools for planning e.g. editathon, for quick just in time learning
Geoff: could highlight examples where you received data, and saw people act on that data
--> Rose: will include that next q
Lila: KPI on # learning patterns created, but more is not necessarily better - have a lot of info out there
Rose: yes, [TMI] can also be an issue internally
Luis: had IdeaLab idea that resulted in searchable index for patterns library

slide 38

[slide 38]

slide 39

[slide 39]

slide 40

[slide 40]

(Rosemary:)
Other successes and misses
e.g. gap between affilates and community
Lila: really interesting to me; I think that for being an online community we don't do enough online events
make events (e.g. Israel hackathon) more accessible for remote attendees
this is a question for Quim too, for the future
[slide 41-43]

Education Program

slide 44

[slide 44]

Floor: interactions KPI - many of these happened at/due to Wikimania and the CEE meeting, could be lower next q

slide 45

[slide 45]

Education extension lacks tech support
Lila: need to run flag up the pole for this
right now not clear who owns this - external/internal, etc
--> Lila: resolve [Education extension's lack of tech support] this q
2. # interactions is not a success metric
I can interact with 1000 people [and not achieve things]
--> Lila: retention rate might be better indicator than # interactions
Floor: agree
made edu collab a much more open and transparent group
Jake: this task flow system is really exciting
Lila: (question about who forms the group)
Floor: collab are community members willing to help
this is about working with them on sharing their knowledge and experiences.
Take a look at the linked Phabricator project
Luis: it's more task management than ticket management
Floor: e.g. "translate this brochure into Spanish"
Lila; exit criteria for members?
Floor: membership basically based on activity / positively contributing

slide 46

[slide 46]

Arab world
Egypt: local members stepped up, forming hub
--> Lila: they [Arab community] should work with content translation team, so many synergies there
Floor: I think they are already connected, using CX tool
Lila: also about strategy
Palestina: look at content acquisition project (wiktionary)

slide 47

[slide 47]

(Floor)
Refine data gathering
--> Lila: these numbers (slide 47) look great, start tracking them over time
but pick what are the important metrics
e.g. teachers engaged and running *active* programs
should know how many people have dropped out (first derivative)
then others indicators will be secondary

slide 48

[slide 48]

slide 49

[slide 49]

will skip next few slides due to time limitations in this meeting

Further slides for Education

Wikipedia Library

slide 54

[slide 54]

Jake:
yoy growth less than expected

slide 55

[slide 55]

EBSCO is big, larger than our other donations
largest q so far
Lila: did we lose partners over time? no
LilaL: need to think how this is integrated into our editing tools, e.g. VE
--> talk to Trevor
Jake: will get to that
one gap there: don't have good KPI
can count citations, but don't know who added them
want to reduce time from signing up to getting access (now: 27 days)

slide 56

[slide 56]

tons of excitement at Wikimania, but then everyone went into vacations, pick up in Sept / Oct [i.e. not yet reflected here]

slide 57

[slide 57]

small part of our work, but hopefully seed for big things
talk to orgs, e.g. invite librarians to curate bibliographies on WP articles
--> Lila: would love to learn more offline about this [collaboration with organizations like libraries]
Jake: where are we likely to succeed?
Lila: also, need more structure - partnerships that are about content acquisition vs. content curation
find where can we scale

slide 58

[slide 58]

slide 59

[slide 59]

Jake: Develop tools: library card platform
Lila: hiring developers ? yes

slide 60

[slide 60]

slide 61

[slide 61]

Lila: talked to DPLA guys? they might have the tech we need
Jake: .... talked but might pull in larger platforms later(?)
Lila: we will become a de facto interoperability standard between these resources
Luis: scope this small
--> Lila: take it offline, talk to Trevor and Wes about this

Community Liaisons

slide 62

[slide 62]

Rachel: KPIs: contributors on product-specific pages- may mean someone thinks that page is somehow useful
Pageviews on major product pages

slide 63

[slide 63]

Defining community/product process
takes longer than expected, cross-functional
now a goal for Q2, coming along
Luis: learning for me: should have engaged more on cross-team issues as executive
Lila: KPIs don't get at what we want to achieve

slide 64

[slide 64]

(Rachel:)
support VE launch to new editors on enwiki

slide 65

[slide 65]

Wikimania sessions
red because only two people engaged on-wiki - but success otherwise
a lot of the concerns community members brought up were about lack of clarity on process
received feedback that roundtable format was appreciated

slide 66

[slide 66]

Tech News
hard to count reach/pageviews because of distribution to widely read channels (incl. en:VPT and Signpost)
Lila: posted to mailing lists?
Rachel: only some
Luis: what sold me on this is that 15 volunteers spend their time translating it

slide 67

[slide 67]

Rachel: ...

--> Lila: widen the audience
--> Lila: mandate for this team:
1. as few as possible of conversations result in (a red status) - i.e. design things that community finds useful
Rachel: from community perspective?
Lila: from WMF's too (we are part of the community...)
2. allow Engineering to accelerate velocity, ship many small things
in a nutshell: build the right thing and make it fast

Further appendix slides