Wikimedia Foundation Annual Plan/Quarterly check-ins/Audiences2 Notes July 2017

Present (in the office): Eileen, Toby, Josh, Nirzar, Zack, Danny H, Jaime, Carolyn Li-Madeo, James Forrester, Heather Walls, Victoria, Amanda, Dario, Zhou, Tilman, Anne, Grace, Katherine

participating remotely: Michael H (scribe), Bernd, Corey, Dan Garry, Joe Walsh, Mark Holmquist, Mel, Mikhail, Piotr, Maggie, Erica, Joady, Olga, Erika, Chris K, Baha, Deb,

deck: https://commons.wikimedia.org/wiki/File:Audiences_2_(New_Readers,_Structured_Data_on_Commons,_Wikidata,_%26_Readers)_Quarterly_Check-in,_July_2017.pdf

Toby: Welcome

Slide 1

Anne:

Slide 2

AG: Background about the New Readers program:

Slide 3

Goals are to increase readership by in (list) countries, looking at offline access and affordabiltiy as main barriers

Slide 4

Offline: looking at people whose internet access is limiited Affordability: where it's too expensive Many solutions address multplie barriers.

Slide 5

Looking at 3 platforms: mobile web, Android (zim format), fully offline (thumb drives, CDs, etc)

Slide 6

Metrics: don't have them yet; interesting challenge trying to collect them.

Slide 7

This q, spent a lot of time understanding the landscape focus on the right places collaboration with CE was super useful

Toby: Community wishlist included that we become more involved in the grantmaking process

Slide 8

Lessons learned:

   Good:

Grant advising informative beyond expectations Android team picking up offline compilations project

Do differently:

Struggles:

   Offline metrics will always be hard to collect

Toby: Good job working with the community Anne: Yeah, that has been great

Slide 9

W0 did add'l launches recently: Iraq, Nigeria

Slide 10

Zack: Fundamentally, we believe that awareness is critical ot our success People can't use us if they don't know we're there

Breeking it down:

   Awareness, attribution, use case

Slide 11

Wanted to do 3 regional campaigns, doing messaging to increase people's recognition of who we are and what we do. Make messging campaigns by, for, with local communities, not just from SF community driven marketing created with local PR agencies Start and end with evaluative surveys Do traffic and social media analysis (since social media ) Toby: not sure if anyone's done this before--working with local communities on messaging

Slide 12

Zack: It turns out that you can learn things you don't know when you do that! (Work with communities on messaging) Iraq is 1st regional campaign, followed by Nigeria, then India; all in flight to some extent Toward end of year, we'll look for another possible country to pilot Also exploring "rapid grants" -- hope to be launched by mid-Oct

Slide 13

Started our process in India last week; we're set up with agency with 8 community volunteers!

Slide 14

Nigeria: 2 videos we're hoping to launch before Wikimania, made in collaboration with 5 volunteers from WMNG

Slide 15

Iraq launched in the spring. Videos have reached audiences and now we need to evaluate their impact. Very close partnership with Global Reach/Zero.

Slide 16

Pageviews spike when Asiacell messages sub Expect another spike

Slide 17

Anne: (skipping Learnings slide due to time)

Slide 18

Rapid Grants

Slide 19

Anne: more on Rapid Grants. 2 pilots.

Slide 20

skipped due to time

Slide 21

skipped due to time

Slide 22

Amanda B: Structured data on commons: funded by a grant over 3 years; we're 6 mos. in. Overall goal: to get file metadata for files on commons structured and machine readable.

Slide 23

Overall goals: make commons more useful by make finding and uploading easier, make reusing it easier.

Slide 24

The idea is that by solving this issue ("liberating" our Commons data), we will empower a lot of people. Doing stakeholder analysis, _____ plan, will do more at the offsite.

Slide 25

2014 is when we first started hearing from the community about how much this work was needed.

First thing we need to do: make metadata around commons editable. Once we do that, we can than make it uploadable, ..., searchable.

Slide 26

This slide describes the work sequence for how we'll get this done. Currently, there is zero structured data on Commons. It's all text blobs. Now we're getting the infrastructure in place, incorporating wikibase. Wididata and platform teams are working on multi-content revisions and database layering so all of it can sit in the DB. Then we'll integrate search and work with the community to start building tools around it.

Slide 27

Timeline: might be tentative! Especially after the offsite as plans start firming up

Katherine: We're already in Q2, shouldn't we be done building MCR?

Amanda: this is calendar year not fiscal year

(and it's Q1  ;) )

But we are well into MCR and should be complete by EOQ.

Slide 28

These are the goals around the first segment: build the infra (detail work on MCR, database schema, Federation test site created)

Toby: This was another awesome cross-team collabo.

Slide 29

AB: This is a research/design goal around how we want the data to be editable, once we have the structure for it. JK working on this while the new product mgr search is underway.

Slide 30

JMo is working with our .. communities and commons and reuse communities to figure out how people want this to work. User-side research.

Toby: Thanks, Amanda, we're glad you're here!

Katherine: Good to see the proposal take shape as a program.

Toby: It'll be v. instructive on how we can scale our efforts to allow community and others to build on top of them.

Slide 31

Josh M: I'll do the high-level overview then turn it over to product owners

Slide 32

~30 humans not because we can't count, but because the team is evolving rapidly!

Slide 33

Lots of transition and planning; org tune-up (we gained some folks from Discovery), we gained Max and Natalia from TPG; Grace joined audiences mgmt Separated cross-dept. programs and products Lots of hiring! PM for structured data work, PM for Android, engrs for various teams

Katherine?: How many open reqs? A: 6

Slide 35

This is our high-level strategy, which has been consistent for last year+ 1) improve the encyclopedia experience 2) reach new readers 3) make content interactive (was "community of readers") -- microcontributions, etc., contributions smaller than full-on editing

Goals based on these: (1) understanding our users and (2) buildling out services to support the individual products (avoid duplicating work).

Slide 39

Olga: web team has mostly been focusing on improving the encyclopedia experience and reaching new readers.

1) page previews feature: getting ready to deploy it as default experience for anon readers on all projects

goal last Q was to deploy to all wps except de and en ran into issues with data analysis/the way we were collecting data. at first unclear what was causing it. so we focused on resolving it. also focused on making the previews look true to the content on each article -- the way the lead para looks on each article. data collection issue is now resolved so we're hoping to deploy next Q

Slide 40

more detail on the trouble we ran into: we were seeing many duplicates. it was preventing us from evaluating the feature's success. the cause turned out to be a bug in firefox; they're fixing it on their end. hopefully launch everywhere but de and en next week. hope to do more focusing on data collection/EventLogging

toby: thanks for sticking with it! this data is vitally important

Slide 41

(Olga:) PDF rendering: the current tech (OCG) has been essentially broken for a while; we're been looking for suitable replacement.

Looked at electron.

Realized that this ties into the new readers focus; began to focus on PDFs as an offline content source. Want to have one style for books, browser prints, and PDFs.

Toby: this is like taking lemons and making lemonade. Ops was asking us to turn off OCG, and the team was able to turn it into something positive.

Slide 42

Demonstrates old vs. new styles

nirzar: Goals: 1) Extend branding 2) ? 3) Presering Latex-like styling

Katherine: will this be for all langs? Nirzar: Yes Toby: localize logo? yes

Slide 43

olga: When we realized we had to switch to a new renderer, we realized we should get feedback from users of the feature; so we provided a link for feedback. Kind of a random idea but it had great success -- let us target the users of exactly that feature and got to hear feedback we dont usually get.

(reading quotes)

It's nice to know that there are people out there who appreciate our work!

Slide 45

We're launching a project with a very large roadmap (~3yrs) to build an offline-capable (web) app. Be able to provide content in conditions of low connectivity, and what this means is that users will be able to read articles while offline. Not a fully offline app as of yet. We'll also be rebuilding with a focus on performance, development speed, and code maintainability.

This Q we've been working on creating an initial prototype and backend work. Next ? we'll be beginning development and the quarter after, we'll start testing.

Probably our most exciting and slowest-paced project.

Toby: Working through some of the arch decisions with Victoria. We hope to use the Tech Council to have some of these discussions in an open way.

Slide 47

Olga: we moved the lead paragraph. Easier to immediately see what an article is about now.

Slide 48

Corey: updates on the 3 major services Reading Infrastructure is working on next 3 quarters.

1) Reading list service. Reading lists are active on the Android app, allows bookmarking and offline management. The service will allow us to synchronize it. last Q, went to Tech Com with an RFC which was approved. Now in dev and will be shipped this Q on Android.

2) Page content service. Adapts Parsoid HTML for use across all of the reading clients. This will allow us to easily share features that we develop across all platforms. E.g. lazy image loading implemented earlier. Also will enable next generation experiences like the new mobile web app

Last Q did phase 1 of our development--service built, and platform engineers auditing their content formats.

4 phases

3) Push notification service. For us, it will enable timely delivery of relevant content. A natural evolution of the Explore feed that we rolled out last year. Big difference: the feed requires opening the app. This plan started last Q but will go to RFC next Q.

Slide 50

Josh: Places (extension of Nearby feature) with Jon Robson: trending articles based on editing instead of reading behavior (where data lags 24h) Research with JMo

Learnign about geography and the relevance of different kinds of trends to different markets. We've found that interest in US and India are different, for example.

Next q Android will switch back into strategy after finishing up offline work

Syncing reading lists is another heavily requested feature we'll be finishing up.

Slide 51

Places: launched on iOS and Apple promoted it. Thought was was really quality and engaging. (They made custom promotional art (shown, on right...), now used on app store listing) We've heard there will be some intl focus on it later this summer

Super positive feedback -- got a five star rating

It's the kind of feature you don't use as often as something like search. (usage shown in red line at bottom of chart). But also has the lowest "bounce rate" (where the user leaves the screen without doing anything) of any part of the app

Slide 52

(Demo of dark mode) Carolyn adapted our existing look and preserved contrast

It turns out there's a non-trivial % of people really sensitive to blue light. Added sepia mode which helps them.

6-8months ago or more, we started working with comms to develop a sticker pack for Wikipedia. (shown) Will become available on imessage and then other social platforms (and in physical form)

Slide 53

Josh: Want to thank JMo again for being great to work with. We originally were going to do this on our own, and he offered to help and continued to be committed. Also, we don't always do recommendations (and evaluating them) well as an org, good to build our skill set here.

Slide 55

Android: focused on improving offline experience, including reading lists (last Q), and this Q, offline libraries (called compilations internally)

Will provide users with the ability to use large packages of content offline.

Slide 57

Adam B.: We've continued to work on TemplateStyles, a feature that allows editors to put in mobile CSS so that things lay out more nicely on mobile devices. Gergo just sent out annoucement, People seem enthusiastic. We are cautiously optimistic. Time will tell.

We're intentionally going slow with this project.

Slide 58

Learnings: we encountered some delays on security review, which is OK. Something we need to get right. There's a defacement danger here. Wanted to get it right. Brian Wolff did an awesome job with the security review.

We hadn't grasped some of the interactions with Flow and VE. Should have done more up front. But as of last week we're on track with regard to those pieces.

Slide 60

We've continued to work on 3D file support; we're nearly feature complete here. Did some outreach on village pumps, etc. Some patent concerns that we need to continue working with Legal about. Need to engage community members in this discussion as well. We'll probably make a recommendation on license approach to get ahead of the patent issue, and see how the commons community responds.

Slide 61

Learnings: Sketchfab - 3D website - They gave us great feedback.

Also got interest from people in this ecosystem about potential donations of material.

The STL file format we are using is the industry standard but is fairly basic compared to other offerings.

Slide 63

Josh: On community of readers/microcontributions:

   Biggest event was Android app rolling out short description editing in-app.  Those descriptions are really vital to app navigation but previously weren't editable, because they're not technically part of Wikipedia.  Worked with both communities to find a way to expose them for editing.  

Promsing form of contribution, does not need lots of context (other tabs etc) This is rolled out to every language except English.

Slide 64

Earlier, the feature had rolled out to a set of smaller languages (see chart) rate of editing here is impressive -- app makes up a substantial % of (manual) description edits in some languages

Slide 66

Deb: During Q4 Discovery split into 2: Backend / platform moved to Tech, Front end / discovery moved to Audiences. But we're still working together.

Today's preso will cover the front end side. (Erika presented backend work last Tues.)

Slide 67

(the team, pictured)

Slide 68

What we do is to make our wealth of knowledge discoverable. We'll talk about the first 4 bullet points

Slide 70

Improving search results was a big project, (originally happened in 2009 and well liked in the community, but melted servers). Last august we started working with the community on presentation issues. Planned to roll out this past April but ran into difficulty with the enwiki editors. The controversy (only with enwiki) was about content from other projects being shown. So we won't show commons, wikiversity, wikinews; (wikivoyage is title only) content on enwiki. The other Wikipedia communities were cool with showing content from all the other projects.

Also worked on finalizing design of the "explore similar" funcitonality.

Slide 71

Now, anytime someone does a search that has results in another project, we'll show it. (subject to the wishes of the community for that Wikipedia)

Slide 72

(chart shows impact on sister projects) Majority of the traffic (to the sister projects from the search results page) is from mobile web, which is surprising, due to being below the fold.

Wiktionary is a big winner -- 26% of all users are going there English shows lowest usage, other languages we're getting more traction

Slide 73

Explore Similar

Tested with usertesting.com 10 testers, 12 countries, 3 langs represented

Green links (shown) show up when the user hovers

a/b test still ongoing for enwiki; should finish up later today

Katherine: I'm curious about the enwiki RFC and whether we want to press for this. Issues for precdent of a hard no vs. issues for future exploration. Historically, we've seen that precedents become kind of community lore. I'd prefer to back away rather than establish a hard precedent.

Deb: enwiki was very vocally opposed at first, others weren't...

Katherine: ? for clarification

Deb: There are actually 2 RFCs...

Toby: Are the RFCs incrementally removing linked content?

Deb: No. First RFC was before we went live. Other one (enwiki) came later -- they don't want to show content from wikibooks. First RFC had no strong answer so we just kept the content in. Kaldari then started a new RFC re: whether to show Wikibooks content.

Slide 74

We talked to the community for almost a year about this, we updated design, etc. Frustrating to get a last-minute RFC after getting positive feedback for nearly a year.

Slide 75

Portal: did lots of maintenance this Q. Article stats, translated text. A few minor bugs remain, we'll do them next Q.

Q2: We want to automate the portal statistics and translation updates. That way this maintenance will go away and turn into just occasional bug fixes.

Slide 77

We are kind of starting fresh and hope to present to the C team this next Q.

Slide 78

Paul Norman has been doing wonderful work. (see map of cleveland, shown)

Slide 79

Getting engineering resources has been difficult with the tune-up. We had a couple of high-profile bugs, one with frwiki, that we just didn't have the resources to do. A volunteer fixed that one. Another volunteer helped us update the service backend.

This week one of our data vendors started requiring an API key, and MaxSem helped out with that.

More of a maintenance plan would have helped.

Slide 80

(plans for next 2 Qs are shown) backend and frontend will need to continue working together. plans are a little more vague in the latter half, as search integrates with structured data

Slide 82

Toby: Lydia is out so I'll give an overview (she should present later b/c it's really dense...)

Slide 84

Milestones!

Katherine: How does the rate of Wikidata adoption compare to the Wikipedia projects?

Tilman: It's hard to compare edits. Not a like comparison necessarily.

Katherine: OK, simple answer is that it's not really comparable.

Slide 85

Toby: We saw a QoQ decline in users, which we usually don't. Looking at what we need to reverse that.

Slide 86

(work around improving usability)

Slide 89

Wikidata is part of the open data scene in Germany, and they have partnerships with:

   Int'l Olympic Committee
   Swiss Railroad
   etc. 
   

Partnerships around open data are something to watch.

Victoria: A question around this is how to make the essence of the data much more obvious?

Slide 101

Toby: People have been building stuff on top of Wikidata more and more. There's real potential here.

Shown: A WLM tool built on top of Wikidata.