Wikimedia Meta-Wiki

Wikimedia monthly activities meetings/Quarterly reviews/Editing/June 2014

< Wikimedia monthly activities meetings | Quarterly reviews
(Redirected from Metrics and activities meetings/Quarterly reviews/Editing/June 2014)

The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's Editing (formerly VisualEditor) team, June 19, 2014, 12:30–14:00 PDT.

Present: Lila Tretikov, Erik Moeller, James Forrester, Roan Kattouw, Trevor Parscal, Gabriel Wicke, Jared Zimmerman, Kaity Hammerstein, Abbey Ripstra, Tilman Bayer (taking minutes), Rachel diCerbo, Rummana Yasmeen, Terry Chay

Participating remotely: Maggie Dennis, Sherry Snyder, Subramanya Sastry, Moriel Schottlender, Dan Garry, Ed Sanders

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Agenda:
Goals and objectives
Progress in 2013–14 Q4
Outline of work for 2014–15
Proposed work for 2014–15 Q1
Questions and general discussion

James: Welcome & introductions
team formerly known as VE team, now just "Editing team"
expanding scope to mobile and wikitext editing
new name focuses more on user than on particular technology

Goals and objectives

edit

My formulation of the goal: to support editors (whether new / existing / to be)
by making the tools they use really easy to use
Actually, besides VE and the usual wikitext editor, there is a second wikitext editor being maintained in parallel, almost no usage
Lila: use percentage?
(James:)
~0.0...% - it's there as a non-JavaScript option
dates from around 2009
Roan: I am one of the people who wrote it, and I want to kill it with fire ;)
(James) focus both on desktop and mobile
direction for wikitext editor unclear, even with respect to community wishes
moderately big team:
9 engineering slots (see slide)
trevor: effectively 4-5 fulltime members, but communications/switching cost of 8
(discussion on work scope for some team members, and some upcoming changes)
Timo moving to Platform, taking some things with him (Roan: also, VE has been initial customer for some things Platform built)
(James:)
we are also depending on Parsoid, and work elsewhere in Engineering
goals for VE: be the best content editor
(i.e. not: discussion editor, code editor, ...)
Lila: how do you define "best"?
James: that people prefer using it
be the main, default editor - not just in technical (configuration) sense, but in user expectation
Trevor: core software of VE is completely standalone, licensed liberally
James: lots of interest so far, but no actual 3rd party users yet
VE objectives:

  • increase use
  • improve performance
  • improve existing tools like for citing, linking
  • improve language support - right now live on 166 language Wikipedias, but that still leaves >100 - some of them by commmunity preference, but most for lack of lang support
  • add missing tools: e.g. tables, galleries. (e.g. can't add row in a table - bad for World Cup coverage ;)
  • improve third party uses: not primary objective though

VE metrics:

  • can be measured in many ways
  • complex landscape makes it hard to establish baseline metrics
  • VE is potentially disruptive change, make sure it does no harm - did not quite meet that objective on beta launch a year ago; around 5% of edits were deteriorating content in some way. But now this is much lower, and decreasing further.

Trevor: also, now seeing some actual community desire to turn it on
James: because of that focus on preventing harm, not much attention on "positive" metrics
example (slide): frwiki
Lila: activated by default there? yes
(James): on wikis where it's only opt-in, much lower, e.g. enwiki 0.4%
around a third of anon edits are with VE, 2/3 still with wikitext
"default" means that both editors are presented side by side via two different edit buttons
Lila: why so low for new editors?
James: outside of content namespace, can't use VE. So as experienced editor, have to learn wikitext anyway. (note: these percentages are for content namespace only)
Lila: still, this is odd, need to understand what's going on there
James: Per-user adoption ratio (example: plwiki)
1:1 for anons, ... (see slide)
overall: unsatisfactory data, doesn't quite tell what's going on
All-edit adoption rate: (see slide, bubbles = projects)
Overall: still very low
Save ratios: ...
Lila: how does it compare to wikitext?
James: we don't know the save ratio for wikitext (anecdotally heard: 1:5 some years ago)
also, no idea what's causing daily periodicity
mid-day UTC is worst time
midnight is best time
(discussion on some speculative reasons, including server load for e.g. API requests changing during the day)
Load performance: (slide)
3 seconds median, up to 25sec for 99th percentile
Lila: oh dear :(
(James:)
Save performance: (slide)
4s median, 8s for 75%ile
can "preload" HTML to wikitext conversion when user begins saving dialog
still, shouldn't be measuring perception for this, but actual processing times
Gabriel: for large pages, dominated by network transfer
POSTs don't compress
Lila: problem is because you save all at once, could save incrementally as user types
Gabriel: would help to paper over some of that while we fix it more generally, but...
Lila: so we just need time to do it?
James: did some client-side compression experiments: takes (e.g.) 0.5s to compress client-side, cut overall save time from 12s to 1s
Lila: so let's do it
Gabriel: would be temporary solution for 2 months
Lila: still, if it's an easy fix
James: with 10k/edits day, would still impact a lot of users
but need to check data - in some cases, compression could actually take longer
Ed (on chat): depends on the browser too. the compression is faster on Firefox, something like that
server side ve.dm.Converter
Trevor: if we had a draft system [with instant saving] in place already....
Roan: I have some radical rewrite ideas that would make that much easier
(James:)
Metrics: What we want to know

  • is editing actually easier with VE?

...

  • ultimately: does VE mean a better service for our readers and editors?

What we plan to track:
identify causes why VE editors get reverted or blocked (e.g. syntax errors, ...)
slice data we already have - e.g. performance by user agent (browser), if we have that information
e.g. Growth does excellent work to get in new editors, impacting metrics - how make sure we only take credit where we caused change?

Q4 progress

edit

Q4 objectives (slide)
(James:)
we didn't achieve many of them
support for citations, ...

  • auto citations: being done by external (OPW) with support
  • image upload (directly) to Commons: on Multimedia team's backlog

Lila: why is that on your list?
James: it's part of the editing experience; lots of editor-side interaction
Lila: OK, but you should only list objectives that you can actually influence
(James:)

  • HTML comments: maybe done (commit this morning)
  • equation editing
  • TOC: 30-40% done
  • TemplateData: 50% done
  • rich nested templates: pushed back
  • inline: partially done
  • page options; e.g. language: deprioritized
  • basic table editing: backlog
  • better browser support: delayed
  • stability/perf: ongoing

Lila: how did you arrive at these?
(James:)
based on user stories, user feedback
e.g. power users saying "x is terrible"

Lila; how many usually engaged?
James: e.g. this morning, held an IRC office hour
with 7-8 people actively asking questions
a few hundred people read the logs, probably
then there are the feedback (wiki) pages
Lila: given that we have 100ks of edits, need to get more
Jared: talked with James about using the model MediaViewer used, with surveys
James: had that a year ago, abandoned because most of the feedback is about WP in general or its content
Lila: ok, I'm just challenging you to think further
James: e.g. get very strong pushback from community that table editing must be supported, but then edit rates are very low
Trevor: actually most table editing is done via templates, VE table support won't change that
but don't know the split of edits which actually change tables
Sherry (on chat): Creating tables via templates is mostly something that you see at the English Wikipedia, not everywhere.
James: often get strong community request to implement support for something, but concrete wishes on details on the implementation only after it's implemented
Lila: needs to be much more targeted: e.g. when user switches from VE to wikitext, ask them why
(James:)
additional work:

  • drag & drop
  • wider rollout of citation editor (to more projects)
  • settings for media items

bug stats:... (see slide)
this quarter for the first time actually reduced # of bugs
Erik: Are you catching the work of VE on mobile?
James: We rewrote most of the system in order to cover mobile. It was a big hit on the velocity, but trying to avoid discussing non user-facing infrastructure changes.
Trevor: Now that we are taking on VE mobile stuff, there will be more convergence work
Lila: Different code bases?
Trevor: It is integrated but there is stuff that uses different classes. Also the UI uses a different theme. Eventually there will won't be separate work. It's like that right now.
Looking to do UI convergence - right now different themes, unnecessary separation
it will no longer be DT vs mobile, just UI in general
James: VE is 70k, VE-MW=40k, VE-MW-mobile=5k on top of that, but we want that down to zero
Roan: but there is a lot of common code already
Lila: Why more patches than bugs?
Roan: Not every patch fixes an official bug
Trevor: also, one bug can mean several patches, e.g. because we have several repositories
James: It doesn't take into account patches to other parts outside VE core
Erik: just be careful to take tech debt into account for Q1 goals
lot of hygiene, integration, cross-functional work

Outline of work for 2014–15

edit

(James:)
Overall narrative (slide)

  • make VE great
  • performance, usability

..
transition mobile editing

specific objectives for Q1

edit
  • Deployment: engage with enwiki, discover pain points, agree on ramp-up criteria
  • IE 9/10 support - has been about 12% of pageviews

Lila: IE 9/10 are so different in terms of e.g. memory management (leaks) etc. - do we really want to do that? could just switch it off
James: right now, even for IE 10, users don't get the VE button
Lila: what if choice between that and a feature that benefits all users greatly?
Jared: also, if team improves wikitext editing too, the gap won't be as big
Erik: right now UXes are very disjoint, every other dual mode editor has better switchover, consistency
we need to look at the numbers, wikitext is still going to be around for years
(James:)

  • Feature: image insertion

Erik: so this is the actual list of goals for Q1?
(James:)
yes, now on to laundry list

Q2 objectives

edit

engage with dewiki, nlwiki, eswiki, like enwiki
improve support for IME, eg. Chinese - big market
Roan: have good support in David
James: table support
Lila: that's done in CSS?
James: HTML - Parsoid does the hard work

  • deployment: non-WPs, say Wikivoyage could benefit a lot from support for adding listings
  • feature: uploading media
  • Wikidata
  • Core: language variants for Chinese
  • release standalone for 3rd parties, say WordPress
  • ...

Metrics: Plausible outcomes end of Q4 (mid 2015):

  • low goal: new features don't degrade perf

plausible / stretch .. (see slide)
adoption and performancec goal numbers (slide)
Erik: this is good, but ...
Gabriel: so this measures VE+Parsoid performances?
James: yes, end to end
but only measure VE perf improvements
Erik: let's sit down on the metrics sometime
feels so far out right now that it is not operationally useful for us

James: additional work areas:

  • fix performance issues on mobile
  • explore performance gains from Parsoid read HTML

Discussion

edit

Lila: on quarterly goals:
figure out baseline and targets
e.g. all VE default wikis have usage of x%
grow from that by say 5%
have performance targets among your key goals
feature wise, you are probably in the right ballpark
Jared: also, don't have the right instrumentation on e.g. tool usage
Lila: agree, have to get the numbers you need
James: we are missing a data person
and, as a team, the experience on how to instrument etc.
had good support from Ori, but not really knowledge transfer
Erik: all of this is technically underway, but it's a resourcing question
Lila: get everybody together and train them
Lila: map out whitespace, lack of ownership
Erik: in conversation with communities, (lack of) data has become central issue
Roan: needed handholding by Ori, and also, I'm not a data analyst
Erik: we only have three analysts
Erik: I'll have a high-level conversation about metrics with James first
Gabriel: also do tests / regressions in continuous integration, so that you can cut out all the variability from other sources and isolate effects
Trevor: we have some high-level asks
hiring: candidates with necessary skill sets are difficult to find
James: one position has been open for an entire year, to replace contributions from Wikia
Trevor: need help to fill reqs with qualified people
one of them specifically for junior person
Jared: given that long time, better to hire someone with cultural fit and train them?
Trevor: rather opposite problem, had people with good skills that had bad cultural fit
e.g. here, as elsewhere in MediaWiki work, high levels of self direction are required
Lila: do we use external recruiters?
Erik: not so much for developers in engineering, but for e.g. PMs and executives. However, pipeline is being rebuilt as we speak
the other reality is that there has been a lot of hiring going on elsewhere in the organization...
Lila: [from my previous workplaces] I'm used to team managers being responsible for hires or onboarding, how is it here?
(Terry's name is mentioned ;)
Terry: for that >1y priority, we actually have an internal candidate we would like, but blocked
optimistic on that one though
Lila: how many engineers under you?
Terry: 27
Lila: how many open requisitions in past year?
Terry: 8 or 9 I think
Lila: so Terry has conflicting demands
so team needs to focus on that too
Erik, Trevor, Terry: that's actually how it has worked out already, kind of
Erik: Ideally, VE team shouldn't own its frontend hiring pipeline
Lila: of course not
Erik: need cross-team/ cross-functional hires, with representation from several teams
right now hiring managers per req
Lila: yes, in a perfect world it would be like that
but in absence of centralized process, need to own it yourself
Trevor: also, our job descriptions may not be optimal, seem to bring in candidates who aren't a match
James: they are very spec based, "x years of experience" etc., rather than "work on y"
Lila: I used to very often source people myself
James: there have been such hires, from existing staff's social network
Roan: there has not been a successful hire where nobody recused themselves ;)

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Wikimedia_monthly_activities_meetings/Quarterly_reviews/Editing/June_2014&oldid=19094701"