Grants:IdeaLab/Longitudinal study of new user experiences

Inspire campaign

Longitudinal study of new user experiences

For a small cohort of new users, identify all the interactions relevant to editor loss and retention over a six month period. Follow up with interviews.

countryUK

themepersonal experience

contact email• mail@lukesurl.com

idea creator• LukeSurl

researcher• LukeSurl

this project needs...

volunteer

researcher

created on09:41, 30 July 2018 (UTC)

Project idea

What Wikimedia project(s) and specific areas will you be evaluating?

Is this project measuring a specific space on a project (e.g. deletion discussions), or the project as a whole?
English Wikipedia. Could be duplicated for other projects, though I would not be qualified myself to help out outside of the project where I work.

Describe your idea. How might it be implemented?

Provide details about the method or process of how you will evaluate your community or collect data. Does your idea involve private or personally identifying information? Take a look at the Privacy Policy for Wikimedia’s guidelines in this area.
Choose a 24-hour period approximately six months ago. Identify *all* new users who began editing within the project on that day. Depending on the number of volunteers for this idea, it may be necessary to select a sub-section of this group for study, which would be done via random selection. Divide this list of users between the volunteers who would then study their actions over the next six months (or until they stopped editing).

These volunteers would then look at all of their sub-cohort's actions over a period of six months and trace the following:

• If/when they stop editing, and if there was any specific trigger for leaving
• What were their intentions for starting editing? (i.e. vandalism, promotion, one specific article, one specific topic, general interest)
• Did their editing behaviour change over these months?
• How disruptive were troublesome users allowed to be before being stopped?
• How did they edit? (i.e. tracing mobile and visual editor users)
• Did they engage with other users? Were their first interactions positive ones?
• Did they engage with services specifically designed for new users (such as the English Wikipedia Teahouse.)

Some of these data would be quantitative, other data would be qualitative. Editors in this cohort who are still contactable would be invited to comment, freely, on their experiences in these six months - though these anecdotes would be noted as being affected by survivorship bias, whereas the rest of this idea is expressly designed to avoid this.

Each volunteer would compile statistical data and write a short report on their sub-cohort, which would then be compiled into one overall report by the project lead.

Note: The date chosen would be outside the range of ACTRAIL (14 September 2017 – 14 March 2018) as this created an atypical experience for new users. If there were a large enough volunteer base, a date within the ACTRIAL range could be used as a comparison.

Are there experienced Wikimedians who can help implement this project?

If applicable, please list groups or usernames of individuals who you can work with on this project, and what kind of work they will do.
This overlaps greatly with WikiProject Editor Retention and the Teahouse on the English Wikipedia, and editors here would be invited to participate.

How will you know if this project is successful? What are some outcomes that you can share after the project is completed?

The main deliverable is the final report, though each individual volunteer's sub-cohort report would also be made available.

As well as quantitative metrics, the project volunteers would - collectively and individually - make suggestions as to actions they believe arise from the quantitative and qualitative data, as a start of a more informed community discussion.

How would your measurement idea help your community make better decisions?

After you are finished measuring or evaluating your Wikimedia project, how do you expect that information to be used to benefit the project?
If the cohort is large enough and chosen truly randomly, we can get a representative sample of new user experiences, free from the various biases (such as survivor bias) anecdotal evidence is likely to have. We can then make more informed decisions about how to make the new user-experience more likely to encourage and retain constructive editing practices.

Do you think you can implement this idea? What support do you need?

Do you need people with specific skills to complete this idea? Are there any financial needs for this project? If you can’t implement this project, can you scale down your project so it is doable?
This project would require probably about dozen volunteers to do the required work. I would be happy to be the project lead, but would be happy to take a subordinate role if a volunteer with more experience in editor retention projects volunteered to lead.

At least one administrator will need to be involved, so as to make sure any relevant deleted edits have been included in the data.

Get Involved

About the idea creator

My name is Luke. I have been editing the English Wikipedia for over a decade across a variety of topics.

Participants

• Researcher As creator. LukeSurl (talk) 14:57, 30 July 2018 (UTC)

Expand your idea

Would a grant from the Wikimedia Foundation help make your idea happen? You can expand this idea into a grant proposal.

Expand into a Rapid Grant

No funding needed?

Does your idea not require funding, but you're not sure about what to do next? Not sure how to start a proposal on your local project that needs consensus? Contact Chris Schilling on-wiki at I JethroBT (WMF) (talk · contribs) or via e-mail at cschillingwikimedia.org for help!