Research:New User Participation in Help Spaces

WSoR 2011

Contact

Jonathan Morgan

University of Washington

---

Research:Projects

This page documents a completed research project.

Topic

This sprint follows up on last week's sprint concerning the spaces where new users ask for help & the sorts of help they ask for. We leverage our findings from last week to create a better set of coding categories and gather a richer data sample. This sprint also continues the work of several other previous and concurrent WSOR sprints that examine the new user experience on Wikipedia.

Specifically, this week we looked at the contribution histories of new users who edited non-article namespace pages (particularly Wikipedia, Help or *_talk) during the 30 days following their first edit. Our rationale for this new sampling strategy is that these are the domains where users tend to ask for help, and since help requests are quite infrequent overall, focusing our efforts on users who edited these namespaces allows us to identify more help requests and get a larger 'n'. Our goal was to identify enough help requests to get a meaningful distribution of request locations and types. We hope that the results of this study will help the Foundation better prioritize potential improvements to the overall help infrastructure (people, practices + technology) of Wikipedia.

(RQ2.16)

Motivation

Overall use of official help spaces on the English Wikipedia has been in decline[1] [2], which makes it especially important to figure out where and why users ARE still asking for help. We'll also be capturing the responses these users receive from more experienced editors (if any) within the first 30 days.

Process

Sampling

All of the users in our sample made an edit to one of these namespaces within their first 30 days. We didn't include users who only edited the Article and File namespaces, in the hopes that this would distill our sample somewhat, since we assumed that the number of users asking for help in these spaces would be low. We also excluded the non-user-editable namespaces like Special.

We grabbed a random sample of 200 new users from each year, 2008-2011. We excluded earlier years in order to concentrate our efforts on the behavior of more recent newbies. During the June 20-24 2011 research sprint, we coded a total of 445 new user contribution histories: 143 from 2011, 199 from 2010, and 103 from 2009.

Namespaces our sample draws from:
Talk
User
User talk
Meta
Meta talk
MediaWiki
MediaWiki talk
Template
Template talk
Help
Help talk
Category
Category talk

Coding

We coded these new user talk pages & 30 day contribution histories using QBox, a Drupal-based online content analysis tool designed by researchers at the University of Washington. We developed a set of coding categories that built on those used in last week's sprint to find out how many users asked for help, where they were asking for help, what kind of help they were asking for, and whether or not they received an answer. We also captured the text of the help requests and responses.

Coding categories

1. General

• Contains Help Request?
• Responding to Notification?

2. User received help? (check all that apply)

• Received response from help request
• Received unsolicited help
• Response Location (Specify)

3. Request Location (check all that apply)

• Own talk page
• Other user talk page
• Article Talk Page
• Help Desk
• New contributors' help page
• Reference Desk
• Tutorial
• Wikipedia Questions
• Editor Assistance
• Village Pump
• New User Log
• AfD_Discussion
• Help Namespace (other)
• Wikipedia Namespace (other)
• Other Namespace

4. Request Type (check all that apply)

• Editing (technical)
• Multimedia (technical)
• Editing (policy)
• Behavior (policy)
• Reference/Content-related help
• Suggesting articles/edits for others to do
• Asking for Tasks to Do
• Other Request (specify)

Results and discussion

Summary of Findings

• Overall, we found a higher percentage of edits which contained help requests than we did during last week's coding project (19% vs. <10%).
• Of those, 63% received some kind of response to their request (compared to <50% in last week's sample)

Help Location

• The places that most new users asked for help were similar to last week's findings. In order, new users asked for help on their own user talk pages, on other users' talk pages, on article talk pages, and on a variety of Wikipedia namespace pages
  • 'Other'locations on the Wikipedia namespace where new users asked for help included: Requests for feedback, Requested articles/Articles for creation and Wikiproject pages
• just over one third of (7/20) users who only requested help on their own talk page received a response, compared to over two thirds (22/30) who requested help on another user's talk page
• Only 7% (6/83) of users who asked for help did so at one of the 'official' help locations (e.g. Help Desk, New Contributors Help Page, Reference Desk)

Help Type

• As in last week's results, the most common types of help new users asked for had to do with the technical and policy aspects of editing. However, this week's dataset revealed a much larger percentage of policy-related requests.
• We are currently reviewing these in greater detail, but anecdotally it looks as though many of the policy-related requests are responses to Article Deletions or Delete nominations and involve issues of Notability and Conflict of Interest. We intend to supplement this week's sprint with some additional analysis of these help requests in future weeks.

Charts & Graphs

Newbie Help Quotes

will add

Conclusions and other Maunderings

I (Jonathan) was surprised by these findings in a few ways. Here are some preliminary observations--I plan to keep an eye on these themes and to flesh some of them out, if they seem productive:

Maundering the First: What's wrong with Help:* ? It's perhaps not a surprise that few people used the official help resources of Wikipedia. Software and web users in general don't use help pages or tutorials in great numbers to begin with. But when you consider that a) many, many new users of Wikipedia struggle with basic community concepts and technical mechanisms, and b) most newbies receive some sort of notification during their first 30 days that directs them to these resources if they have a question... well, these numbers seem even lower than expected.

My own intuition on this is that first of all, the Help resources are confusing and intimidating. Wikipedian Ironholds has put together an excellent proposal to clean up Help and make it more user-friendly in a variety of ways, from standardizing page layout and navigation to creating separate tutorial 'tracks' for user with different needs (abject newbies vs. 10+ edit editors vs. researchers, say). This is an excellent start, but these data tell another story too: people tend to ask for help in spaces that they know (e.g. their own talk page) and/or in spaces where they know that actual people are likely to see them (e.g. other users' talk pages).

To a newbie, Wikipedia doesn't look like a community: it looks like a ghost town. They don't know how to get to 'where the action is' and they don't see action when they get there anyway: they see pages and pages of one-off messages and threaded discussions. Jumping in and contributing to these discussions can be intimidating (especially when so many of a typical new users' edits meet elicit criticisms or warnings). For users with technical questions, there's an even more fundamental challenge: it's tough when you're asked to edit a page in order to ask a question about editing a page! Perhaps it feels 'safer' to ask questions in the relative privacy of one's own or someone else's talk page, rather than in a more public Help forum.

Maundering the Second: Good faith, bad topic? Not all the data is in on this one, but we did see quite a few new users struggling to understand why they couldn't add articles about their company, or themselves. These editors weren't all blatant vandals or spammers (though there were plenty of those, too). Many of them poured a lot of effort into creating articles about something they were passionate about, and were willing to work with veteran editors to make that content conform to Wikipedia's standards--with varying degrees of success.

Maundering the Third: Lazy Newbies? I also saw quite a few new users request other people add content to articles, or create whole new articles. Not sure whether this has been typical throughout Wikipedia's history (more data needed), but it was striking to me that these users didn't try to create the content themselves--after all, they had gone through the trouble of signing up for a user account! What causes these users to hold back, and what would it take, I wonder, to coax theminto making the edits themselves?

Future work

As mentioned above, we intend to analyze and publish additional findings on the kinds of technology- and policy-related requests that editors made, since these made up the majority of all help requests. We would also like to continue coding these data if time permits, since it would be interesting to see if there is a shift in request types and locations over time--perhaps reflecting a shift in the needs or motivations of newly-minted Wikipedians in, say, 2011 vs 2008.