Research:New user help requests/Full report

This page in a nutshell: Based on a series of qualitative studies of where newbies ask for help, and what kind of help they ask for, we find that newbies seldom use the standard help resources available to them. We also find that newbies struggle with both technical and policy issues when editing Wikipedia, and that even basic tasks like knowing where to ask for help, using the helpme and hangon templates, and spotting responses to help requests present challenges for new users.

WSoR 2011

Contact

Jonathan Morgan

University of Washington

Research:Projects

This page documents a completed research project.

Topic

New users receive a lot of inbound communication from the Wikipedia community in the form of (often templated) welcome and warning messages to their talk pages. These welcomes and warnings usually contain a variety of helpful links to resources, many of which were created to help teach new editors how to be successful contributors to Wikipedia. A lot of this information resides in 'help' namespace. However, despite the plethora of these helpful links, it is not known to what degree new users actually use these help resources.

Furthermore, as others have noted, the help infrastructure on Wikipedia is fragmented and confusing. For instance, with major help portals--Wikipedia:New Contributors Help Page vs. Help:Contents--divided between different namespaces, it may be difficult for new users to know where to ask a question in the first place. The help resources also suffer from inconsistent page layouts and navigation. And perhaps more fundamentally, participating in help usually requires editing pages with Wikitext, which is often challenging to new users.

(RQ2.16)

Motivation

We undertook a series of studies in June and July 2011 to try to answer two fundamental questions about new users' use of help resources:

What kinds of questions do new users have?
Where are they asking them?

Because of the open-ended nature of these questions, we chose to undertake a qualitative, exploratory analysis rather than a large-scale quantitative study. We were primarily interested in developing a better idea of the scope of help request types and locations. However, because we counted the frequency of request types and locations, we also present some basic descriptive frequency counts. Although these counts are drawn from a small sample size, they are still suggestive: new users by and large don't ask for help in help spaces, despite exposure to help resources through user talk page templates and the 'help' link in the global Mediawiki navigation menu. Instead, they generally ask for help in the spaces they are most familiar with: their own user talk pages, the pages of other editors with whom they've previously interacted, and on article talk pages.

Studies

This report is a synthesis of data from three different week-long research 'sprints':

Sampling

For the purposes of these studies, a 'new user' was defined as someone who had been an editor for less than 30 days. The three studies used slightly different sampling strategies to gather new users for analysis. Studies 1 and 2 drew from a random sample of new users from 2004-2010 (divided into half-year cohorts). Study 3 used a similar strategy, but with a fresh sample and one additional critierion: users selected for analysis in Study 3 must have made an edit to a non-article namespace sometime in their first 30 days. We added this additional criterion in order to gather a potentially richer sample once we discovered that help requests of any kind were in fact very infrequent, as in fact was any kind of participation in non-article spaces during an editor's first 30 days.

Qualitative Coding

We used a Drupal-based online content analysis tool called QBox, which allowed us to view each new editor's user talk page as it appeared 30 days after their first edit, as well as their contribution history up until that time, and to classify their features and take notes. We analyzed the messages on the user's talk page (if any), and also any edits they made to namespaces other than 'Article' during that time period, for help requests. The codebooks used in the individual studies are available behind the links above.

In order to gather the widest variety of data, we defined 'help' very widely to mean "Any question or request for information or assistance related to reading or editing Wikipedia pages, whether or not it was directed at a particular person."

Summary of Findings

% of new users who participat in the Help namespace during their first 30 days, 2004-2010. From June 6 - 10 Sprint

Places new users asked for help, 2009-2011. From June 20 - 25 Sprint

Types of help new users requested, 2009-2011, From June 20 - 25 Sprint

Overall, help namespace participation by new users is extremely low. Only about 3% of users make an edit to a page in Help: during their first 30 days.
Help requests overall within the first 30 days is still low, but substantially higher: about 9%
The percentage of new users who made at least one edit to a non-article namespace within their first 30 days who also requested help was higher, about 20%.
More than half of all help requests that relate to norms, policies or guidelines for editing involve issues of notability or conflict of interest.

Themes

Editing Policy - Notability and Conflict of Interest

Notability and conflict of interest were the most frequent points of policy that new users struggled to understand and implement. In fact, of the 32 policy-related help questions coded in the June 20 - 25 sprint, 18 of them (56%) were related to notability or conflict of interest (often both at once).

In many of these cases, the new user had created an article, or added sources, which were subsequently deleted because they were judged to be non-notable or self-serving. These new users often worked patiently with Wikipedians to correct these policy issues. Wikipedians often exhibited good faith in dealing with these newbies, in some cases working hard to both educate them on the points of Wikipedia policy and help them correct the issues that had caused their article to be rejected. However, users were seldom able to get their articles restored after removal for lack of notability or CoI.

This finding may illustrate a major challenge for editor retention: newbies who create articles have already demonstrated a willingness to devote significant effort into creating content on Wikipedia. Newbies who are willing to follow up after than content is deleted and try to bring it up to Wikipedia's standards are in some ways showing extraordinary good faith and patience. Every one of these users who becomes frustrated by the community's interpretation of notability guidelines and leaves is a potential loss to the community.

Markup Complexity

Unsurprisingly, new users struggled with writing in Wikitext. Struggles with the complexity of the markup comprised roughly 22% of all help requests made in the June 20 - 25 study. In some cases, the difficulty of Wikitext even hampered the help request itself. For example, see this user, who tried to place a {{helpme}} on their userpage, but used parentheses instead of curly brackets.

Knowing where to ask for help

Despite the welcome or warning templates that many new users receive, many still do not know where to go for help, or else intentionally avoid public help spaces such as the Help Desk and the New Contributors' Help page.

Unfortunately, many new users ask for help on their own user page, where the response rate is less than 33% (June 20 - 25 study). It is encouraging, however, that so many new users ask for help on other users' talk pages, where the likelihood of getting a response is high (over 66% in the same study).

However, even newbies who reach out to other users for help often have trouble knowing where to look for responses to that help. This user left three separate messages on other users' talk pages, but may have failed to see the responses left for them on their own talk page by a very helpful Wikipedian.

New User Help Quotes

"Help! When I did an edit for Slovak-Hungarian War - Bombing of Iglo, it appeared on the page in comic-sans typeface extending way out of the margin, surrounded by a funny dashed box. Why didn't it appear in normal format? How do I fix it?" own talk page

"Hey, Aristophanes. You wrote on my talk page when I first made my Wikipedia account. I was just wondering how to put all of those cool links and stuff on the side of one's user page. Could you help me out, here?" other user's talk page

"I just joined, so maybe I ought to have adressed this issue some other way. The page says: For more than 6,000 years, China's political system was based on hereditary monarchies (also known as dynasties). The first of these dynasties was the Xia (approx. 2000 BC) but it was the later Qin Dynasty that first unified China in 221 BC. This means that the 6000 years mentioned are contradicted by the following sentence, reek of propaganda to me and it requires cleanup. Don´t know how, haven´t made enough contributions yet so can´t do it myself." article talk page

"Seems empty. I don't know what article to edit." own talk page

"Could I please have a copy of the original page added to my page? Also, was the reason it was deleted simply due to the lack of a categorisation tag? We are a unique social networking website / company with a concept different to any other coupon marketing website. There are plenty of similar sites to us, with a similar amount of notoriety that are on Wikipedia. Could I at least have the article back so I can create a special page and seek advice on cleaning it up in preparation for it to become a proper article? I wish to do this the right way and abide by the Wikipedia guidelines. :)" other user's talk page

"HELP! Somebody deleted a page I was in the middle of making, How do I get it back so I can continue working on it? Do I have to start from scratch?" talk:Main Page

"I would like to make a userbox." WikiProject_Userboxes/Members

"Hi there, I hope I'm doing this right but wikipedia isn't so easy to work out! I got your note about Darren Turner and CRS Racing's wikipedia pages. I'll just deal with Darren for the minute. I edited his page as it was almost two years out of date and contained lots of errors, including his place of birth. I fully appreciate that I am not impartial as I work for Darren but I made sure the text was biographical, not promotional. My version showed a clear history and a clear description of what Darren is doing right now. From an encyclopaedic (real word?) point of view, my version was correct. Also the photo I uploaded was indeed copyright free. I appreciate that I need to learn how to format text and add links etc. but I don't stand much of a chance if the updates I make are instantly reverted to the old incorrect information. I'm happy to keep learning and adding to the page but it's pretty soul destroying when your work is reverted. Please can we go back to the text I added and I will endeavour to do all the additional bits and bobs.... Look forward to hearing from you..." other user's talk page

"Hello, you recently deleted an article I edited on Backyard Baseball 2001. It was right for you to delete it, however it was my fault for forgetting to cite a source. Please put the article back as soon as possible, then let me know. Once I get the message, I will gladly put the cite and references on the page. Thank you." other user's talk page

"Hi guys, first time editor here - please don't bite :))) I'm looking to add to the Brian McGinlay article but unsure how to proceed as lots of info seems to be suppressed already. Can anyone offer any help as to explain the withdrawal from the 1986 world cup and his 1992 retirement in a wiki-friendly way?" talk:Wikiproject Football