Grants:IEG/Reimagining Wikipedia Mentorship/Midpoint

This project is funded by an Individual Engagement Grant

Welcome to this project's midpoint report! This report shares progress and learnings from the Individual Engagement Grantee's first 3 months.

Summary

In a few short sentences or bullet points, give the main highlights of what happened with your project so far.

Conceptual planning for the Co-op is complete. The wireframes for our space, which provide general layouts, are nearly complete. We have recently begun concrete discussions about theming.
We have conducted five interviews with mentors and learners and will continue to interview editors. We have compiled a report summarizing our preliminary findings from our interviews.
We are still seeking out a dedicated programmer for our team, though we do have individuals committed to working on bot maintenance and the matching components needed for the Co-op.
Our current graphic designer, Dustin York, will be unable to work with our team for the second half of the grant. Consequently, we will need to hire an additional designer to the team to complete the remaining design work.
We have so far recruited five dedicated and interested editors to assist with mentoring during our pilot. We will continue recruiting mentors in accordance with our goals of having roughly 20 mentors to help during this period.

Methods and activities

How have you setup your project, and what work has been completed so far? Describe how you've setup your experiment or pilot, sharing your key focuses so far and including links to any background research or past learning that has guided your decisions. List and describe the activities you've undertaken as part of your project to this point.

Conceptual planning

Excerpt of an editing skill tree with a loose prerequisite system, an early system in Co-op development. We decided to scale down this model due to its complexity and difficulty in presenting a navigable list of skills.

The goals of the Co-op are closely aligned with the Adopt-a-user program, which became the foundation for us to start our background work in defining and organizing editing skills. We evaluated various approaches to adoption, determined some best practises, and summarized what types of lessons are taught in various adoption programs. Some of the programs were those of Yunshui (Adoption page), Worm That Turned (Adoption page), Matty.007 (Adoption page), and Jackson Peebles (Adoption page). A key point we noticed was how every adopter had a different approach to teaching the lesson itself, even though the actual skills taught remained consistent.

Apart from the Adopt-a-User program, we also looked into other resources which would be helpful for the Co-op:

The Education Program
Video Tutorials
The Teahouse, as a potential resource for adopting practices friendly to learners.
The Wikipedia Adventure, which could be a potential resource material for creating lesson material.

Another major component of planning was how to organize what can be learned about editing Wikipedia. We initially decided to opt for a structured approach that divides editing skills in terms of complexity, from Beginner level to Advanced. Based on this approach, we achieved an initial list of approximately 50 skills. Some skills acted as loose "prerequisites" for others. The list was then modified accordingly based on the feedback and suggestions of established mentors (Go Phightins!, Yunshui etc) to make it more cohesive and relevant. More recently, however, we have scaled back the list of skills to a handful of discrete categories (e.g. Communication, Writing, Technical, etc.) and are planning to allow editors to more specifically describe their needs. This scaling back was done because the sheer number of skills presented design challenges that were difficult to overcome in a space designed for newer editors. However, this list of skills may be useful in creating suggestions for editors on what they can learn about through mentorship.

Research and measurement

Design and programming

Recent draft version of wireframes for the front page of the Co-op with comments and revisions. The text/phrasing on the wireframes is not finalized, and should basically be treated with the same gravtias as lorem ipsum dolor fillers.

We have been making considerable headway on our design work, namely developing the high-level wireframes for our space. These drafts describe the "what goes where" questions that not only will facilitate our programming needs, but have provided the groundwork for our ongoing conversations surrounding the theming for the Co-op. We are also beginning to develop artistic elements, for which the original "hands" image (pictured left) has continued to serve as an inspiration. Unfortunately, we have received news that Dustin will be unable to continue in his role as designer for the second half of our project. He is committed to finish a number of priority elements of the space, such as the wireframes and a main thematic image, but it is likely we will need to look for another designer to complete other design tasks.

Ambidextrie, by Erik Wannee. There are certain elements about this image (and others) that our team has continually found favorable (e.g. colored hands, a bold and bright color palette, that it generally looks fun) and will ultimately be incorporated into our main thematic design.

In terms of programming, our original plan was to hire a single programmer to take care of bot maintenance tasks, developing a matching tool, and to program the actual interface of the Co-op. In actuality, bot work and the matching tool are to-be-covered by two volunteers on the project (Jonathan Morgan and Jeph Paul, respectively), but we have not yet been able to hire an interface programmer for the team at this time. That said, we are at a point where have started discussions surrounding the developing of the other components. I (Jethro) am currently trying to use a number of project spaces by which to identify potential candidates such as the Bot Approvals Group and Mediawiki Mentorship program who have the necessary on-wiki and programming experience to build the interface we require.

Recruiting volunteers for pilot

A critical part of our project is ensuring that we have a sufficient number of volunteers to mentor during the pilot period, which we have set from the beginning of December 2014 to the end of January 2015. Our plan was to personally invite individual editors we knew well and had a strong commitment and positive reputation for working with new editors. We have started to begin doing this recruiting by contacting individual mentors through The Teahouse, WikiProject Editor Retention, some of the active IRC chats. Much of this has been done on an individual basis, particularly with editors that our team is familiar, often having worked with them in on of the aforementioned projects. We have therefore not been comprehensive in recruiting individuals from these spaces, and there are other places, such as the Help Desk and Adopt-a-user that we have yet to explore in terms of formally inviting people to participate in our pilot.

At this time, we have five mentors signed up for our pilot who are willing to work with a small number of editors for the pilot. It is our hope that many of these individuals will be interested in continuing to work with us beyond the pilot. Soni and I (Jethro) are also volunteering to mentor, totaling seven:

Folks we have recruited are enthusiastic and curious about the project. User:Buster7 had this to say when he volunteered to mentor for our pilot:

"...to simplify the whole process of “mentorship” as we know it today." Sounds interesting!

Midpoint outcomes

What are the results of your project or any experiments you’ve worked on so far?

Please discuss anything you have created or changed (organized, built, grown, etc) as a result of your project to date.

See the above list of mentor participants in the Co-op so far for our pilot.
Also see above for wireframes example that illustrates the work we have put into fundamental design planning for the mentorship space.
Two main design-related conclusions from our interviews so far based on our preliminary interview report:

We may want to consider Co-op being the step 2 or 2b in relationship to The Wikipedia Adventure and Teahouse in the newcomer on-boarding experience. Maybe everyone should go to TWA first, then ask a few questions on Teahouse, and then, for more specific questions where a mentor is needed, go to Co-op. In fact, it may make sense to suggest that newcomers, after visiting Teahouse, consider trying Co-op.

In the interviews, TWA appears to be a good starting point, preparing the newcomers to ask more specific questions of experts in the Teahouse or 1-on-1 settings.

Some important conclusions from our interviews with learners include:

The biggest difficulties our learners faced was with wiki markup (syntax), and relatedly, a reluctance to edit pages because they might "mess things up."

The Wikipedia Adventure helped learners most with the syntax, research skills, and alleviated the daunting nature of editing.

Learners reported that they may have been able to figure out their editing needs without help spaces like TWA, but it would have been "frustrating" and "a nightmare."
Learners felt like TWA was a good place to start, but that a mentor would be helpful as they were focused on content building as a part of GLAM-related work.

Further feedback from learners and mentors regarding their experiences using the Teahouse, The Wikipedia Adventure, and the Adopt-a-user program can be found in the preliminary report.

Mentors normally communicated with learners on-wiki, but in some circumstances, also occasionally used e-mail and Facebook to communicate.

Conclusions from our interviews with mentors include:

Hosts at the Teahouse reported that learners provide enough information about what they want to know about half of the time, and that it often takes a little more dialogue to ascertain their needs.

One adopter from the Adopt-a-user program reported that communcation was very difficult for new editors:

The way we communicate now is like leaving handwritten notes in the margins of a newspaper.

The adopter also noted that if there was an easier way for editors to communicate, it would've be easier to figure out what adoptees wanted to learn.

Teahouse hosts felt like the informality of the space is an important factor for why it is successful: It meets learners where they are because they are not required to have down pat the fundamental editing skills, which is not the case for areas such as the Reference Desk.

We have finalized our research questions and measures to address them. Our measures intend to address the following questions and can be examined more fully in the aforementioned document:

How well does the Co-op work?
What predicts how well the Co-op works for particular learners?
What features work best in the various help spaces on the English Wikipedia?
Why do learners seek out and continue mentorship?

Finances

Please take some time to update the table in your project finances page. Check that you’ve listed all approved and actual expenditures as instructed. If there are differences between the planned and actual use of funds, please use the column provided there to explain them.

Then, answer the following question here: Have you spent your funds according to plan so far? Please briefly describe any major changes to budget or expenditures that you anticipate for the second half of your project.

Our spending has largely been within our original budget. Changes to our budget have exclusively dealt with our design contractor for this grant. We exceeded our budget for our design needs on the basis that we required a designer with some experience working with the mediawiki framework and understood its limitations and that we wanted to ensure that we had enough time (from August through December) with our designer to prepare the design elements appropriately.

In terms of future spending, we have yet to hire a dedicated programmer for our space. It's certainly possible we may need additional funding for this position once we identify a candidate. This need may be based on their qualifications, our time commitment needs, and the candidate's own needs. However, our graphic designer recently informed our team that he will be transitioning out for the second half of our grant, which will free up the second half of his disbursement. Some or all will be needed for a new graphic designer. Furthermore, many of our original programming tasks for this position have since been taken up Jonathan Morgan (User:Jtmorgan) (bot work) in a volunteer capacity and Jeph Paul (developing the means by which users are matched). While this does mean that the original position's task load is lessened, additional time will be needed for collaboration between our programmers. Consequently, the effect this will have on the budget is not immediately clear.

Learning

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you are taking enough risks to learn something really interesting! Please use the below sections to describe what is working and what you plan to change for the second half of your project.

What are the challenges

What challenges or obstacles have you encountered? What will you do differently going forward? Please list these as short bullet points.

Finding appropriate programming and graphic design candidates has been challenging. The limited amount of time we have for this grant necessitates that we must identify candidates who already have experience working with mediawiki software and its interface (and therefore knows its limitations). Thus far, I (Jethro) have used personal contacts and certain project pages on en.wiki (like the Bot Approvals Group) to find programming candidates. I plan to use the resources on this learning pattern to expand my search.
Our team has had difficulties in recruiting for interviews. This was a deliberate choice for our research, because we desired to have richer feedback. We have only been marginally successful at recruiting a broad spectrum of editors across the different help spaces we identified in our planning period. We originally decided to do interviews for Co-op participants; we still will ask mentors at the Co-op to participate in these interview, but we feel that adding a survey will be a better approach given the low response rate we've received in interviews. Gabe has also suggested that to the extent that we can recruit editors we know and have worked with, we are likely to receive more favorable responses for interviewing.

What is working well

What have you found works best so far? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

Grants:Learning patterns/Finding programmers for a WMF grant (added additional considerations)
Grants:Learning patterns/Conducting a semi-structured interview (added additional considerations)
Grants:Learning patterns/Keeping documentation of discussions with team

Next steps and opportunities

What are the next steps and opportunities you’ll be focusing on for the second half of your project? Please list these as short bullet points. If you're considering applying for a 6-month renewal of this IEG at the end of your project, please also mention this here.

For Soni:

Continue recruitment of mentors for pilot program
Development of a Resource Center for all lessons as a standard lesson
Mentoring in the Pilot program
Begin to engage mentors in the space introducing them to the Co-op, its workflow, and gather feedback.

For Gabrielm199:

Consider redeploying the initial survey to gather more data about experience in existing mentorship spaces.
Formalize a respondent recruitment strategy.
Develop interview protocol for post-pilot interviews with mentors.
Pre- and post-survey for learners in the pilot
Finish up recruiting for interviews with mentors and learners across various mentorship spaces.
Continue data analysis as data comes in

For I JethroBT:

Hire our programming contractor so that the interface can begin to be built and adequately tested.
Hire a new graphic design contractor to complete additional design elements needed for the space.
Finish up interviews for existing mentorship spaces
Continue to recruit mentors for the pilot.
We are interested in applying for a 6-month renewal of this IEG provided that the pilot shows evidence that the space is successful and has a reasonable number of involved editors who are interested in maintaining and participating in the space beyond scope of the grant. This grant would allow us to further flesh out and expand the space to allow all editors to use the space, and also allow us to prepare a package for other Wikimedia projects to create a local Co-op of their own.

Grantee reflection

We’d love to hear any thoughts you have on how the experience of being an IEGrantee has been so far. What is one thing that surprised you, or that you particularly enjoyed from the past 3 months?

From Soni: So far my experience as a Grantee has been more than great. This has been a wonderful learning experience for me in terms of understanding how Grants and proposals approved by the Wikimedia Foundation are approved, and working so far has been more than productive.

One thing that I personally found as a pleasant surprise was how pro-active the Grants team, especially Siko were towards helping, especially during brainstorming sessions and by providing specific insights on topics that we had missed. The informality of the entire interactions, combined with the helpfulness, was one pleasant surprise and warmly welcomed.

From Gabrielm199: I have enjoyed conducting interviews and gathering data, although getting respondents has been a challenge and I would like to find a way to overcome this hurdle. Working on the user experience design of the interface has been enjoyable because it ties into the findings from my data collection. Being able to immediately see where my data collection might have an impact on the experience of newcomers and mentors in Wikipedia has been exciting.

From I JethroBT: Despite some setbacks with regard to interviewing and contractors, I've enjoyed working on this project and very much enjoy being able to help guide discussions in our research, design, programming, conceptual planning, and outreach arms in preparation for our pilot. I would also like to echo the generous support our team has received from our advisors, Siko and Jake, not only in terms of their time but also for maintaining our perspective and priorities when it feels like a ton of things need to get done at once. I was also very surprised about how productive our team during Wikimania, because from past experience, I know there can be a tendency to get swept up in outings and fun and networking at these events. We used our time very well there.