Wikimedia Meta-Wiki

Assessment of Belfer Center Wikipedian in Residence program

Harvard University's Belfer Center for Science and International Affairs hired a Wikipedian in Residence in 2012 - 2013. This page is for evaluation of the program's design and execution. There are also more granular reports of the Wikipedian in Residence's specific activities; see below.

Belfer Postmortem

edit

Project Report from the Wikimedia Foundation

edit

Purpose of this document

edit

The purpose of this document is to postmortem the Belfer Wikipedian-in-Residence project undertaken by the Wikimedia Foundation (WMF) and the Belfer Center between February 2012 and August 2013. The project has been criticized by Wikimedia community members and WMF staff as having violated community/editorial best practices. The purpose of this report is to document what happened, and to surface lessons learned and best practices derived from it, both for the WMF and any parties considering similar projects in future.

This report was written primarily by Sue Gardner, with help and vetting for factual accuracy from Erik Moeller, Lisa Gruwell, Geoff Brigham, Asaf Bartov and Siko Bouterse. Any errors and omissions are likely Sue's.

Summary

edit

In February 2012, at the request of a long-time WMF funder, the Wikimedia Foundation undertook a project to create a Wikipedian-in-Residence position at the Belfer Center for Science and International Affairs at Harvard. The Stanton Foundation funded the WIR's salary with a restricted grant to the WMF and worked with the WMF to craft a job description. The WMF advertised the job description, pre-interviewed candidates, and did some light supervision of the Wikipedian-in-Residence during his tenure. The Belfer Center interviewed candidates and made the final hiring determination, and hosted the Wikipedian-in-Residence and gave him some direction.

During the recruitment process WMF staff and community members raised strong concerns about the position as initially described in the job description, because it contained editing as a core job responsibility, which runs counter to community best practices and risked undermining the integrity of the Wikipedian-in-Residence model. They helped draft a new version of the job description that mirrored previous WIR positions and did not include any reference to article editing. The Wikimedia Foundation assured involved staff and community members that the role would not include editing as part of its core work.

The Stanton Foundation then maintained that improving articles was a core part of the job and asked the WMF to revert the job description back to the original one, which the WMF did without informing the staff and community members who had strongly recommended removing any reference to editing. The WMF conducted a first round of interviews after which it recommended four experienced Wikipedians as candidates to the Belfer Center. Those candidates were rejected by the Belfer Center due to a lack of strong academic background on international security issues. Then, the job requirements were reconstructed to prioritize subject-matter expertise over Wikipedia editing experience. The WMF then forwarded three new candidates to the Belfer Center who all had strong academic backgrounds in international security, but no significant prior Wikipedia experience.

In August 2012 the Belfer Center hired an international security expert for the Wikipedian-in-Residence role, who spent the next year in the role. During that time, he wrote about 66 articles on topics related to international security. He disclosed his affiliation with the Belfer Center on his userpage and seems to have generally adhered to Wikipedia policy in his article writing. However, it is generally considered inappropriate for Wikipedians-in-Residence to primarily write articles as a core activity of their residency, because of the possibility that their contributions may lean towards being non-neutral in substance or tone. The Belfer Wikipedian-in-Residence project was constructed to include a core editing component, despite community members and WMF staff urging otherwise during the development of the project.

The WMF believes it is a bad practice for those in Wikipedian-in-Residence positions to edit Wikipedia as a core activity of their residency, and regrets that it constructed the Belfer project to include editing as part of the position's core work. The WMF has made a number of decisions, detailed below, designed to prevent similar situations from playing out like this in future.

Lessons learned

edit
  1. At the point when it became clear that this project was not a simple pass-through grant but required programmatic work, the Executive Director should have transferred responsibility for it to a programmatic area. In general, it's a good practice to separate fundraising and programmatic work, because programmatic staff have programmatic expertise that fundraising staff lack. (For example in this instance, programmatic oversight would have likely resulted in regular public reporting.) Having programmatic work overseen by the fundraising department was a mistake.
  2. In the preliminary stages of the project, people who understood the Wikipedian-in-Residence model (Siko Bouterse and Asaf Bartov as WMF staff members, Lori Phillips in her one-year role as WMF’s U.S. Cultural Partnerships Coordinator, and Pete Forsyth and Liam Wyatt as volunteers) provided significant input on the job description and the hiring process, and revised the job description to be consistent with community expectations of a Wikipedian-in-Residence role. However, when the Belfer Center rejected candidates with strong Wikipedia experience, and when the the Stanton Foundation made it clear that it expected article editing to be a core job responsibility, the WMF acceded to that request, replacing the job description with a new version provided by the Stanton Foundation and the Belfer Center. The WMF didn't give that new version enough scrutiny before agreeing to it, and didn't inform the people who'd been advising us. This was a mistake.
  3. As the project played out and the Wikipedian-in-Residence began writing articles, it was flagged internally at the WMF that paid editing is controversial in the Wikimedia community. Despite these concerns being raised, the residency continued unchanged. This was a mistake.

Decisions made

edit
  1. In the future, the Wikimedia Foundation will not support or endorse the creation of paid roles that have article writing as a core focus, regardless of who is initiating or managing the process.
  2. The Wikimedia Foundation rarely accepts restricted or semi-restricted grants, or pass-through fiscal sponsorships. The Executive Director and Chief Revenue Officer agree that in the future, any grants that are not unrestricted will receive a special high level of scrutiny before being accepted.
  3. The ED commits that grants that require programmatic work will be reviewed by programmatic staff before being accepted, and will be executed by programmatic staff.
  4. The ED plans, with the C-level team, to develop a better process for staff to escalate and express concerns about any WMF activities that staff think may in tension with, or in violation of, community policies or best practices. It will take some time to develop a simple, robust process: we aim to have it done by 1 May 2014.

This is the end of the WMF report. Please do not edit the WMF report, but please feel free to add additional material below or on the talk page.

Pete Forsyth notes

edit

As the WMF report above notes, I (Pete Forsyth) was one of a few people who criticized WMF's role in the Belfer Wikipedian in Residence (WIR) program from an early stage. In fact, as far as I know, I am the only person independent of the organizations involved (WMF, Belfer, and Stanton) who had substantive and repeated contact with almost everybody involved in the program. (Liam Wyatt's role was significant as well, as described below, but not as extensive as mine.) I believe the following points fill in important gaps in the Wikimedia Foundation report above. I do not attempt to draw broad conclusions in this section.

I also consulted with a number of people both within and outside the Wikimedia movement, at various stages. Those discussions are the only reason any of this has come up for public review.

Here is a timeline of the significant points of contact I had with the program and its precursors:

  • I was hired as WMF's Public Outreach Officer in 2009, in what was then known as the "Programs" department. In that role, my primary function was to design the Wikipedia Public Policy Initiative, which was funded by Stanton.
  • In that role, and also in my later WMF role in Foundation Relations (early 2011), I developed an understanding of Stanton’s reasons for caring about and funding Wikipedia.
  • Sue Gardner introduced me to Lisa Gruwell, who was hired after I left, in fall 2011; Gardner emphasized my expertise about Wikipedians in Residence, and urged Gruwell to consult with me about Stanton's desire to fund a WIR.
  • I spoke in depth about Wikipedians in Residence with Liz Allison of Stanton on one and only one occasion (November 2011), when I learned that her perspective on the ethics of organizational engagement with Wikipedia was fundamentally incompatible with mine. We did not discuss Belfer.
  • Gruwell sought my input into the Belfer program, for the first time, about three days before the job posting was about to go live (April 2012).
  • I found the job posting highly problematic, and alerted Lori Byrd Phillips (then the WMF's U.S. Cultural Partnerships Coordinator, and one of the first WIRs). She, in turn, brought in Wyatt (the very first WIR, whose work had been featured as the centerpiece of the WMF's 2010 Annual Report). We all agreed it was vital to avert what we saw as a major threat to Wikipedia and to the integrity of the WIR concept.
  • The three of us participated in a 25 message email discussion, spanning close to two weeks. We and several Wikimedia staff members analyzed the problems with the program as designed, and discussed the best way to proceed. The discussion was primarily for Gruwell's benefit; she participated actively, posting four messages herself, and also actively engaging in a rewrite of the job description conducted in Google Docs.
  • Gruwell rebuffed my strong recommendation that an expert Wikipedian (me or somebody else) be assigned to guide the Wikipedian in Residence. The job description was edited to remove the most egregious of its many problems.
  • Also in April 2012, the Wikipedia Signpost published an interview in which I said: "Ideally, we [Wikipedians] develop skills that help us minimize the impact of [our] conflicts [of interest] on a project devoted to neutral information. People in a professional role are no less capable of developing those skills than anyone else, but it would certainly be helpful if other organizations' policies toward Wikipedia were to develop in ways that fully support Wikipedia's mission."
  • I heard nothing else about the Belfer program until April 2013, when Gruwell told me that the WIR was nearing the end of his term and leading Wikipedia workshops for academics. Up until that point, my best guess was that the program had been abandoned, and I had felt that was perhaps the best possible outcome given the circumstances.
  • In July 2013, I summarized my concerns about the program in an in-depth conversation with Gardner.

Please do not edit this timeline, but please feel free to add additional material below or on the talk page.

Timothy Sandole's reports

edit

Timothy Sandole provided a final project report and weekly memos to the Belfer Center staff he worked with (James Smith, Graham Allison) and to Sara Lasner at the Wikimedia Foundation. These reports weren't initially intended to be published, but given the concerns raised, he gave permission to publish them, and they're available on Wikimedia Commons:

Further material

edit

The Harvard University fellowship[1] from 2012-13 has generated some controversial discussion[2] due to perceived lack of transparency (lack of disclosure on Wikipedian's user page about WMF sponsorship, lack of public report or follow-up), and a heavy focus on editing and internal discussions and events at the host institution (as opposed to e.g. outreach/capacity building).

Internal assessments

edit

What follows is a chronological directory of responses and analyses of what went wrong with Belfer, what went right, and how to improve, from members of the Wikimedia community.

External coverage

edit

What follows is a chronological directory of coverage of the Belfer situation from people outside of the Wikimedia movement (media coverage, etc.)

Issues / concerns

edit
  • Experienced Wikipedians felt that their feedback early on (e.g. not to place high emphasis on editing) was dismissed in favor of getting the project off the ground quickly.
  • The project report from the Wikipedian in Residence was not initially made public.[3]
  • The applicants were not properly trained, or they received misleading information, i.e. the assertion of the job posting that "Wikipedia" hires people.[4]
  • There were publicly contradictory statements about whether the WMF had control over recruitment and whether the WMF managed the position.[3][5]
  • The selection of the Wikipedian in Residence in question was never announced.
  • There was no unambiguous plan or statement published with regard to funding, either before or during the project.
  • A blog post by community member Odder noted some of the above concerns, and noted that the Fellow did not mention on his userpage that funding for his position came through Wikimedia.[6]
  • Other community members wonder how the WMF will avoid knowingly doing the wrong thing again.[1] [2]
  • Some have expressed concern about the personal relationship (by marriage) of the directors of the Stanton Foundation (Liz Allison) and the Belfer Center (Graham Allison), and questioned the integrity of the program on that basis. [3]
  • The Stanton Foundation has historically been a major supporter of the Wikimedia Foundation, and recently gave the Wiki Education Foundation a large grant to support their programmatic activities.

For more see archives of wikimedia-l discussions:

Lessons learned

edit
How should the Wikimedia Foundation adjust its practices?
  • The Belfer position was created because of the desires of a donor (and was part of a restricted grant?) The Wikimedia Foundation has historically been cautious about letting donors guide where their funds go and about accepting restricted grants, but that caution should probably be increased. Given the potential fundraising capacity of the Foundation, there are probably very few if any situations where accepting restricted grants or letting donors guide what their grant money goes to is in the WMF's best interests.
How should organizations, in general, design and execute WiR programs?
  • Wikipedians in Residence are facilitators and trainers, more than they are writers or editors. WiRs should strive to model ethical and effective engagement with Wikipedia, to their host organizations and to others in their spheres of influence. Most significantly, they should seek the collaborative improvement of Wikipedia content, according to Wikipedia's content standards and practices; they should not be primarily writing Wikipedia content, and it should be clear from how the organization designs and presents their position that they are not seeking to advance a specific editorial objective.

In June 2014, Community Liaison @Mdennis (WMF): added the grantmaking department's interpretation of Sue's words above: User talk:Mdennis (WMF)/Archive 1#Pings

Community Resources/Grantmaking team's update

edit

Before and after this post mortem, WMF has funded (directly, and through our affiliate partners) a number of residencies. When advising grantees and potential applicants in any WMF grants program, we encourage grantees to cover the costs of Wikimedians in Residence (WiRs) through host institutions, or to engage volunteers in this work.

There have been more than 40 WiR programs involving affiliates who received funding through the FDC/Annual Plan Grant process. We do not do an in-depth review of each of the WiR programs after they are completed. We don’t have the capacity to do so, and don’t believe it is compatible with the goals of the APG program, since grantees receive general support funding through this program and are trusted to make decisions about how to use their funding effectively for mission-aligned activities (as specified in their grant agreements). If any issues surface related to Wikimedians in Residence being paid inappropriately for editing activities, we would address them on a case by case basis. There have not yet been any issues of which we are aware.

In the past, Project & Event Grants (PEG) and Individual Engagement Grants (IEG) have followed the same guidelines for funding Wikimedians in Residences (WiRs) -- WiRs whose focus is to collect resources, train/engage GLAM and Wikimedian community, and organize events around integrating content are eligible for funding. WiRs whose sole focus is to digitize resources and/or personally create content are not eligible for funding. In cases where funding for a residency is provided directly by WMF or a grantee receiving funding from WMF, we ask grantees to make job descriptions of Wikimedians in Residence that do not include paid editing of the Wikimedia projects.

These guidelines are also outlined in the Rapid Grants and Project Grants programs. The PEG/IEG programs have funded only a handful of WiRs (or similar positions), which have gone under careful review by the community and respective grants committees. Examples of the types of WiRs we have funded through PEG and IEG include GLAMing Madrid, Metro Institutional Growth and Community Fellow, UNESCO Wikimedian in Residence, and Wikipedian in Residence for Gender Equity. In all of these cases, the WiR has served as a trainer, facilitator, and networker between the GLAM institution and community.

KLove (WMF) (talk) 16:24, 17 October 2016 (UTC)[reply]

Outcomes

edit

Other Wikimedia Foundation-funded Wikipedians in Residence

edit

Prior to the Belfer Center position, there were two Wikipedian in Residence positions funded by the Wikimedia Foundation:

In 2015, Wikimedia Foundation funded Wikipedian-in-Residence programs again, for a total of several tens of thousands dollars.

References

edit
  1. ↑ WMF blog post, 26 April 2012
  2. ↑ Wikimedia-l email list discussion, March 2014.
  3. ↑ a b User page statement by Sandole, March 2014.
  4. ↑ WMF job advertisement, before August 2012
  5. ↑ Changing definition of role, Wikimedia Foundation Report, August 2012.
  6. ↑ The pot and the kettle, the Wikimedia way, March 2, 2014.
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Assessment_of_Belfer_Center_Wikipedian_in_Residence_program&oldid=24927505"