Grants:IdeaLab/Effective Engagement with Health Topic Experts using Guided Checklists

statuswithdrawn

Individual Engagement Grants

Effective Engagement with Health Topic Experts using Guided Checklists

summaryToday Wikipedia is a primary source of health information on the internet. But many of Wikipedia's medical articles are incomplete or contain outdated information. Health topic experts are interested in improving the health content on Wikipedia through engagement with Wiki Project Med, Wiki Ed Foundation, and other global initiatives.

targetWikipedia English

strategic priorityimproving quality

amount$6,500 USD

grantee• FloNight

advisor• Thepwnco

contact• Sydney Poore/FloNight (talk)• sydney.poore@gmail.com

project manager• FloNight

this project needs...

volunteer

created on22:58, 8 April 2016 (UTC)

round 1 2016

Friendly space expectations

Project idea

What is the problem you're trying to solve?

Wikipedia is the most popular source of health information on the internet. Wikipedia is often one of the first results when searching online for health information.

Despite the high use of Wikipedia by consumers, students, and health care professionals, too much of Wikipedia's health content is incomplete, outdated, or is written using low quality sources. The majority of the Wikipedia health content is not regularly peer reviewed by Wikipedians. Wikipedia English has over 35,000 health related articles with the majority of them being start class or stubs. See Summary Assessment Table.

Health topic experts are being recruited to contribute to Wikipedias by Wiki Project Med, Wiki Ed Foundation, and other global initiatives through Wikimedia affiliates, or the WMF Education program. There is currently a growing interest from institutions of higher education, medical research organizations, medical publishers, and government agencies to contribute to Wikipedia.

New approaches to editing such as the visual editor, and training materials specific to the medical topic have decreased the number of poor edits to medical articles. But there still continues to be a big gap between the expert knowledge of the new recruits and quality of content that is being added by them. Because health topic experts are busy people, they need to see positive results to consider the time contributing worthwhile.

Despite training by experienced Wikipedia medical editors and Wikipedian in Residence, many health topic experts are lacking understanding about quality review process on Wikipedia, and have a difficult time understanding the most effective use of their time and efforts on Wikipedia.

What is your solution?

The health care experts and other people interested in editing Wikipedia's health content would benefit from a better process to assess the quality of medical content that would guide them towards making higher quality contributions/edits.

This proposal is influenced by the learning from Grants:IEG/Full Circle Gap Protocol: Addressing the ‘Unknown Unknowns’/Final which engaged topic experts on the gender gap and then worked with instructors and students in Wiki Ed Foundations courses. It will built from their experience working with topic experts to find gaps in content.

Specifically, the learning from Grants:Learning patterns/Engaging non-Wikipedian academic experts to identify content gaps will be used to draft interview sessions and guided checklists for health topic experts. Additionally, other work on content gap analysis will be reviewed such as Wikipedia:Wikipedia:Society for Marine Mammalogy/Content gap analysis

Further, I'll do a thorough literature review of existing peer review instruments and guidelines for reviewing medical content. Below are sample results of a quick search.

• How to Critically Appraise an Article

• Review Form

• How to review a surgical paper: a guide for junior referees

• A Systematic Guide for Reviewing a Manuscript

Relevant concepts will be carried over into a Guided Checklist for Wikipedia health articles.

Project goals

The goal of this project is to create a easy to use and effective quality assessment tool for Wikipedia medical content. The end goal would be checklist that health topic experts would use to review health content and guide their decisions about how to make the highest quality edits.

The health topic quality checklist for use by health topic experts (and others) would be created by:

1. Using the current quality assessment methods and tools on Wikipedia English.
2. Interviewing at least 5 active medical editors from Wikipedia English
3. Interviewing at least 5 active medical editors from other language Wikipedia
4. Interviewing Wiki Ed Foundation representative/staff
5. Interviewing at least 5 health topic experts from Cochrane and other health research institutions.

Project plan

Activities

• The primary activity would be creating 2 guided checklists for health topic experts by doing research and interviews with Wikipedia medical editors and external health topic experts, and a community consultation.
• The secondary activity would be streamlining the quality assessment pages of Wikipedia English WikiProject Med to make the quality assessment tools easier to find and utilize.

Timeline: Draft

• Month one:
  • Brainstorming session at Wikimania Medical Pre-conference.
  • Begin formalizing partnerships with subject matter experts

• Month two:
  • Continue formalizing partnerships with subject matter experts
  • Begin formal community consultation with medical editors
  • Begin Structured interviews with subject matter experts by google hangout or Skype

• Month three:
  • Continue Structured interviews with subject matter experts by google hangout or Skype
  • Continue Structured interviews with active Wikipedia medical editors

• Month four:
  • Design checklist
  • Tie in with Year in Science focus on Medicine in September
  • Participants will trial the use of the Checklist

• Month five:
  • Other health topic experts and medical editors invited to use the Checklist

• Month six:
  • Evaluation survey to participants
  • Community consultation wrap up
  • Final Report

Budget

• $5,500 - Project manager at 10hr/week, $25/hr for ~220 hours
• $500 - A $100 honorarium for five health topic experts advisers
• $250-500- Wikipedia swag for 10 medical active editor advisers who participate
• Total: $6,500 USD

• Travel to Wikimania and Medical Pre-conference for myself paid with an existing Wikimania Scholarship.
• Travel to Wikimania Medical Pre-conference for external health topic experts paid for by external funding.
• Expenses for room, refreshments, wifi for brainstorming session at Wikimania Medical Pre-conference built into the existing conference.

Community engagement

The WikiProject Medicine and other health related WikiProjects on Wikipedia English will be notified and invited to participate. Wiki Project Med Foundation and Wiki Ed Foundation will be contacted and invited to participate. Cochrane, UK Cancer Research, and NIH, health experts at universities will be invited to participate.

Community Consultation

A part of the project will be a community consultation on Wikipedia English with medical editors about quality assessment of health information on Wikipedia English, and the use of a Guided Checklist to get expert advice about the quality of medical content and the best way to integrate the Guided Checklist into the workflow.

Diversity of participants

Although the main scope of this proposal is focused on introducing the Guided Checklists into Wikipedia English, Wikipedia medical editors and health topic experts from other languages will be invited to participate in the community consultation, and informed about the final conclusions.

Wiki Project Med is a global organizations with members from around the world. Wiki Project Med partners with organizations with an international reach such as Cochrane, the World Health Organization, Translators without Borders, and national government health agencies. Medical editors from around the world collaborate in joint projects. Through international collaboration the Translation Task Force has translated over 1,300 articles in more than 90 languages.

Wikipedia Year of Science 2016

The Wiki Ed Foundation in cooperation with Wikipedians is sponsoring Wikipedia Year in Science 2016

• The Year in Science Portal on Wikipedia English

• The focus of YoS during September is Medicine.

During the month of September I would tie in with the YoS medicine theme to promote the importance of health topic experts collaborating with Wikipedia.

Sustainability

The goal of the project is to have a new tool for engagement with health topic experts that would be included in training sessions and offered to Wikipedians in Residence at health related organizations, Wiki Ed Foundation for use in training instructors, and other training sessions for health topic experts.

Ideally, the Guided Checklist could offered to use in the Global Event and Project Dashboard for use by trainers planning medical events.

Impact

Wikipedia medical articles are widely viewed. Popular medical pages on Wikipedia English. So, improvements to the quality of the medical articles will directly improve the health information of readers.

Measures of success

Two health topic checklists are created. One for general medical topics, and one for pharmaceutical drugs.

In a second phase of the project, health topic experts and medical editors will measure the success of using the checklist. A survey will be done of health topic experts using the checklist to measure satisfaction with using the checklist and the quality of contributions of people using a checklist.

Goal of 10 new health topic experts using the Guided Checklist and making high quality edits to 10 medical articles.

Project team

Sydney Poore is on the Board of Wiki Project Med Foundation, a Wikipedian in Residence at Cochrane, and an editor of Wikipedia English since 2005.

Community notification

Please paste links below to where relevant communities have been notified of your proposal, and to any other relevant community discussions. Need notification tips?

Notified Talk:Wiki Project Med , and w:en:Wikipedia talk:WikiProject Pharmacology, Wikipedia English WikiProject Med.

Additional notification at w:en:Wikipedia talk:Year of Science/Projects.

Endorsements

• seems like a very good idea Ozzie10aaaa (talk) 10:25, 9 April 2016 (UTC)
• I support this project. Right now it is still an outline. This project covers one of the essential basic tasks which will have to be done by someone eventually. Sydney is an ideal person to manage this. The desired project outcomes are reasonable to expect given a time investment and community participation. I have no doubt that the community would want to participate in giving comment to this. I do not anticipate that anything could go wrong with this project. Blue Rasberry (talk) 14:51, 9 April 2016 (UTC)
• Guided checklists would be very helpful for Wiki Ed's student editors from medical schools as well, and we'd love to see the outcome of this project, especially as part of the Year of Science. I have great confidence in Sydney's ability to create a useful tool for others to use. --LiAnna (Wiki Ed) (talk) 17:11, 11 April 2016 (UTC)
• Strong proposal, and Sydney has the skills to do it well and make the results useful.--Ragesoss (talk) 20:04, 11 April 2016 (UTC)
• Firmly support this project. In the past we've had subject experts turned off Wikipedia because of discussions about authority and primary sources. I am fairly certain that the deliverables from this project will go some way to improve this. JFW (talk) 20:15, 13 April 2016 (UTC)
•  Support Tobias1984 (talk) 18:20, 26 April 2016 (UTC)
• Greater onboarding of experts and students is required. This looks like a step in that direction. Doc James (talk · contribs · email) 09:45, 1 May 2016 (UTC)
• This is a great proposal, and an important project. I'm glad to see the working with academic experts learning pattern from the Gap Finding pilot project is being further developed. There are more lessons to be learned. I will look forward to hearing how the check list development goes. What will work for this specific group of experts? How will the "knowledge transfer" go? I'd like to suggest careful documentation of the process from the project manager. Shameran81 - Monika Sengul-Jones (talk) 17:26, 3 May 2016 (UTC)
• I believe this project has excellent potential, and would work very well with the project I wish to run — which also depends on collaboration with health organizations: Grants:IEG/Health images for all. I believe there we could achieve quite a bit of synergy if both were accepted. CFCF 💌 📧 14:52, 9 May 2016 (UTC)
•  Support Sydney can make it work. Raystorm (talk) 16:49, 30 May 2016 (UTC)
• Because Wikipedia's health coverage, as the introduction to this idea says, is a popular source for health-related information. It should be improved, and this idea would help with that improvement. Chickadee46 (talk) 23:20, 5 June 2016 (UTC)
•  Support Very strong support, makes a lot of sense Cmungall (talk) 05:32, 13 September 2016 (UTC)

Participants

• Volunteer
  • I'd be happy to do some of the leg-work within the UK, if needed. RexxS (talk) 14:56, 9 April 2016 (UTC)

• Advisers
  • I have expertise (and 40 years experience) in Education and Training, and am a regular editor of medical articles. I'd be happy to provide advice, if needed, on training materials, technical editing, project design and areas of medicine where I have expertise. I am happily retired and would prefer not to receive any honorarium, etc. RexxS (talk) 14:56, 9 April 2016 (UTC)