Grants talk:Project/Rapid/Touro College/publishing wiki education research in healthcare

Comments

Anyone may comment here. Blue Rasberry (talk) 15:49, 8 August 2018 (UTC)

• May I suggest that because the research is of interest to anyone doing Wikimedia-based educational outreach, you should consider publicising it through Chapters and User Groups, all of whom will be involved in outreach. I know that WMUK is always interested in receiving blog posts that have a different "angle" on Wikimedia-related topics, for example, so there may be other avenues by which you can make the community aware of your work. --RexxS (talk) 16:47, 8 August 2018 (UTC)

Marginal benefits?

What does this get us that putting it on something like W:Social Science Research Network or W:ArXiv does not? I don't see why it's a necessary expense. --50.201.195.170 23:05, 9 August 2018 (UTC)

Aha! https://meta.wikimedia.org/wiki/Learning_patterns/Paying_for_an_open_license_for_academic_research_related_to_wiki --50.201.195.170 23:13, 9 August 2018 (UTC)

Yes the learning pattern tells the story.

I do not like being mixed in political controversy but everyone is part of this. Academic publishing is a industry which directs several billion dollars every year. 100s of 1000s of people are on each side of the controversy with this huge amount of money. I do not like sending Wiki community funds to commercial academic publishers and yes, I wish that publishers like ArXiv or some open access publication was popular in every field, but I feel that many fields are not at this point yet.

The $2000 I am requesting makes this paper accessible without a paywall to more readers. It puts the article in a more traditional journal reaching a more established audience. Someday I want to use more economical and sustainable publishing models but right now my opinion is that Wikipedia as a movement needs to publish in the most respected journals, especially in cases where the research describes an activity which we want everyone to replicate. This particular article describes some outcomes of Wikipedia editing by students in classrooms. I want every class replicating this model. Blue Rasberry (talk) 20:41, 13 August 2018 (UTC)

Is there a preprint of the article available? Or do you plan on submitting it to a publisher who is hostile to preprints? HLHJ (talk) 18:56, 18 August 2018 (UTC)

@HLHJ: I strongly favor using preprints and almost feel that if preprints are not permitted then I should not take WMF money for this.

Yes I want to share a preprint, no I have not. I am conflicted here and sorting this out. I want to use an open publisher. My advice is that the wiki community should demand preprints for these kinds of grants. Blue Rasberry (talk) 23:09, 20 August 2018 (UTC)

@HLHJ: The preprint is at "Improving the Quality of Consumer Health Information on Wikipedia" and published by Journal of Medical Internet Research (Q6295534). In addition to having that journal host the preprint we also submitted to that journal. Please see the Wikidata-based profile of that journal through d:Wikidata:Scholia. Blue Rasberry (talk) 15:51, 10 October 2018 (UTC)

Thank you, Bluerasberry! I've read it, and it looks useful. If I may slightly misuse this forum to comment on the content, I'd be really interested in stats on the post-editathon activity of the newly-trained editors. I'm sorry I didn't respond in time to support your request, and wish you well with it. HLHJ (talk) 01:37, 22 October 2018 (UTC)

@HLHJ: Your questions are not a misuse because this is the only forum which exists at this time. I am making another project page to document some new outcomes but I do not have it yet. When I do I will carry your comments over.

I can only share community lore and guesses in this space and cannot provide data.

My guess is that the student retention rate is <2%, because so far as I know 2% is the highest retention rate of any wiki outreach project targeted non-self-selected new users. I might further guess that the student retention rate for this project is 0.

Wiki Education Foundation oversees about 400 United States based classes every 4 months. They are the think tank saying that wiki classroom outreach provides the most value by seeking to retain instructors and schools rather than individual editors. For this project, these instructors are on their 4th year of engagement and they just joined WikiConference North America, further developing their wiki engagement. I have spent more time on the relationship with the instructors and school than I have on the students, following the lead of Wiki Edu. That said - if anyone wanted to check retention themselves, they could using the public dashboards for these classes or any classes.

I have another pet project to discuss grant metrics. At Grants:Metrics#Three_shared_metrics see that the WMF puts attention on metrics other than retention. I would guess that this recommendation guides about US$10,000,000 in funding. Even while they make this recommendation, I think that data or community discussion which backs this policy recommendation does not exist. Those metrics do support the outcomes of classroom outreach, which creates short-term experiences, but a different outreach strategy could support longer-term editor engagement which we measure with some kind of retention metric. This is weird for a few reasons. (1) WMF does not really support classroom outreach - Wiki Edu does. (2) The Wiki community wants retention, but WMF requires metrics which assume low-retention outreach strategies. (3) Low wiki community participation in the policy development, yet extremely high global and expensive wiki community participation in outreach strategy development.

If you want to guide this conversation in a particular direction then I would join you. There are lots of ways to go to achieve progress. Blue Rasberry (talk) 18:45, 23 October 2018 (UTC)

@Bluerasberry: That is flat-out weird, thank you for telling me about it. For anyone else reading this, Research:Editor retention has the data supporting the idea that Wikipedia does not have a problem with getting new editors, but with retaining them. Retention may well be biassed, in which case poor retention would exacerbate systemic bias; we haven't looked yet. It's not even as if retention is technically harder to measure than number of new editors; you would need to pick a metric, which might be politically difficult, but that's the only disadvantage I see. Wikipedia's Eternal August deeply worries me.

What to do about it is another matter. Discussing the grant guidelines seems like a good step (is that ten US megadollars per year?). There is a more nuanced set of metrics: Learning and Evaluation/Global metrics. Wikiproject Editor Retention has a barnstar for editor retention; I have not seen it used. I have drafted, offline, an informational user-warning set for people reverting fixable good-faith new edits, as there seems to be some community feeling that there is a point beyond which discouraging potentially good new editors is disruptive. Modifying editing tools to make it relatively easier to fix fixable new-editor edits seems difficult, but promising as an approach. Community wishlist is coming up; would, say, flags on the edits of high-attrition-phase users help? Some sort of positive-feedback notification for those who help them through this phase? You know the wikis much better than I; what do you see as the approaches with the best chance of working? HLHJ (talk) 03:19, 25 October 2018 (UTC)

Authors

Who are the authors? (IRL and here)?--50.201.195.170 23:05, 9 August 2018 (UTC)

en:WP:Touro lists several events...--50.201.195.170 23:13, 9 August 2018 (UTC)

Yes this is my oversight. I am Lane Rasberry the experienced Wiki editor. The others are Shira Weiner, who has a profile at New York University, and Jill Horbacewicz, who has a profile with Touro College. All of this research happened through Touro College and not NYU. The professors are not wiki editors themselves even though their students did edit in the class. Blue Rasberry (talk) 20:35, 13 August 2018 (UTC)

Comments from WMF

Hi Bluerasberry. Thank you for this proposal and your continued efforts to make more Wikimedia-related research open and accessible. We have a few questions about the project:

1. Can you provide more information about what the article is about? Is it a case study of a Wikimedia education program focused on medical content? Since paying open access fees is not a very scalable use of resources, we want to make sure that the content of the article is both widely relevant to the Wikimedia community and that the outreach plan will reach it's intended audience, outside and inside the Wikimedia movement. What new information will readers learn that they previously did not have access to before?
2. Where will the article be published and why did you choose that particular publication?
3. Have you requested support from Touro College since the project was based there?

Thanks, Alex Wang (WMF) (talk) 16:42, 6 September 2018 (UTC)

@AWang (WMF):

1. Yes, the article is a case study of a Wikipedia education program focused on medical content. I published a similar paper and got similar open access funding from the WMF for en:Wikipedia:Why Medical Schools Should Embrace Wikipedia. The most obvious difference with this case is that it demonstrates a much less resource intensive medical collaboration and still got good results. Unusual aspects of this cohort from a wiki perspective include it being a 3-year study and it presents some metrics which are outside the scope of what is typical for WMF reporting, like persistence of edits. Unusual aspects from an academic perspective are that it is publication from a wiki insider's perspective - most papers at en:Wikipedia:WikiProject Medicine/Research publications are written without input from anyone in the wiki community. The Wikipedia Education program recruits very little wiki leadership from academics and that is a deficiency to address as part of an outreach strategy. I would like to start a trend for more academics to feel comfortable making most of the program decisions in their Wiki collaborations and speaking to them on their own terms - academic publication - seems to me to be the most natural way to do that. This project also was unusually led with significant academic input matched with wiki oversight. My view is that most of these ideas are new and unlike Wikipedia community discussion or anything previously published.
2. Publication is difficult. The goal is to choose the most popular, most respected, most values-aligned (open) journal possible. We have been shopping the article for a few journals. Some considerations we have are who has been friendly to wiki publication in the past, who is likely to get good peer review, who can be most open in the most ways, who is indexed best (in PubMed of course but also more), and who has the best promotion.
3. Yes Touro has committed $1000. The likely fee seems like $3000. Blue Rasberry (talk) 18:18, 6 September 2018 (UTC)

Approval

Hello Bluerasberry, thank you for providing answers to our questions. I am approving your grant request. Best regards, WJifar (WMF) (talk) 21:41, 21 September 2018 (UTC)

Request to change organization receiving funds

There are two organizations listed in this grant request. One is the University of Virginia, and the other is Touro College. Both organizations have a role in this project.

This request was drafted as a request for money to go to the University of Virginia. I would like to change this to go to the other named organization with the other named individuals, which Touro College and the professors there. Other terms and commitments stay the same.

• See the changes.

Behind the scenes we have been discussing our schools' own bureaucracies and how to make the payment to the published. Right now it seems that Touro College will pay the publisher, so we would prefer that money go to that institution.

I am not pinging the WMF on this yet because there is still bureaucratic review in process.

Thanks. Blue Rasberry (talk) 19:40, 9 October 2018 (UTC)

Okay, I got some review from Touro College. They have agreed to receive these funds if the WMF can approve the change in terms from University of Virginia receiving this to Touro College. @MJue (WMF):, I have been in email contact with you about this grant. Are you able to approve the change which I linked above in this thread? Thanks. Blue Rasberry (talk) 21:57, 9 October 2018 (UTC)

Hello Bluerasberry, I'm responding on behalf of Morgan. Thank you for making the necessary changes in regards to the funds receiving entity. Since both are similar educational entities we don't need to grant additional approval. @MJue (WMF): will be in touch with necessary next steps. Best regards, WJifar (WMF) (talk) 22:40, 9 October 2018 (UTC)

Paper published today

• Weiner, Shira Schecter; Horbacewicz, Jill; Rasberry, Lane; Bensinger-Brody, Yocheved (18 March 2019). "Improving the Quality of Consumer Health Information on Wikipedia: Case Series". Journal of Medical Internet Research 21 (3): e12450. doi:10.2196/12450. 

I announced it at English Wikipedia's WikiProject Medicine.

• en:Wikipedia_talk:WikiProject_Medicine#Case_study_of_wiki_in_several_cohorts_of_classes

Blue Rasberry (talk) 17:35, 18 March 2019 (UTC)

Audio interview about the program and research behind this grant

File:Wikijab018-jill-horbacewicz-shira-schecter-weiner.mp3 from WikiJabber. Blue Rasberry (talk) 13:34, 3 May 2019 (UTC)