Grants:Project/Learning with Wikipedia Based Assignments
Project idea
editWhat is the problem you're trying to solve?
editWhat problem are you trying to solve by doing this project? This problem should be small enough that you expect it to be completely or mostly resolved by the end of this project. Remember to review the tutorial for tips on how to answer this question.
Thousands of instructors use Wikipedia as a teaching tool throughout the world. At the same time we don't fully understand the benefits for the students, the instructors, and for Wikipedia (and the world) in general. Although there is ample anecdotal evidence that Wikipedia is helpful to teach writing skills, collaboration skills, information literacy, digital literacy, and other important transferrable skills, there is little empirical research on this subject.
In Fall 2016 the Wiki Education Foundation started this process investigating Student Learning Outcomes (SLO) with Wikipedia-based assignments. Over 1600 students participated in our surveys, and we conducted 13 focus groups in North American higher education classrooms using Wikipedia based assignments. This research acts as an example, model, and framework for future research, but it is far from complete and requires more time and effort to continue.
What is your solution?
editFor the problem you identified in the previous section, briefly describe your how you would like to address this problem. We recognize that there are many ways to solve a problem. We’d like to understand why you chose this particular solution, and why you think it is worth pursuing. Remember to review [Grants:Project/Tutorial|the tutorial]] for tips on how to answer this question.
The Fall 2016 study was a pilot study, attempting to gather preliminary information on a variety of research areas that were of interest to educators, researchers, and the Wikipedia community. It provided us with a variety of preliminary findings, all of which require corroborating with more evidence, narrowing down contexts, and triangulating our understandings of student responses.
This study will benefit researchers and students by narrowing the focus of previous research and increasing the amount of data on this underrepresented subject. More data on this subject will allow a better understanding of the student's contexts that effect attitudes, productivity, digital literacy, collaboration, and skills transfer. This information is beneficial to Wikipedia and Wikipedia researchers because it helps triangulate attitudes and contexts for understanding how students engage Wikipedia, but it also helps create knowledge about increasing student learning while collaborating on and creating Wikipedia articles. In the end this research will help improve Wikipedia (and the world) because it will help to improve teaching and learning using Wikipedia, particularly in regards to creating ideal contexts for new editors.
We will publish the data, the survey design, the interview questions, and our own analysis of this data openly.
This will not only help educators and educational researchers, but it will help Wikipedians across the world understand the attitudes and contexts of new participants in the Wikipedia community. This data will be invaluable in helping to understanding the Wikipedia learning process, as well as helping to unlock broader knowledge about student engagement with Wikipedia in general.
Others will not only be able to use the data but also use the tools to conduct their own research that can use this as a model and build off of it.
Finally, this grant will allow us to improve our tools that we utilize to gather the data, leading towards better data for everyone using the dashboard tool.
Background / Previous Research
edit- Fall 2016 Pilot Study
In Fall 2016 the Wiki Education Foundation assisted over 6000 students used a Wikipedia-based assignment in lieu of a traditional paper assignment. WWe conducted a mixed methods research study that assessed SLOs while using Wikipedia-based assignments. We conducted research on information literacy and research skills, alongside attitudes toward the assignment and toward Wikipedia, as well as reflections on their experience. Over 1600 students participated in the surveys, along with focus groups conducted in 13 different classes.
- Preliminary Findings
Despite mixed initial reactions to hearing they would be using Wikipedia in the classroom (30% negative, 30% neutral, 40% positive), we found that a majority of students spent more time (31% more time versus 20% less time), were more satisfied with their work on the Wikipedia assignment (50% more satisfied, 37% the same, and 13% less satisfied), and found the assignment more valuable in a variety of ways.
Instructors highly valued using Wikipedia in the classroom, particularly when rating the value of a Wikipedia assignment against a traditional assignment in developing digital literacy (96% more/much more valuable), for learning about the reliability of online sources (85% more/much more valuable), and for learning about writing clearly for the general public (79% more/much more valuable). Students survey responses skewed slightly more to the center, with a high percentage (~30%+) students selecting “about the same” for their valuation. Similar to instructor responses, students were most confident about Wikipedia being more valuable for the reliability of online sources (63% more/much more valuable), developing digital literacy (70% more/much more valuable), and learning to write clearly for the general public (72% more/much more valuable). In fact, no Wikipedia assignment was perceived as “less valuable” than a traditional paper assignment, from learning about the topic, to developing critical thinking, to computer skills, peer review, or working on a team. There are a variety of different contextual clues to understanding the most ideal student learning experience (length of assignment, type of editor interaction, subject matter, etc) that we are currently investigating to help understand the most ideal situation for students to produce excellent Wikipedia articles.
Additionally, one of the more telling questions asked students to use three words to describe how they would describe Wikipedia before the assignment, as well as use three words to describe it after. There is an easily discerned trend, which make a distinct break from “informative, unreliable, and confusing” before to “informative, reliable, and collaborative” afterwards. This question points us to future research that can help understand attitudes towards Wikipedia, particularly this transformation from "unreliable and confusing" to "reliable and collaborative," narrowing down the understanding of how students grasp the collaborative space of Wikipedia.
- Where we are at
We have an incredible amount of data, a solid model and guidance for continued research, and an ongoing stream of students to help us understand attitudes and contexts of students working with Wikipedia.
We are in the final stages of preparing the data for open publication, which includes de-identified student survey data (linked to Wiki Education courses), de-identified instructor survey data, survey tools used in the data collection, as well as preliminary analysis of the data.
Additionally, we are currently preparing three different conference presentations and four papers on these findings.
Project goals
editWhat are your goals for this project? Your goals should describe the top two or three benefits that will come out of your project. These should be benefits to the Wikimedia projects or Wikimedia communities. They should not be benefits to you individually. Remember to review the tutorial for tips on how to answer this question.
- Overall - Improved understanding of learning and teaching with Wikipedia through repeatable (and repeated) research and research data dissemination on learning with Wikipedia
- Survey design for both repeated and semester-specific research questions
- Publication of survey tool in open format
- Continued data collection and analysis
- Publication of data in open format
- Presentation of data and findings at national and international conferences
- Open publication of analysis and presentations
- Improved survey system for long term research support
Project impact
editHow will you know if you have met your goals?
editFor each of your goals, we’d like you to answer the following questions:
- During your project, what will you do to achieve this goal? (These are your outputs.)
- Once your project is over, how will it continue to positively impact the Wikimedia community or projects? (These are your outcomes.)
For each of your answers, think about how you will capture this information. Will you capture it with a survey? With a story? Will you measure it with a number? Remember, if you plan to measure a number, you will need to set a numeric target in your proposal (e.g. 45 people, 10 articles, 100 scanned documents). Remember to review the tutorial for tips on how to answer this question.
We recognize that academic research has a long tail for success measurements, as dissemination of information does not necessarily map to performance indicators. However, we believe that we can measure success in the following ways:
- Survey participant count - We are aiming for N>1600, goal of N=2000
- Successful interviews - Considering these are from student volunteers, we are hoping that SP18 we will be able to conduct 20 student interviews
- Presentation at research conferences - Present at four conferences between FA17-SP18.
- Publication of research in peer-reviewed journal(s)
- Publish at least two articles in peer reviewed journals regarding student learning with Wikipedia.
Do you have any goals around participation or content?
editAre any of your goals related to increasing participation within the Wikimedia movement, or increasing/improving the content on Wikimedia projects? If so, we ask that you look through these three metrics, and include any that are relevant to your project. Please set a numeric target against the metrics, if applicable. Remember to review the tutorial for tips on how to answer this question.
Project plan
editActivities
editTell us how you'll carry out your project. What will you and other organizers spend your time doing? What will you have done at the end of your project? How will you follow-up with people that are involved with your project?
As with the previous research, we will work with Zach McDowell at the University of Illinois Chicago to (re)design the research project, including survey design and interview questions.
- Fall 2017
- Seek approval from University Human Subjects Review Board for study
- Continued analysis of FA16 research data
- Continued presentation and research publication
- Redesign survey. Focus on contexts, attitudes, and perceptions of skills transfer, digital literacy, and attitudes toward Wikipedia
- Conduct surveys and collect data
- Collect volunteer emails for Spring 2018 interviews
- Spring 2018
- Open publication of Fall 2017 data
- Interview Fall 2017 volunteers
- Analysis of Fall 2017 data
- Presentation of findings
- Conduct surveys and collect data
Budget
editHow you will use the funds you are requesting? List bullet points for each expense. (You can create a table later if needed.) Don’t forget to include a total amount, and update this amount in the Probox at the top of your page too!
- Costs
- Researcher (also known as Principal Investigator/PI) cost: $12500
- Research Assistant (RA) Cost: $12500 per semester X 2 semesters: $25000 (see below)
- Revamp survey tool: $10000 (estimated 200 total hours, partially offsetting development cost)
- Survey Incentives: $4000 ($2 per survey respondent, allocated by lottery)
- Travel and accommodation for up to 2 researchers to attend Wikimedia and education conferences, x4 trips x2 researchers = $8500
- Assumptions
- Cost for researcher reflects one course "buyout" - additional time is assumed as pro bono / volunteered by researcher as long as research assistant is funded.
- This assumes a negotiated rate with University between $10500 and $12500 per semester for 20hr/week RA cost / one course buyout.
- Potential Conferences: WikiConference North America, Wikimania, Educause, SXSWEdu, Association of College and Research Libraries, etc.
- We recognize that this budget may not support international trips for two researchers (eg: Wikimania in Cape Town). Potential savings from staff and researcher time can be applied to shore this up, or only one attendee might attend international conferences.
Community engagement
editCommunity input and participation helps make projects successful. How will you let others in your community know about your project? Why are you targeting a specific audience? How will you engage the community you’re aiming to serve during your project?
The Wiki Education Foundation continues to keep the community updated through a variety of outreach exercises, including participation in conferences, and constant blog updates. This research will be disseminated through these and other channels, as this research reaches beyond just Wikipedia and into a variety of other education communities.
Dr. Zachary McDowell is a long time educator with Wikipedia and has presented at numerous academic conferences as well as WikiConference North America (and hopefully at Wikimania this year) on teaching with Wikipedia. It is our plan to engage the Wikipedia Education Collaborative in redesigning this research, as well as engage more Wikipedia researchers to help answer questions about student participation in Wikipedia alongside larger questions of education outcomes.
Get involved
editParticipants
editPlease use this section to tell us more about who is working on this project. For each member of the team, please describe any project-related skills, experience, or other background you have that might help contribute to making this idea a success.
Dr. Zachary J. McDowell
Zach McDowell is an Assistant Professor in the Department of Communication at the University of Illinois, Chicago. In FA16-SP17 he worked with the Wiki Education Foundation as a Research Fellow conducting research on Student Learning Outcomes. Zach earned his PhD from the Department of Communication at the University of Massachusetts Amherst. Teaching with Wikipedia for six years, Zach has been involved with Wikipedia and the Wiki Education Foundation in a variety of roles, including Campus Ambassador and Teaching Fellow. A staunch supporter of Open Access and Open Education, Zach's research focuses on how access transforms cultural production.
Community notification
editPlease paste links below to where relevant communities have been notified of your proposal, and to any other relevant community discussions. You are responsible for notifying relevant communities of your proposal, so that they can help you! Depending on your project, notification may be most appropriate on a Village Pump, talk page, mailing list, etc. Need notification tips?
On Wikimedia Projects
edit- Education mailing list
- Wiki Research mailing list
Social Media
editEndorsements
editDo you think this project should be selected for a Project Grant? Please add your name and rationale for endorsing this project below! (Other constructive feedback is welcome on the discussion page).
Support I think this project is very useful and has high impact potential. The applicant has a lot of experience in research projects and education. The results could improve both knowledge and practices in the field of educational outreach. Seeris (talk) 17:10, 15 March 2017 (UTC)- Support The idea of having quantitative data supporting the positive uses of Wikipedia is exciting and could be used to support any number of other projects. Gamaliel (talk) 18:45, 15 March 2017 (UTC)
- Support This could lead to resources to envision new directions in developing programs. --Joalpe (talk) 21:14, 15 March 2017 (UTC)
- Support Having good data on Wikipedia Based Assignments could motivated more instructors to give them to their students and thus bring more students to writing for Wikipedia. ChristianKl (talk) 08:57, 24 March 2017 (UTC)
- Support I've been happy to participate in this research so far (i.e., feedback on design, one of my classes was debriefed, and some analysis) and would like to see it continue. Beyond appreciating when and how it can be a good assignment, I hope insights will be shared on how to improve the experience for students and instructors. -Reagle (talk) 12:31, 24 March 2017 (UTC)
- Support I have been fortunate to work with Dr. McDowell during the initial pilot study (providing some feedback on design and collaborating on an (in-progress) article manuscript that draws from select data). Throughout this, I have been consistently impressed with his knowledge and expertise in leading a large-scale research project, as well as his collaborative work ethic. Continued research in this area is important to academics because it allows for the discovery of "best practices" for designing effective Wikipedia-based assignments that accomplish specific learning outcomes. But this type of research is also highly significant for making realizations about how Wiki Education initatives can best serve to improve and diversify Wikipedia itself. Matthewvetter (talk) 17:52, 4 April 2017 (UTC)
- Support Zach has extensive experience working in WikiEd initiatives and is an accomplished academic researcher who can persuasively translate between community-focused practice and academic theory. Madcoverboy (talk) 20:04, 4 April 2017 (UTC)
- I used the Wiki-Edu platform for a class, and I'd like to learn more about its efficacy. Shibusawa2 (talk) 21:02, 4 April 2017 (UTC)
- I am confident that this research will benefit Wikipedia, as it will support increased use by educators and students. Arrotramel (talk) 13:25, 5 April 2017 (UTC)
- Support The proposal has a lot of potential and would bring great benefits to Wikipedia. --Samuele2002 (Talk!) 22:32, 5 April 2017 (UTC)
- Support Mssemantics (talk) 23:36, 5 April 2017 (UTC)
- Is it correct to assume that the study is in North America only? Thanks, GerardM (talk) 04:33, 6 April 2017 (UTC)
- Yes, at the moment it is only in North America because that is where Wiki Education currently operates. However, the tools and methods will be released openly so others can replicate this in other geographic areas, and I'm very interested in collaborating in other regions on this topic. Zach (Wiki Ed) (talk) 12:47, 6 April 2017 (UTC)
- Education Programs need evidence that support their expected impact. This research will help making the case for education programs the world over (as well as guiding future research). VMasrour (WMF) (talk) 23:00, 11 April 2017 (UTC)
- Support Richard Nevell (WMUK) (talk) 13:40, 13 April 2017 (UTC)
- Support Reem Al-Kashif (talk)
- Support -- Frhdkazan (talk) 07:10, 2 May 2017 (UTC)
