Research:Codex/Methodology

Scope

• Anything that may provide insights about how to build, nurture and grow successful online communities who produce and curate free educational resources.

Differences with traditional literature reviews

Vested interest

Contrary to scholars—whom often strive to remain neutral towards the subject of their research—Wikimedia communities have a vested interest in understanding how they function in order to survive, thrive, and work towards the shared vision of the movement. Having an interest in a specific outcome, does not mean abandoning the scientific method; although, it may play a role in prioritizing certain research questions over others—if specific questions have a greater potential to provide insight that is of higher relevance to the Community's particular interest—even if, those questions may be deemed of lesser interest or novel, from a scholarly standpoint.

Acknowledging this vested interest is also critical to identifying and mitigating biases and prejudice. This is particularly important since this document is intended for a broad audience composed, not only of academics, but also of practitioners.

Audience

Academic literature reviews are usually written for the benefit of other researchers, most notably newcomers, to the field. Depending on the subject of research, literature reviews may also be written with practitioners in mind, but often as merely a positive side–effect.

This initiative aims to benefit practitioners first, and researchers, second. Therefore, a particular effort is made to use jargonless language, as much as possible. Some complex topics, obviously require specialized knowledge and vocabulary; however, such concepts are explained in simple terms (if possible) and refer to relevant Wikipedia articles, for further reading. The Codex, also includes a glossary of common terms, used by academics in their respective fields of research.

Inclusion criteria

Authors of traditional literature reviews, define and specify protocols, to determine the extent of their work. For example, systematic reviews may select: [all publications] → that match a [specific keyword] → in a [given citation database] → for a [defined period].

Reviewers, outline inclusion criteria, to make sure their review is comprehensive; but also, to restrict the scope due to practical concerns——they need to be able to review their selection within a reasonable time frame, with limited resources. They also often restrict their selection to peer-reviewed research in keeping with academic standards.

This initiative doesn't have the same restrictions. Here, we're interested in anything that may provide insights about how Wikimedia communities function, and how best to support each of them. This may include scientific articles published in peer-reviewed journals; but also, books, conference proceedings, blog posts, or research reports. The document may mention Wikipedia explicitly, yet it may also focus on a more general view of online communities, social psychology, or technology.

Also, our effort here is not aimed at exhaustiveness. Some research areas related to Wikimedia sites, while worthwhile to scholastic endeavors, lack insights relevant to our particular interests. This notably includes, all the research that merely uses Wikipedia's content as a corpus or data source, in order to answer research questions unrelated to online educational communities.

Living reference

Traditional literature reviews, habitually result in a published artifact; typically, as an article in a peer-reviewed journal. Such publications are valuable—but, as snapshots in time, they are also inherently out-of-date before they're even published. This drawback may be mitigated by considering successive literature reviews, if they exist.

The goal of this Codex is to be a living, up-to-date reference to, the state of knowledge about online educational communities. When recent research provides new insights on a particular topic, updating the Codex will be as simple as editing the relevant wiki page.

Limitations

Timeline

The quantity of research to be potentially reviewed as part of this effort requires an important time investment. As an first step, work is currently (as of early 2016) focusing on topics of higher importance to the Wikimedia Foundation (contributor roles and system dynamics). Many stub pages have also been created on other topics, to make it easier for casual contributors to participate.

Specialized knowledge

The broad scope of this project makes it difficult for its contributors to be familiar with all the disciplines, theories, and methods involved in the multitude of cited publications. Therefore, there is a risk that some research be misunderstood.

The open, collaborative and iterative nature of wikis may alleviate this issue by encouraging other contributors (who might be more familiar with a specific topic) to improve on the original text. Calls for collaborative proofreading may be made on the wiki-research mailing list, to invite other researchers to edit the pages directly, or use talk pages to discuss misinterpretations and approximations.

Guidelines

• A topic page should only include content that has been previously published, whether it's in a peer-reviewed journal, a conference paper or on a wiki page. Exceptions must be argued on the associated talk page.
• A topic page must be a direct subpage of Research:Codex.
• A topic page must include a summary section called "Summary", a section listing research questions called "Research questions", and a section listing open research questions called "Open questions". This allows for easy section transclusion.
• A topic page may include a section called "Current research" listing related ongoing research efforts, and a section called "Sources to add" listing references that have not yet been included in the text.
• A topic page should include a references section called "References".
• Statements on a topic page should be dated as often as possible; e.g. "Based on data from 2004–2009…" or "As of March 2011…".
• Statements on a topic page must not extrapolate from the results they cite. Statements should use credibility indicators to indicate the trustworthiness of the claim; e.g. "It is likely that…", "It is possible that…", "It is argued that …".
• References should use a standard citation template, and should include a link to the WikiPapers page, Acawiki page, or Newsletter review for the resource (using {{Wikipapers}} or {{Acawiki}}), if they exist.
• The primary link of a reference should point to the full text of the source, if publicly available.
• A Codex page must include the {{Research codex}} header template, the {{Research codex navigation}} navigation footer template, and be placed in the Research Codex category.
• Sections and templates should be placed in the order outlined on the page template.
• A topic page is considered a Codex page. Codex meta pages (e.g. Background, Methodology, Glossary, and Overview pages) are not considered "topic pages".
• These guidelines may change over time.