Referees

This is a proposal for a system of refereeing for Wikipedia articles without in any way restricting the current practice or culture. Please feel free to edit it like any other Wikipedia page.

It's intended as an alternative to Wikipedia needs editors, which inspired it. See also The future of Wikipedia#Editorial process which is quite relevant, also The elements of an encyclopedia project and Recruiting editors, but there's not a lot there!

This issue is more generally discussed in w:Wikipedia approval mechanism, which contains a summary of this proposal and also a number of other proposals, some along the same broad lines.

There is much in common with some of these, and much of what is not in common is quite consistent and could be added to this proposal. There are also some conflicts between these various existing proposals, and between them and this one, for example the suggestion of separate namespaces for approved articles which this proposal suggests is unnecessary. See also w:User:Andrewa/Wikipedia approval mechanism for an annotated copy of just the proposals which preceded this one (and which you are also welcome to update).

What this proposal addresses

For Wikipedia to be of high quality the articles should be:

• Accurate.
• Comprehensive.
• Well presented.
• Citable.

This proposal addresses three of these four directly, the exception being the comprehensiveness. It will not directly help to make the list of articles more complete, nor their individual content more complete. It might indirectly help, by attracting more and a greater range of people to contribute. The other three are directly addressed.

This proposal partly or wholly addresses the following overlapping issues, among others:

• Quality control:
  • Edit wars.
  • NPOV issues.
  • Accuracy.
  • Adherance to standards.
• Approval mechanism:
  • Credibility.
• Content filtering:
  • A G-rated Wikipedia.
  • Avoiding any suggestion of censorship.
• Wikipedia 1.0:
  • Paper version.
  • CD Version.
• Specialist encyclopedias.
• Preserving the existing interfaces, policies, and practices.
• Preserving and building on the existing culture:
  • Avoiding building little topic kingdoms.
• Reasonable and achievable in terms of:
  • Software development.
  • Hardware capacity.

The issues partly addressed at present may be fully addressed with a bit more work on my part, or yours. Have a go!

What this proposal does not address

1. It is assumed that there is the need for an approval system. This is not a closed issue by any means, and needs to be addressed somewhere. But it is not addressed here. This proposal rather seeks to answer the question: Assuming we do have an approval system, what should it look like?
2. As noted above, the question of completeness is not directly addressed.

Appearance

Whenever a refereed article from the main article namespace is displayed, a prominent link will be displayed, reading either:

This article has been reviewed.

or

A previous version of this article has been reviewed.

For articles not reviewed, neither will appear, and this proposal makes no changes to current practice.

This link will lead to a page that will give the current reviewed status of the article, explain briefly what the status means, and provide a link to a more general explanation of the system that is the same for all articles. The review status page will come from a new namespace (perhaps called Referees), while the more general article will be in an existing namespace. So, for example, the link from the Widgettest article would go to Referees:Widgettest, and this page would link in turn to perhaps Wikipedia:Referees for a more detailed explanation.

Suppose Widgettest has reached the highest standard of review. The start of Referees:Widgettest would then read something like:

This article has been refereed. This means that it has been reviewed and approved by one or more people with relevant qualifications who played no major role in writing it in the first place.

There would then a list of people who have reviewed the article at any stage in its history, with only the briefest of their qualifications, perhaps academic honorifics and relevant positions only. Each would link to a concise account of the qualifications of the reviewer, and also to the latest version of the article which this particular reviewer had actually reviewed if not the current version. For example:

Current Version:

• Referees (alphabetical):
  • M. Mattressface, MSc, Chairman, International Widget Council
  • Professor Wathishere
• Specialist Reviewers (alphabetical):
  • Douglas Ooglebox, BA
• Reviewers (alphabetical):

Previous Versions:

• Referees (alphabetical):
  • Michel Oftenlost PhD, FRS (03:07, 01 Apr 2001)
• Specialist Reviewers (alphabetical):
• Reviewers (alphabetical):
  • Andrew Alder BA (05:17, 30 Oct 2003).

(End of example)

There are obviously some niceties that could be observed. We could eliminate headings of empty sections, and eliminate the word 'alphabetical' for sections with only one name.

It would also be nice if on browsers which allowed it, putting the cursor over a link to a previous version highlighted the names of all those reviewers who had reviewed this exact version. This highlighting is an efficient way of presenting a lot of information, as everyone who has ever reviewed this article generally appears once and once only on the page, and so does the latest version that they have personally reviewed (the exception is if they have reviewed a previous version at a higher level of authority to the current one which they have also reviewed, in which case they may appear at two or even three levels). People whose browsers can't use this feature would not be prevented from doing anything, it's just convenience.

There is a lot of information here, and a lot omitted. We don't have the entire history of who approved what, nor do we need it (although we will want to provide a complete approval history page which does exactly that for the curious). All the normal viewer wants to decide is what the best version is for their purposes, and exactly how authoritative it is, and then they want to read it.

Behind all this

We need a special set of privileges to make this meaningful. A person with a reviewer privilege, when logged on and viewing an article, would see a button allowing them to mark it to say they'd reviewed it.

Whenever a new version is created... that is whenever an article is updated in any way... all its approvals are set to off. But they are not lost from the previous versions of course. That's the whole point. Until at least one person reviews the new version, it will now display the message saying that a previous version has been refereed.

We will eventually need three categories or levels of reviewers.

The highest category, here called referees, ideally would have PhD degrees in the area that includes the article. Even universities don't always reach this ideal, in Australia it is not unnusual for people with masters level degrees to referee PhD degrees in their particular area of expertise, normally alongside others who have doctorates in more loosely related areas. So, we will probably also need to be a little more lenient than this too. We should be in no hurry to appoint them. It may be rather traumatic if we get into the position of wanting to revoke this privilege!

In some fields, the most authoritative people may have no academic qualifications at all. But if they are the citable authorities, they are the ones we want.

The second category is here called specialist reviewer, and it includes people with relevant qualifications and experience at less than PhD level. Again, let's not rush, but we'll find lots of current Wikipedians suitable for this. At this level it is even more likely for the best people to have no academic qualifications at all. We are interested here in their expertise rather than their authority. However neither should we discriminate against those with academic qualifications. Imperfect as they may be, they do often mean something.

There are in all nine introductory messages that will be needed for pages from the Referee namespace, which should be software generated. The cases to be considered are:

1. This version of the article has been refereed. Say this and explain refereeing.
2. This version of the article has been specialist reviewed, no version has been refereed. Say this and explain specialist review.
3. This version has been specialist reviewed, previous version has been refereed. (Say this and) Explain both refereeing and specialist review.
4. This version has been reviewed, no version has been refereed or specialist reviewed. Explain review.
5. This version has been reviewed, previous version has been specialist reviewed, no version has been refereed. Explain review and specialist review.
6. This version has been reviewed, previous version has been refereed. Explain review and refereeing.
7. This version has not been reviewed at all, previous version has been but no version has been specialist reviewed or refereed. Explain review.
8. This version has not been reviewed at all, previous version has been specialist reviewed but no version has been or refereed. Explain review and specialist review.
9. This version has not been reviewed at all, previous version has been refereed. Explain review and refereeing.

Whew! The reason this looks so complicated is because we want it to be simple to use. Basically, we are cutting out of the Referees page that relates to a particular article the one or two descriptions that aren't relevant to this article in its current state. All the descriptions are available on the general description page for those who are interested.

The links below these descriptions show some details.

The third and lowest category is just reviewer. I suggest we give this to whoever asks for it for the moment, like sysop. But I also suggest we develop some guidelines for the use of the privilege, and even for who should apply. These guidelines (which I haven't even started writing) are the most important part of the whole exercise probably.

Any reviewed article should be NPOV, attractive, accurate, meaty (and perhaps even complete, in the sense of not lacking anything important), grammatical, and probably some other things besides. No reviewer should sign-off on an article which isn't all of these. Of course some reviewers will be better at this than others, at all levels. And, the independence thing only really kicks in at the highest level of referee. The lower levels shouldn't review an article which is all or even mainly their own work, but they can and should contribute a lot more to it than a referee could.

These privileges should be all-or-nothing. This avoids the enormous overhead of trying to identify the fields of expertise of our reviewers at any level, and to restrict them to these areas. That's their responsibility. Others may comment on it, yes, and should if they feel specialists or referees are venturing outside their fields, or even if reviewers are failing to do the job properly.

So, let any sysop set any user to reviewer status, or specialist, or referee. You can't be more than one, but when you review an article if say you have specialist status you can choose to review it just as a reviewer, and should if it's outside your area of expertise. Likewise a referee could choose to downgrade their vote of confidence to either of the lower levels if appropriate.

This could be done in at least two ways. One is that a person with referee privilege, for example, could see three extra buttons when looking at a page, allowing them to mark it as reviewed by them at any one of the three levels. There should also be an easy way of any person withdrawing, upgrading or downgrading their endorsement from any version, at any time. So, if they'd already reviewed this version of the page, one of the buttons would then be to revoke their endorsement from this version.

Alternatively, there could be just one link for all reviewers, say Mark this article as reviewed. This would lead to a page with appropriate options, and has the advantage of reminding reviewers of their responsibilities. This page could read:

Reviewing (name of article) (version of article)

• I wish to approve this version of this article as follows:
  • I have read the article, and it has accurate, adequate and appropriate content, and complies with any other Wikipedia standards and guidelines for this type of article that I think are relevant.
  • In addition to the above, I have expertise in the area covered by this article.
  • In addition to the above, my expertise is supported by relevant qualifications, and I have taken no major role in writing this version of the article.
• I now have doubts about this version, and wish to remove my previous approval.
• I see problems with this version, and wish to disagree with any approvals granted to it by others.

(end of example)

This example actually shows five exclusive options, as the second-level of the top option is only available when the top option is selected. In practice not all of these options would be shown, only the relevant ones, but in the order shown. They would be 'radio buttons', and the default would be the top button that does display, ie in the example it would be to approve at the reviewer level. The order has been chosen to make this work.

The maximum number of buttons that might show is four, the minimum is two, depending on the privileges of the user. Which buttons show is dependent on previous approvals, and the wording of the buttons would need to be changed a little depending on which buttons were displayed.

There should be some means of flagging dissent. It will complicate the infrastructure a little. An extra button, available to any class of reviewer, should allow them to list themselves in a third section of the Referees page of an article as disputing part of it. Note that this is a section of a page would only ever be displayed if someone else has reviewed the article favourably, so this doesn't replace the current NPOV and other warnings at all. It is in addition to them. And, it complicates things in ways not at first obvious, I'll go into that more if we seem about to do it.

There is no need for chatter or reasons. Just a yay or nay. Discussion belongs on the existing talk pages.

This is not all that is needed for QA or release control. There are several other features and facilities we need to do a printed or static release, or anything else for which we need to baseline a version. The facilities described here would be a good basis for such a release control heirarchy, there's perhaps about half the functionality needed in fact.

Triggering reviews

Reviews would be triggered in lots of ways. One of them would be a set of three pages similar to the existing Peer review page, one for each level. These could be combined into one page, perhaps just called 'Review', there are advantages either way. 'Review' for basic level, 'Specialist review' and 'Refereeing' for the higher levels is the proposal at present. It could logically be called 'Request for review' but keeping it short should reduce the use of pipes or redirects.

Baselining

See Referees/1#Baselining.

Please add any updates to this and following sections to the subpage, reflecting just the heading structure below.

Wikipedia 1.0

See Referees/1#Wikipedia 1.0.

Other uses of the baselining software

See Referees/1#Other uses of the baselining software.

A G-rated Wikipedia

See Referees/1#A G-rated Wikipedia.

Specialist encyclopedias

See Referees/1#Specialist encyclopedias.

Collegiate approval

See Referees/1#Collegiate approval.

Edit wars

See Referees/1#Edit wars.

Comments welcome

See Referees/1#Comments welcome.

Quality standards?

See Referees/1#Quality standards?.

Separate Namespace

See Referees/1#Separate Namespace.

Comment by 128.175.112.225

See Referees/1#Comment by 128.175.112.225.

Comments not on the subpage

This section is for people who want their comments to appear on this page, not the subpage. Please don't merge my comments back here, I want to keep this page down to a size where everyone can edit it. But feel free to add to the bottom of either page, or edit any section in either. Andrewa 15:26, 12 Nov 2003 (UTC)