Objective Revision Evaluation Service/wp10
Article quality in Wikipedia is of critical concern. Many wikis implement a features article process (e.g., en:Wikipedia:Featured article) for identifying high quality content and many WikiProjects use quality assessments to prioritize and direct work. But this assessment work is extremely time intensive and assessments will become out of date. This model makes predictions about article quality to support these processes.
Contexts
editEnglish Wikipedia (enwiki)
edit
See en:Wikipedia:Version_1.0_Editorial_Team/Assessment for discussion of the rating scale. See also work by Warncke-Wang et al.[1]
https://ores.wmflabs.org/v2/scores/enwiki/wp10/?model_info
ScikitLearnClassifier - type: RF - params: random_state=null, oob_score=false, warm_start=false, center=true, balanced_sample_weight=false, bootstrap=true, max_depth=null, criterion="entropy", balanced_sample=true, scale=true, min_weight_fraction_leaf=0.0, verbose=0, min_samples_split=2, min_samples_leaf=8, max_leaf_nodes=null, n_estimators=320, max_features="log2", class_weight=null, n_jobs=1 - version: 0.3.1 - trained: 2016-03-13T14:09:52.020303 Table: ~B ~C ~FA ~GA ~Start ~Stub ----- ---- ---- ----- ----- -------- ------- B 378 263 85 164 123 13 C 178 452 35 126 182 8 FA 54 7 652 154 1 1 GA 43 52 279 603 4 0 Start 57 161 1 14 682 106 Stub 1 7 0 0 123 873 Accuracy: 0.619 ROC-AUC: ------- ----- 'B' 0.829 'C' 0.844 'FA' 0.95 'GA' 0.913 'Start' 0.908 'Stub' 0.984 ------- ----- F1: ----- ----- Start 0.639 GA 0.591 B 0.435 Stub 0.871 FA 0.679 C 0.47 ----- -----
French Wikipedia (frwiki)
edit
See fr:Projet:Évaluation for a discussion the rating scale.
https://ores.wmflabs.org/v2/scores/frwiki/wp10/?model_info
ScikitLearnClassifier - type: RF - params: warm_start=false, max_depth=null, criterion="gini", oob_score=false, random_state=null, class_weight=null, max_features="auto", center=true, n_estimators=501, verbose=0, min_weight_fraction_leaf=0.0, min_samples_split=2, bootstrap=true, n_jobs=1, balanced_sample=true, scale=true, min_samples_leaf=8, balanced_sample_weight=false, max_leaf_nodes=null - version: 0.2.0 - trained: 2016-03-13T14:37:22.965656 Table: ~a ~adq ~b ~ba ~bd ~e --- ---- ------ ---- ----- ----- ---- a 32 84 48 88 19 4 adq 26 217 11 81 0 0 b 23 21 144 43 65 11 ba 30 83 21 171 2 2 bd 2 1 50 3 161 56 e 0 0 11 1 53 226 Accuracy: 0.531 ROC-AUC: ----- ----- 'a' 0.719 'adq' 0.891 'b' 0.838 'ba' 0.848 'bd' 0.905 'e' 0.959 ----- ----- F1: --- ----- e 0.766 a 0.165 b 0.486 adq 0.586 ba 0.491 bd 0.562 --- -----
Russian Wikipedia (ruwiki)
edithttps://ores.wmflabs.org/v2/scores/ruwiki/wp10/?model_info
ScikitLearnClassifier - type: RF - params: n_estimators=501, oob_score=false, scale=true, min_samples_split=2, verbose=0, class_weight=null, random_state=null, criterion="gini", max_depth=null, n_jobs=1, min_weight_fraction_leaf=0.0, balanced_sample=true, min_samples_leaf=8, max_features="auto", max_leaf_nodes=null, balanced_sample_weight=false, center=true, warm_start=false, bootstrap=true - version: 0.0.1 - trained: 2016-06-07T17:34:08.180792 Table: ~I ~II ~III ~IV ~fa ~ga ~sa --- ---- ----- ------ ----- ----- ----- ----- I 36 46 24 3 23 47 39 II 28 75 31 6 8 20 37 III 7 48 117 36 2 0 22 IV 1 7 57 157 1 0 5 fa 6 1 1 1 158 50 4 ga 19 5 7 3 50 143 16 sa 6 12 3 0 0 17 207 Accuracy: 0.561 ROC-AUC: ----- ----- 'I' 0.73 'II' 0.782 'III' 0.868 'IV' 0.956 'fa' 0.939 'ga' 0.888 'sa' 0.956 ----- ----- F1: --- ----- IV 0.724 fa 0.683 II 0.376 I 0.224 ga 0.55 III 0.496 sa 0.72 --- -----
References
edit- ↑ Appendix A in Warncke-Wang, M., Ayukaev, V. R., Hecht, B., and Terveen, L. (2015). The Success and Failure of Quality Improvement Projects in Peer Production Communities (PDF). CSCW., as well as Warncke-Wang, M., Cosley, D., and Riedl, J. (2013). Tell Me More: An Actionable Quality Model for Wikipedia. OpenSym/WikiSym.