Research talk:Autoconfirmed article creation trial/Work log/2017-12-11

Monday, December 11, 2017

Today I'll work on combining our knowledge from last week about how quickly newly registered accounts create articles/drafts with our data on contributor survival to answer the question of how survival has developed before/during ACTRIAL. I'll then use the same datasets to find measures of content quality in order to understand how that has developed.

Account age during ACTRIAL

In the previous work log, we looked at historic data on account age when creating an article or draft for accounts younger than 30 days, and created these two histograms showing the age distribution for Main (ns=0) and Draft (ns=118) namespaces:

To what extent has this changed during ACTRIAL? We used the page creation tables in the log database (these tables also provide data to our Page Creation Dashboard) to grab the same data we used previously: all non-redirect page creations in the Main and Draft namespaces that were made between 2017-07-21 and 2017-10-31 (inclusive) by users who did not have the autopatrol-right, and where the account was less than 30 days old. Once we had the dataset, we labelled all creations after midnight on 2017-09-15 as "during ACTRIAL". From there we can then create a set of four histograms, split in rows by namespace and in columns by time period to show the development prior to and during ACTRIAL:

There are several things to note about these histograms. First of all, we can clearly see the effect of ACTRIAL on account age for article creations: the four day delay to reach autoconfirmed status to be able to create articles is evident. Secondly, we can see a shift of creations from the Main namespace to the Draft namespace during the trial as the number of creations in the Draft namespace drastically increases. Note that the time period in each column is not too dissimilar, there are 56 days from the start of the dataset to ACTRIAL starts, and there are 47 days from the start of ACTRIAL to the end of the dataset. Lastly, we can see that the distribution of account age does not appear to change significantly. The distributions of account age pre-ACTRIAL are similar to the distribution of account age in the Draft namespace during ACTRIAL.

We also inspected whether accounts tend to create a single article/draft in this dataset as well, and found similar results to what we saw previously. These results indicate that measuring survival by looking at creations during the first week is reasonable, both historically as well as during ACTRIAL.