Research:Article Creation Workflow/Data and metrics

This page describes the data and metrics plan for the Article Creation Workflow (ACW) feature.

Test phases

During phase 1 we will focus on the usage of the landing page to understand how many among those users who are directed to the landing page enter a specific funnel or successfully complete them. We will also measure the effectiveness of of ACW at getting anonymous users to register an account.
During phase 2 we will study the quality and survival of pages created via different ACW funnels.

Bucketing

We will display the new experimental ACW landing page to a small sample (1%) of all users trying to create a page by following a redlink (whether coming from (1) a search result page, (2) another article containing a redlink or (3) the interstitial displayed when typing a URL with a non-existent page name). We will redirect the remaining 99% to the default edit screen.

Data

All data for test phase 1 are obtained via the clicktracking extension. The full specifications of the data collected are here.

Metrics

Entry points

Impressions of the ACW landing page as a function of the redlink referral:

search
browse
url

Clickthrough rates

Volume of clicks per impression of the ACW landing page:

log in (anonymous only)
sign up (anonymous only)
article wizard
create draft (registered only)
create directly (registered only)

Conversion rates

Ratio of successfully completed edits per impression of the edit screen via the following funnels:

create draft
create directly

Notes

We will use as a baseline the conversions of users directed to the default edit screen.
The order of the elements on the landing page will be randomized (with the exception of the exit button which will be always at the bottom).