Schema talk:MobileWikiAppSessions

Maintainer:	Mikhail Popov & Dmitry Brant
Team:	Mobile Apps
Project:	Mobile Apps
Status:	active
Sampling:	1:1
Purge:	Auto-purge just eventCapsule PII after 90 days, keep the rest indefinitely

This page holds a JSON schema that specifies a data model for EventLogging.

Logging and client-side validation error rate on Grafana

Server-side validation errors in Logstash

[[Category:Schemas (Mobile Apps team)]]

Sampling, and page load latency

The sampling for this schema is currently 1:1 (the older comments below are out of date, and kept for historical purposes).

Important note about the pageLoadLatency field: To properly calculate the overall latency, we must remove rows where the latency is zero. This is because it is possible to have "sessions" in the app where the user never loads an article (e.g. only browses the feed, or browses saved articles only), in which case the latency will be logged as zero. Therefore these rows must be filtered out. Dmitry Brant (talk) 19:02, 18 November 2022 (UTC)

Sampling on production vs. beta

Just a note for myself and anyone else using this schema that (to my understanding), it is sampling 1% of app IDs for production versions, but is apparently not sampled for beta versions. The majority of events currently come from beta (see below). This means one should use something like "WHERE userAgent LIKE '%-r-%'" to restrict events to production. (On the other hand, it's nice that there are no Googlebot user agents present in the current table - MobileWikiAppSessions_14031591 -, cf. phab:T117631.)

mysql:research@analytics-store.eqiad.wmnet [(none)]> SELECT MID(LEFT(userAgent,INSTR(userAgent," ")-1),INSTR(userAgent,"/")+1) AS app_version, COUNT(*) AS sessions FROM log.MobileWikiAppSessions_14031591 WHERE timestamp BETWEEN '20160509' AND '20160516' GROUP BY app_version ORDER BY sessions DESC;
+---------------------------------+----------+
| app_version                     | sessions |
+---------------------------------+----------+
| 2.1.144-beta-2016-05-09         |   208091 |
| 2.1.144-r-2016-05-09            |   116571 |
| 2.1.143-beta-2016-03-16         |   114359 |
| 2.1.141-r-2016-02-10            |    74273 |
| 2.1.143-r-2016-03-16            |    63345 |
| 2.1.141-beta-2016-02-10         |    50684 |
| 2.1.136-beta-2015-12-09         |     4983 |
| 2.1.139-beta-2016-01-22         |     4846 |
| 2.1.134-beta-2015-11-12         |     4580 |
| 2.1.138-r-2016-01-21            |     3852 |
| 2.1.136-r-2015-12-09            |     3598 |
| 2.1.137-beta-2016-01-11         |     3163 |
| 2.1.135-beta-2015-12-03         |     2567 |
| 2.1.142-beta-2016-03-07         |     2192 |
| 2.1.140-beta-2016-02-04         |     1753 |
| 2.1.141-amazon-2016-02-10       |     1683 |
| 2.1.134-r-2015-11-12            |     1550 |
| 2.1.133-r-2015-10-26            |     1529 |
| 2.1.133-beta-2015-10-26         |     1508 |
| 2.1.132-beta-2015-10-23         |      665 |
| 2.1.137-r-2016-01-11            |      591 |
| 2.1.241-beta-2016-03-03         |      327 |
| 2.1.139-r-2016-01-22            |      191 |
| 2.1.143-fdroid                  |      164 |
| 2.1.138-amazon-2016-01-21       |      131 |
| 2.1.138-beta-2016-01-21         |       52 |
| 2.1.136-amazon-2015-12-09       |       28 |
| 2.1.141-fdroid                  |       28 |
| 2.1.133-fdroid                  |       23 |
| 2.1.136-fdroid                  |       10 |
| 2.1.134-amazon-2015-11-12       |        7 |
| 2.1.133-amazon-2015-10-26       |        4 |
| 2.1.139-fdroid                  |        4 |
| 2.1.140-fdroid                  |        4 |
| 2.1.143-releasesprod-2016-03-16 |        3 |
| 2.1.137-fdroid                  |        3 |
| 2.1.134-fdroid                  |        3 |
| 2.1.141-dtac-2016-02-10         |        2 |
| 2.1.139-beta-2016-01-29         |        2 |
| 2.1.137-releasesprod-2016-01-11 |        1 |
+---------------------------------+----------+

Regards, Tbayer (WMF) (talk) 22:55, 16 May 2016 (UTC)

fromBack not counted in totalPages

Another note about a non-obvious property of the schema which one should be aware of during analysis:

Pageviews resulting from the back button (fromBack) are not included in the total page view count for the session. I.e. totalPages = Search + Random + Langlink + Internal + External + History + Saved + Nearby + Disambig.

Regards, Tbayer (WMF) (talk) 04:50, 9 June 2016 (UTC)

Definition of a session

Currently, when a user stop using the app for more than 30 minutes (with basically any interaction with the app counting as usage), then we consider the current session as over, and send the event once the user interacts with the app again (code).