Wikimedia CH/Project/Argo Wikimetrics

This page is about Argo Wikimetrics, supported and hosted by Wikimedia CH. It contains project technical documentation.

This is an attempt to write useful application and sysadmin documentation.

General info

Blog article:

https://synapta.it/blog/fare-community-estraendo-dati-wikidata/

Project homepage:

https://argo.wikimedia.swiss/

Initial support page:

Grants:IdeaLab/Wikimetrics for local users

Maintainer: Wikimedia CH/Tech boost.

Started by: Synapta in 2017 (not active anymore). [1]

In use: yes (2022).

Purpose

Argo Wikimetrics can be used to find users with a specific geographical focus, often related their full-time or temporary (familiar, free time or professional) residence.

This will improve the network of users related to a specific geographical area and create a much close-knit community.

https://meta.wikimedia.org/wiki/Grants:IdeaLab/Wikimetrics_for_local_users.

The impact is a more effective community networking, both for onwiki or offwiki activities. Users with a very strong geographical background can be contacted to help with content creation (missing images, sources in local library) and to boost the territorial presence of Wikimedia initiatives, sometimes allowing lower costs and bigger efficiency. You can get an insight of a specific place (better facilities or contacts in the local administration) in a much direct way if you can find someone on wiki who knows it already.

Source code

The source code is now hosted on Wikimedia GitLab (previously on Synapta's GitHub):

gitlab:repos/wikimedia-ch/argo-wikimetrics

Server overview

WMCH LAN IP	192.168.128.18
Application	127.0.0.1:8082 (NodeJS)
Database	127.0.0.1:8082 (Blazegraph)
Frontend webserver	0.0.0.0:80 :443 (nginx)

Application service

The application is runs using a Python supervisor. Configuration file:

/etc/supervisor/conf.d/argo.conf

Python supervisor systemd service:

systemctl status supervisor

Application logs

Check here:

/var/log/supervisor/argo-scheduler-stderr-*.log
/var/log/supervisor/argo-scheduler-stdout-*.log
/var/log/supervisor/argo-server-stderr-*.log
/var/log/supervisor/argo-server-stdout-*.log
/var/log/supervisor/argo-blazegraph-stderr-*.log
/var/log/supervisor/argo-blazegraph-stdout-*.log

Nginx frontend proxy

All WAN requests pass through nginx. Configuration:

/etc/nginx/sites-enabled/argo.wikimedia.swiss.conf

Languages

To work, Argo Wikimetrics needs direct access to wikitech:Wiki Replicas.

Since 2022, this is the current language configuration:

Language	Database name	Local SSH tunnel
Italian	itwiki	13306
French	frwiki	13307
German	dewiki	13308
English	enwiki	13309
Romansh	rmwiki	13310
Allemannic	alswiki	13311

The second column it's useful since each language is connected to a Wikimedia Replica database with the same name (for example enwiki_p). Moreover, each database can be accessed only from an hostname like enwiki.analytics.db.svc.wikimedia.cloud. Note that these hosts are not available to the WAN. So, we need to keep some SSH tunnels to that LAN through login.toolforge.org. See wikitech:Help:Toolforge/Database for details. See phab:T318191 for details about this over-complication.

The third column it's the local TCP port to which it's possible to start MySQL connections going to that specific language.analytics.db.svc.wikimedia.cloud MySQL server.

You can discuss in phab:T313737 if you think that things can be simplified.

Database network details

It seems the application does direct MySQL queries to Wikimedia Toolforge using the Wikitech development environment.

Here the systemd service:

systemctl status wmflabs-autossh

To do so, the user ilario is used to login via SSH in dev.toolforge.org.

Then it tries to keep a local port opened to:

itwiki.analytics.db.svc.eqiad.wmflabs:3306

That is the cluster dedicated to slow MariaDB queries that can run for long time.

This connection be configured from this file:

/home/adminubu02/.ssh/config

Example:

Host wmflabs
    HostName      dev.toolforge.org
    User          <omissis>
    IdentityFile  <omissis>
    LocalForward  13306  itwiki.analytics.db.svc.wikimedia.cloud:3306
    LocalForward  13307  frwiki.analytics.db.svc.wikimedia.cloud:3306
    LocalForward  13308  dewiki.analytics.db.svc.wikimedia.cloud:3306
    LocalForward  13309  enwiki.analytics.db.svc.wikimedia.cloud:3306
    LocalForward  13310  rmwiki.analytics.db.svc.wikimedia.cloud:3306
    LocalForward  13311 alswiki.analytics.db.svc.wikimedia.cloud:3306

wikitech:Help:Toolforge/Database

Database

It has a local PostgreSQL database called argo with 5 tables. Management:

$ sudo -u postgres psql
> postgres=# \c argo
> postgres=# \dt

         List of relations
 Schema |   Name   | Type  | Owner
--------+----------+-------+-------
 public | articles | table | argo
 public | counters | table | argo
 public | edits    | table | argo
 public | jobs     | table | argo
 public | users    | table | argo
(5 rows)

Contact / Report a bug

Software bugs:

Create Task in #WMCH-Argo (kind subscribers: Ilario V., Valerio Bozz.)

(phab:tag/wmch-argo)

This tool is feature-complete but feel free to contact Wikimedia CH to consider developments or fixes.

Thank you!