The 2020 survey collected feedback from Toolforge project members and Cloud VPS project administrators on how the services offered can be improved to help their development and maintenance needs.
The participants of the survey are the members of the Wikimedia Toolforge project and Cloud VPS project administrators. 1825 participants were contacted via email. The emails were harvested from the Cloud Services' LDAP system ("developer accounts") and the Wikitech database. The final list was created by finding the union of the members of the Toolforge project and all users with "administrator" rights on one or more Cloud VPS projects. Users who have opted-out of email contact by other users in their Wikitech preferences or the survey specific opt-out page were excluded. Specific opt-outs for the Annual Survey can be made by users at wikitech:Annual Toolforge Survey/Opt out .
137 email recipients participated in the survey which represents 7.5% of those who were contacted.
The 2020 survey was active between 2021-01-14 and 2021-02-05 .
Questions and answers
edit
Answer
%
0-1
25%
2
23%
3-6
34%
7+
17%
edit
Answer
%
0-1
37%
2
25%
3-5
25%
6+
13%
edit
Answer
%
0
17%
1
36%
2-3
24%
4-8
13%
9+
11%
Additional demographics
edit
Answer
%
Web apps (tools that are accessed through a web browser)
77%
Bots (including pywikibot)
63%
APIs for use by other tools
32%
On-wiki gadgets
24%
Microsites (small static websites)
18%
Dashboards and other data visualizations
14%
Extensions
2%
Data analysis
1%
Other
2%
Answer
%
Wikipedia
64%
Wikidata
36%
Wikimedia Commons
29%
All Wikimedia projects
26%
Wikisource
9%
Wiktionary
7%
Wikinews
6%
Wikiquote
6%
Wikibooks
4%
Wikiversity
4%
Wikivoyage
3%
Phabricator
3%
MediaWiki
2%
Wikispecies
2%
Gerrit
1%
Logstash
1%
Meta
1%
Toolforge
1%
Wikitech
1%
Answer
%
Access to Wikimedia-specific resources; including wiki replicas
74%
Cost
60%
Ease of collaborating with other Wikimedians
54%
Philosophical or ideological reasons
47%
Ease of use
34%
Privacy and security considerations
32%
Wikimedia staff
2%
Help with maintenance
1%
Reliability
1%
Answer
%
Mysql/Mariadb (Not Including Toolsdb Or The Wiki Replicas)
63%
Redis
24%
Sqlite
18%
Filesystem
9%
Mongodb
6%
None
5%
Memcached
4%
Postgresql
2%
Toolsdb
2%
Elasticsearch
1%
Kafka
1%
Wiki Pages
1%
Answer
%
1
20%
2-3
23%
4-5
24%
6+
33%
Answer
%
Python 3
67%
Php
44%
Nodejs
16%
Python 2
15%
Java
8%
Perl
6%
Bash
5%
Ruby
3%
Rust
3%
Javascript
3%
Other
8%
Answer
%
Almost all of the work
60%
More than half of the work
14%
About half of the work
7%
Less than half of the work
15%
I don’t know
4%
Answer
%
Git
86%
I Do Not Use Source Control
13%
Mercurial
1%
edit
edit
Answer
%
1
22%
2-3
24%
4-5
29%
6+
24%
edit
Answer
%
Hosting one or more tools or other public services such as a web app; bot; dashboard; API; etc.
72%
Testing and experimenting with software
44%
Running a backing service (database; cache; etc.) for a tool or other public service
37%
Running one or more MediaWiki instances
35%
Conducting data analysis or other large computational tasks
9%
edit
Answer
%
I do not know
14%
No
63%
Yes
23%
edit
Answer
%
No
39%
Yes; but I usually don't read messages
16%
Yes; and I usually read messages
45%
edit
Answer
%
No
57%
Yes; but I usually don't read messages
13%
Yes; and I usually read messages
30%
edit
Answer
%
Phabricator
73%
IRC
66%
Mailing list/Other email
30%
Telegram
2%
Other
3%
edit
Answer
%
IRC
63%
Phabricator
59%
On-wiki talk pages
39%
Mailing list/Other email
34%
Facebook
7%
Telegram
7%
Discord
6%
Slack
6%
Matrix
2%
Real life meet-ups
2%
Zulip
2%
Other
3%
edit
Not a lot of people answered this question, so percentages are avoided here.
StackOverflow
In code repository
Debian wiki
Digital Ocean's tutorials
Kubernetes website
Meta wiki
Sourceforge Not a lot of people answered this question, so percentages are avoided here.
Local user groups and events
Telegram
Telegram #wmhack
Twitter
Wikimedia Community Discord Qualitative questions
edit
A series of qualitative questions were asked in order to gauge general satisfaction with Toolforge and Cloud VPS.
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
82%
100%
83%
65%
92%
83%
92%
78%
95%
83%
89%
82%
93%
64%
90%
96%
80%
83%
80%
82%
94%
86%
59%
Disagree/Strongly Disagree
5%
0%
13%
8%
3%
4%
4%
5%
5%
6%
7%
0%
7%
14%
3%
4%
0%
8%
7%
0%
3%
5%
6%
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
63%
76%
68%
50%
69%
52%
69%
65%
80%
49%
79%
71%
73%
47%
69%
77%
47%
75%
59%
55%
81%
66%
41%
Disagree/Strongly Disagree
10%
0%
20%
12%
8%
16%
12%
8%
0%
16%
7%
7%
0%
27%
5%
12%
0%
8%
9%
36%
0%
9%
18%
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
64%
67%
61%
64%
76%
30%
60%
72%
100%
39%
73%
77%
93%
54%
57%
80%
73%
58%
60%
64%
84%
65%
59%
Disagree/Strongly Disagree
2%
5%
4%
0%
0%
9%
0%
0%
0%
6%
0%
0%
0%
8%
0%
0%
7%
0%
3%
0%
0%
2%
0%
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
53%
48%
57%
63%
57%
39%
68%
57%
72%
36%
81%
56%
79%
43%
47%
72%
67%
75%
49%
27%
77%
59%
18%
Disagree/Strongly Disagree
7%
10%
4%
8%
3%
13%
4%
3%
6%
11%
4%
4%
0%
14%
8%
4%
0%
0%
10%
0%
0%
6%
12%
Cloud Services documentation is easy-to-find.
edit
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
59%
48%
64%
65%
67%
48%
65%
62%
80%
49%
71%
68%
73%
50%
69%
65%
53%
67%
58%
50%
66%
63%
29%
Disagree/Strongly Disagree
16%
5%
28%
15%
8%
41%
12%
5%
5%
28%
14%
0%
13%
31%
13%
8%
13%
17%
19%
20%
3%
15%
18%
Cloud Services documentation is comprehensive.
edit
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
58%
48%
56%
62%
66%
44%
60%
62%
80%
44%
67%
71%
73%
50%
66%
58%
47%
83%
55%
70%
63%
61%
38%
Disagree/Strongly Disagree
13%
10%
16%
15%
9%
26%
4%
14%
0%
21%
7%
7%
7%
25%
8%
12%
7%
8%
14%
10%
9%
12%
19%
Cloud Services documentation is clear.
edit
Answer
All
1 Year
2-3 Years
4-5 Years
6+ Years
0-1 Tools
2 Tool
3-6 Tools
7+ Tools
0-1 Maintain
2 Maintain
3-5 Maintain
6+ Maintain
0 Hours
1 Hour
2-3 Hours
4-8 Hours
10+ Hours
Toolforge
Cloud VPS
Toolforge+Cloud VPS
Tools
No tools
Agree/Strongly Agree
48%
38%
56%
40%
61%
23%
62%
54%
70%
28%
59%
68%
67%
44%
47%
69%
40%
58%
47%
50%
56%
52%
25%
Disagree/Strongly Disagree
17%
24%
12%
20%
11%
31%
12%
14%
5%
23%
11%
11%
13%
25%
13%
15%
13%
8%
20%
0%
13%
15%
31%
The survey included several free form response sections. Survey participants were told that we would only publicly share their responses or survey results in aggregate or anonymized form.
Responses to this question were varied, here is a summary of the comments clustered in different areas:
Documentation :
Improving documentation for beginners
Updating and removing outdated documentation
Better documentation for batch jobs / cron workflows Support :
More responsive help desk
Help to beginners, people getting started
Centralized support site/forum Workflow :
Deleting tools
Automated deployments
Development / Production environment parity for developing Platform :
Replicas
Local access
Cross DB joins
Reliability
Stability (breaking changes)
Performance
Kubernetes
Usage for jobs is hard
Memory quotas
Tooling
NFS performance
Language / Version support
Node.js versions
Ruby on Rails support
Rust support
Docker support
Logging
Logs and emails from cron jobs
Monitoring
Health checks and alerts
Show general service availability
edit
Thank you for those who answered positive comments and praise, it is very much appreciated.
There are less responses in this question, so the summary will be less detailed.
Improve documentation
Better deployment workflows
Docker support
Managed Kubernetes clusters
Backups
Monitoring services
edit
Hard to understand
Beginner documentation is very hard to understand
Needs more graphics
Cloud VPS docs are harder to understand than Toolforge's
Lacking
What should I use? What is supported?
Reference lists for features, instead of all tutorials
More documentation about effective workflows
Out of date
Structure
Self-study is very hard, complicated to navigate the docs (guided-study is not so bad)
i18n There were many praise responses, thanks a lot, they are very much appreciated.
Besides those, there were some comments about language/framework support, using propietary software for this surveys instead of OSS, and a variety of comments about topics mentioned in the other freeform questions.
The responses here were very varied, the ones mentioned several times were:
Improve language support and use newer versions of languages and applications
Memory intensive work on Toolforge. People end up moving to Cloud VPS for these
Remote access issues (SSH, SFTP, etc.)
Analysis work and very CPU/IO intensive programs
Problems with blocked IPs The rest were single comments and in interest of anonymity won't be detailed here.
Thanks to everyone that took the time and participated in the 2020 survey. Your input is extremely valuable and very much appreciated.
See you soon in the 2021 survey!
.svg)
.svg)
.svg)
.svg)
.svg)
