Grants talk:Project/Rapid/GeneralNotability/Bullseye

Comments from I JethroBT (WMF)

Hello GeneralNotability and thanks for this proposal to support CheckUser, Stewards, and other relevant users access to the Bullseye tool. I appreciate your interest in sharing and making accessible tools and services that can help support the work individuals in these roles. Here are my comments and questions:

• Are there any usage statistics available for Bullseye from these types of users (on en.wiki or elsewhere) up to now?
• Relatedly, have you posted notifications about this proposal in some of places you mentioned? This community engagement is something we ask all applicants do, before a funding decision is made. This is especially important around funding requests supporting the development or access to tools, because we want to be sure that there is adequate interest in using the tool to support project activities or needs. I am also grateful to see a strong endorsement from L235 on the proposal.
• Also, due to the API query limits, some features are restricted to administrators or to checkusers/stewards; by purchasing keys I can open these features to more editors. What features will be opened up to editors? Are there any privacy risks associated with this kind of access to the tool from non-admins/checkusers/stewards? It sounds like not, but wanted to confirm the kinds of features available.
• Based on your knowledge of interest in the tool right now, I'd also like you to provide an estimate of unique participants on the platform (i.e. admins, checkusers, stewards). This can be a minimum or a range, even limited to just en.wiki and/or Meta). It is OK if this is very rough or based on incomplete information (these estimates often are), but I am wanting to understand how many people are likely to benefit from using Bullseye to support their work.
• In addition to general usage statistics, I also recommend adding metrics or evaluation around the proportion of users not only on accessing Bullseye, but use it to support a subsequent administrative action (e.g. blocks, page protection), issuing a warning, or similar tasks. If exact metrics are not possible/practical to gather, you can generally ask users something like "About how often was Bullseye needed to support an administrative action you took?" (Never / Rarely / Sometimes / Often / Always). Would you be willing to follow-up with users to understand how the tool is supporting their work more directly?

Thanks for the proposal, and please let me know if I can clarify anything around my feedback. I JethroBT (WMF) (talk) 15:46, 15 October 2021 (UTC)

I JethroBT (WMF), sure:

• I've intentionally been careful in how I do usage statistics (I don't want to tie IP lookups to specific users, since that would have privacy implications if those were correlated with checkuser actions). I've thrown together a rough check by searching the webserver logs; it looks like the low end is 20 IP lookups/day, high end is a bit over 100. I'd like to get those numbers up, of course, and I do think I'll put some kind of aggregate per-user statistics in place ("lookups per month" for example). My experience as a newly-minted checkuser is that I've opened several tabs of bullseye every time I perform an investigation.
• I have not posted yet, since I was not sure of the norms around announcing these (whether it might be considered canvassing for endorsements, for example). I will post notifications later today.
• Right now, Shodan data is restricted to administrators, global sysops, checkusers, and stewards, due to the low number of queries I have available for that service. I would open that data up to everybody. Spur is restricted to checkusers and stewards for similar reasons, I would open it up to administrators and global sysops as well at first, and am considering opening it to all users. I do not believe there are any privacy implications with letting the average user have access to this tool; the only information gathered from Wikipedia is "is this IP blocked," and I believe all data sources can be accessed for free by users. This just consolidates those data sources into one useful overview.
• There are currently 97 users registered with the tool. The majority of those have access to the checkuser tool in some way (ombuds, stewards, checkusers) - not surprising given that it was advertised on the checkuser mailing list. I haven't checked every user individually, but based on name recognition, at least 2/3 of the current users fall into that group, and most of the rest are administrators.
• I think the survey would be the best approach here, and I will definitely be willing to do this - I'll plan to do this following approval (perhaps one survey every three months).

GeneralNotability (talk) 16:12, 15 October 2021 (UTC)

I have announced the grant at the following places: stewards' noticeboard, enwiki Village Pump (technical), enwiki sockpuppet investigation clerks' noticeboard (frequented by checkusers as well), enwiki WikiProject on Open Proxies. GeneralNotability (talk) 17:09, 15 October 2021 (UTC)

@GeneralNotability: Thanks for these clear responses. I understand the uncertainty around norms in announcing. For reference, we encourage applicants to share information about their proposal with relevant communities broadly. This approach differs to some community policies (like canvassing) about certain community discussions. There are a few reasons for this-- the grant review process does not operate like a vote or a consensus-driven process, community feedback isn't necessarily required to support funding, but most important, we want communities to have some awareness and opportunity to support or provide feedback on proposals we are reviewing. I JethroBT (WMF) (talk) 21:34, 18 October 2021 (UTC)