Fundraising 2011/Engineering/Fundraiser test checklist

Here's a list of things that need to be dealt with/checked up on during fundraising tests. Last year's list can be seen here: Fundraising_2010/Tech_testing.

Before

Check PayPal dev forums etc. to ensure that there are no outtages (planned or unplanned) during testing time): [1]
Ensure that there are no emergencies happening on the cluster
- Check with operations (#wikimedia-operations on IRC is a good place to do this)
Ensure that there are no conflicting software deployments, architecture changes, etc. happening
- See [2]
- Be sure to actively engage anyone who has planned deployments around the time of the test to make sure you/they have the all-clear
Make test donations with using planned landing pages etc (be sure to test all payment methods).
- Check the tracking bits
Make sure you're hanging out in #wikimedia-fundraising on IRC

During

Keep an eye on #wikimedia-fundraising
Watch Ganglia [3]
- Particularly keep an eye on Grosley [4] (and once the payments cluster gets re-added to Ganglia, keep an eye on that!)
  - Note that you can now see live graphs of donations being consumed on Grosley
- Watch for anomalies on the cluster in general
Keep an eye on the 'donations' queue in ActiveMQ [5] - it should steadily increase and then drop to 0 as the queue consumer runs (roughly every 5 minutes)
Keep an eye on Jenkins [6]
- Particularly watch build statuses for the 'Donation Queue Consume' job, and ensure that the job is running every ~5 minutes
- NB: you can click on a job in Jenkins and see the console output. This is handy to do every now and then just to ensure that the queue consumer is running sanely. Normally if there's a problem, the build will fail and you'll get an email, things will explode, etc., but every now and then things can fail silently so I like to periodically check this.
If you have sufficient access, monitor logs.
- On loudon: /var/log/payments_cluster/mw/minfraud (transactions as they're coming in, communication with minfraud)
- On loudon: /var/log/payments_cluster/mw/payflow (MinFraud/PayflowPro response times)

NB: During high load (like during FR tests), you should be seeing log entries from each payment box that is pooled (typically payments1, payments2, payments3, payments4). The payment box is noted in the fourth column of the log file, eg:

Jul 27 00:18:47 payments3 payflowpro_gateway: 7479351855332103 Finished querying Maxmind

After

Make sure banners have come down.
Make sure the donations queue is empty, or in the process of being fully emptied
Give the all-clear to anyone waiting on the FR test to be done before doing a deployment, etc.
Check in with the rest of the fundraising team to see about any complaints/issues that arose

Check Tracking bits

note: you only need to check these for WMF fundraiser/tests. For chapter FR tests, this does not apply.

Overview

We use Google Analytics-style identifiers to track a user's flow through the donation pipeline:

utm_source: this is the most critical and it's made up of three parts: <banner name>.<landing page name>.<payment instrument>
- eg: B_KA2LP_Julytest_US.L11_KA2LP_Julytest_2.pp
- <payment instrument> will be one of:
  - 'cc' (credit card)
  - 'pp' (PayPal)
  - 'rpp' (Recurring PayPal)
utm_medium: this will almost always be 'sitenotice' or null. For fundraising tests, it should always be 'sitenotice'.
utm_campaign: the name of the campaign
- eg C_KA2LP_Julytest_US

How to double check

This year we've added nice new features to CentralNotice which automates the insertion of the various tracking bits into banners/landing pages which makes much of this process a lot easier.

Credit card

Go through the flow (banner -> landing page -> CC form)
Use firebug (or equivalent) to view hidden form fields on the CC form. You should see proper values in: utm_source, utm_medium, utm_campaign, language, referrer, contribution_tracking_id
Make a note of your contribution_tracking_id
Check values in the database (see below)
Submit a test donation

PayPal

Go through the flow and stop at the landing page
Proceed to PayPal donation page
Open up Firebug and click on the net tab (NB: if the net tab is not enabled on the PayPal donation page, enable it, and repeat the process)
The first item in the Net tab should be a request that looks like 'POST webscr'. Expand that item.
Click on the 'Post' tab for this particular request
In the 'Parameters' list, make a note of the number in the 'custom' field - this is the contribution_tracking_id
Check values in the database based on this id (see below)
Submit a test contribution

Checking contribution_tracking

You'll need read access to the Drupal database on db9.

$ mysql -u drupal -p -hdb9.pmtpa.wmnet drupal
mysql>
mysql> select * from contribution_tracking where id='5160532'\G
*************************** 1. row ***************************
             id: 5160532
contribution_id: 1109730
           note: NULL
       referrer: http://en.wikipedia.org/wiki/Main_Page
      anonymous: 1
     utm_source: B_KA2LP_Julytest_US.L11_KA2LP_Julytest_2.pp
     utm_medium: sitenotice
   utm_campaign: C_KA2LP_Julytest_US
         optout: 0
       language: en
             ts: 20110722185153
    owa_session: NULL
        owa_ref: NULL
1 row in set (0.00 sec)

Of course, replace the 'id' value in the above query with the contribution_tracking_id. Note that 'contribution_id' should only have a value in it if a donation has actually been made.