Testing Everything? How to Organize to Prevent Chaos
Published: November 3, 2016
Author: Megan Gritzke
At 3Q Digital – and at most quality digital agencies – you hear the word “test” thrown around a lot, and for good reason. Every hypothesis, every variation, every new targeting option, every new beta, etc., should be tested before anyone invests time and money into building it out.
A philosophy of testing everything isn’t just good; it’s fundamental. But what we don’t talk about enough is how to make sure the results of all those tests aren’t blending into each other and muddying the picture.
Earlier this year, we realized that we were running a lot of tests simultaneously for a client but were lacking formal documentation. A quick huddle with our client contact kicked off a project for us to help keep track of all past/present tests and to ensure that a) the test details were recorded and b) learnings from the test were kept for easy reference.
Testing organization 1.0
We started by creating an Excel doc with nine specific columns:
- Opportunities and challenges
- Outcome – for all tests we include a link to test results file so we can review performance data
- Next steps
- Start/end date
- Date recommendations will be delivered to client
- Items needed from client
Testing organization 2.0
It quickly became evident that this would be a living document that evolved over time. As we expanded into more channels and verticals, we continued to add on, adding a tab for testing priority and subsequent tabs for separate testing initiatives. We’ve also added columns for Test Type and Initial Performance/Impact.
The document is now interactive; the client frequently comments on the document with questions or requests, which helps us keep track of changes. The heaviest lifting was at the beginning because we had to backfill with historical tests completed months ago. But the frequency of testing requires constant, real-time updates.
Our contact has shared this document with the client’s internal marketing team as a way to showcase what is going on within the SEM program. Other concrete benefits include:
- Keeping track of what you have already tested and when (really important for a test-heavy account, especially since we have multiple tests running at once across separate channels and accounts)
- Ability to access test results all in one place (our contact always asks if we have tested something before and how it performed; with this document we can quickly look it up)
- Visibility for people who are not in the account day-to-day to get a good idea of what we are doing; this is hugely important for account transitions.
Ultimately, this organization has made our testing more effective, more clear, more resonant, and easier to communicate. Undertaking the clean-up might sound (and actually be) painful at first, but it’s undoubtedly worth the uplift.