How to create a thorough innovation and experimentation process

Joseph Kerschbaum
SVP, Search & Growth Labs

Length

5 min read

Date

14 February 2022

In the first installment of this series, we discussed creating a culture of innovation, which must be supported by a structured process. This process should provide a framework for each phase of experimentation: planning and launching tests, monitoring progress, analyzing data, and broadcasting results throughout the company. Each phase of the process is critical to driving innovation at scale within your organization.

A thorough process is necessary for a successful experimentation program, but it’s important for that process to be supported by thoughtful documentation. Haphazardly launching new experiments within media channels will lead to lackluster results and it will be difficult to scale learnings across your organization.

Create an experiment roadmap and supporting documentation

In terms of innovation, you need to know where you’re going. You may not know the exact destination, since experimentation can take you down many paths – but you need to have a general sense of direction and a compass to help you navigate. The best way to not lose sight of your goals is to create a roadmap. This roadmap can lay out which experiments you want to run and the correct progression of those tests.

A roadmap will also help determine the priority for your experiments. As we mentioned in our previous article, consistent communication and a clear vision are critical for driving innovation. You want team members to understand where the experimentation program is going and how they can contribute to its success. A roadmap provides a high-level overview of your program, and specific experiment briefs can provide details that will frame each test.

The scientific method for experimentation requires an observation, a hypothesis, an experiment, and a conclusion. You should frame each new test with a thorough experiment brief. This document should:

Describe what you want to test
Detail why the experiment is important
Identify who should be involved
Address how outcomes will be measured

Determine your measurement process and analyze data

Your first data point on the road to launching an experiment is gathering feedback from other teams. To foster a culture of belonging, everyone who is involved in an experiment should feel like they are part of the cultivation process.

Next, you need to determine the data required to accurately measure the impact of each experiment. In paid media, the metrics usually include sales, revenue, and site traffic. However, you can run experiments anywhere in the sales cycle and the objective of an experiment could be more complex attribution. Within your experiment brief, detail how you will measure performance and what success looks like.

We all want experiments to run as quickly as possible so that we can get a glimpse of our results. It’s kind of a rush to see how things turn out! However, you can’t rush experiments that need to run until enough data is collected to determine a significant result. The expected timeframe for the experiment should also be included in your brief!

Having a repeatable process will allow you to more easily scale your experimentation roadmap, accelerating innovation.

Determine test results and broadcast outcomes

Of course, every experiment must come to a close. Some tests will provide conclusive results (positive or negative) and others will be unclear. For any experiment that is undecided because the data did not reach statistical significance, then you should review the testing parameters and try again. Each successful experiment should open the door for another. This could mean running the same experiment within a different account or diving deeper into a specific tactic. Your experiment roadmap should be a living document and always evolving!

As we mentioned previously, communication is critical when driving innovation within an organization and you should regularly broadcast results. This way, teams throughout the entire company can learn from completed experiments, review results, and consider these tactics for their media account, and results on a regular cadence help inspire innovative dialogue.

What’s next?

Innovation is a result of enthusiastic culture and repeatable processes, and this is only the beginning. Once you have innovative ideas and you’ve proven their effectiveness through testing, then you have to broadcast them across your organization. As the scope of innovation widens to a broader audience, that’s when its impact can weaken. As your new idea reaches out across your company, the more it can be diluted and lose its original power for change. This is when your teams need to focus their efforts to drive sustainable innovation within their organizations. In our next article, we’ll discuss how to do just that.