How to Formulate a New Experiment Idea

Keen Funnel Official Blog

Keen Funnel Official Blog

PR and Communication Department

Learn the template we use at GrowthHackers

 

An experiment can be understood as actions taken in the hopes of validating a hypothesis. In a business context, it is a practical test that by taking action A, it will help reach goal B. Companies that are learning-driven are deploying experiments to solve business problems across every stage in their customer journey.

“[…] a company’s ability to create and refine its products, customer experiences, processes, and business models — in other words, to compete — is deeply affected by its ability to experiment. In fact, no innovation can exist without first being an idea that is subsequently shaped through experimentation. Today, an innovation project can involve hundreds or thousands of experiments, all with the same objective: to learn through rounds of disciplined testing, whether a business idea holds promise for addressing a customer need or problem.” Experimentation Works book, by Stefan Thomke

While members of growth teams are more likely to be on the same page on how to formulate and propose new ideas when you democratize ideation and break silos, idea submissions from other departments will pop up in all sorts of formats. Navigating through a non-standardized idea backlog might make prioritizing and operationalizing experiments challenging.

This article will help with an essential part of experimentation that allows your team to build an idea backlog full of insights and hypotheses ready to be prioritized and tested according to your business goals. To do so, I will dissect the template for a new experiment idea we use here at GrowthHackers.

“Ideas are the rocket fuel of growth. And you need a pipeline of them delivering a steady flow. Ideas should be submitted to an idea pipeline, following a templated format by which they should be submitted. It’s important to standardize the format so that ideas can be quickly evaluated, without the team needing to ask lots of questions.” Sean Ellis and Morgan Brow, Hacking Growth book

To do so, I will be using a fictitious meal delivery company to illustrate concepts. In this case study, our meal delivery company has checked their app usage data and separated their users into cohorts based on app usage, profile and purchase history and learned that the cohort with the most engaged and highest spending users take a few actions in common after they install the app, and have a similar profile.

They have defined their north start metric as the weekly number of orders. To support the north star metric, they have set a few objectives. To reach one of them, a member of the growth team had an idea.

North Star Metric: Number Of Orders

Objectives:

– Get existing customers to order more

– Increase the number of orders from new customers

1. Idea Name

1*jWEk77A7KA8sKzy0b2Q7Cg
Idea Name

Ok, so the first element of the new idea template is what the name suggests. The idea should be named as clearly and concisely as possible. We find that keeping an idea name brief makes discussing and locating an idea in the future much easier. For the purpose of this exercise, we will call this idea “Favourite Restaurants Notifications”.

2. Idea Description

1*pUtJm 5CscirwBoO3EHobA
New Experiment Idea Description

Next, it is time to describe your idea in detail, with the what, the whos, the whens. It should contain how this experiment will be run, what is the sample for the experiment, what will be created, what resources are needed, and under what merits the idea should be taken forward. It is the rationale behind this idea: what proprietary data, benchmark, or customer interview justifies this test? What kind of test will be run?

“In determining whether an experiment is needed, managers must first figure out exactly what they want to learn and measure. Only then can they decide if testing is the best approach to achieve their answer and, if so, what the scope of the experiment should be.” Experimentation Works book, by Stefan Thomke

For our meal delivery app, the description should look like this: “We have identified that the user cohort who orders more often do so from the same restaurants they have ordered from previously, therefore, we should test notifying users when there is a promotion in a restaurant where they have placed a previous order and rated 4 stars or above. This test will be done over a period of 2 weeks with a control group from cohorts A and B and analyzed against the results from the control over the same period.”

3. Hypothesis

1*gE
New Idea Hypothesis

The experiment hypothesis is the expected cause and effect resulting from your experiment, what you are hoping to achieve if your test is successful. Quantifying the expected gain can help teams to have a clearer idea of success.

Calculating an expected lift can be done by using results from previous experiments, external benchmarks, or by an estimate based on the scale of your test. Including a quantitative prediction is preferable, but not mandatory, especially for teams who do not have a history of previous tests.

The more you test, the more accurate your hypothesis will become. Your hypothesis has to be phrased in a way that allows for it to be proved or disproved.

1*OEYx2GS3Ny0UnMr9 kTlfA
Source: Experimentation Works book, by Stefan Thomke

As an example, our made-up delivery app company could have as a hypothesis “If we can get other customers to behave like cohort A by reminding them of the restaurants they like, we can expect a lift in the number of orders”.

4. Objectives and Metrics

This is where you disclaim what metrics should be tracked to evaluate the outcome of your test. This metric should be connected to your objectives, which should, in turn, support the achievement of your North Star Metric. To learn more about how to set your goals, have a look at this article: The 3 Layered Cadence of KPIs of a Growth Strategy.

1*Uzc yqvrGE3JeM9aclns7Q
Metric Goal

According to Sean Ellis, you should “identify metrics to be tracked by looking at the metrics “downstream” from the experiment that will be impacted”. In our case study, the metrics to be measured could be:

– Number of orders made from notifications from this campaign

– % of users who received a notification and completed an order right after

– Average number of orders from the experiment group

– Average number of orders from the control

These metrics will help the team assess if they have validated their hypothesis.

5. Prioritization

To help organize your experimentation pipeline, you can also suggest a prioritization grading according to your chosen prioritization framework, which can be adjusted during the growth meeting, when you are discussing the logistics and resources available to your team. Here at GrowthHackers, we use the ICE score prioritization framework, where we grade ideas for Impact, Confidence, and Ease.

 
ICE Prioritization Framework

6. Related Ideas

When suggesting an idea that relates to a past experiment, or to other ideas that are still waiting in your pipeline and relate to the same goal, it is a good idea to either tag ideas so they can be easily grouped, or link the other ideas.

By doing so, you will be able to filter related ideas that pertain to the same growth lever and objective and easily get a full picture of past tests and existing ideas for that topic when suggesting a new idea.

Below you can see what this information looks like in our new idea template at Experiments by GrowthHackers. Once your team is briefed on your current goals and knows how to suggest ideas that can be understood by anyone in the team, you will soon have an idea pipeline to prioritize and test according to your objectives and resources.

 
New Idea Card at Experiments by GrowthHackers

How does your team suggest ideas in your company? Have you tried a similar process to make ideation easy?

Share it in the comments!

Read more insightful articles

  • All
  • Branding
  • Content Marketing
  • Digital Marketing
  • Email Marketing
  • Graphic Design
  • Growth Hacking
  • Influencer Marketing
  • Keyword Research
  • Lead Generation
  • List Building
  • Local Marketing
  • Mobile Marketing
  • Online Advertising
  • Pay-Per-Click Advertising
  • Search Engine Marketing
  • Search Engine Optimization
  • Social Media Marketing
  • Video Marketing
  • Web Development
  • Web Performance Optimization
All
  • All
  • Branding
  • Content Marketing
  • Digital Marketing
  • Email Marketing
  • Graphic Design
  • Growth Hacking
  • Influencer Marketing
  • Keyword Research
  • Lead Generation
  • List Building
  • Local Marketing
  • Mobile Marketing
  • Online Advertising
  • Pay-Per-Click Advertising
  • Search Engine Marketing
  • Search Engine Optimization
  • Social Media Marketing
  • Video Marketing
  • Web Development
  • Web Performance Optimization

How to Create a Documented Content Strategy That Works

Read More →

About Us

We are a full-service digital agency that drives results for companies through tech-enabled marketing solutions.

We unlock revenue growth for your business with our world-class expertise. Our team experts have assisted thousands of businesses with digital marketing projects aimed toward increasing lead generation, quality and quantity of website traffic, phone calls volume, conversion rate, and transactions.

With a custom strategy, we’ll help your business impact the metrics that matter most, from traffic to revenue.

Recent Posts

Follow Us

Sign Up For Our Newsletter

Subscribe to always stay in touch with us and get the latest news about our company and all of our promotions


Stay connected with us

You are in good company here

error: Content is protected!