Building and Testing an Optimization Plan

When it comes to constructing a Conversion Rate Optimization Plan, people typically take one of two approaches: applying popular Conversion Rate Optimization tactics or building a Conversion Rate Optimization plan.

When applying CRO tactics, you…

  • Are equipped with a toolbox of suggestions and quick fixes—like changing button color
  • Rely on tips and tricks that have worked for others, hoping for the best
  • Are only focused on elemental concerns
  • Pay little attention to analyzing customer behavior
  • Have a starting place, but no clear plan of action… so you make more guesses or seek out more tips and tricks

By contrast,

When building a CRO plan, you…

  • Attempt to figure out what the numbers mean before trying to fix them
  • Form a hypothesis (or hypotheses) based on those test results
  • Construct a plan of action to test those hypotheses
  • Take the knowledge gained and use to it form new hypotheses
  • Understand this is a consistent, structured, and ongoing process of making your website better over time

For example, a new source of traffic is added to a test page, and the conversion rates drop. If you are working from a set list of optimization tactics, you immediately begin tweaking page elements in search of “fix.” Conversely, when working from a strategic optimization plan, your first action is to attempt to figure why those numbers changed. Do the needs of this new traffic source differ from those of your established sources? If so, how? These questions are then followed by tests that attempt to answer them or, at the very least, help determine which tests to run next.

When you implement a conversion strategy, you do so knowing that a single tweak won’t fix all your website’s problems. You understand that even if a test fails to support your original hypothesis, the knowledge gained from the test still contributes to growing, changing understanding of how you can better serve your users.[1] Tweet this!

So, now that you understand the importance of having a strategic Conversion Optimization plan, let’s discuss how to go about constructing that plan. We’ve chosen to divide this planning into “phases” rather than “steps” in order to emphasize the circular nature of a sound optimization strategy. Rather than thinking of these phases as a strict progression from one to the next, keep in mind that you will revisit each one in order to continually address the needs of your users over time.

Phase 1: Lay the Groundwork

We’ve already covered the importance of identifying what “conversion” means to you, but we’ll restate it again because it’s that important. Before beginning any optimization strategy, you have to know what you’re measuring and attempting to optimize. It’s also important to understand what drives these conversions.

For example…

You run a wedding planning business, and you have a form on your website that allows visitors to schedule a free fifteen minute video consultation. This is the conversion you want to measure and optimize.

But what drives this conversion? Testimonials from happy customers? Adspace on wedding blogs? Large photo galleries showcasing weddings you’ve coordinated?

It could be each of these things, or something else entirely. The only way you’ll know is to isolate each variable on its own and measure how users behave under each set of circumstances.

Here is what a test plan for your wedding website might look like:

  • Your goal is to increase your number of free video consultations, which, though free, often result in people choosing to use you as their wedding planner
  • You hypothesize that adding testimonials may increase the number of free consultations scheduled
  • You decide to compare conversion rates for when testimonials are prominently displayed versus when they are not (A/B split testing)
  • You’ll measure the number of consultations scheduled from each page to see if your hypothesis was correct

You will do this for each variable you want to better understand.

Phase 2: Establish a Baseline

As we’ve discussed throughout this guide, a sound conversion strategy is based on some important metrics and tons of user input. But in order to work from that information, first you need to understand where you’re starting from. This is called your baseline. Only by establishing your current performance can you measure the changes you make to find improvement.

You won’t know if your optimizations actually improve unless you have numbers to compare them to. To establish your baseline for comparison, you will…

  • Refer back to the goals you identified in Step 1
  • Look at the metrics related to these goals. What is your current conversion rate? Which are your best sources of traffic for this conversion?
  • Run a user survey to figure out how well you are meeting these goals and what you could be doing better
  • Employ user testing around these goals to establish how successful your site is at meeting them

Your basic toolbox will include:


Software to track and report on what’s happening on your site day in and day out. You want an analytics package such as Google Analytics, KISSMetrics or similar that has advanced analysis tools like audience segmentation and conversion tracking. Segmentation can produce data for different sets of people, and you can isolate hiccups or trouble spots in your conversion funnel.

User Surveys

Analytics can only communicate so much about your users’ needs; you need something that gives you the ability to gain insights directly from users in the moment, to hear their concerns in their own words; there is no such thing as too much user feedback.

User Testing

Software like Optimizely and other testing tools allow you to directly observe how users are interacting with your site. You can test potential changes and document how they play out in real life.

You now have the baseline against which all future changes will be measured. Whenever you alter something, compare performance before and after. How have your metrics changed? Your survey results? The ways in which users interact with your site? This is how you figure out if you make things better or worse.

Phase 3: Form Some Testable Hypotheses

Now it’s time to look at the baseline we established in the previous phase and identify your biggest barriers to conversion. What you want to do is identify the problem areas, implement those tools we just talked about to investigate, and then design some potential tests.

For example…

Via your analytics tool, you learn that the bounce rate for your wedding planning website is on the rise, so you use the page report feature to isolate it to your photo gallery—a popular but clearly under-optimized page. At this point, you might decide to implement an on-page survey on that page in particular asking users what they’re looking for and whether they were able to find it. You could also run some user tests to see what people are doing while there. You could install CrazyEgg to see where people people are clicking, or how far they’re scrolling on the page. Additionally, you could ask a few of your customers at the store to look at the page and watch them try to navigate through it.

Take the information you get from testing and user surveying, and use it to form a hypothesis that attempts to explain why no one is sticking around on that page. Next, come up with some alternate versions of the photo gallery page.

This leads us to the next phase.

Phase 4: Design Your Tests

In this phase you’re going to take everything you’ve learned so far and design a test strategy. Start by making a list of your priorities. Which points of concern come up again and again in user surveys? What seem to be your sites biggest issues, and which ones do you need to address first?

Above all else, it’s important to be methodical here. Double and triple check your numbers and and keep a written record of absolutely everything. Tweet this!

Here are a few points to consider when designing your test…

  • Start small—look for something that won’t be too complicated to change and measure, but with real potential for improving conversion rates.
  • Begin with simple A/B tests. Don’t make too many changes at once—or you won’t know if it’s the more prominent call to action or the new and improved testimonials that result in the improvement in your conversion rate.
  • Think outside the box—if visitors aren’t clicking “Schedule a Free Consultation,” the answer isn’t necessarily to make it red.
  • Get a second opinion. If you designed the page, you may not readily spot the problem.
  • Consider modeling results to see what kind of impact they might have. For example, if you can reduce your bounce rate by 10% at a certain point in the funnel, how many more conversions might that lead to? What effect would this have on your profit margin?
  • You might have to lower the “risk bar” for your users.[2] Maybe you’re asking too much too soon? Free trials and promotions can help to earn your users’ trust sooner.
  • To give your own numbers some perspective, you might want to look for benchmarks for other companies in your industry.
  • Double and triple check that you have sufficient tracking in place so that you are able to interpret the effects of your test.
  • Set a sample size and stick to it! Whatever you do, don’t end your test early because you think you’ve found a winner. You have to let your test run its course to be sure—more on this in Chapter 10. [3]
For example…

You believe the high bounce rate on the gallery page of your wedding planning website results from a lack of contact and brand information on this page. This page gets a lot of hits from external sources like Pinterest and wedding blogs, many of whom skim your photos and then leave unaware of the services you offer.

You decide to create an alternate version of the photo gallery page, adding a banner that reads, “Planning a wedding? I’d love to help. Click to schedule your free consultation,” sending them to your standard contact form.

Phase 5: Run Your Tests

You will measure success against the baseline you established in Phase 2. The data resulting from your test, when compared to your baseline, will tell you where to go from here.

If this test was a success, then great. Now, you can either cross this concern off your list and move on to the next one or continue refining and re-testing this page, making it as awesome as possible.

If this test wasn’t a success, don’t be discouraged. All this means is that it’s time to go back to Phase 4, reexamine the data, and design a new test. You learn as much from a negative outcome as you do from a positive outcome.

Keep in mind…

Regardless of the outcome of your initial round of testing, you should think of optimization not as an end goal but as an ongoing process. Because the way we do business is always evolving and customers’ needs change over time, you will never reach the point where you’ve run “enough” tests.

When you’ve improved a specific sticking point in your user experience, pat yourself on the back for a job well done, and then go back to Phase 3 and ask yourself what else can be improved upon.[4] Tweet this!

Chapter 4 Notes