Optimizely: a powerful tool for your site optimization efforts
Updated March 14, 2016

Optimizely: a powerful tool for your site optimization efforts

Anonymous | TrustRadius Reviewer
Score 7 out of 10
Vetted Review
Verified User

Overall Satisfaction with Optimizely

Used for simple A/B tests and to force visitor traffic to pages that have specific messaging or UI elements

Pros

  • Very easy to configure experiment targeting, traffic allocation, and other settings you would want to customize for your testing program
  • Allows easy selection of experiment types like Multivariate, Multipage, A/B
  • Permits simple UI changes using its code editor and other options which allows for decreased reliance on development resources. Product and Design teams can launch new experiments easily on their own
  • Support is first rate: knowledgeable, great communicators who take the time to explain things so that you can learn and apply to future experiments. Optimizely customer support is among the best I've encountered from a web software service

Cons

  • Improvements can be made with experiment results reporting specifically customization of data within the data export or whatever is viewed on the summary page
  • Different organizations could have different thresholds for statistical significance. Optimizely is set to 95%. It would be nice to have this value configurable.
  • It is difficult to ascertain whether a losing variation is a statistically significant loss (only shows low chance to beat baseline). Having this would assist with pausing variations that are not performing very well.
  • Customizable, distributable reports would be a great feature to add. Currently you must click through each individual experiment to see experiment details. Seeing a summary through email or selecting multiple experiments to view on one page would be a huge improvement for scanning health of tests.
  • Optimizely has helped us optimize on micro conversion goals throughout our site
  • Our macro conversion goals have also been optimized using Optimizely
  • Homegrown A/B platform and Adobe Test & Target
Not as strong.
Review how front end or back end-heavy your tests plans will be. Optimizely allows for easy implementation of front end changes which won't bog down development resources. They should be able to focus on bigger fish than changing button colors, texts, or adding images to pages.

Using Optimizely

5 - Growth, Digital Product, Engineering
1 - A/B testing knowledge, Front End JS/CSS, QA
  • Test new ideas/variations against current control version
  • Force visitors to see certain site changes
  • Reports to understand impact of site change
  • Segment traffic from the various inbound channels
  • Analytics
  • Performance indicators
Overall a great experience from ease of use and flexibility for the tool. Reporting improvements could be made, but what exists currently is enough to give businesses a sense of the directional movement of a change to the site.

Evaluating Optimizely and Competitors

  • Price
  • Product Features
  • Product Usability
Simple and effective tool to use for quick site changes and A/B tests.
Talk with power users for each product more, and understand what feature they wish they could have but currently don't.

Optimizely Support

Enterpriise support is great when they respond to a request, but if only on standard plan good luck.
ProsCons
Knowledgeable team
Support understands my problem
Slow Resolution
Poor followup
Problems left unsolved
Escalation required
Slow Initial Response
No - Not sure if the benefits are justified for this particular product and organization.
When critical tests were impacting our paid campaigns due to issues with the way Optimizely handled our page, then my request was escalated and team worked to address right away.

Using Optimizely

Decently usable but if doing advanced stuff it's not easy.
ProsCons
Like to use
Well integrated
Familiar
Requires technical support
Slow to learn
  • Button color and copy
  • Segmenting to different audiences
  • Working with updating images in a slider carousel
  • Reporting

Optimizely Reliability

Sometimes when too many A/B experiments are running, weird things happen on the site. It'e best to only run a few amount of experiments concurrently.
For the most part Optmizely's availability has been solid.
Sometimes there are glitches for variations in production, but depends on the change.

Comments

More Reviews of Optimizely Web Experimentation