![No, You Shouldn't Use Visual Editors of Testing Tools [Rant]](https://cxl.com/wp-content/uploads/2017/05/visualeditor-1024x189.jpg)
“Our A/B testing tool’s visual editor allows marketers to set up tests without needing developers!”
A version of this is communicated by pretty much every testing tool out there. Their websites show off screenshots or videos of visual editors. But it’s wrong. You shouldn’t use visual editors to run your testing program.
Saying that non-devs can set up AB tests now is misleading and wrong. Yes, you can change button color and make copy changes easily, but that’s not what one should do. You should test what actually matters and what actually needs to be tested (based on your conversion research).
Note: there might be cases where you need to test something quick and easy. If that’s really whats you think would be best, visual editors will help you do it.
If you are limited by what the visual editor allows you to test, you will start testing stuff that makes no difference. Instead of testing what matters, you test what your lacking skills enable you to test. So the lack of your skills dictates what you test. Obviously, this approach is wasting everyone’s time and money.
Your tests should tackle real issues, your test variations should cover a wide range of ideas – and no idea for a treatment should be limited by the lack of coding skills.
The moment you try to set up a more complicated test via visual editors, there’s close to 90% chance that it messes up the code – and variations look wonky and won’t work properly on half the browsers and devices. QA is hard as it is, and now you let people with no coding skills – and most likely clueless about quality assurance testing – set up tests?
I see surprised folks every day in my training program. They ask me “what, I need to learn to code to set up tests?” Yes, yes you do. “But, but – …”.
And I’m not even going to rant about test targeting rules (regex anyone?), goal configuration and stuff that can also be done completely wrong.
Promoting visual editors is uneducating marketers. It’s a disservice to the community.
It’s fine to have them – but they should come with a disclaimer: “can’t be used to set up any real tests.”
Update. This sparked interesting Twitter conversations.
Should you use visual editors of testing tools?
Related Posts
-
A/B testing is highly useful, no question here. But a lot of businesses should not be…
-
If I were to tell you - what would you do with that information? Honestly,…
-
Both your visitor and Google prefer your site to be fast. Increasing site speed has…
-
Do you need to be a big company and need large volumes of traffic to be data…
We’ve gained some really key insights on how to communicate various products’ value add by letting our marketing team test on their own during private beta periods. Without visual testing tools, they wouldn’t have been able to run these tests.
Ultimately, the final versions of those pages were built by our engineers for public launch, but rapid testing enabled us to learn quickly. Even better- we saved time by letting these tests run while our dev team was focused on getting the product ready for public release.
Over the years, I’ve found marketing and engineering teams need to collaborate to drive the best outcomes. It’s not one or the other, but both together.
That’s a very specific use case and is not about running a testing program via visual editor. That’s pretty much the only legit use case I can think of.
That’s the argument I was discussing with my mates the other day and this is the first time (finally!) I see an experienced marketer being honest and talking about one of the hardest parts of running A/B tests. Thank you, Peep. Seriously… thank you.
The other point is that these editors generally apply unoptimized and unorganized jQuery scripts that could negatively affect loading performance of your experiments and cause element flashing and other issues that a layman wouldn’t be able to fix easily (and visual editor can’t offer much support for that).
Learning the basics of frontend programming was EXTREMELY useful for my profession and made everything so much easier in terms of A/B test setup – and it’s just the basics so a little big of programming knowledge should be highly encouraged to all marketers that want to succeed with conversion optimization and A/B testing.
Peep,
I agree that the best tests are the ones where there are large changes, which typically can’t be solved by using a visual editor alone.
But what is the best way for CRO agencies to work w/ clients to get larger tests with code changes launched. Who does the actual dev work: the agency or the client?
Does your team basically do all of the coding themselves & then hand it over to the client, saying “upload this”? Or do you lay out the changes you want to make & have the client’s dev team handle the changes.
Obviously it depends on the client, but typically the agencies set up all the tests themselves.
Really?? So in a lot of cases your agency will actually have direct access to the client’s code & upload variation code?
No access to the website code. You have access to the testing tool and deploy custom front-end code (jquery, css etc) for variations there
But visual editors like VWO allow you to add custom front end code like jQuery and css. Sounds like you’re saying you use the tool you’re ranting against. Am I missing something?
For reference:
https://vwo.com/knowledge/using-vwo-code-editor/
This says you can use the editor to make javascript/css changes
You’re missing the point. All the testing tools, including VWO, have both visual editors AND code editors. You should use code editors to build tests. Not the WYSIWYG editors inside those tools.
Oh ok I understand now. I think the confusion is from the fact that CEO refers to its code editor as a feature WITHIN the visual editor. (For example, the instructions for editing variation code in their knowledge based is filed under “Using VWO Visual Editor”.) So when you were saying that the visual editor is not worth using, I took that to mean that you were saying the code editor ( an advanced feature within it) is also not worth using.
I agree. The visual editor is absolutely no replacement for a proper web developer.
That’s not to say Small tests can’t be done with the editor, but the smaller tests aren’t going to win you any wars.
This blog normally seems really smart, which is why I follow it, but this post… I don’t even get it.
First of all, how anyone with any experience can suggest that testing visual treatments of pages isn’t worthwhile just boggles my mind. Since the examples of marketers creating massive, game-changing gains based on visual design tests are endless and happen all day every day, the only explanation I can come up with for your promoting this obviously wrong assertion is that you’re trying to sell your coding services? Seems too obvious of a ploy though. Your audience is sharper than this, I think? I don’t get it. In my many years doing online marketing, if I didn’t have quick access to visual tests, I would have failed at virtually every project, period. Instead, every single time I take on a project, we start with results that are somewhere between crappy and passable, and we test our way to success… using visual optimization tools. It works every time. To suggest that it doesn’t is either stupid, terribly misinformed, or purposely misleading. Which is it?
Second of all, this article lacks any clarity about what you’re actually suggesting as an alternative. “Your tests should tackle real issues,” you write. But you leave whatever those tests may be to the readers imagination. Please explain what tests are so much more important and valid than those than can be done via a visual editor?
Finally, the assertion that ‘there’s close to 90% chance that [your visual editor] messes up the code’ is absurd. There are dozens of extremely high-quality tools out there for testing all sorts of page variations, and plenty of them create perfectly acceptable code. In fact, an argument could easily be made that many of these tools reliably create higher quality code than you can count on from the average hand-coder…
So: what is this article for exactly? Did you just have another proposal turned down by a marketing team that thought VWO or Unbounce might just do the trick for them for a fraction of your price, without making them rely on another flakey coder for every change they want to make? Did another prospect value speed of implementation over your religious devotion to some kind of code-commandments that you received from on high?
Everyone has a bad day. Stick to what has made this blog valuable over time, not these whiny rants about nonsense.
Okay, hold your horses cowboy.
1. This post is not about visual treatments. It’s about using (or rather not using) the built-in visual EDITOR to set up those tests. I’m all for visual treatments.
2. I don’t sell coding services
3. The question of “what to test” is the most important question in optimization. The answer comes from conversion research – qualitative and quantitative data gathering and analysis. Based on the research outcomes – which should identify where the problems are, what the problems are and why these issues are problems to begin with – you can start coming up with test hypotheses for addressing those issues/problems. The hypotheses also need to be prioritized based on the severity of the issues they address, amount of traffic they affect and ease of implementation.
You can look into ResearchXL model for research and PXL model for prioritization.
What you test now will be based on this research and prioritization effort. Let’s say you have a list of 100 test hypotheses now, prioritized. What it takes to set up those tests should not matter. If you go -hmm, I could set up tests 3, 34 and 72 with the visual editor – so let’s test those instead!
Visual editor is for simple changes – which is fine for some cases – but you should not be limited in your testing to what can be accomplished by the visual editor. Testing tools are fully aware of this – most of their tests are set up via custom code.
4. No serious testing program is run on the backs of visual editors. Also VWO is offering to code up tests for all their users since they’re also well aware of it.
Gotta admit I agree 100% with Dave Johannsen.
This strikes me as clickbait at best. Also reminds me of the days when everyone kept telling me how it was impossible to code a decent website with a WYSIWYG editor. Well I created a successful campaign org’ with one, and along the way learned enough to end up as a sales consultant and now semi-retired. Would I use such an editor today? Not for a website, but for explaining something I still do :)
Seriously, this article is just a head-shaker.
The article is a bit of a rant and takes the point to far but there’s definitely truth in it. In many cases, visual editors take away focus and changes the thinking from proper research to ‘let’s run some AB tests. This also touches upon another point that’s important. Clients, especially uneducated ones, have a lesser perception of our work and what it entails.
Peep, You know I’m a totally FAN! but I have to agree with Dave Johannsen, as well… I was looking for a part in the article that reads something like… “here’s what you should do instead and why”. You were very clear in your article title that it is a RANT, therefore you get a pass from me – we all have venting days. However for your followers, ok me, I look to you for deeper insights that tell me about methods that are better and why. Remember, your audience is full of newbies and experts. From this newbie I wanted to read the rant, but also the solution or the better ways to A/B Test.
How to get more wins: https://cxl.com/how-to-come-up-with-more-winning-tests-using-data/
I strongly disagree, I used userfeel.com and test results were revieling! My e-shop had a significant improvement in usability since the corrections we did.
Not sure what you’re on about. This post doesn’t mention user testing even once.