Why market testing is worth the effort

Market testing is worth the effortOver the years, I have known a number of clients who didn’t do market testing or didn’t think it was worth the effort. Often these are overworked employees of medium-to-small companies who have a lot of balls in the air; how can you justify spending hours to analyze a past campaign when it’s all you can do to get the current one out the door? I’m frustrated by this attitude as a copywriter because results are what I get paid for; if I can’t prove my effort outperformed your control then you’re less likely to hire me for a future project.

I like clients who live and die by market testing and are willing to follow its learnings even if results conflict with their gut or the preferences of their boss. Such a client recently asked me to write a number of variations of an email inviting investors to an introductory workshop. This organization has plenty of data tracking how registrations for this event turn into a future revenue stream. Increasing registrations costs nothing more than the few dollars you pay the copywriter to come up with a fresh message and the benefits go directly to the bottom line.

I crafted the test messages based on input from focus groups and polling of prospects who had registered for a previous event but did not show up; if we could find anomalies in these groups compared to the profile of their typical student and speak to those, maybe we could increase the perceived value of the workshop and make them more likely to register and then attend. I also did a series of messages based on an earlier successful test in which we emphasized that the event lasted just three hours and made that seem like a trivial commitment and a good use of their time.

Result: virtually all my emails beat the control and the most successful nearly doubled it measured by the percentage of recipients who signed up for the workshop. The creative was the only variation in the promotion, proving that yes, people really do read the copy. The messages will be retested, refined and rolled out, potentially bringing in a lot of motivated new prospects to be nurtured and developed into committed, profitable students. Market testing is definitely worth the effort.

If you need to sell the value of market testing internally, you might use the example of MoviePass. This company had the bright idea to buy unsold tickets in movie theaters and then wrap them into a membership plan where you can watch x movies a month for a fee of y. According to a recent interview with the CEO of its parent company, MoviePass did test y but apparently not x. They just decided to offer unlimited movies… which meant MoviePass would end up buying its members a full priced ticket if a discount was unavailable for a popular movie. The CEO didn’t see a problem with a burn rate which was then $21 million a month even though MoviePass only had $43 million in cash on hand.

Shortly after this interview (which went live on July 18 of this year), MoviePass announced the number of movies you could see per month would be reduced from unlimited to… three. The result was a sharp decline in its stock price and a feeding frenzy from competitors and the media. All of which could have been avoided with some simple market testing.

How to interpret “illogical” market testing results

From time to time I’ve made analogies between marketing and home shop experiences and pointed out the simple wisdom that can be found through working with your hands. So bear with me for a moment here as I wind up to a theory on illogical market testing results.

Cindy and Chris at Northside Service Company were miracle workers in bringing my 25-year-old NuTone intercom system back to life. However, when all was said and done there were some glitches. You are supposed to be able to push a button to initiate a call from a remote unit, the recipient is able to talk hands free, and you as the initiator can continue to talk by pressing a button each time you want to speak.

My system wasn’t working like that. You could initiate a call and hear another person and they could hear you, but after that first time you pressed the button subsequent presses did nothing; the signal didn’t go through. There was another issue, minor but consistent across the system: the “end call” button which returned you to whatever you were doing beforehand (like listening to FM radio) didn’t work.

This one-and-you’re-done setup worked fine for summoning kids to dinner or answering somebody who pressed the doorbell. And standing at a wall intercom and talking back and forth in your own home seemed a little Austin Powers-ish. Nonetheless, I wanted to get to the bottom of this.

Through testing I found ONE remote intercom system, out of 16, that worked as it was supposed to. You could press and release that talk button and continue to communicate, and the “end call” button worked as it was supposed to. So I took it apart. And it turned out it was mis-wired. Two wires going to terminals marked “red” and “red/white” were reversed.

I tried mis-wiring a second remote and it, too, began to work properly. I thought about the master controller and what might be wired incorrectly. No schematics are available after a quarter century, certainly not on the NuTone website. Perhaps I made a mistake when I took out the pull connectors to send it in for repair. But I was pretty careful and the pull-off single wire connectors were grounds, so they should not affect the electrical switches.

My guess is that something deep within the system was mis-wired at installation and the original owner put up with it the entire time they lived in the house. There is no other logical explanation because the problem began with an illogical mistake. It takes only a minute or two to remove a remote from the wall and rewire it, so that’s what I’m going to do with the remaining 14 remotes. I’ll also put a note inside the housing of the main controller for a future owner of the house.

Getting to the marketing analogy, many many years ago I worked on a test mailing that involved a bunch of shredded U.S. currency visible through a window in the outer envelope. The product was a newsletter on reducing your taxes, and the message of the involvement device was that you might as well be tearing up your own money for all the unnecessary taxes you’re paying. This test was a disaster. The numbers indicated that absolutely nobody had opened the package and considered responding.

And that was odd. The involvement device may have been a bit sensationalist and there were probably some quirks to the copy, but it followed a solid platform related to the features and benefits of the publication we were selling. I could accept a terrible result of responses that were 20% of the control’s, but 0%? Something is wrong.

My hunch is that somebody at the mail house (or possibly the post office, but less likely since that would be a hanging crime) took a fancy to my gimmick and simply appropriated the few thousand pieces involved in the test. Illogical and far fetched, but can you give me a better explanation?

Happy April Fool’s Day, by the way, but the above is no joke.

Why Cadillac “Dare Greatly” campaign doesn’t

While many brands made a political statement with ads on the 2017 Super Bowl, Cadillac saved theirs for the (surprisingly non-political) Academy Awards telecast. “We are a nation divided. That’s what they tell us, right?” the ad begins. “But what they don’t tell you, what doesn’t make the news, is this: We carry each other forward.” And we have a series of clips of Americans supporting or carrying others, sometimes literally.

Next we go to a montage of historical figures standing next to their Cadillacs. “No matter who we are, or what we believe, or where we come from, we’ve had the privilege to carry a century of humanity,” the ad continues. The “carry” metaphor is not very subtle, but it speaks well to Cadillac’s history as a symbol of having made it. I grew up in a suburb of Dallas with a very eclectic population mix—from college professors to poultry distributors, and from aristocrats with a long southern heritage to first generation immigrants.

Some folks thought owning a Cadillac was a way to show off your prosperity; my Uncle Jim bought a new one every year. But others would never own a Cadillac because it was not appropriate to be so ostentatious. I suspect the new Cadillac campaign is speaking to the first group. You may live in fear of deportation, you may belong to an ethnic group that is the subject of hate crimes, but if you have money for the down payment at least you can drive a nice car.

As it wraps things up, Cadillac stretches its metaphor a bit too far in my opinion. “But maybe what we carry isn’t just people. It’s an idea. That while we’re not the same, we can be one. And all it takes is the willingness to dare.” And we close with the Cadillac logo and the tag line, “Dare Greatly.” (The tag is not new, but was introduced with this campaign a couple of years ago.) Again, what we’re talking about is bourgeois prosperity. It’s the same kind of daring that might cause you to go for it with a nicer anniversary gift or a pricier steak than you had planned because, hey, this is America and you can. It’s really not that daring, but maybe it hits the Cadillac buyer’s sweet spot.

The branding continues to veer off course with the next spot, “Pioneers”. “We’ve always been dreamers. We’ve been a symbol of the future…. A standard… a star. But our past is just that, past. What lies ahead is in our hands.” And the ad goes on to introduce concept vehicles including self-driving and electric Cadillacs. The drivers in these vehicles are younger, much younger.

The third ad, “Pedestal”, starts with a vehicle literally on a pedestal surrounded by gawkers. A well-dressed 30ish woman comes forward, mesmerized. “We know how it feels to be treated like a trophy. Driven to awards shows… parties… and across so many silver screens…. But a Cadillac is no trophy, no museum piece. This is our future, and it will inspire every car that follows…. Intermission’s over. This is how we drive the world forward.” And now the mesmerized woman is behind the wheel while multiethnic pedestrians gawk at her good fortune.

The problem here is that Cadillac is trying to transform its brand and appeal to a new audience, yet its history is the reason for its cred. They should have done what Lincoln and Chrysler did, in high concept campaigns of the past. Tell your story, then shut up. The campaign can continue with product-focused features and performance ads, and buyers will connect the dots.

I’ll close with Electro Cadillac, presumably a current owner, who says much the same thing in a comment under the YouTube “Pioneers” video:

“I can’t believe this… This commercial is basically telling you that all the achievements and great cars that were made in the past, history (which, let’s be honest is much better than the weak plastic bullshit we have today) let’s just throw all that in the garbage… I enjoy driving and all this ‘futuristic’ crap is getting on my nerves with every single day going by…

“One thing that is fundamental guys, Cadillac hear me out: You can’t beat the old school. Those flimsy plastic bumpers will never compare to those good old chrome steel ones.

“And here is another thing. I am watching the Oscars as we speak, and your Escala commercial says this: ‘Our cars will never be like before’. Well that means that you’re not going to be Cadillac anymore.

“Here is a new slogan for you: There is no future without the past.”

Celebrity athletes free to express their political opinions? Now that’s scary.

When Under Armour’s CEO Kevin Plank described Donald Trump’s business-friendly attitude as “a real asset” to the country, the Golden State Warriors’ Steph Curry told The San Jose Mercury News, “I agree with that description, if you remove the ‘et.’” The sneaker brand promptly terminated its relationship with the popular MVP, explaining that “we don’t care who you are, you don’t disrespect the office of the President of the United States.”

Except that’s not what happened.

Plank took out a full page ad in the Baltimore Sun (Under Armour is based in Baltimore) to explain that “in a business television interview last week, I answered a question with a choice of words that did not accurately reflect my intent.” He then reached out to Curry as well as actor/wrestler Dwayne Johnson and ballerina Misty Copeland, who had also criticized him, to make sure they understood. And Steph Curry is still with Under Armour and presumably free to speak his mind.

In the past the conventional wisdom, and certainly the preference of marketers who pay for their endorsements, was that celebrity athletes should stay out of politics. (Celebrity actors are, of course, a different story.) Why risk offending millions of people who might boycott your brand as a result in the same way that Trump supporters are now boycotting Nordstrom’s and T.J. Maxx? But the flip side is that, when young people put on a pair of Steph Curry sneakers, they’re not just wearing shoes. They’re paying extra for identification with a brand that stands for ability, brilliance, success through striving and just plain coolness.

If that brand lets the celebrity add a political perspective, it may be brave and ballsy but it may also be coolly calculating. Think about the rationalizations that might be made by Midwestern white kids who continue to wear Curries. Maybe they’ll say, “hey, I appreciate a guy who’s complex and able to live with contradictions… like me.” Or maybe they’ll take Curry’s example as a license to form their own opinions and think for themselves. That’s scary, and exciting.

Note: this post heavily draws from a long and thoughtful piece which appeared in the New York Times on February 26, “Celebrity Endorsers Turn Political, and Keep Their Deals”. Please read that piece, by Zach Shonbrun, for more perspective.

Thank you internet! Thank you Chris and Cindy Peters for fixing my NuTone intercom!

NuTone IMA-4006 controller
My NuTone IMA-4006 Music Intercom. Not only did they rebuild the control unit, Northside Service Company cleaned it up so it looks like new.

I recently moved into a house that has a NuTone IMA-4006 Music Intercom system installed. You can press a button and be heard in another room (or from the porch if you are ringing the doorbell) and also play music throughout the home. I know that sounds quaint in these days of earbuds and instant messaging, but there is a remote unit in every room so it was pretty hard to ignore.

The most recent owners, who lived here for 15 years, had never tried the NuTone intercom system. I experimented by turning a few knobs and got nothing but hum. But at least there was power. Was it possible the system could be restored to working condition? My local NuTone service center said “we don’t know anybody who works on them anymore.” So it was off to the internet.

A search quickly put me in touch with Northside Service Company, a factory-authorized NuTone Service Center in San Ramon, CA. Their site features dozens of links to videos, manuals and articles to help you make the most of your obsolete equipment. I filled out a web request form and a few minutes later the phone rang. It was owner Chris Peters, calling to discuss my system. It turns out that the remote units rarely fail so if I would send him my control station he would rebuild it at a cost that was not cheap, but far less than buying a comparable system today or taking out all those speakers and patching the holes walls.

I disconnected the many wires following Chris and Cindy Peters’ very clear step by step video, then packed it up according to another video of instructions. A couple of weeks later, I got the unit back along with a bag of parts that had been replaced. Not only did they rebuild the unit, they replaced the doors that cover the controls and often break off (the hinges are no longer available so Northside had them custom manufactured) and at my request added an A/V jack and sent me a couple of new lighted doorbell buttons.

It took me a few days to get up the courage to re-install the unit and test it, but I did and everything works as advertised. I’m back in business, feeling very much like a wired denizen of the early digital era.

This is a story that could not have happened without the internet—which helped me find Northside Service Company, and enabled Chris and Cindy to build a site that was incredibly useful and also showed me they knew what they were doing. But the capper was the personal service. I mentioned that Chris called me immediately when I submitted a request. (It was on New Year’s Eve as I recall.) When I had a question about an extra wire during re-installation he answered me by email within an hour. This is a business model many other specialized service providers (copywriters come to mind) can learn from, and emulate.

Lessons not learned in 2016

Brexit and Donald Trump’s election were, according to fivethirtyeight.com, well within the margin of error in polling predictions and so were shocking only because people were mentally and emotionally incapable of thinking these events would take place. This made me think of some experiences with focus groups and direct mail back in the days when both were a bigger thing than they are now.

When shown a number of creative options, focus groups would inevitably veer away from the more promotional formats, especially when those formats had big screaming headlines and prominent offers. Yet every time the formats were tested head-to-head in the mail, the promotional format won.

My point: there is no substitute for real-world, boots-on-the-ground testing so long as you have two or more worthy options to consider. People will lie when they think the wrong answer might embarrass them, but not when they’re alone with the offer and their credit card.

Search marketers know this and so do online marketers who constantly refine their landing pages through multivariate testing. But I see many traditional marketers who don’t bother to test or—maybe worse—set up an a/b split and then fail to capture the results or are too busy to analyze them.

I always like to present a more and a less conservative option for any campaign. Of course, I am disappointed when the client chooses the conservative approach and even more disappointed if they test both and the conservative one wins. But the marketplace doesn’t lie. If you ignore this truth, you better be prepared to live with the consequences as well as explain them to your boss or client.

Free marketing advice from Warby Parker

The other day I was on a plane and got in a conversation with my seat mate. When she found out I worked as an advertising guy she told me her husband designed neckwear and wanted to sell his ties via the internet. What free marketing advice did I have?

My first thought was, uh oh. Fashion is a very fickle industry. I had some experience early in my career when I was an ad manager for a large department store. In that bricks-and-mortar era a men’s fashion manufacturer had to sell a network of retail buyers each season, starting with the MAGIC Show  (is it still around?) and other industry events and and once you had a few retailers signed up, manufacture and distribution was the next channel. Maybe online sales have broken down some of those barriers, but the subjectivity of the ultimate buyer probably hasn’t changed.

Then it occurred to me: Warby Parker. Here’s another niche fashion product that seems to be very successful, based on the frequency with which I see their Facebook ads. So I advised her to study Warby Parker, or another single-line internet retailer, to see what they do. If it seems successful, then consider emulating their strategy.

I don’t think this is bad advice. One of the great things about working in marketing is its transparency. It’s not like the technology industry where a company’s special sauce is kept under lock and key so competitors won’t steal it. To the contrary, retail advertising is in plain view and the more you see it the more successful it probably is.

“What advice can you give me as a marketing pro” has just been added to the topic list for my DMA Ignite session on Monday, October 17 at the DMA &Then conference. This session is evolving into a sort of town hall meeting in which creative practitioners and ad managers will share their ideas and frustrations with their peers. Come join us at 4 pm at the Los Angeles Convention Center. And in the meantime, if you have any free advice of your own, please comment below.

How not to localize direct mail

Your personalization vendor has a great idea: add variable fields to localize direct mail by mentioning the reader’s geographical location. This is especially effective if you’re a national brand and you want to connect with prospects on a grassroots level. Hence this month’s example from the Salvation Army:

Saratoga Cares
Salvation Army wants me to know Saratoga Counta Cares

The biggest problem with this effort is something that can happen to any marketer–which is why I am sharing this even though I have beat up on these poor folks and their localization in the past. I know where I live, so making a reference to that place has no meaning unless you add another attribute. For example, “last night 368 people in Saratoga County went to bed hungry. Here’s your chance to help them get a good meal”. Just saying “this is a mailer about Saratoga County” is a huge so-what because almost every piece of localized mail, whether it’s a bill or a message from a civic group, says the same thing.

A secondary problem is that “Saratoga County” is a meaningless term. Saratoga County is a gerrymandered entity and I feel no affinity for my fellow citizens down in Waterford or across the lake in Edinburg. I am a resident of Saratoga Springs, a city, so please identify me/yourself as such. If you’re going to localize, take the time to research local usages like this and avoid faux pas. (A favorite, which I searched unsuccessfully for just now, was a liquor ad localized to San Francisco from a few years back in which the tippler looking for an excuse to drink checks off “Saw a hippie at Haight”. It should be “in the Haight” of course and that boner immediately brands the marketer as a carpetbagger.)

Here’s another interesting example of personalization/localization. A Canadian ad firm drove Porsches to people’s homes, parked them in the driveway and took a picture, then drove away and mailed the prospect a picture of the car in their own driveway with the caption “it’s closer than you think”. In friendly Canada it drew accolades and  32% response rate. In the U.S. it would have drawn lawsuits. Know your local audience.

Copywriters, the end is near

I was paying my quarterly visit to a client when the online marketing manager mentioned he needed quite a few SEO articles written. I asked if I could help. No need, he said, I’ll just order them on TextBroker for ten cents a word and run them through copycheck to be sure they weren’t plagiarized.

I’ve always had a smug attitude toward the link bait that pops up when you search on a technical topic. It’s pretty obvious they’re partly machine-written and/or English is not the writer’s primary language. So to have my own client go this path was a bit unsettling. I signed up for my own TextBroker account and commissioned an article on Bengali cuisine for my food blog. The author would have to say what makes Bengali cuisine unique (specific spices and ingredients used) and provide a defining recipe as an example. All this for a maximum of $75.

2 days later, _Liz came through. She hit all the markers and the article is good enough that with a couple of tweaks I’m going to use it. The cost? $65.08, for something that would have taken me the equivalent of $500 or more at my hourly rate to research and write. By the way, why did I choose that topic? Because I assumed the writers were South Asia based and I might get lucky and find a real subject matter expert. But TextBroker tells me their writers are 100% in the USA.

Here’s the other reason we copywriters might as well trade in our keyboards for flip-flops. The Wall Street Journal, which I rely on for sports perspective for some reason, has twice reported in the last few days on organized races with declining registration. First 10Ks, now “mudder” type obstacle courses. The explanation? Those darn millennials. Studies show they don’t enjoy competition as much as previous generations, hence less interest in organized competitions.

Of course, the reason we as copywriters get paid what we do is that we convince readers they can rise above the competition—whether you define that by economics, status or ability to do a job better—with the help of our client’s product or service. If the reader no longer cares, where does that leave us?

A long weekend in New Orleans is looking mighty good right now.

If you can’t read the results, why test it?

FICO envelope front
Envelope variations for Chase Slate FICO package, plus duplicates received at my address

My early direct mail copy chiefs beat this mantra into me: only test one thing at a time. If you change the offer and simultaneously change the letter lead, how do you know which change was responsible for any lift?

I thought of this as the drumbeat of “Free FICO Score” offers from Chase Slate continued, and I realized one of the envelopes looked different. Same color scheme, virtually the same OE copy on the front but arranged in a slightly different way. The first really noticeable change is on the back. One says “no annual fee” and the other has a lineup of four unidentifiable awards. (We know which would win in that test, don’t we, since benefits always eclipse chest pounding.)

FICO envelope back
FICO outer envelope back with testing variations

Inside the “alike but different” motif continues. One letter starts “Transfer high rate balances from other credit card issuers and save money.” The other, “From balance transfers to new purchases, Chase Slate makes saving simple.” Exact same facts, but one is about “you” and the other about “the card” so again, it’s pretty clear which would win if tested on its own. (That old copy chief of mine would have had the second writer start over, rather than testing the two leads.) However, the “you” copywriter is paired with the art director who put the shields on the back of the OE, so we’re tied.

Two FICO letters
Which of these letters is more persuasive? Why?

And it continues throughout the package, with design and copy slightly different without changing the facts or the basic presentation. What’s happening here is that two teams were tested against each other to see which one is “better”—a costly experiment on Chase’s part. This isn’t the same as a direct mail package test in which creative teams come up with completely different ideas from scratch. It’s an expensive waste of time.

My advice to Florian Egg-Krings, who signs both letters (no testing variations there): test spelling out “credit score” on the OE rather than calling it “FICO score”. Take one key benefit—I’d probably go with the reasons you’d want a monthly credit report and how great it is to get it for free—and lead one letter with that, then keep the other about your laundry list of benefits. Now you’ve got something worth testing.