User testing – an evergreen idea

A tale for all marketers to remember user testing before they roll out major tactics.

“Comcast and the happiness of all my TVs”

Here we go again with Jennifer Bater, amateur service tech!

What can be observed as a marketer, new product developer and strategist about the challenge of transitioning large swaths of customers to a new technical arrangement? A transition customers must make?

Such as Comcast going all digital here in Denver?

Right off, you have to figure customers are not going to be thrilled. My two secondary sets are not digital… each set needs a box. Blech…..more stuff plugged in, more remotes, more batteries to go dead etc. Blech.

Shades of the digital transition that blew up my kitchen TV. I’ve never been able to remedy that…..(No really small TV sets available with digital incorporated, the freestanding digital boxes can’t seem to tune in the kitchen, no coax in the kitchen. But that’s another story.)

I got a letter from Comcast. I ignored it. The main set with the super STB (Set Top Box – I’m onto this lingo) is working.

This is all about marketing strategy, customer communications, customer service, perceptions of value, etc. I’ll recap at the end on my thoughts.

Before the new gear arrived:

• Comcast called me (and miraculously got me) and clarified the letter. I had glanced down at the letter and my eyes only saw another $1.99 a set. (The rep admitted that many people only saw the $1.99 point. But apparently, I get two free boxes before the $1.99 kicks in. This I didn’t understand in the letter. Letter pre-tested?)

• The second dilemma was that I didn’t know what a Digital Transport Adaptor (DTA) was from the letter. I still don’t really know what it is other than it’s a low end STB but not called that. It’s a DTA. (The rep couldn’t really explain it either, but what the heck?)

• I liked the call. I actually had the letter right in the kitchen trying to decide what to do.

• The rep seemed weirdly interested in which sets these DTAs were for. Which rooms are these for? Is the reason that people are stockpiling devices? Signal theft/resale market? Are they trying to smoke out customers who would ask for a box in “reserve”?

My DTAs arrive:

• Lovely packaging. Very slick!

Very attractive packaging/But what's with the "white cable"?

• I start pondering the instructions. They are more than 3 feet long, nearly a foot wide and two-sided. A little daunting. But to their credit, the type is nice and big.

"Lengthy" instructions

• Each set must be set up before any are activated. There was a stern message to this effect. Why?

o Do they have to be synchronized?
o What does the phrase “set up” mean?
o In many ways, I’d rather do one TV at a time and complete the task vs. doing two in parallel. Maybe that’s just me.

• You need to be ready with your serial numbers to do activation and there is a very convenient spot on the giant instruction form to complete this task. But, I can’t write on the paper to get ready for activation. Too shiny. The pen doesn’t work well. More on that in a minute.

• Plus, remember I have to note where each DTA is on the shiny paper. Is this like Google where they are collecting data on the side? Is the little box taking pictures of the room? Do they think my house is so vast that I’ll forget where these boxes are?  Hmmm.

• And then there is Secret Step 4B. You must get the remote set up as it’s the only way to operate the DTA. It’s not optional…..I don’t know why it’s not explicitly in the instructions flow. This is not something you can do later or if you want to. I feel clever that I realized this. Somewhat offsets the fact that I didn’t realize that the original letter offered me two DTAs for free.

At this point, I am also thinking that I spend $120 a month for cable…..and you can’t come out and set this up for me for a nominal sum…..$25 for the visit. I’d pay it. $100 to set this up, then I have to think.

The Install

Here I go – service tech, Jennifer….no, let me pause and have a cup of tea.

Now I’m ready:

1. Go to the guest room set. I get the cables hooked up, the DTA powered up, and then the remote doesn’t work according to the instructions on the 3 foot piece of paper. I go to the detailed instructions and “win” on the first NEC code. I am feeling good.

I believe the guest room set is waiting for activation.

However, the guest room set did flash the message that “you are experiencing a service interruption” and need to call Comcast. I don’t think Comcast really means this, but I wish the message said, “Congratulations, you are ready for activation on this set. Don’t activate until all sets are ready.” This would make me happier.

2. Go to the office set. I’ve got the 2nd serial number noted down by pressing hard on the shiny paper. But then I notice that I’ve written 12 digits and not 13 digits like the first box, so back to the first number where I have misread my attempt to write on the shiny paper. Back to the other set to double-check the DTA serial number. Sure enough, I misread the number I wrote on the shiny paper.

3. Really go to the office set. Off I go, confidently ready to set up set two.

4. “No job is so important that we cannot do it safely”. I’m feeling very confident as I unpack the power cord for DTA #2 and stretch it out.

Unfortunately, I was almost knocked unconscious when I tip over a heavy candle that clunked me on the side of the head and that fell off and nearly broke my reading glasses. I had to have a pause for ice. (This tells you why marketers have to review their instructions with Legal fifty times. I also, at this point, am willing to pay $50 to have someone else do it.)

The candles are heavier than you think!

5. I “win” again! The remote works on the first NEC code. Both sets are on and I am ready for activation. Am I sure both sets are on? Let me back up and check again that both sets are on. This requires climbing up and down 3 sets of stairs.

6. Best part so far: I liked the Comcast web experience. It was obvious where to put in my special number and once I did, the web site KNEW MY DTA SERIAL NUMBERs. Yippeee!!!

So I don’t have to try to read the numbers I wrote on the shiny paper.

When I pushed the button to activate, it backed me up to have me check that the sets were connected and tuned to channel 3. Which they were.

But, I believe it should also have asked me if the sets were turned on. (Or maybe it would work if the sets were off?  Not clear.)

And, it does ask me to Take Our Survey, but I can’t do that right now, I don’t know if I’ve got service working……Hold your horses!

7. Next steps: The web site says it could take up to 45 minutes to have this process go through. The shiny brochure says “a few minutes”. Hmm. I wish the brochure had said it could take up to 45 minutes as I would have had a clearer picture of the time this requires. Goodness, no wonder they don’t want to roll a truck. This could be an hour at the customer’s home.

8. First check-in: It’s been a “few minutes”, let me go see if the light on the front of either set is solid. Up the stairs I go, BINGO! The set is on and playing video. I have lots more channels than I had before without a box. I have the GOLF CHANNEL!

Golf Channel! (complete with the perky white cable!)

9. Check on Number 2 DTA: Down the stairs I go. Sure enough. Set 2 is working!!! With Bloomberg TV. Did I have that before? Certainly not on set 2 in the office.

And Bloomberg too!/See those dangerous candles lurking!

10. The survey: Good, crisp survey. Good job. I was able to mention a few things that hung me up.

But I was bummed out on the question about whether I had watched the online video as an aid to connecting the equipment. I saw no mention of that video. Bummer. I think that could have helped me. Did I once again misread something? There is no mention on the 3 foot long instructions of a video. I would have watched the video.

Also, since I had to check “30 minutes or more” vs. two choices for less time spent, this has to confirm that I was in the slow group on setup. But I did do two devices. Maybe I get credit for that.

Overall, I rate this as a very good experience with Comcast. Yes, I had a few hiccups but I accomplished this in about an hour. Not including the pause to ice my head.

And my lessons learned for all marketers and new product developers –

a) There is true value of naïve testers – you get so close to things that you forget that regular citizens don’t know what DTAs are, for example.

And that reading skills aren’t what they should be. I see the main letter in the box said “go to for live online support” – would that have gotten me to the instructional video? It doesn’t sound like a video?

b) There is real value in human factors testing – I think that would have caught the shiny paper problem. As well as:

a. The remote not being right in the instructions flow.
b. The funny on screen message about calling about a service interruption.
c. The fact that both of my DTAs were set physically to channel 4, even though the instructions seemed to suggest that channel 4 was the rarity.

i. Why wasn’t the box set to channel 3 then?
ii. Were they all shipped set to channel 4? So, should the instructions have been different?

c) The reminder to all of us as marketers and new product people to do a beta test with regular customers and see how they do.

a. Modify methods as needed
b. Change processes that cause extra customer service cost (Was I the only one inclined to call about the on screen message about “service interruption”?)

d) Track metrics and collect customer feedback. Good survey by Comcast. I would think that they have a target cost per average customer and that they are collecting metrics to see if they met their cost and service call goals.

e) Are incentives a piece of the puzzle when you want customers to do something for you? I felt like a deserved a present from Comcast for having served as amateur customer service tech. I know I got a few more channels but I felt like I merited a credit on the bill or a free VOD for taking an hour of my time.

I’m also hearing advertising in the Denver market about what a “hassle all of this is” by at least one satellite provider. So maybe a small present was merited. Of relatively high perceived value, but relatively low cost for the service provider.

I’m sure I have more thoughts to offer but I must go watch Bloomberg TV in the office and The Golf Channel in the guest room!

Comments are closed.