656453 - [Mobile] Launch Mobile Survey on Mozilla.com

Status: VERIFIED FIXED

(www.mozilla.org :: General, defect, P1)

Description

[Laura Forrest]

Objective: Figure out leading use cases for mobile visitors to Mozilla.com to help inform the redesign of the mobile UX. 

To do this, we'd like to launch a survey on Mozilla.com that only shows to visitors on Mobile devices asking them why they're visiting the site. 

(Wiki: https://intranet.mozilla.org/Webanalytics/Mobile)

I've created this survey using SurveyGizmo http://sgiz.mobi/s3/293dbff586e0. There's a number of technical ways to launch/host this including:
-web link
-Javascript imbed
-Website pop-up or pop-over
-Static HTML imbed
-iFrame imbed

Survey UX: there's a couple different options here depending UX disruption level we're comfortable with, WebDev preferences and SurveyGizmo technology. 

I propose something like having this survey appear about 5 seconds into a mobile visitors session as the main element on the page, then disappear once survey is complete, with option all-along to not participate in survey (I can add a "will you take a quick survey" question into the survey above as the first question if needed). The survey is already mobile optimized.

Chrissie or Jason - Thoughts on preferred UX?

Sampling Rate: Would be good to serve this out to 25% of website visitors using mobile devices (amounts to ~17K/week).

Comment 1

[Laura Forrest]

Moving to 2.6 since 2.5 is a big one. 

Steven - once the dust subsides a bit let me know your thoughts around serving this out.

Comment 2

[Jason Grlicky [:grlicky]]

Thanks so much for getting this together, Laura! That survey looks awesome, and I think your proposal for the Survey UX sounds like a good starting point. Having some opt-out option available at all times would be great, and great to track too (so survey participation is something we can work to improve as well!).

I'm not familiar with web survey best practices, though - what is the reason to wait for 5 seconds? To make sure that the user has a chance to see that they're at the right site?

As a final thought, would it be a good idea to reword the introductory sentence to include some concrete estimate of the time the survey will take? Everyone says their surveys are quick, so it might be more powerful to say "2 minutes of your time?", "3 questions", etc. The existing sentence definitely works well too, though.

Comment 3

[Laura Forrest]

(In reply to comment #2)
> Thanks so much for getting this together, Laura! That survey looks awesome,
> and I think your proposal for the Survey UX sounds like a good starting
> point. Having some opt-out option available at all times would be great, and
> great to track too (so survey participation is something we can work to
> improve as well!).

Agree. User should be able to easily close the survey throughout.  

> I'm not familiar with web survey best practices, though - what is the reason
> to wait for 5 seconds? To make sure that the user has a chance to see that
> they're at the right site?

That's exactly right - so they can recognize that they ended up on the right site. If the survey is the first thing they see I think it would make for a confusing experience, unless we are able to really brand it to match the website, which is pretty hard to do since the survey tool we're using has limited customization features.

> As a final thought, would it be a good idea to reword the introductory
> sentence to include some concrete estimate of the time the survey will take?
> Everyone says their surveys are quick, so it might be more powerful to say
> "2 minutes of your time?", "3 questions", etc. The existing sentence
> definitely works well too, though.

Good idea, I'll add it in. This survey literally will take only 5 seconds to complete.

Comment 4

[Laura Forrest]

Hey Steven - Do you think this can go in the 2.6 release for next Tuesday?

Comment 5

[Steven Garrity [:sgarrity]]

(In reply to comment #4)
> Hey Steven - Do you think this can go in the 2.6 release for next Tuesday?

That should be ok, yes. Can I get some clarification on the implementation?

Should this appear on every page, or just the home page? Should this obscure the rest of the page or just appear as a button in-line in the page?

Comment 6

[Laura Forrest]

(In reply to comment #5)
> (In reply to comment #4)
> > Hey Steven - Do you think this can go in the 2.6 release for next Tuesday?
> 
> That should be ok, yes. Can I get some clarification on the implementation?
Happy to hear it!

> Should this appear on every page, or just the home page? 
Every page but only upon the visitors entry to the site. 

Should this obscure
> the rest of the page or just appear as a button in-line in the page?
Good question. One suggestion is to have a module that says something like "Please take our short survey" "Ok"/"No Thanks" appear as the visitor enters the site in a way that covers most of the page but not all of it - so the user clearly can tell that they're on Mozilla.com. If they select "Ok" the survey starts, and then they're dropped back onto Mozilla.com at the end. 

I can see some sort of button working too - so long as it's noticeable enough to mobile visitors. 

Let me know what you think would work best considering the UX and technical possibilities. I'm happy to have a quick chat over IRC/AIM too.

Comment 7

[Steven Garrity [:sgarrity]]

Attachment: Mockup of survey button

Comment 8

[Steven Garrity [:sgarrity]]

Attachment: Mockup of survey button (half-size)

I've added a quick mockup of a way would could insert the survey prompt into any page without intruding too much on the experience (changes are people didn't come to the site to tell us why they came to the site).

I've attached two mockups, the large is just a full-size screenshot of a typical android screen resolution. The 'half-size' just shrinks that same mockup down 50% to give you a better (rough) idea of how large things will look on an actual phone.

Wording is just something I wrote for the sake of the mockup - would need careful review (probably a re-write). Brevity is key though - even a few more words requires more lines (precious vertical space) or smaller text, which is tricky on a mobile display.

Thoughts?

Comment 9

[Jason Grlicky [:grlicky]]

Steven - I really like this mockup! We'd need to make sure that the "No thanks" link has a large hit area so that it's easy to touch for users. Laura - what are your thoughts on the "Got a minute to help us out?" copy?

Comment 10

[Laura Forrest]

Steven - This looks great, thanks so much for throwing it together. As for copy, I'm cc-ing Matej here to see if he has any input. 

@Matej - this is a link to a single question survey, so it really is super fast to take. Do you have any better copy ideas? Thanks!

Comment 11

[Matej Novak [:matej]]

I think it looks pretty good as is, but if it's really one question, I'd make that explicit:

Got a sec to help us out?

Answer one quick question.


Or, a cheekier version:

A one question survey?

Sure, I'd love to help.

Comment 12

[Jason Grlicky [:grlicky]]

Matej: <3

Either option seems great to me!

Comment 13

[Laura Forrest]

I vote cheeky!

Thanks Matej!

Comment 14

[Steven Garrity [:sgarrity]]

The text/css is implemented in trunk in r89597. No cookies (to dismiss and hide when you've done the survey, yet).

You can see how it looks so far here: http://www-trunk.stage.mozilla.com/en-US/m (looks better on a phone than on the desktop).

Comment 15

[Jason Grlicky [:grlicky]]

Looks great to me (checked on iPhone4:Safari/Droid2:Default Browser,Firefox)! Nice work Steven :D

Comment 16

[raymond [:retornam] (inactive)]

qa-verified http://www-trunk.stage.mozilla.com/en-US/m/ checked on iPhone4, Firefox for Android

Comment 17

[Steven Garrity [:sgarrity]]

As of r89713 in trunk, we're now using cookies to hide the survey prompt after you've either started it or dismissed it.

Laura, after completing the survey, there's a message that says you'll be redirected back to mozilla.com, but it doesn't seem to actually do that.

Comment 18

[Laura Forrest]

(In reply to comment #17)
> As of r89713 in trunk, we're now using cookies to hide the survey prompt
> after you've either started it or dismissed it.
> 
> Laura, after completing the survey, there's a message that says you'll be
> redirected back to mozilla.com, but it doesn't seem to actually do that.

Good catch - I just changed to include a link back to Mozilla.com.

Comment 19

[Steven Garrity [:sgarrity]]

Merged to stage in r89737.

Comment 20

[raymond [:retornam] (inactive)]

doesn't link to Mozilla.com after completing the survey

Comment 21

[Laura Forrest]

(In reply to comment #20)
> doesn't link to Mozilla.com after completing the survey

Couldn't find a way to redirect automatically but I did add in instructions and a text link so we should be set to push this live. 

Anthony - Can you push as part of 2.7?

Comment 22

[Laura Forrest]

This survey has been ready to go for a while and can be pushed out with the 2.8 release, or sooner. 

Since we're eagerly awaiting the findings of these results before we make drastic changes to our mobile experience, I'm changing this to a P1.

Comment 23

[James Long (:jlongster)]

pushed to production r90569

Comment 24

[raymond [:retornam] (inactive)]

verified fixed http://www.mozilla.com/en-US/m/