MozillaWiki

Firefox/Projects/browser ballot mechanical turk: Difference between revisions

Page Discussion

Firefox/Projects/browser ballot mechanical turk (view source)

Revision as of 06:46, 13 January 2010

10,174 bytes added ,  13 January 2010
+doc
(+doc)
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Background  ==
== Background  ==


To compliment the testing and research done by Critical in Europe, a series of Mechanical Turk tests will be conducted to test various aspects of the EC's Ballot design. The goal is to determine both how to design the ballot in the most neutral way possible, and for Mozilla to determine the most successful summary and image for the Firefox section of the ballot.
To compliment the testing and research done by Critical in Europe, a series of Mechanical Turk tests will be conducted to test various aspects of the EC's Ballot design. The goal is to determine both how to design the ballot in the most neutral way possible, and for Mozilla to determine the most successful summary and image for the Firefox section of the ballot.  


== Series One: Comparing Slogans and Designs ==
== Series One: Comparing Slogans and Designs ==


=== Test 1: Comparing Firefox Banners  ===
=== Test 1: Comparing Firefox Banners  ===


This simple 6-question test asked about people's reactions to the top Firefox logos. 40 people participated.
This simple 6-question test asked about people's reactions to the top Firefox logos. 40 people participated.  


==== Questions asked ====
==== Questions asked ====


# Which of the following five versions drew your attention first?
#Which of the following five versions drew your attention first?  
#:[[File:Five options ec ballot proposal.png]]
#:[[Image:Five options ec ballot proposal.png]]  
# Why did that version draw your attention first? (freeform)
#Why did that version draw your attention first? (freeform)  
# Which of the following five versions do you prefer?
#Which of the following five versions do you prefer?  
#:[[File:Five options ec ballot proposal.png]]
#:[[Image:Five options ec ballot proposal.png]]  
# Why do you prefer this version? (freeform)
#Why do you prefer this version? (freeform)  
# Which browser do you use most often?
#Which browser do you use most often?  
#* Internet Explorer
#*Internet Explorer  
#* Firefox
#*Firefox  
#* Safari
#*Safari  
#* Opera
#*Opera  
#* Other
#*Other  
# Why do you use this browser? (freeform)
#Why do you use this browser? (freeform)


==== Results ====
==== Results ====


[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdERuUS1YNnpvWU5aUTkyZ1l0dDFjRHc&hl=en Raw results in Google Doc Spreadsheet]
[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdERuUS1YNnpvWU5aUTkyZ1l0dDFjRHc&hl=en Raw results in Google Doc Spreadsheet]  


{| width="200" cellspacing="1" cellpadding="1" border="1" summary="Which of the following five versions drew your attention first?"
{| width="200" cellspacing="1" cellpadding="1" border="1" summary="Which of the following five versions drew your attention first?"
|+ Test 1 Question 1: Which drew your attention first?
|+ Test 1 Question 1: Which drew your attention first?  
|-
|-
|  
| <br>
| '''Frequency'''  
| '''Frequency'''  
| '''Percentage'''
| '''Percentage'''
Line 56: Line 56:
| 17.50%
| 17.50%
|}
|}
[[File:T1q1 ecmechturk.png]]
 
[[Image:T1q1 ecmechturk.png]]  


{| width="200" cellspacing="1" cellpadding="1" border="1" summary="Which do you prefer?"
{| width="200" cellspacing="1" cellpadding="1" border="1" summary="Which do you prefer?"
|+ Test 1 Question 3: Which do you prefer?
|+ Test 1 Question 3: Which do you prefer?  
|-
|-
|  
| <br>
| '''Frequency'''  
| '''Frequency'''  
| '''Percentage'''
| '''Percentage'''
|-
|-
| [[Image:Design1 ecballot thumb.png]]  
| [[Image:Design1 ecballot thumb.png]]  
| 14
| 14  
| 35.90%
| 35.90%
|-
|-
Line 78: Line 79:
|-
|-
| [[Image:Design4 ecballot thumb.png]]  
| [[Image:Design4 ecballot thumb.png]]  
| 13
| 13  
| 33.33%
| 33.33%
|-
|-
| [[Image:Design5 ecballot thumb.png]]  
| [[Image:Design5 ecballot thumb.png]]  
| 6
| 6  
| 15.38%
| 15.38%
|}
|}
[[File:T1q3 mechturk.png]]
 
[[Image:T1q3 mechturk.png]]  


== Series Two: Comparing Slogans Individually  ==
== Series Two: Comparing Slogans Individually  ==
Line 91: Line 93:
In five separate tests, a complete ballot was presented similar to the one that Microsoft proposed. On each, a different Firefox slogan was displayed. Each test contained the following four items, presented to each of 40 users:  
In five separate tests, a complete ballot was presented similar to the one that Microsoft proposed. On each, a different Firefox slogan was displayed. Each test contained the following four items, presented to each of 40 users:  


=== Questions Asked ===
=== Questions Asked ===


Imagine you just bought a new computer running Microsoft Windows 7. You turn it on, and you see the following screen. Please read the description of each browser carefully, and then answer the following questions.
Imagine you just bought a new computer running Microsoft Windows 7. You turn it on, and you see the following screen. Please read the description of each browser carefully, and then answer the following questions.  


[[Image:Phrase1 for ec ballotscreen sample.png|thumb|center|300px]]  
[[Image:Phrase1 for ec ballotscreen sample.png|thumb|center|300px]]  
Line 122: Line 124:
<br>  
<br>  


{| cellspacing="1" cellpadding="1" border="1" summary="Which of the following five versions drew your attention first?" style="width: 355px; height: 756px;"
{| cellspacing="1" cellpadding="1" border="1" style="width: 355px; height: 756px;" summary="Which of the following five versions drew your attention first?"
|+ Test 2 Question 1: Which is most convincing?  
|+ Test 2 Question 1: Which is most convincing?  
|-
|-
|  
| <br>
| '''Frequency'''  
| '''Frequency'''  
| '''Percentage'''
| '''Percentage'''
|-
|-
|  
| Your online security is Firefox's top priority. It's built by the Mozilla Foundation to make browsing better.  
====== Your online security is Firefox's top priority. It's built by the Mozilla Foundation to make browsing better. ======
 
| 8  
| 8  
| 20%
| 20%
|-
|-
|  
| Firefox adapts to how you use the Web. Find out why more people have downloaded it than any other browser.  
====== Firefox adapts to how you use the Web. Find out why more people have downloaded it than any other browser. ======
 
| 13  
| 13  
| 32.5%
| 32.5%
|-
|-
|  
| Firefox is free and made to help you get the most out of the Web. Find out why more people have downloaded it than any other browser.  
====== Firefox is free and made to help you get the most out of the Web. Find out why more people have downloaded it than any other browser. ======
 
| 7  
| 7  
| 17.5%
| 17.5%
|-
|-
|  
| More people have downloaded Firefox than any other browser. Find out why!  
====== More people have downloaded Firefox than any other browser. Find out why! ======
 
| 7  
| 7  
| 17.5%
| 17.5%
|-
|-
|  
| Built by the Mozilla Foundation to make the Web better, Firefox won't take over your computer or compromise your privacy.  
====== Built by the Mozilla Foundation to make the Web better, Firefox won't take over your computer or compromise your privacy. ======
 
| 5  
| 5  
| 12.50%
| 12.50%
|}
|}


[[File:Test2q1mechtest1.png]]
[[Image:Test2q1mechtest1.png]]  


== Series Three: Comparing Logo Designs Individually  ==
== Series Three: Comparing Logo Designs Individually  ==


In five separate tests, a complete ballot was presented similar to the one that Microsoft proposed. On each, a different Firefox slogan was displayed. Each test contained the following four items, presented to each of 40 users.
In five separate tests, a complete ballot was presented similar to the one that Microsoft proposed. On each, a different Firefox logo design was displayed. Each test contained the following four items, presented to each of 40 users.  


=== Questions Asked ===
=== Questions Asked ===


Imagine you just bought a new computer running Microsoft Windows 7. You turn it on, and you see the following screen. Please study each item on the ballot carefully, and then answer the following questions.  
Imagine you just bought a new computer running Microsoft Windows 7. You turn it on, and you see the following screen. Please study each item on the ballot carefully, and then answer the following questions.  
Line 191: Line 183:
==== Results  ====
==== Results  ====


===== Design 1 =====
===== Design 1 =====
[[Image:Design1 ecballot thumb.png]]
 
Raw data:  
[[Image:Design1 ecballot thumb.png]]  
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdDd4THZDVVZ0ZWhlZHpEVW5hemFRVVE&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 1, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 13
| 32.5%
|-
| Internet Explorer
| 8
| 20%
|-
| Firefox
| 14
| 35%
|-
| Opera
| 4
| 10%
|-
| Choose Later
| 0
| 0%
|}
 
<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 1, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.56%
|-
| Chrome
| 8
| 20.51%
|-
| Internet Explorer
| 10
| 25.64%
|-
| Firefox
| 18
| 46.15%
|-
| Opera
| 2
| 5.13%
|-
| Other
| 0
| 0%
|}
 
===== Design 2  =====
 
[[Image:Design2 ecballot thumb.png]]
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdDlfR2Z6cS1xMjdQRjI2T3FRVFdadXc&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 2, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 10
| 25%
|-
| Internet Explorer
| 8
| 20%
|-
| Firefox
| 18
| 45%
|-
| Opera
| 3
| 7.5%
|-
| Choose Later
| 0
| 0%
|}
 
<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 2, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 9
| 22.5%
|-
| Internet Explorer
| 10
| 25%
|-
| Firefox
| 19
| 47.5%
|-
| Opera
| 2
| 5%
|-
| Other
| 0
| 0%
|}
 
<br>
 
===== Design 3  =====
 
[[Image:Design3 ecballot thumb.png]]
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdFNIa09KWnktQmoxSVFjY083VWM5eHc&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 3, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 3
| 7.5%
|-
| Chrome
| 11
| 27.5%
|-
| Internet Explorer
| 5
| 12.5%
|-
| Firefox
| 16
| 40%
|-
| Opera
| 5
| 12.5%
|-
| Choose Later
| 0
| 0%
|}
 
<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 1, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 10
| 25%
|-
| Internet Explorer
| 5
| 12.5%
|-
| Firefox
| 21
| 52.5%
|-
| Opera
| 3
| 7.5%
|-
| Other
| 0
| 0%
|}
 
===== Design 4  =====
 
[[Image:Design4 ecballot thumb.png]]
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdHRWbFlGYUMtRGxMYTZLbGFnSlBUZHc&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 4, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 2
| 5%
|-
| Chrome
| 6
| 15%
|-
| Internet Explorer
| 7
| 17.5%
|-
| Firefox
| 22
| 55%
|-
| Opera
| 3
| 7.5%
|-
| Choose Later
| 0
| 0%
|}
 
<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 1, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.63%
|-
| Chrome
| 7
| 18.42%
|-
| Internet Explorer
| 5
| 13.16%
|-
| Firefox
| 22
| 57.89%
|-
| Opera
| 3
| 7.89%
|-
| Other
| 7
| 0%
|}
 
===== Design 5  =====
 
[[Image:Design5 ecballot thumb.png]]
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdENOaWQwdl9WcDVSQlZ4UjgwNDdEUlE&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 5 Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 3
| 7.5%
|-
| Chrome
| 11
| 27.5%
|-
| Internet Explorer
| 5
| 12.5%
|-
| Firefox
| 18
| 45%
|-
| Opera
| 3
| 7.5%
|-
| Choose Later
| 0
| 0%
|}
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 1, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 11
| 27.5%
|-
| Internet Explorer
| 6
| 15%
|-
| Firefox
| 20
| 50%
|-
| Opera
| 2
| 5%
|-
| Other
| 7
| 0%
|}
 
== Series Four: Comparing Slogans Individually  ==
 
In five separate tests, a complete ballot was presented similar to the one that Microsoft proposed. On each, a different Firefox slogan was displayed. Each test contained the following four items, presented to each of 40 users.
 
'''Slogans:'''<br>
 
#Your online security is Firefox's top priority. It's built by the Mozilla Foundation to make browsing better. <br>
#Firefox adapts to how you use the Web. Find out why more people have downloaded it than any other browser. <br>
#Firefox is free and made to help you get the most out of the Web. Find out why more people have downloaded it than any other browser. <br>
#More people have downloaded Firefox than any other browser. Find out why! <br>
#Built by the Mozilla Foundation to make the Web better, Firefox won't take over your computer or compromise your privacy. <br>
 
=== Questions Asked  ===
 
Imagine you just bought a new computer running Microsoft Windows 7. You turn it on, and you see the following screen. Please study each item on the ballot carefully, and then answer the following questions.<br>
 
#Which item on this screen would you choose?
#*Install Safari
#*Install Google Chrome
#*Install Microsoft Internet Explorer
#*Install Firefox
#*Install Opera
#*Select Later
#Why would you choose this item? (freeform)
#Which browser do you currently use most often?
#*Safari
#*Google Chrome
#*Microsoft Internet Explorer
#*Firefox
#*Opera
#*Other
#Why do you use this browser? (freeform)
 
==== Results  ====
 
===== Slogan 1  =====
 
''"Your online security is Firefox's top priority. It's built by the Mozilla Foundation to make browsing better."''<br>
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdFVvYXNteWkzam5tYU03ZnNick1vYWc&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Slogan 1, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 13
| 32.5%
|-
| Internet Explorer
| 7<br>
| 17.5%
|-
| Firefox
| 16
| 40%
|-
| Opera
| 3
| 7.5%
|-
| Choose Later
| 1*<br>
| 2.5%
|}
 
*&nbsp;"Selecting a browser is not my first priority while using my computer for the first time."<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Slogan 1, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 9
| 23.68%
|-
| Internet Explorer
| 6<br>
| 15.79
|-
| Firefox
| 22<br>
| 57.89%
|-
| Opera
| 1
| 2.63%
|-
| Other
| 0
| 0%
|}
 
===== Slogan 2  =====
 
''"Firefox adapts to how you use the Web. Find out why more people have downloaded it than any other browser."''<br>
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdHNMb0xIdllhV284eTV5WUhmYkYwRUE&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Design 2, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 10
| 25%
|-
| Internet Explorer
| 7
| 17.5%
|-
| Firefox
| 18
| 45%
|-
| Opera
| 3
| 7.5%
|-
| Choose Later
| 1*<br>
| 2.5%
|}
 
*&nbsp;"I want moretime to chose one within these items" (IE user, says uses IE because it's "microsoft's product. microsoft is the most popular company among the it companies")<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ S 2, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 9
| 24.32%
|-
| Internet Explorer
| 4
| 10.81%
|-
| Firefox
| 21
| 56.76%
|-
| Opera
| 3
| 8.11%
|-
| Other
| 0
| 0%
|}
 
<br>
 
===== Slogan 3  =====
 
''"Firefox is free and made to help you get the most out of the Web. Find out why more people have downloaded it than any other browser."''<br>


(Table for Design 1)
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdFV4YjRrek1lQlVZaDI4ZkttRE5mRVE&hl=en raw data]
(Graph for design 1)


===== Design 2 =====
{| width="200" cellspacing="1" cellpadding="1" border="1"
[[Image:Design2 ecballot thumb.png]]
|+ Slogan 3, Question 1: Which item on this screen would you choose?
Raw data:  
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 18
| 45%
|-
| Internet Explorer
| 4
| 10%
|-
| Firefox
| 14
| 35%
|-
| Opera
| 4
| 10%
|-
| Choose Later
| 0
| 0%
|}
 
<br>
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Slogan 3, Question 3: Which browser do you currently use most often?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 14
| 35%
|-
| Internet Explorer
| 5
| 12.5%
|-
| Firefox
| 19
| 47.5%
|-
| Opera
| 2  
| 5%
|-
| Other
| 0
| 0%
|}
 
===== Slogan 4  =====
 
''"More people have downloaded Firefox than any other browser. Find out why!"''<br>
 
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdGhBWmIxRWctSXFwNWMxSlF3NXBGQ0E&hl=en raw data]
 
{| width="200" cellspacing="1" cellpadding="1" border="1"
|+ Slogan 4, Question 1: Which item on this screen would you choose?
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 1
| 2.5%
|-
| Chrome
| 16
| 40%
|-
| Internet Explorer
| 5
| 12.5%
|-
| Firefox
| 14
| 35%
|-
| Opera
| 4
| 10%
|-
| Choose Later
| 0
| 0%
|}


(Table for Design 2)
<br>
(Graph for design 2)
(table for main browser used)
(graph for main browser used)


===== Design 3 =====
{| width="200" cellspacing="1" cellpadding="1" border="1"
[[Image:Design3 ecballot thumb.png]]
|+ Slogan 1, Question 3: Which browser do you currently use most often?
Raw data:
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 10
| 26.32%
|-
| Internet Explorer
| 9
| 23.68%
|-
| Firefox
| 16
| 42.11%
|-
| Opera
| 3  
| 7.89%
|-
| Other
| 0
| 0%
|}


(Table for Design 3)
===== Slogan 5  =====
(Graph for design 3)
(table for main browser used)
(graph for main browser used)


===== Design 4 =====
''"Built by the Mozilla Foundation to make the Web better, Firefox won't take over your computer or compromise your privacy"''<br>
[[Image:Design4 ecballot thumb.png]]
Raw data:


(Table for Design 4)
*[http://spreadsheets.google.com/ccc?key=0AmikoOWTD0bIdFQyS3dkb2hwVFFKUGVZWFhVX0V3VHc&hl=en raw data]
(Graph for design 4)
(table for main browser used)
(graph for main browser used)


===== Design 5 =====
{| width="200" cellspacing="1" cellpadding="1" border="1"
[[Image:Design5 ecballot thumb.png]]
|+ Slogan 5 Question 1: Which item on this screen would you choose?
Raw data:
|-
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 3
| 7.5%
|-
| Chrome
| 9
| 22.5%
|-
| Internet Explorer
| 12
| 30%
|-
| Firefox
| 9
| 22.5%
|-
| Opera
| 6
| 15%
|-
| Choose Later
| 1
| 2.50%
|}


(Table for Design 5)
{| width="200" cellspacing="1" cellpadding="1" border="1"
(Graph for design 5)
|+ Slogan 1, Question 3: Which browser do you currently use most often?
(table for main browser used)
|-
(graph for main browser used)
| <br>
| '''Frequency'''
| '''Percentage'''
|-
| Safari
| 0
| 0%
|-
| Chrome
| 9
| 23.08%
|-
| Internet Explorer
| 12
| 30.77%
|-
| Firefox
| 14
| 35.9%
|-
| Opera
| 4
| 10.26%
|-
| Other
| 0
| 0%
|}


==Ballot Designs==
== Ballot Designs ==


(moved to page [https://intranet.mozilla.org/Ballot_dump under LDAP]
(moved to page [https://intranet.mozilla.org/Ballot_dump under LDAP]
== Current Report ==
Can be downloaded [http://people.mozilla.com/~jboriss/ballot/ec_ballot_report3.pdf here (pdf)].
Jboriss
Confirmed users
954

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/188225...193905"