Evangelism/WebsiteCompatibilityInternational

International Web Site Compatibility and Tech Evangelism

Goal

To improve Firefox website compatibility and the use of open standards outside of the US and Europe.

Strategy

Take the data that's being collected in reporter and use it as a lever to improve website compatibility for Firefox and open standards-based development. Invite local volunteers to use reports for their countries as a way to help websites in those countries improve compatibility. Give them the tools to be able to work together on these problems. Inform our localization contacts of this as a resource and offer them help in building communities around testing and compatibility.

Stakeholders for Feedback

• justin
• sethb
• asa
• chofmann
• robert accettura
• kenk
• morgamic
• beltzner
• arun
• jay
• jane
• gen, dynamis, kohei
• li, jack
• djst
• (india?)

Overall Plan

Stage One

• Open up communications for the Evangelism team. Start using the external mailing list on a regular basis including posting plans such as this one. Have an open irc channel for people who want to join up with us (#devrel.)
• Talk with Ken to identify key data elements, output and key metrics that can help us drive change.
• Scope out changes to reporter to support per-country information. (Turns out reporter already records locale information.)
• Scope out changes to Firefox to support voluntary per-country information to improve fidelity.
• Scope out changes to support geoip support so we can get country information.

Stage Two

• Look at the current reporter database to see if there are any changes required to improve scaling with today's data
• Set up a dashboard of reports that include:
  • Number of daily reports (total)
  • Number of daily reports (per-browser)
  • Per-country reports (total)
  • Per-country reports vs. local market share
  • Top 10 sites per country (zeitgeist-style)
• Expose this information to the public to help drive awareness and value of the data.

Stage Three

• From first set of data and scoping from stage one, execute on changes to Firefox to improve data fidelity.
• Involve support in this process to see if there are things that can improve end user support.

Stage Four

• Start inviting localizers to start listing places that they know about.
• Set up a wiki place for this stuff to live on developer.mozilla.org including per-country pages.
• Look at setting up per-country mailing lists for evangelism or ways we can manage that through the wiki. (What does localization do?)
• Identify leads in every country to lead efforts.
• Identify and localize docs required to launch a larger effort in identified countries.
• Identify a set of docs that local people can use to describe tech evangelism issues.

Stage Five

• From local communities see if there are "expert tools" (add-ons) that we can create that will allow those communities to do focused testing.

Stage Six

• Look at outreach through PR and Marketing in various countries once we have data on particular countries and a local community to handle questions, community development and press.

Metrics

How many leaders we have in various countries and if they are able to run an effective program for local website compatibility?

This is more about local leadership than anything else. Are we giving them good materials? Are we giving them good data they can work against?

Number of compatibility problems reported per-country?

We can compare reporting information with top-site information on a per country basis to determine how deep into various countries we're getting and how compatible we are with top content in any particular geography.

Market share results?

Does increased compatibility turn into increased market share in a particular market? Does building a local team to handle compatibility also drive awareness, PR and market share as a result of word of mouth work?

Does this give us more tools for end-user support around the world?

It turns out that we can use this data to help end users solve support problems at both an aggregate level but also at a local "I'm having problems with this website" level. We also might be able to use this data directly in Firefox to display reported problems to users with a particular site if and when they use the broken site reporter tool.

WebDev Work Items

• Update schema to include most recent version information for reports
  • might be saved in the database already when reported by the client but not exposed in the interface
• Schema analysis to support l10n
  • The data is currently storing locale information
  • Would be good to gather geoip information since locale doesn't reflect country
  • Not sure if the description is unicode-safe
• Rewriting reporter templates to support gettext in static files
• Schema changes if needed, including modifying service and front-end to work with new schema
• Database optimization and performance tuning as needed
• Front-end scaling (yslow scores) and visual refresh
• Load testing, QA, and Unit Tests

Firefox Work Items

• Iterate to see when people are using Report Broken Website when they shouldn't be or what it's teaching us about how people are using the browser.
• Work with metrics, webdev + evangelism to see if the form in the browser should be updated.
• Take part in metrics design to understand if changes are helping users or not.

Community Building Exercises

• Can run a simple testing day to report broken web sites for a release and/or a beta. Could use that data in conjunction with the RRRT to see if things pop up as important around the world.
• Can offer an extension that offers more capabilities for local power testers. For example, could keep a list of urls that we generate from the top 1000 sites for a particular country and have you be "responsible" for testing those sites and reporting problems. Reports from those people would include more information about possible solutions.
• Good way for localizers to get involved and build larger communities of their own.
• Need to have a site for particular country people to get organized and involved. (Or let them develop them on their own in the MCS model or use local social networking sites? They very a _lot_ per-country.)
• Recognize "power testers" on a testing site per-country as special for the work that they do. "tested 100 sites", etc.
• Offer the ability via an extension to have a list of urls and sites you need to test on your corporate intranet.

Observations and Ideas

• There might be more than one type of user using this - end users vs. technical users vs. testers. Do those individual users have different needs when reporting broken web sites?
• Lots of reports do include "I tested this in another browser and it does work" - might want to make that an easy to use checkbox in the interface for reporting a broken web site.
• The most reported site in the US was google. Lots of people are reporting that the safe browsing screen is "broken". Interesting, that.
• Locale has very little to do with actual country. Would be good to use geoip information and/or ask in the interface for country information to improve fidelity. Need to be clear in privacy on this.
• Allow web sites to get an RSS feed for reports about their site. Get them involved directly.
  • Also could do something like "live feed for top 100 sites in the US" for testing team and RRRT

Feedback

• Consider more direct outreach campaigns from the start of Stage 1. There is likely a web developer audience in non-US/European locales that need to join our campaign from step one. (sethb)
• Stage 4 asks to reach out to localizers. Consider expanding this to all constituents relevant in local markets. This includes localizers, campus reps, fans of Firefox, conference organizers, known web developer or other industry contacts, government contacts, etc. (sethb)
  • Would love help on figuring out how to do these two. I would like it, too, assuming we're committed to doing this but I don't have a sense of how to do it. Looking for advice and help. (blizzard)
• Consider a tool a bit more dynamic than a wiki to capture websites that need evangelizing. Perhaps a simple web app with a database collecting information and then a great way to display it. Would this be hard? Don't know... (sethb)
  • The reporter site can do this and won't be a wiki. It's entirely data driven. Here's an example of a report for www.google.com for one day: http://tinyurl.com/c3rz4s
• Should look into combining information with herdict: - lots of good visualization and data ideas here. (blizzard)
• As for Reporter addons, people think that their report will be checked and used by Mozilla. They don't know how to search the database. They also don't know they can see the reports on http://reporter.mozilla.org/. This is because reporter UI don't tell "you can see your report here" nor "you can search reports here". I think if reporter include such a information (link to search interface or metrics), locale web evangelism community will know about it and start using it. That is, if we want to develop local evangelist team, we should update reporter UI to make people know how to see the reports. (dynamis)
  • of course, we should prepare introduction page for search page, top 10 sites (per locale), and other metrics and should link to the page from reporter addons UI. (dynamis)
• As for type of the problem, in Japan (and I believe in China, Korea etc too) mojibake (garbage text) is the most famous broken case (caused by lack or miss of charset in the content type header). I thinks it's nice to add mojibake in the type list in the reporter (at least for multibyte locales). (dynamis)
• Some problems are caused by addons and reporter should also include installed addons list (not only build config). (dynamis)
• I think volunteer may use Greasemonkey to test, verify and develop code to fix a listed broken page. This code needs to be uploaded to our server database. A mechanism shall be established at back-end to review and verify his code. Meanwhile, we develop an add-on, shipping with Firefox, whenever user open an incompatible page, this add-on will grab the code from server, and fix it. A question is that what a kind of mechanism could ensure this program sustainable? for example, one fixed some pages in CNN.com, a few days later, CNN changed their pages ... (jguo)