Support/Q2HelpfulnessTests

This quarter we gave ourselves the goal of increasing the average helpfulness score of knowledge base articles by 2%. I think there are two basic ways to approach that – make articles more helpful (obviously) and get more people to an article that helps them on the first shot (thereby decreasing the likelihood of rating an article unhelpful).

Our plan is to run multivariate tests on the support home page (getting people to the right article) and on 2-3 of the top support articles (making things more helpful).

Improving performance

Improving performance - The initial idea was test whether using a simplified header and footer to increase page load times will lower their bounce rate. In addition, improving the load times should make using and navigating the site easier. After looking at the size and loading times for various elements of the home page, it appears that removing the large background image (54k) at the top of the page will make the biggest impact. Simplifying the navigation and removing the mobile section (in the in-product version of the page), for example, only saves 13.5k and is mainly due to having removed the image in the mobile section.

Browsing Categories

Another idea that might help get people to the correct article faster would be to offer a better browsing option. Currently the home page is designed around searching with an additional list of popular articles for users who prefer to browse. One idea we'd like to test is to replace that list of popular articles with a few category sections. They could have headings like Crashes, Troubleshooting, or Tutorials with a short description of what other things are included. These would then link to a category page.

We might also test whether categories work better above search

Article tests

For our tests on KB articles, we want to use some of our most trafficked pages. We also want to choose an article with a high helpfulness rating and one with a low rating.

• The Private browsing article is our 2nd most popular article (58763 views, wk of 4/12) and has a helpfulness rating of 85%.
• The Profiles article is our 5th most popular article (24640 views, wk of 4/12) and has a helpfulness rating of 37%
• The Pop-up blocker article is our 9th most popular article (17954 views, wk of 4/12) and has a helpfulness rating of 33%.

If we only pick two pages to test we might go with Profiles over Pop-up blocker because it gets almost 40% more traffic and has a similar helpfulness rating.