Support/Kitsune/features/search-ranking

Purpose

• Improve CTR to about 80%
• Increase the quality of results (based on heuristics)
• Increase helpfulness rating for articles when compared with users sent from Google vs SUMO Search (vs clicking an article on the home page)
• Decrease the rank of searches people click (e.g. the first, 3rd, 8th)
• Decrease the ratio of forum questions per thousand visitors

Outcome

Better search results

Definition

Feature overview

Reports

• Report on the bail rate (people who click on the "Ask a Question" button) and the articles with the highest bail rate.
• A best bet should be regarded as a Click through
• Visualization for searches that have the lowest CTR (frequent searches with a low CTR)
• A report of the top 200 searches per week with their CTR, what the top 5 articles were for those searches (including the share of clicks per result)
  • Should also allow us to manually enter what the accuracy was for each search and calculate the overall accuracy
• Best bet/ tailored search audit report: How often was a certain best bet triggered and what was the CTR.
  • Clicking on a best bet should show you what search terms were used to trigger it.
• A report that lists words the dictionary didn't recognize by frequency (to add those non-words as synonyms)

Features

• Capture the path someone has taken before they used the search

We need two types of features:

1. Type A: A way to boost a whole range of articles based on certain criteria
2. Type B: A way to manipulate search results for a very specific query selector

Type A

• Boosting certain type of results (KB, forum)
• Boosting based on tags.
• Boosting trending articles/threads.
• Boosting articles/threads based on helpfulness rating.

Type B

• A way to build selectors for tailored search: [synonym] 'stem match' "string match" AND OR NOT ()- Active and Passive
  • [synonym]: Anything in brackets should be seen as equivalent to the search string
  • 'stem match': stemming should be considered in matching
  • "string match" should match exact string
  • AND OR NOT: Logical operators
  • (): To separate queries that are connected by the operators
  • Active and Passive: Active queries manipulate the search results, passive queries are only for metrics tracking.
• Topic restrictions: only show certain results when certain topics are active (may come from path taken or from facet selected)
• Time constraints: We need to be able to activate queries for a certain time period.
• We need a tool that lets us build up the results list for certain queries. Needs to let us define
  • only a few articles to show on top of otherwise regular results
  • a complete list of results to be shown (with nothing added after that)
• A way to change the display name or description for ranked searches
• Best bets: a special box that shows up on top of the search results with arbitrary content.
• A process wizard builder.

Design

Functional Spec

Things that will need to be figured out

• Which metric sources do we already have access to, and which ones do we still need?