Support/Kitsune/features/search-ranking

Purpose

• Improve CTR to about 80%
• Increase the quality of results (based on heuristics)
• Increase helpfulness rating for articles when compared with users sent from Google vs SUMO Search (vs clicking an article on the home page)
• Decrease the rank of searches people click (e.g. the first, 3rd, 8th)
• Decrease the ratio of forum questions per thousand visitors

Outcome

Better search results

Definition

Feature overview

Reports

• Report on the bail rate (people who click on the "Ask a Question" button) and the articles with the highest bail rate.
• A best bet should be regarded as a Click through
• Visualization for searches that have the lowest CTR (frequent searches with a low CTR)
• A report of the top 200 searches per week with their CTR, what the top 5 articles were for those searches (including the share of clicks per result)
  • Should also allow us to manually enter what the accuracy was for each search and calculate the overall accuracy
• Best bet/ tailored search audit report: How often was a certain best bet triggered and what was the CTR.
  • Clicking on a best bet should show you what search terms were used to trigger it.
• A report that lists words the dictionary didn't recognize by frequency (to add those non-words as synonyms)

Features

• Capture the path someone has taken before they used the search or any facets a user has selected prior to running a search.

We need two types of features:

1. Type A: A way to boost a whole range of articles based on certain criteria
2. Type B: A way to manipulate search results for a user/admin defined query or set of queries

Type A

• Boosting results based on article source (KB, forum, marketing?)
• Boosting based on tags.
• Boosting trending articles/threads.
• Boosting articles/threads based on helpfulness rating.

Type B

• An interface to build query logic for tailored search: [synonym] 'stem match' "string match" AND OR NOT ()- Active and Passive
  • [synonym]: Match on the word or any synonyms including stemming
  • 'stem match': Match only the word in and stems. Do NOT match synonyms or stem matches of synonyms.
  • "string match": should match exact string. No synonym OR stem matches
  • AND OR NOT: Logical operators
  • (): To separate queries that are connected by the operators
  • Active and Passive: Active queries manipulate the search results, passive queries are only for metrics tracking.
• Topic restrictions: only show tailored searches when certain topics are active (may come from path taken or from facet selected)
• Time constraints: The option to set start and end dates/times for tailored searches.
• The tool should also let us define the results to return when each tailored search is invoked. Needs to let us define
  • only a few articles to show on top of otherwise regular results
  • a complete list of results to be shown (with nothing added after that)
• A way to change the display name or description for ranked searches
• Best bets: a special box that shows separately from the search results list with user/admin defined content. ie, links to articles, informational text, external links,etc
• A process wizard builder.

Design

Functional Spec

Things that will need to be figured out

• Which metric sources do we already have access to, and which ones do we still need?