View source for Firefox/Projects/Network Error Pages/Test Plan

= Network Error Pages =
*'''Development Status:''' - UNKNOWN (6/24/2010) 
*'''Feature Testing:''' - STALLED (6/24/2010) 
*'''Team:''' cbartley (dev), ashughes (qa)
*'''Tracking Bugs''': {{bug|482874}}
=== Feature Description  ===
When a requested web page cannot be loaded, Firefox should display an error page that not only describes the problem, but which also presents the user with tools to resolve the issue. It also makes sense to extend this functionality to webserver 404 (File not found) responses as well.

= [[#Feature Release Readiness Assessment_Details|Feature Release Readiness Assessment]] =

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone.

{| class="fullwidth-table"
|-
! Milestone
! Assessment
|-
| [[#Beta1]]
| '''NO GO'''
|-
| [[#Beta2]]
| N/A
|-
| [[#Beta3]]
| N/A
|}

[[#Feature_Description|top]]
= [[#Feature_Documentation_Details|Feature Documentation]] =

{| class="fullwidth-table"
|-
! Item
! Description
! Status
|-
| [[#Project_Wiki]]
| Wiki Links to all feature related entries
|
|-
| [[#Developer_Links|#Developer_Links (blogs)]]
| Developer links to feature related sites
| 
|-
| [[#Other_Docs]]
| Web links to feature related sites 
|
|-
| [[#Developer_QA_Review]]
| Details from developer and qa discussions regarding feature test strategies and issues.
|
|}

[[#Feature_Description|top]]

= [[#Feature_Bug_Management_Details|Feature Bug Management]] =

{| class="fullwidth-table"
|-
! Item
! Description
! Status
|-
| [[#Bug_Tracking]]
| Top level bugs tracking feature  
|
|-
| [[#Bug_Verification]]
| Feature bugs that need verification  
| 
|-
| [[#Bug_Triage]]
| Links triage bug tasks  
|
|}

[[#Feature_Description|top]]

= [[#Feature_Test_Items_Details|Feature Test Items]] =

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. 

{| class="fullwidth-table"
|-
! Test Item
! Description
! Covered By
! Status
|-
| [[#404_Pages|404 Pages]]
| Page not found error page
| Developer Tests, QA Tests, Beta tester coverage
| 
|-
| [[#Suggestions|Suggestions]]
| Alternate site suggestions
| Developer Tests, QA Tests, Beta tester coverage
|
|-
| [[#Search|Search]]
| Pre-populate site-search box & button, primarily used on 404 error pages, terms derived from URL 
| Developer Tests, QA Tests, Beta tester coverage
|
|-
| [[#Localization]]
| Feature localization 
|
|
|-
| [[#Accessibility]]   
| Feature accessibility
| 
|
|-
| [[#Plugins]]
| Plugins compatibility  
|
|
|-
| [[#Addons]]
| Addons compatibility  
|
|
|-
| [[#Topsites]]
| Top internet sites compatibilities  
|
|
|}

[[#Feature_Description|top]]


= [[#Feature_Tests_Details|Feature Tests]] =
== [[#Automated_Tests_Details|Automated Tests]] ==
{| class="fullwidth-table"
|-
! Item
! Description
! Status
|-
| [[#Developer_Tests]]
| Links to automated developer tests
|
|-
| [[#Mozmill_Tests]]
| Links to automated mozmill feature test cases  
| 
|}

== [[#Manual_Tests_Details|Manual Tests]] ==
{| class="fullwidth-table"
|-
! Item
! Description
! Status
|-
| [[#Smoke_Tests]]
| link to smoke tests
|
|-
| [[#Regression_Tests]]
| link to BFT and/or regression tests
|
|-
| [[#Functional_Tests]]
| link to FFT and/or complete functional tests
|
|}

[[#Feature_Description|top]]

= [[#Community_Test_Events_Details|Community Test Events]] =
{| class="fullwidth-table"
|-
! Item
! Description
! Status
|-
| [[#Testdays]]
| Links to test day event results for feature
|
|-
| [[#Bugdays]]
| Links to bug day event results for feature
|
|-
| [[#Meetups]]
| Links to Meetup events for feature
|
|}
[[#Feature_Description|top]]

= [[#Feature_Documentation|Feature Documentation Details]] =
== Project Wiki ==
* [https://wiki.mozilla.org/Firefox/Projects/Network_Error_Pages Project page]

[[#Feature_Documentation|top]]

== Developer Links ==
* [https://wiki.mozilla.org/Firefox/Projects/Network_Error_Pages#Design Design mockups]
* [https://wiki.mozilla.org/Firefox3.1/Sprints/Network_Error_Pages/Error_types Error types]
* [http://jboriss.wordpress.com/2009/03/10/improving-everyones-favorite-feature-404/ boriss' blog post]

[[#Feature_Documentation|top]]

== Other Docs ==
* Provide links to all feature related developer links to blogs and other internet sites

[[#Feature_Documentation|top]]

== Developer QA Review ==
* Do we have automated tests for the feature?
* What do they cover?
* What do they not cover?
* How well do they cover the feature?
* What are the important areas we should focus on?
* What are the dependencies?
* What is our comfort level with this feature in its current state?
* What feedback would you like from QA?

[[#Feature_Documentation|top]]

= [[#Feature_Release_Readiness_Assessment|Feature Release Readiness Assessment Details]] =
* Not sure of status for ANY Firefox 4 iteration

== Beta1 ==

== Beta2 ==

== Beta3 ==

= [[#Feature_Bug_Management|Feature Bug Management Details]] =
== Bug Tracking ==
* Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.

{| class="fullwidth-table"
|-
! Query Name
! Description
|-
| {{bug|482874}}
| Top-level tracking bug
|}
  
[[#Feature_Bug_Management|top]]

== Bug Verification ==
* Feature bugs that need verification  

[[#Feature_Bug_Management|top]]

== Bug Triage ==
* Bug triage information

[[#Feature_Bug_Management|top]]

= [[#Feature_Test_Items|Feature Test Items Details]] =

== 404 Pages ==
* Override 404 pages containing less than 250 characters served by Apache (minimizes chances of overriding custom error pages)
* Do not override 404 pages containing greater than 250 characters
* Ability to view the original 404 error page
* Ability to view the Firefox 404 override page manually when it did not automatically override 
* Do the error pages validate?
* Is the spelling/grammar correct?
* Is the spelling/grammar correct for localizations?
* Does the layout look decent? 

[[#Feature_Test_Items|top]]

== Search ==
* Pre-populate site-search box & button, primarily used on 404 error pages, terms derived from URL
** Appears for 404 error overrides
** Does not appear for DNS Not Found errors
** Search box is pre-populated when it appears
** Clicking the search button triggers a search
** Use the user's preferred search engine if it makes sense to do so
*** Google, Yahoo, MSN make sense
*** Wikipedia, Amazon, Ebay do not 
* Global Web Search
** Similar to Search but this works for both 404 and DNS error pages, search terms derived from URL.

[[#Feature_Test_Items|top]]

== Suggestions ==
* Alternate URLs i.e. "Did you mean...", drawn from Places DB using "edit distance" criteria, only appears if match is pretty close
** Appear when there is a very close match
** Does not appear when there is not a close match
** Suggested URLs are well-formed
** Suggested URLs work 

[[#Feature_Test_Items|top]]

== Localization ==
* Details of feature localization test requirements 

[[#Feature_Test_Items|top]]

== Accessibility ==   
* Details of feature accessibility test requirements

[[#Feature_Test_Items|top]]

== Plugins ==
* Details of plugins compatibility test requirements 

[[#Feature_Test_Items|top]]

== Addons ==
* Details of addons compatibility  

[[#Feature_Test_Items|top]]

== Topsites ==
* Details of top internet sites test requirements

[[#Feature_Test_Items|top]]

= [[#Feature_Tests|Feature Tests Details]] =
== [[#Automated_Tests|Automated Tests Details]] ==
'''NOTE: Most of these may be best served as an automated and manual test'''
* If a 404 error page meets the constraint it should be overridden.
* If a 404 error page does not meet the constraint, it should *not* be overridden.
* Make sure Suggestion appears when there is a very close match.
* Make sure Suggestion does not appear when there is not remotely close match.
* Verify that the URL is well-formed, and that the anchor element displaying it works.
* Make sure Search appears for 404 error overrides.
* Make sure Search does *not* appear for DNS Not Found errors.
* Verify that the search box is pre-populated when it appears, and verify that the button actually triggers the search.
* Verify that the search uses the user's preferred search engine when it makes sense (e.g. Google or Yahoo, but not Wikipedia).
* Verify that the search is actually constrained to the site.
* Make sure that Global Websearch appears both for 404 error overrides and DNS Not Found errors.
* Make sure that the Global Websearch search box is pre-populated with search terms, and verify that the button actually triggers the search.
* Make sure the Global Websearch search uses the user's preferred search engine when it makes sense (e.g. Google or Yahoo, but not Wikipedia).
* Verify that the UI control (button, link, etc.) appears in the original error page, and verify that it works.
* [Manual Error Page] Trigger a conventional 404 page that will not be overridden (just too large, etc.).  Verify that the UI control (button, link, etc.) is visible in the browser chrome, and verify that it will show the 404-override page.
* Do the error pages validate? (I don't know if this is critical, but if we can automate it, it won't hurt to test.)

=== Developer Tests ===
* Links to automated developer tests

[[#Feature_Tests|top]]

=== Mozmill Tests ===
* Links to automated mozmill feature test cases  

[[#Feature_Tests|top]]

== [[#Manual_Tests|Manual Tests Details]] ==
'''Potentially all of the above plus...'''
* Is the spelling and grammar correct? (Obviously localization dependent).
* Does the layout look decent? (Keep in mind that error pages may vary in layout from error to error since only some features will apply to each error page).

=== Smoke_Tests ===
* links to litmus smoke tests or description

[[#Feature_Tests|top]]

=== Regression_Tests ===
* links to litmus BFT and/or regression tests description

[[#Feature_Tests|top]]

=== Functional_Tests ===
* links to litmus FFT and/or complete functional tests description

[[#Feature_Tests|top]]

= [[#Community_Test_Events|Community Test Events Details]] =
== Testdays ==
* Links to test day event results for feature

== Bugdays ==
* Links to bug day event results for feature

== Meetups ==
* Links to Meetup events for feature