Loop/Test: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(adding links to external content (WIP))
(Link: Automated Testing)
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Overview =
= Overview =
The purpose of this wiki is to provide a reference for testing Hello Firefox (Loop) from both client & server side.
This document is not intended to be exhaustive, but more a 'how-to' reference of test tools, deployment verification, end-2-end (e2e) testing, etc.  For test strategy, please refer instead to: [[Loop/QA]]


NOTE:
== Summary ==
This content needs further consolidation, but placing existing wiki content here for now...


= Deployment Verification =
The purpose of this wiki is to provide a reference for manually testing Hello Firefox (Loop) from both client & server side.  This document is not intended to be exhaustive, but more a 'how-to' reference of test tools, stack check, end-2-end (e2e) testing, etc.   
== Description ==
Deployment verification tests are a kind of 'kick-the-tires', server-side test to verify that a service is up-and-running
where we expect it to be, with the right configuration, URLs, etc.   
I.e. verifying everything we can short of doing a load or e2e test.


== How-to ==
These tests should be performed in both STAGE and PRODUCTION environments.


see:
[[#toc]]


[[QA/Services/LoopTestEnvironments]]
== Notes ==
* Everything documented here is in the process of being automated.
* This content needs further consolidation, but placing existing wiki content here for now...
* For test strategy, please refer instead to: [[Loop/QA]]


= Loadtesting =
== Description ==
Loadtest are server-side tests performed against loop-server to verify scalability of the service.
What happens when the service is subjected to load?
* Does the service crash?
* Does it throw errors?
* Does it perform slowly?
* etc.


== How-to ==
[[#toc]]
These tests should only be performed in STAGE environment.


see:


[[QA/Services/LoopServerLoadTesting]]
= Testing Categories =
 
Testing can be divided roughly into 4 categories:
* [[#Stack Check|Stack Check]]
* [[#End-2-End Testing|End-2-End Testing]]
* [[#Loadtesting|Loadtesting]]
* [[#Functional Testing|Functional Testing]]
 
 
[[#toc]]
 
 
== Stack Check ==
 
'''Description'''
 
Stack check serves as a kind of 'kick-the-tires', server-side test to verify that a service is up-and-running, where we expect it to be, with the right configuration, URLs, etc. 
i.e. verifying everything we can short of doing a load or e2e test.
 
# verify that the correct release has been deployed
# verify that basic services are up-and-running
 
 
'''How-to''' ---> [[Loop/Test/Stack Check]]
 
 
[[#toc]]
 
 
== End-2-End Testing ==
 
'''Description'''
 
End-2-end (ore: "e2e") tests serve as a kind of comprehensive, client-side 'smoke test' using a service-enabled product to verify proper <u>basic function</u> from the user's perspective.  By performing the same actions a user would, you can also ensure that the client is correctly consuming all backend services.


= End-2-End Test =
== Description ==
End-2-end (e2e) tests serve as a kind of client-side 'smoke test' using a service-enabled product to verify proper basic function from the user's perspective.


End-2-end tests can also be used as a kind of sanity test for both STAGE and PRODUCTION service deployments.  This kind of testing may require some configuration changes to 'point' the client application to either the STAGE or PRODUCTION service, but (in general), the test methodology would remain the same.
End-2-end tests can also be used as a kind of sanity test for both STAGE and PRODUCTION service deployments.  This kind of testing may require some configuration changes to 'point' the client application to either the STAGE or PRODUCTION service, but (in general), the test methodology would remain the same.


== How-to ==
These tests should be performed in both STAGE and PRODUCTION environments.


see:
'''How-to''' --->
* [[Loop/Test/E2E-Hello]]
* [[Loop/Test/E2E-msisdn-gateway]]
 
 
[[#toc]]
 
 
== Loadtesting ==
 
'''Description'''
 
<i>verify that service can handle adequate load/traffic</i>
# verify that the service is robust (won't crash under load)
# verify that the service can scale (load-balancing, etc.)
 
 
'''How-to''' ---> [[Loop/Test/Loadtesting]]
 
 
[[#toc]]
 
 
 
== Functional Testing ==
 
'''Description'''
* verify functionality of all client-side features  (<span style="color:#FF0000">not covered in this wiki</span>)
 
'''Other Wikis'''


[[QA/Loop/E2E]] - This page documents the test strategy of verifying Firefox Hello is working end-to-end.
<i>These may be out-of-date</i>
* [[Loop/QA]]
* [[QA/Loop]]


[[Loop/Try_Loop]]


[[Loop/QA/MSISDN_testplan]]
[[#toc]]
 


= Reference =  
= Reference =  
[[Loop/Architecture]]
 
[[Loop/QA]] - obsolete?
== Wikis ==
[[CloudServices/Loop]] - update test links
* [[Loop/Architecture]]
* [[Loop/Test/Monitoring]]
* [[Loop/Automated_Testing]]
* https://wiki.mozilla.org/Loop/Architecture/ID
* https://wiki.mozilla.org/QA/Services/FxATestEnvironments
* https://wiki.mozilla.org/QA/Services/TSVerifierSyncTestEnvironments
* https://wiki.mozilla.org/CloudServices/Loop
* https://wiki.mozilla.org/CloudServices/Loop/Server
* https://wiki.mozilla.org/Loop/Telemetry
* Loop MVP Features: https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AlJmiyFngeSwdGJEVnItQ2NVbGx6NXdjbVBUYmIxSGc#gid=0
* https://people.mozilla.org/~dhenein/labs/loop-docs/
* https://people.mozilla.org/~dhenein/labs/loop-mvp-spec/
* https://wiki.mozilla.org/Loop/Architecture/MVP
* settings: https://wiki.mozilla.org/QA/Services/LoopTestEnvironments#Loop_Mock_Server_Stage_Details
* Puppet Config: https://github.com/mozilla-services/puppet-config
* Cloud Formation: https://github.com/mozilla-services/svcops
 
 
[[#toc]]
 
 
== Docs ==
* https://docs.services.mozilla.com/loop/index.html
* https://docs.services.mozilla.com/loop/apis.html
 
 
[[#toc]]
 
 
== Github Repos ==
* https://github.com/orgs/mozilla-services/teams/loop
* https://github.com/mozilla-services/loop-server
* https://github.com/mozilla/loop-client
* https://github.com/mozilla-services/msisdn-gateway
* https://github.com/mozilla-services/msisdn-gateway/blob/master/API.md
* https://github.com/mozilla/videur
* https://github.com/mozilla/fxa-auth-server/wiki/API-extensions-for-supporting-MSISDN-verification-in-FxA-auth-server
* https://github.com/mozilla/gecko-projects/tree/elm/browser/components/loop/standalone
* https://github.com/mozilla-services/svcops
* https://github.com/mozilla-services/puppet-config
 
 
[[#toc]]
 
 
== Bugzilla ==
* https://bugzilla.mozilla.org/enter_bug.cgi?product=Loop&component=Server
** Other Products
** Product: Loop
** Component: General/Client/Server
 
* https://bugzilla.mozilla.org/enter_bug.cgi?product=Mozilla%20Services@component=MobileID
** Client Software
** Product: Mozilla Services
** Component: MobileID
 
 
[[#toc]]

Latest revision as of 17:30, 2 July 2015

Overview

Summary

The purpose of this wiki is to provide a reference for manually testing Hello Firefox (Loop) from both client & server side. This document is not intended to be exhaustive, but more a 'how-to' reference of test tools, stack check, end-2-end (e2e) testing, etc.


#toc

Notes

  • Everything documented here is in the process of being automated.
  • This content needs further consolidation, but placing existing wiki content here for now...
  • For test strategy, please refer instead to: Loop/QA


#toc


Testing Categories

Testing can be divided roughly into 4 categories:


#toc


Stack Check

Description

Stack check serves as a kind of 'kick-the-tires', server-side test to verify that a service is up-and-running, where we expect it to be, with the right configuration, URLs, etc. i.e. verifying everything we can short of doing a load or e2e test.

  1. verify that the correct release has been deployed
  2. verify that basic services are up-and-running


How-to ---> Loop/Test/Stack Check


#toc


End-2-End Testing

Description

End-2-end (ore: "e2e") tests serve as a kind of comprehensive, client-side 'smoke test' using a service-enabled product to verify proper basic function from the user's perspective. By performing the same actions a user would, you can also ensure that the client is correctly consuming all backend services.


End-2-end tests can also be used as a kind of sanity test for both STAGE and PRODUCTION service deployments. This kind of testing may require some configuration changes to 'point' the client application to either the STAGE or PRODUCTION service, but (in general), the test methodology would remain the same.


How-to --->


#toc


Loadtesting

Description

verify that service can handle adequate load/traffic

  1. verify that the service is robust (won't crash under load)
  2. verify that the service can scale (load-balancing, etc.)


How-to ---> Loop/Test/Loadtesting


#toc


Functional Testing

Description

  • verify functionality of all client-side features (not covered in this wiki)

Other Wikis

These may be out-of-date


#toc


Reference

Wikis


#toc


Docs


#toc


Github Repos


#toc


Bugzilla


#toc

Retrieved from "https://wiki.allizom.org/index.php?title=Loop/Test&oldid=1082747"