Marketplace/Tarako/Testplan
Summary
Tracking bug
Team/Status
Feature | Status | Dev Manager | Developers | QA Lead | QA tester(s) | Status |
Tarako Marketplace | Under Implementation | David Durst [:ddurst] | Chris Van Wiemeersch [:cvan] Chuck Harmston [:chuck] Mathiew Pillard [:mat] Kevin Ngoke [:ngoke] Davor Spasovski [:spasovski] |
Krupa Raj [:krupa] | Victor Carciu [:vcarciu] Iulian Timis [:julianT] Madalin Cotetiu [:madalinc] |
n/a |
Testing Matrix
browsers: None
platforms: Firefox OS
access: within the app
connectivity: EDGE, wifi and 3G
Specifications/Requirements
Project Plan: https://wiki.mozilla.org/Marketplace/Tarako
Technical Design: https://wiki.mozilla.org/Marketplace/Tarako/Technical_design
UX Mocks: https://www.dropbox.com/s/uiaplb5ovsakjb4/tarakoPresentation.pdf
PRD: https://docs.google.com/a/mozilla.com/document/d/10VdjkaqdC-x8QONjzCJftakVIX35bH0jRwZt2i96KAA/edit#
Relevant Bugzilla links
- Tracking bug
- To file new bugs, use the following template
- Please remember to attach logcats and screenshots to all bugs.
Test set up
- If your tarako device is not recognized by your macosx, do the following
- open adb_usb.ini under ~/.android
- add 0x1782 in there and save
- adb kill-server
- adb start-server
- Don't forget to install certs (if not packaged apps won't work). Here are the instructions (on OSX):
- git clone https://github.com/briansmith/marketplace-certs.git
- cd marketplace-certs/
- Put certdb.tgz from https://www.dropbox.com/s/agkljpkh8fia2em/marketplace-stage.cert.tar.gz in the marketplace-certs folder
- tar xvf certdb.tgz
- ./change_trusted_servers.sh <device_ID> "https://marketplace-dev.allizom.org,https://marketplace.allizom.org,https://marketplace.firefox.com"
- ./push_certdb.sh <device_ID> certdb.tmp
- Adb reboot
- NOTE: https://bugzilla.mozilla.org/show_bug.cgi?id=996427
- Install the test app by using the following steps:
- Load http://people.mozilla.org/~cwiemeersch/mktdev.html on your tarako browser
- Under Custom, enter: https://darjeeling.paas.allizom.org/manifest.webapp
- Click Install
Feature Scope
What's in scope?
- Marketplace as a hosted app
- Browsing through app catalog when offline/online
- Installing apps when online
- Queuing apps for installs when offline
- Feedback Submission
- Report Abuse
- Packaged Apps
- Search for apps
- Recommended Search(?)
What's out of scope?
- Login any (including via Persona)
- Access to list of user's installed apps via a 'My Apps' page
- User Reviews (View/Add/Edit/Delete)
- Paid apps
- Region detection
Test Coverage
What's in Scope?
- All user-facing functionality will be thoroughly exercised
What's out of Scope?
- Performance testing
- Load times will be measure via Eideticker: https://bugzilla.mozilla.org/show_bug.cgi?id=994552
- Load testing
- Security testing
Entry Criteria
- Tarako Marketplace (dev) is set up and ready to test
- The Tarako Collection is set up
- Tarako Marketplace dev app can be installed on your FirefoxOS phone
Exit Criteria
- No failing tests in the Smoketest run
- All other bugs have been reviewed and deemed non-blocker
- There are no bugs for Tarako which haven't been triaged
Risks
- Insufficient time to QA an entirely new app
- [Mitigation] lol, krupa! it's like you never learn.
- QA has 2 devices to test: 1 in California and 1 in Romania (shared by three testers)
- [Mitigation] Send a few more phones to Romania.
- Testing is not done with Indian SIMs
- [Mitigation] This has been assessed as a low risk.
Tests
Smoke tests
Smoke tests will be run daily prior to the full functionality testrun. Any failing test in the smoketest will block 04/18 release. Smoke tests are maintained at https://docs.google.com/spreadsheets/d/18MjqHDCLT0LQjebUf_T-Lf3cTosMD6YmiL8kYmEPpFk/edit#gid=178471723 for now. They will be moved to Moztrap at a later date.
Full functionality testrun
Note that all these tests will be moved to Moztrap at a later date.
End-to-End testrun
Devhub changes are yet to land and are out-of-scope for v0.
Reports
A daily email will be sent using the following template
Other References
https://etherpad.mozilla.org/mkt-perf-workweek
https://wiki.mozilla.org/FirefoxOS/Tarako
https://wiki.mozilla.org/B2G/QA/Tips_And_Tricks