QA/Browser Technologies/2011-11-17

« previous mtg | Browser Technologies Home | next mtg »

Discussion Items

• Action items from last week
  • Take a look below, and see what needs follow up. Please add new notes for this week.

• BHAG discussion (Matt)
  • Sync - Automate our world with Mozmill/TPS (tracy)

• Mobile (kevin)

• Services (James)
  • John Morrison has joined the Services QA team. Will be sharing duties in Sync, Sync Server, and BrowserID.
  • BrowserID build-out of Dev, Stage, Test/CI, and Prod is moving forward. Dev and Stage will be completed first, followed by Prod, then Test/CI.
  • BrowserID Dev, QA, and OPs teams are ramping up to prepare for the full release to Production by the end of Q4.
  • Discussions have started across teams to come up with automation solutions for BrowserID: client-side and server-side.

• Community Growth
  • How are we tracking with our community team goals?

Project Status

Fennec

Execution (kevin, aaron)

• Native Fennec
  • Tracking feature testing
  • In process of merging to mozilla-central

Specialized (naoki)

• Crash report simplified down to native UI
  • bug 702624 - Crash Reporter crashes
    • will most likely disappear if screenshot is removed

• Performance:
  • Manual Perf testing found regression in 9th ~ 10th build bug 702829
  • Automation : still needs to get logged publically, almost there.

• Responsiveness group might get started by Taras? Need to ping him on that.

• Newsletter Template selected, content edited, need to combine and push out thurs/friday

Sync

Client (Tracy)

• No client trains the past two weeks
• Using extra time to catch up on test case writing, Fennec native Sync test planning and other misc tasks

Server (James)

• Sync Server 11-03-2011 deployment of server-core (2.6.x) to syncstorage
• Maintenance Release: Sync Server 11-09-2011: Restart of syncstorage, keyexchange, ap, reg, and sreg gunicorns to change CEF logging destination
• Maintenance Release: Sync Server 11-15-2011: LDAP master failover

Test pilot (tracy)

• no extension work planned
• new broken extension study due soon. Will smoketest it when available.

BrowserID (james)

• Completed Train 12: 11-03-2011: bug 699962 - QA and deploy BrowserID train-2011.11.03 to production
• Completed Train 13: Bug 701934 - QA and deploy BrowserID train-2011.11.10 to production
• Train 14 will include some testing on the new Dev and Stage environments

Pancake (Naoki)

• Unit Test Automation through Addon/Jetpack
  • Met with Max, got them to run, but something still looks odd
• doing Addon testing
  • still getting 500 error
• requested virtual environment to do testing for pancake to see server side issues as well; possible server side automation
• Future Pancake version ideas are being hashed out in Toronto (iBarlow, Madhava, Stuart)
• Social Aspect (twitter/facebook search) being added to staging

WebAPI (John)

• test plan
  • WIP available at https://wiki.mozilla.org/QA/Browser_Technologies/WebAPI
• test pages
  • now have battery, Vibrator, Camera (web sms, telephony, mozindexxeddb are in the works)
• native apps
  • now have (for android) battery, device info (Geolocation, Accelerometer are next on the list)
• Litmus [1]
  • Battery suite is complete. Vibrator, camera are next
  • CC is still having some issues regarding rendering all testcases
  • have been using Android framework for Android (go figure) am considering using QT for native apps for Windows, Mac, Linux
• automation frameworks
  • Robocop got demo from Trevor regarding; hope to integrate it with my battery/info classes and begin playing around soon
  • Marionette got first WIP build from MDAS; it's on my short list. Since it uses Python; would be interesting to try to integrate it with Lettuce (BDD)

Round Table

• Let's make sure Native Fennec and Native Fennec Sync projects stay informed of each others status
  • Tracy will begin attending Mobile team meetings.

Notes

Action Items

Action Items from This Meeting

• jbonacci and nhirata to discuss at least a sync newsletter if not a btech newsletter

Past Action Items

• [DONE] mobile team to meet at 1:00
• [DROPPED] b-tech team: please try out crowdbot for fennec xul; and give feedback
  • martijn : send out link to the crowdbot to the team
• tchung : talk about impact of other communities in the QA community meeting
• tchung : give bullets for browser tech on visibility for outreaching; show communities for each individual goals for people.
• tchung and mevans to talk about crowdbot off line
• [DONE] jbonacci : dashboard would be good idea. Spreadsheet is fine as well; relevant bugs, status, checklist items.
  • https://wiki.mozilla.org/QA/BrowserID/OPsBuildOut

Raw Notes

Crowdbot dropped : 
new zippity working; pulling together the new zippity
Not sure where tony is on the other items.
jbonacci finished his dashboard; link in the past action items section.

BHAG:
tracy : 
	mozmill and tps, we can nearly automate all test cases.
	need another person or way more time to get this done.
mobile : 
	automation solutions
	engagement / Community
browser id : 
	automation solutions
	engagement / Community
pancake : 
	automation solutions
	engagement / Community	

Need to get audience for Services side
- who the community is 
- how to reach out to them
- it's a little different from client

Services need more technical community that's interested in bouncing ideas in hacking and testing which may be tricky
Come up with some ideas for single or small groups through various sites that specialize in server side testing
Trying to reach a community, not sure of
trying to think of what's already running in terms of conferences or community with server ops maybe?
hacking community : hacking for security privacy issues.

Again trying to find the people would be hard

For Browser Tech, not sure.
- hard to accomplish each of these goals due to youth of projects (for mobile, sync/browser id, pancake)
- one of the goals for next quarter is to get pancake out

Anything for process?
- chance to reset
- figure out the processes : risk areas, etc.
- rotational program for each person for a round robin [ that way we don't get tester fatigue; coverage ]
- newsletter for each group

- newsletter : cross training, bi weekly.  based on btech meeting?  should we have it through out.
	- try to take as little time as possible to spend on the newsletter

- do we need a SLA?  Need to formulate structure for SLA : response to testing ?
- Some kind of more formalized process with the dev

Mobile : 
- Robocop would help out with.  
- exploratory in the beginning, manual verifications of nonautomated tests, remaining 10 %
- Crowdtest : martijn Crowdbot ported to native UI?  Need agreement from Martijn
- Zippity ported already
- Something around Devices?  
- Performance : automation, compare against other browser.
- getting community with devices that we don't have ; community device owner
- getting devices to contributors; people that have devices that we don't have.
- Resources : move 90 % automation?  -> verify the automations, migrate tests litmus to automation, write litmus to automation, maintainence, increase in waverly help
- if we have some slack we can shift people to identify testing, browser id, etc.

Services:
- QA is running a fully integrated Test system, consisted of VM's and physical hardware
	- sync server environments, mix of VMs and physical hardware that's already happening
	- could be a scaling thing, in terms of how to cover this?
	- automation picture is bigger?
	- cut thing?  need tony to check out
	- rest of these seems like processes, which most of them we already starting to do.
- he wants us to be doing; we have various test environments: 
	- requesting QA have their own environment for testing
	- borrowed stuff from Ops team
	- 2012 : make our own sync environment : browser ID environment rather than staging environment
	- No server side automation.
	- nightly unit testing; that's only nightly?  per train?  not sure if it's nightly testing?
	- will soon.  on the sync server side it is happening Tarak has it set up
	- Browser ID ones are javascript
	- sync server side it's python
	- sync is not moving to pyramid
	- sync is all javascript?  TPS stuff; some kind of nightly test that's being done... disregard.  it's client side.

BHAG:
Need a clearer idea what BHAG is
	- context: for the next year
	=> 2 big goals : automation, and community
Since browser ID has started, most of the stuff has been running on VM.  They are creating full facing environments for release
for staging, production, dev.  Work in progress.  All 4 environments will be up and running by end of Nov.  (est)
	=> end of Q4 go live
	

	How do we track the community
	=> we have one active community member for mobile team
	=> sync no progress as of yet.
actionitem: => sync newsletter to be discussed between james/oki
	
Status : read wiki
	
Round Table : fennec table to make sure that sync and fennec stay in sync.
Are they bundling as two separate applications?
Not sure how it's going to go.
The release of first version of native is sync is a blocker if it isn't there.
- it might not be a hard blocker for having sync there.
- idea is tighter knit strategy for this.