QA/Browser Technologies/2011-12-01

« previous mtg | Browser Technologies Home | next mtg »

Discussion Items

• BHAG discussion (Matt)

Project Status

Fennec

Execution (kevin, aaron)

• Mobile
  • Mobile Developers are looking for additional testing in regards to recently landed patches (bug 627842) re: improving font size readability on Firefox for Android. Please file bugs if pages you view have odd looking font inflation (size) issues.
  • Encourage and continue to audit and track testing of feature coverage here.

Specialized (naoki)

• CrashKill :
  • reporting using P1 ~ P5 status is over. Using Topcrash, reproducible and QA+ for tracking, not sure how Topcrash/reproducible is going to change anything since that's always the way that it's been done
  • going to start graphing the number of crasher bugs in component areas to raise awareness of crash bugs

• Performance
  • Automation seems about on par with manual testing for local pages, not sure about automation. Probably need to compute a graph with mean and variance.
  • QA to own testing for the video performance testing; need to talk to Erin about video equipment/current methods
  • initial query of where we can go with tpan/tzoom; possible hooks within Robotium; also need to see how Eideticker might help. Eideticker may be done towards the end of this year.

• Memshrink :
  • currently nothing to report; need to investigate RSS/Profiling

• Newsletter
  • Need to talk to Al about emailing through generic email account
  • this week's newsletter to be also tweeted about, posted on yammer, facebook, and several other places (including the normal channels of planet, qmo, btech, mobile)

Sync

Client (Tracy)

• no client trains
• been working on test cases and testing tips videos

Pancake (Naoki)

• Project Status:
  • Discover tab implemented and pushed to staging; twitter /facebook login also pushed, but incomplete.
  • Addon implemented; blocking bug of initial bookmark+history being pushed to server bug 679133
  • Addon to get fixed, need server changes to support the change. Once it gets fixed, some retesting and release of the addon with the reset of server to take place.
  • Revamp of concept being worked upon still

• Testing:
  • Testing has been done on the addon and staging server: see bottom of QA notes
  • Testing environment : Need to setup VM testing environment

• Automation
  • Unit Test Automation demoed to tchung; need more tests added
  • Selenium test from AMO received, need to port to Selenium 2 and section off the BrowerID login portion for Browser ID (ETA end of next week?)
  • Instruments portion : delayed

Notes

Action Items

Action Items from This Meeting

• jbonacci and nhirata to discuss at least a sync newsletter if not a btech newsletter

Past Action Items

• [DONE] mobile team to meet at 1:00
• [DROPPED] b-tech team: please try out crowdbot for fennec xul; and give feedback
  • martijn : send out link to the crowdbot to the team
• tchung : talk about impact of other communities in the QA community meeting
• tchung : give bullets for browser tech on visibility for outreaching; show communities for each individual goals for people.
• tchung and mevans to talk about crowdbot off line
• [DONE] jbonacci : dashboard would be good idea. Spreadsheet is fine as well; relevant bugs, status, checklist items.
  • https://wiki.mozilla.org/QA/BrowserID/OPsBuildOut

Raw Notes

Crowdbot dropped : 
new zippity working; pulling together the new zippity
Not sure where tony is on the other items.
jbonacci finished his dashboard; link in the past action items section.

BHAG:
tracy : 
	mozmill and tps, we can nearly automate all test cases.
	need another person or way more time to get this done.
mobile : 
	automation solutions
	engagement / Community
browser id : 
	automation solutions
	engagement / Community
pancake : 
	automation solutions
	engagement / Community	

Need to get audience for Services side
- who the community is 
- how to reach out to them
- it's a little different from client

Services need more technical community that's interested in bouncing ideas in hacking and testing which may be tricky
Come up with some ideas for single or small groups through various sites that specialize in server side testing
Trying to reach a community, not sure of
trying to think of what's already running in terms of conferences or community with server ops maybe?
hacking community : hacking for security privacy issues.

Again trying to find the people would be hard

For Browser Tech, not sure.
- hard to accomplish each of these goals due to youth of projects (for mobile, sync/browser id, pancake)
- one of the goals for next quarter is to get pancake out

Anything for process?
- chance to reset
- figure out the processes : risk areas, etc.
- rotational program for each person for a round robin [ that way we don't get tester fatigue; coverage ]
- newsletter for each group

- newsletter : cross training, bi weekly.  based on btech meeting?  should we have it through out.
	- try to take as little time as possible to spend on the newsletter

- do we need a SLA?  Need to formulate structure for SLA : response to testing ?
- Some kind of more formalized process with the dev

Mobile : 
- Robocop would help out with.  
- exploratory in the beginning, manual verifications of nonautomated tests, remaining 10 %
- Crowdtest : martijn Crowdbot ported to native UI?  Need agreement from Martijn
- Zippity ported already
- Something around Devices?  
- Performance : automation, compare against other browser.
- getting community with devices that we don't have ; community device owner
- getting devices to contributors; people that have devices that we don't have.
- Resources : move 90 % automation?  -> verify the automations, migrate tests litmus to automation, write litmus to automation, maintainence, increase in waverly help
- if we have some slack we can shift people to identify testing, browser id, etc.

Services:
- QA is running a fully integrated Test system, consisted of VM's and physical hardware
	- sync server environments, mix of VMs and physical hardware that's already happening
	- could be a scaling thing, in terms of how to cover this?
	- automation picture is bigger?
	- cut thing?  need tony to check out
	- rest of these seems like processes, which most of them we already starting to do.
- he wants us to be doing; we have various test environments: 
	- requesting QA have their own environment for testing
	- borrowed stuff from Ops team
	- 2012 : make our own sync environment : browser ID environment rather than staging environment
	- No server side automation.
	- nightly unit testing; that's only nightly?  per train?  not sure if it's nightly testing?
	- will soon.  on the sync server side it is happening Tarak has it set up
	- Browser ID ones are javascript
	- sync server side it's python
	- sync is not moving to pyramid
	- sync is all javascript?  TPS stuff; some kind of nightly test that's being done... disregard.  it's client side.

BHAG:
Need a clearer idea what BHAG is
	- context: for the next year
	=> 2 big goals : automation, and community
Since browser ID has started, most of the stuff has been running on VM.  They are creating full facing environments for release
for staging, production, dev.  Work in progress.  All 4 environments will be up and running by end of Nov.  (est)
	=> end of Q4 go live
	

	How do we track the community
	=> we have one active community member for mobile team
	=> sync no progress as of yet.
actionitem: => sync newsletter to be discussed between james/oki
	
Status : read wiki
	
Round Table : fennec table to make sure that sync and fennec stay in sync.
Are they bundling as two separate applications?
Not sure how it's going to go.
The release of first version of native is sync is a blocker if it isn't there.
- it might not be a hard blocker for having sync there.
- idea is tighter knit strategy for this.