Breakpad/Status Meetings/2013-October-23

Meeting Info

Breakpad status meetings occur on Wed at 11:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad Mountain View: Dancing Baby (3rd floor)

Socorro Release Tracker
Socorro Bug Tracker

64

• Some bugs need pushing out - let's review
• Next week is a Firefox release week. Let's talk about the risk profile of the release:
  • Ship new processor?
  • Turn on jMDSW?
  • Anything else of concern?
• bugs

Bugzilla query error

Array ( [type] => error [message] => http-timed-out [params] => Array ( [0] => https://bugzilla.mozilla.org/bzapi/bug?product=Socorro&target_milestone=64&include_fields=id&include_fields=summary&include_fields=assigned_to&include_fields=target_milestone&include_fields=priority&include_fields=status&include_fields=resolution&include_fields=whiteboard ) ) 1

PR Triage

• https://github.com/mozilla/socorro/pulls

Other issues

• Socorro Maintenance on 2013-10-22 @ 5-6:30PM
  • Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=925033 / Etherpad: https://etherpad.mozilla.org/failover-prep
  • Outcome: failover was rolled back
  • Issues:
    • https://crash-reports.mozilla.com/ accidentally hardhatted for 10 minutes due to solarce selecting the wrong virtual server (intended to hardhat crash-stats.mozilla.com:80) in Zeus and not noticing the name. rhelmer noticed drop in traffic in Ganglia, I reviewed my changes and discovered the error and fixed. Total time of sending incoming crash reports to hardhat approx 12 minutes. See http://sp-admin01.phx1.mozilla.com/ganglia/graph.php?r=day&z=xlarge&c=Socorro+Collectors&m=load_one&s=by+name&mc=2&g=network_report
    • Database Errors in socorro2? mpressman can fill in details

• Follow up on your action items from work week -- https://wiki.mozilla.org/CrashKill/StabilityWeek2013

Ops Updates

• Reduced availability over next few weeks as webops has to do a lot of preparation work for the 11/16 SCL3 outage

Actively working on

• Shipping new processor with jsonMSDW
• Various misc bugs
• migrate crash-analysis.mozilla.com to django web servers, will file bugs today
• Updating documentation

ElasticSearch Prod Cluster capacity - https://bugzilla.mozilla.org/show_bug.cgi?id=907673

• Financial review underway https://bugzilla.mozilla.org/show_bug.cgi?id=910747, needs input from Bob Moss
• Added both dev servers to prod cluster on 2013-09-23
  • cluster much more evenly balanced

Socorro Chief - https://bugzilla.mozilla.org/show_bug.cgi?id=864823

• stage is working but all the commander script does is call shell scripts
• working on some cleanup bits in the shell script - https://bugzilla.mozilla.org/show_bug.cgi?id=912647
• Then, working on a couple PRs to get the scripts chief is using into the repo
• Then, the commander script needs to be added to the repo and its functionality moved up into python
• no change since last week

PHP Decommission - https://bugzilla.mozilla.org/show_bug.cgi?id=892194

• underway

PTOs

• solarce at ricon.io on 10/28-10/31, will be afk at conf 29, 30 and afk travel day 10/31
• lonnen at mozfest, in NY the week after

Links

• q4 goals (draft)