Breakpad/Status Meetings/2015-05-27

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 11:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

AWS

• PostgreSQL RDS DB has been migrated
  • takes ~6 hours for "core" DB, to be usable, ~30 hours for historical data
  • migration is scripted https://github.com/mozilla/socorro-infra/pull/119
  • can we catch up and stay up-to-date by running processor only in AWS for now?
    • e.g. "re-"process from S3 and backfill to cover gap
    • push from PHX crashmovers -> AWS RMQ to keep up
  • rhelmer might need help getting crontabber running in AWS

• ES migration
  • plan is to use special-purpose processors in AWS
    • https://github.com/mozilla/socorro/pull/2773

• how bad is running multiple crontabbers?
  • could we get duplicate data, e.g. ADI?
  • this will happen the way deploys happen right now
    • autoscaler spins up new instance before shutting down old one

• load testing
  • make sure to snapshot DB first, and use non-prod S3 bucket so as not to pollute prod!
  • how many crashes should we try submitting?
  • can we get datadog statsd landed first?
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1167767

DB

• PHX PostgreSQL memory has been stable, but watching it
  • https://graphite-phx1.mozilla.org/render/?width=777&height=573&_salt=1431452379.214&from=-2weeks&target=hosts.sp-processor*_phx1_mozilla_com.memory.memory.used.value

Project Updates

Socorro Bug Tracker
this week's bugs

PR Triage

• https://prs.paas.allizom.org/mozilla/socorro

Travel, PTO, etc

• rhelmer out a few hours each on Thursday and Friday