Breakpad/Status Meetings/2017-06-14

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

• we didn't do a -prod deploy last week (sadface)
• this is a release week, so we're not doing a -prod deploy this week
• elasticsearch migration
  • we now have a working ES 5.3 cluster in ops infrastructure for prototyping reindex-from-remote
  • working on migrating data--should be ready today
  • miles put the cluster behind a NAT server to reach out into webeng environment
• infrastructure migration
  • working on getting webapp and processor working in ops environment
• symbols
  • environments for load test are built
  • logging and systemtest have issues still

• priority adjustment -- elasticsearch has to go out asap
  • we need to reduce the amount of work to get ES out as much as possible
  • we're going to just reserve the -stage environment for a week -- no new dev, no merges, no nothing unless it's ES related
  • adrian is switching projects so peterbe/willkg will fill in

• elasticsearch keeps blowing up and we want metrics on the size of documents going in
  • willkg will write up bug to add metrics for document sizes

Security Updates

• will look at tecken
• Is this a security solution to long-lived API tokens? https://github.com/mozilla/mozilla-django-oidc/issues/133#issuecomment-308224298

QA Updates

• grumpy started his first load tests of symbol server. Currently blocked waiting on confirmation that fix for NoSuchBucket has been implemented and pushed to the staging environment
• talked to tarek about adding incremental load ramping to molotov (https://github.com/loads/molotov/issues/61)
• setting up a spreadsheet for tracking load testing on tecken

Project Updates

Deployment Triage

• Socorro: https://whatsdeployed.io/s-7M7
• Antenna: https://whatsdeployed.io/s-Up2

PR Triage

• http://prs.mozilla.io/mozilla/socorro

Major Projects

Deprecation rampage

• (Adrian) it seems we can run our django web app with just about 11 tables (that is, if we remove 3 useless features)
  • compare that to the ~160 tables we currently have in our database
  • that's not counting django-related tables (stuff like auth, meta-data, etc. )
• started experimenting with managing those tables inside django
• adrian will do a braindump in some good place and put into bugs at some point
• willkg can "take over" this project because it's related to processor work, but he will probably put this on hold for a bit

Processor rewrite

• (willkg) continued working on the spec; going through existing Processor and spec'ing out all the parts and working on reducing Phase 1 of the rewrite to bare minimum

Upgrading elasticsearch

Covered in ops update.

Symbol Server 2017

• Discovered a "bug" in Socorro's symbol upload
  • Had to re-upload LOTS of .sym files that haven't changed between builds
• Tecken Stage is ready for load testing
  • peterbe working on a doc https://docs.google.com/document/d/1UGz4sY-WESTr_x0_6j9duKhfJHSPsRrKnhCY1PyskLQ/edit#
• Symbol upload now works with retries on various network errors
• Auth0 works! (at least on Dev)

Other Business

• Adrian is moving to work on Pontoon, starting soon
  • deprecation rampage -> willkg/peterbe
  • elasticsearch update -> willkg/miles
  • will be gone for 1-3 months or something like that
• STICKERS!

Travel, etc

• mbrandt, PTO 6/16
• miles, PTO (6/14, 6/15)
• willkg, PTO (7/10 -> 7/14)
• grumpy, PTO for August