Breakpad/Status Meetings/2010-Jan-06: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
(Moving hadoop out of 1.3 section)
 
(31 intermediate revisions by 4 users not shown)
Line 1: Line 1:
The 1.3 push - what being done
= Socorro 1.3 =
The 1.3 push - what being done - what needs to be done


== Display URL field ==
[https://bugzilla.mozilla.org/show_bug.cgi?id=415027 Bug 415027] Display URL field when viewing individual crash reports
[https://bugzilla.mozilla.org/show_bug.cgi?id=415027 Bug 415027] Display URL field when viewing individual crash reports
* ozten
* ozten


== ADU ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=470620 Bug 470620] Create a new stat: crashes per user
* [https://bugzilla.mozilla.org/show_bug.cgi?id=534697 Bug 534697] UI New Trend Report - ADU
* [https://bugzilla.mozilla.org/show_bug.cgi?id=526673 Bug 526673] ADU foreign key mapping
* [https://bugzilla.mozilla.org/show_bug.cgi?id=537337 Bug 537337] start regular inserts of ADU data into staging - FIXED
* these are all about the ADU report
** web service is done, tested and committed to trunk
*** a IT staging request has been made for the database changes is complete: [https://bugzilla.mozilla.org/show_bug.cgi?id=537338 Bug 537338]
*** an IT request has been made to stage the new python web service: [https://bugzilla.mozilla.org/show_bug.cgi?id=537844 Bug 537844]
*** documentation has been added to [http://code.google.com/p/socorro/wiki/SocorroUpgrade SocorroUpgrade]
*** Lars has asked Daniel to get the raw_adu flowing into staging asap and into production
**** Daniel implemented to staging export.  Waiting on go-ahead and verification of credentials to push to production.
**** a IT request was made and fixed to get him access [https://bugzilla.mozilla.org/show_bug.cgi?id=537338 537338]
** '''someone needs to be responsible to make the production IT push request'''


[https://bugzilla.mozilla.org/show_bug.cgi?id=470620 Bug 470620] Create a new stat: crashes per user
== Extensions table ==
[https://bugzilla.mozilla.org/show_bug.cgi?id=534697 Bug 534697] UI New Trend Report - ADU
[https://bugzilla.mozilla.org/show_bug.cgi?id=526673 Bug 526673] ADU foreign key mapping
[https://bugzilla.mozilla.org/show_bug.cgi?id=537337 Bug 537337] start regular inserts of ADU data into staging
* these are all about the ADU report
* web service is done, tested and committed to trunk
** a IT staging request has been made
** documentation has been added to [http://code.google.com/p/socorro/wiki/SocorroUpgrade SocorroUpgrade]
** asked daniel to get the data flowing into staging
*** a IT request has been made to get him access
* '''someone needs to be responsible to make the production IT requests'''


* [https://bugzilla.mozilla.org/show_bug.cgi?id=520230 Bug 520230] extension table columns are too short
** this change may have to be deferred to 1.4 because the schema change requires hours of time with the database down.


[https://bugzilla.mozilla.org/show_bug.cgi?id=520230 Bug 520230] extension table columns are too short
== CFRelease - compound signature ==
* documentation has been added to [http://code.google.com/p/socorro/wiki/SocorroUpgrade SocorroUpgrade] for both the processor code change and the database change
* [https://bugzilla.mozilla.org/show_bug.cgi?id=525378 Bug 525378] Make CFRelease part of a compound signature
* '''no staging request has been made'''
* no response on the bug mean nothing has been done - shift to 1.4?
** since this requires a change to the database schema, waiting on getting a timing estimate
* since its just a config change, this can really be done any time


[https://bugzilla.mozilla.org/show_bug.cgi?id=525378 Bug 525378] Make CFRelease part of a compound signature
== Correlation reports ==
* no response on the bug mean nothing has been done
* Punting to 1.4. I didn't finish in time. [[User:Aking|Aking]] 18:24, 6 January 2010 (UTC)
* since its just a config change, this can really be done any time
* [https://bugzilla.mozilla.org/show_bug.cgi?id=525785 Bug 525785] Integrate correlation reports into Socorro UI


[https://bugzilla.mozilla.org/show_bug.cgi?id=525785 Bug 525785] Integrate correlation reports into Socorro UI
== Speeding up minidump_stackwalk ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=526512 Bug 526512] Reduce process time per report


[https://bugzilla.mozilla.org/show_bug.cgi?id=526512 Bug 526512] Reduce process time per report
== Speeding up the web service ==
[https://bugzilla.mozilla.org/show_bug.cgi?id=533757 Bug 533757] Perf slow web service calls /200911/topcrash/sig/trend/history  
* [https://bugzilla.mozilla.org/show_bug.cgi?id=533757 Bug 533757] Perf slow web service calls /200911/topcrash/sig/trend/history
** a first attempt at this is coded, tested and committed to trunk
** a further step involving partitioning the TCBS table is in testing (1.4)
** the first set of changes will ship along with the ADU webservice changes
** no further staging or production IT request are necessary for this


== url sanitizing ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=532500 Bug 532500] Enhancement to how url is sanitized for urldims table. 
* Truncating long URLs requires a new config value: truncateUrlLength in topCrashesByUrlConfig.py
** this change has been coded, tested and checked into trunk
** documentation has been added to [http://code.google.com/p/socorro/wiki/SocorroUpgrade SocorroUpgrade]
** an IT staging push request has been made: [https://bugzilla.mozilla.org/show_bug.cgi?id=537841 Bug 537841]
** ''''someone will have to make the associated production push request''''


[https://bugzilla.mozilla.org/show_bug.cgi?id=532500 Bug 532500] Enhancement to how url is sanitized for urldims table
== TopCrashersSigHistoryAPI special quoting ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=533611 Bug 533611] TCBS - missing totals by OS on blank signatures
* [https://bugzilla.mozilla.org/show_bug.cgi?id=534063 Bug 534063] TopCrashersSigHistoryAPI needs special quoting
** this is coded, tested and checked into trunk
** this will get pushed along with the ADU enhancements, so no further IT staging or production requests are necessary here


[https://bugzilla.mozilla.org/show_bug.cgi?id=533611 Bug 533611] TCBS - missing totals by OS on blank signatures
== Collector Throttling Granularity ==
[https://bugzilla.mozilla.org/show_bug.cgi?id=534063 Bug 534063] TopCrashersSigHistoryAPI needs special quoting
* [https://bugzilla.mozilla.org/show_bug.cgi?id=534656 Bug 534656] collector needs finer throttling granularity
** coded and tested
** documentation has been added to [http://code.google.com/p/socorro/wiki/SocorroUpgrade SocorroUpgrade]
** staging request has been made - and it is done
** '''somebody will have to make the production push request'''


== Search by Build ID ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=411354 Bug 411354]
** an apparent last minute addition to 1.3
** in order to work properly, the solution must have support of indexes in the reports partitions or there will sequence scan timeouts
** a single index on the 'reports' table will not work
** '''this will need attention immediately if 411354 is to get pushed in 1.3'''


= Socorro 1.4 =


[https://bugzilla.mozilla.org/show_bug.cgi?id=534656 Bug 534656] collector needs finer throttling granularity
* Schedule a triage so we can decide what's in and what is not
 
Proposed
== Add integration with Hadoop cluster to crash-report web heads ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=538206 Bug 538206]

Latest revision as of 21:43, 7 January 2010

Socorro 1.3

The 1.3 push - what being done - what needs to be done

Display URL field

Bug 415027 Display URL field when viewing individual crash reports

  • ozten

ADU

  • Bug 470620 Create a new stat: crashes per user
  • Bug 534697 UI New Trend Report - ADU
  • Bug 526673 ADU foreign key mapping
  • Bug 537337 start regular inserts of ADU data into staging - FIXED
  • these are all about the ADU report
    • web service is done, tested and committed to trunk
      • a IT staging request has been made for the database changes is complete: Bug 537338
      • an IT request has been made to stage the new python web service: Bug 537844
      • documentation has been added to SocorroUpgrade
      • Lars has asked Daniel to get the raw_adu flowing into staging asap and into production
        • Daniel implemented to staging export. Waiting on go-ahead and verification of credentials to push to production.
        • a IT request was made and fixed to get him access 537338
    • someone needs to be responsible to make the production IT push request

Extensions table

  • Bug 520230 extension table columns are too short
    • this change may have to be deferred to 1.4 because the schema change requires hours of time with the database down.

CFRelease - compound signature

  • Bug 525378 Make CFRelease part of a compound signature
  • no response on the bug mean nothing has been done - shift to 1.4?
  • since its just a config change, this can really be done any time

Correlation reports

  • Punting to 1.4. I didn't finish in time. Aking 18:24, 6 January 2010 (UTC)
  • Bug 525785 Integrate correlation reports into Socorro UI

Speeding up minidump_stackwalk

Speeding up the web service

  • Bug 533757 Perf slow web service calls /200911/topcrash/sig/trend/history
    • a first attempt at this is coded, tested and committed to trunk
    • a further step involving partitioning the TCBS table is in testing (1.4)
    • the first set of changes will ship along with the ADU webservice changes
    • no further staging or production IT request are necessary for this

url sanitizing

  • Bug 532500 Enhancement to how url is sanitized for urldims table.
  • Truncating long URLs requires a new config value: truncateUrlLength in topCrashesByUrlConfig.py
    • this change has been coded, tested and checked into trunk
    • documentation has been added to SocorroUpgrade
    • an IT staging push request has been made: Bug 537841
    • 'someone will have to make the associated production push request'

TopCrashersSigHistoryAPI special quoting

  • Bug 533611 TCBS - missing totals by OS on blank signatures
  • Bug 534063 TopCrashersSigHistoryAPI needs special quoting
    • this is coded, tested and checked into trunk
    • this will get pushed along with the ADU enhancements, so no further IT staging or production requests are necessary here

Collector Throttling Granularity

  • Bug 534656 collector needs finer throttling granularity
    • coded and tested
    • documentation has been added to SocorroUpgrade
    • staging request has been made - and it is done
    • somebody will have to make the production push request

Search by Build ID

  • Bug 411354
    • an apparent last minute addition to 1.3
    • in order to work properly, the solution must have support of indexes in the reports partitions or there will sequence scan timeouts
    • a single index on the 'reports' table will not work
    • this will need attention immediately if 411354 is to get pushed in 1.3

Socorro 1.4

  • Schedule a triage so we can decide what's in and what is not

Proposed

Add integration with Hadoop cluster to crash-report web heads