Revision as of 20:08, 26 January 2011

Migration

Wow. Every person on this call (and some that are not) did a stellar, stellar job.
Follow up items?
- We need a data ageing policy for the current HBase and PostgreSQL data.
- Monitor performance needs radical improvement
- Ongoing improvements in monitoring the system
- Connection cleanup for PostgreSQL connections ... 1.7.7 or 1.7.8?
- Discuss steps required for "full throttle" processing.

HBase config tuning: Visited StumbleUpon yesterday and spent a few hours going over our PHX cluster. Came out with a series of useful recommendations for changes we should try out to improve the performance and robustness of the cluster. Big question for Socorro is how we can test these changes. We can take a copy of the data we have and load it up on the secondary cluster with config changes, but some of the changes will only be measurable with new traffic flowing in. Wondering if we can fork collectors over to the other cluster or spin up some collectors on my spare boxes or something else. While we are currently running fine, some of these changes are likely to be important for continued stability over the next couple of months.
Archived data removal: Want to delete the data we archived back in June. Also want to look at implementing the deletion of raw dump data that has not been accessed in the last 6 months. When can we finalize a plan for these two things? Bonus, yay, we can actually test the effects using secondary cluster.

Theme: ship UI improvements for Platform
bugs
Freeze: 2/17???
Ship: 2/24???
These dates can be a week later
Main features:
- Explosive bug tracking (all)
- Work on duplicate crashes (all)
- Post migration tidying up (all)
- Better safety for postcrashemail (rhelmer)

Just FYI
Ship by end of March
Theme: architectural improvments
Main features:
- Monitor improvements (lars)
- MDSW binary symbols (ted)
- On crash emails (rhelmer)

@@ Line 2: / Line 2: @@
 * Wow.  Every person on this call (and some that are not) did a stellar, stellar job.
 * Follow up items?
+** We need a data ageing policy for the current HBase and PostgreSQL data.
+** Monitor performance needs radical improvement
+** Ongoing improvements in monitoring the system
+** Connection cleanup for PostgreSQL connections ... 1.7.7 or 1.7.8?
+** Discuss steps required for "full throttle" processing.
 ;HBase config tuning
 :Visited StumbleUpon yesterday and spent a few hours going over our PHX cluster.  Came out with a series of useful recommendations for changes we should try out to improve the performance and robustness of the cluster.  Big question for Socorro is how we can test these changes.  We can take a copy of the data we have and load it up on the secondary cluster with config changes, but some of the changes will only be measurable with new traffic flowing in.  Wondering if we can fork collectors over to the other cluster or spin up some collectors on my spare boxes or something else.  While we are currently running fine, some of these changes are likely to be important for continued stability over the next couple of months.