Confirmed users, Bureaucrats and Sysops emeriti
1,081
edits
(→IT) |
(→IT) |
||
Line 126: | Line 126: | ||
=== IT === | === IT === | ||
* We are in the middle of a massive data center move from one of our original data centers, SJC1 to our other data centers. Contractually, this move has to be completed in the middle of the quarter and comprises roughly 1300 servers and virtual machines. While migration work has been going on for a year now to get websites and services out of there in advance of this move, some services are hanging on to the end and may require downtime to get them physically moved (or virtually migrated) to their new home. Please be patient with us as this move takes our resources away from other projects, and as some of these services incur downtime for their move. | <b>Data Centers</b> | ||
* <b>SCL3/SJC1:</b> We are in the middle of a massive data center move from one of our original data centers, SJC1 to our other data centers. Contractually, this move has to be completed in the middle of the quarter and comprises roughly 1300 servers and virtual machines. While migration work has been going on for a year now to get websites and services out of there in advance of this move, some services are hanging on to the end and may require downtime to get them physically moved (or virtually migrated) to their new home. Please be patient with us as this move takes our resources away from other projects, and as some of these services incur downtime for their move. | |||
* Data center migration in China for Mozilla Online continued this week, with Shyam on site finishing the load balancer and vm hosting, and working on moving key sites. Mail should take another week and we will start wrapping up and heading home. | |||
<b>Websites</b> | |||
* This week we stood up a cluster (admin, dev, stage, and prod) along with global caching for the mozilla-hosted Firefox 3.6 start page: https://wiki.mozilla.org/Websites/Firefox_3.6_Start_Page | |||
* This week we setup bedrock (the new site behind www.mozilla.org) to be multi-hosted between Phoenix and Santa Clara. In addition it is also cached for europe out of Amsterdam. We will be making more use of CDNs in the next step. | |||
<b>Infrastructure</b> | |||
* Mozilla Foundation and Mozilla Corporation email accounts are on a new Zimbra infrastructure (and the migration happened with no impact to you!). See http://blog.mozilla.com/it/2012/04/06/re-imagining-zimbra-email-at-mozilla/ | * Mozilla Foundation and Mozilla Corporation email accounts are on a new Zimbra infrastructure (and the migration happened with no impact to you!). See http://blog.mozilla.com/it/2012/04/06/re-imagining-zimbra-email-at-mozilla/ | ||
* This week we turned up hundreds of new systems (more than I can collectively count amongst all teams in IT). For example, 128 nodes with new Sandy Bridge Xeon procs have been deployed to the SCL3 data center, and 64 to the PHX1 data center. | * This week we turned up hundreds of new systems (more than I can collectively count amongst all teams in IT). For example, 128 nodes with new Sandy Bridge Xeon procs have been deployed to the SCL3 data center, and 64 to the PHX1 data center. | ||
* We replaced the NIC cards in our PHX1 load balancers this week with new vendor-recommended cards to overcome problems that have plagued us for months. So far, we have not had any lb-related problems in PHX1. However, part of this success is due to moving the load of Addons version checking and about:home snippets to a new set of Netscaler load balancers which handle the heavy-loaded sites better but don't provide some of the features we enjoy out of our Stingray load balancers. | * We replaced the NIC cards in our PHX1 load balancers this week with new vendor-recommended cards to overcome problems that have plagued us for months. So far, we have not had any lb-related problems in PHX1. However, part of this success is due to moving the load of Addons version checking and about:home snippets to a new set of Netscaler load balancers which handle the heavy-loaded sites better but don't provide some of the features we enjoy out of our Stingray load balancers. | ||
<b>Developer Services</b> | |||
* '''hg.mozilla.org''' | * '''hg.mozilla.org''' | ||
** A lot of work was wrapped up in prepping a new environment for hg.mozilla.org in our new data center. | ** A lot of work was wrapped up in prepping a new environment for hg.mozilla.org in our new data center. | ||
*** We should be announcing downtime soon to make the switch this week. | *** We should be announcing downtime soon to make the switch this week. | ||
** <i>Please</i> do not abort a push to try (with ctrl-c) once it has begun. Due to the added pushlog in commit hooks, it causes hg to break and requires manual intervention to fix. | ** <i>Please</i> do not abort a push to try (with ctrl-c) once it has begun. Due to the added pushlog in commit hooks, it causes hg to break and requires manual intervention to fix. | ||
=== Release Engineering === | === Release Engineering === |