|
|
| Line 1: |
Line 1: |
| = Roadmap = | | = Roadmap = |
| == Q1 2013 roadmap == | | == Q1/Q2 2013 roadmap == |
| In Q1 2013 the Identity DevOps team will be moving services out of the physical datacenter SCL2 and into Amazon Web Services. | | In Q1 2013 the Identity DevOps team will be moving services out of the physical datacenter SCL2 and into Amazon Web Services. |
|
| |
|
| 2/4
| | 3/19 : VPC outbound internet access via NAT instance |
| * 1/28 - 2/4 : roadmap defined and signed off
| | 3/21 : Webhead AMI |
| * 1/28 - 2/4 : technology stack justification written and shared
| | 3/22 : Zeus routing logic converted to nginx |
| | 3/26 : keysigner AMI |
| | 3/27 : dbwriter AMI |
| | 3/29 : Cross DC VPN for DBs |
| | 4/2 : db AMI |
| | 4/3 : Handoff environment to QA |
| | 4/5 : Nagios monitoring |
| | 4/9 : Create region 2 |
| | 4/10 : QA approval of region 1 |
|
| |
|
| 2/11
| | 4/30 : Final day to turn down servers at SCL2 |
| * 2/4 - 2/11 : chef server built and working
| |
| * 2/4 - 2/11 : established 1 region VPC
| |
| | |
| 2/18
| |
| * 2/4 - 2/18 : completed a mini provisioning test and plan
| |
| * 2/11 - 2/18 : written the webhead chef provisioning logic
| |
| * 2/11 - 2/18 : written the nginx chef provisioning logic and carried over existing nginx routing logic
| |
| * '''milestone : chef can fully provision webheads'''
| |
| | |
| 2/25
| |
| * 2/18 - 2/25 : zeus routing logic is converted into nginx logic
| |
| * 2/18 - 2/25 : written the nagios chef provisioning logic
| |
| * 2/18 - 2/25 : basic webhead nagios checks created
| |
| * '''milestone : admin can see the monitored availability and performance of the webhead'''
| |
| | |
| 3/4
| |
| * 2/25 - 3/4 : ELB is setup and sending traffic to the webhead
| |
| * 2/18 - 2/25 : basic webhead nagios checks against the ELB created
| |
| * '''milestone : internet client can fetch persona main page from AWS traversing ELB'''
| |
| | |
| 3/11
| |
| * 3/4 - 3/11 : written swebhead chef provisioning logic
| |
| * 3/4 - 3/11 : ELB configured for swebhead cluster
| |
| * 3/4 - 3/11 : written db chef provisioning logic
| |
| * 3/4 - 3/11 : ELB configured for db cluster
| |
| * 3/4 - 3/11 : written keysign chef provisioning logic
| |
| * 3/4 - 3/11 : ELB configured for keysign cluster
| |
| * 3/4 - 3/11 : established VPN to PHX1
| |
| * 3/4 - 3/11 : basic swebhead db and keysign nagios checks created
| |
| * '''milestone : internet client can login using persona in AWS'''
| |
| | |
| 3/18
| |
| * 3/11 - 3/18 : written bigtent and squid proxy chef provisioning logic
| |
| * 3/11 - 3/18 : ELB configured for bigtent and squid clusters
| |
| * 3/11 - 3/18 : basic bigtent and squid proxy nagios checks created
| |
| * '''milestone : internet client can login with a yahoo address using yahoo bigtent'''
| |
| | |
| 3/25
| |
| * 3/18 - 3/25 : load tested/validated that region 1 is ready for prod traffic
| |
| * 3/18 - 3/25 : full security group logic is in place replicating existing physical network
| |
| * '''milestone : security : network security is in place and all tiers use proxies for communication'''
| |
| * '''milestone : load testing complete for region 1'''
| |
| | |
| 4/1
| |
| * 3/25 - 4/1 : moved master from PHX1 to region 1 AWS
| |
| * dynect is changed to balance between AWS region 1 and PHX1. SCL2 sits running as a backup
| |
| * '''milestone : all db writes are now going to AWS'''
| |
| * '''milestone : AWS region 1 is live in production, SCL2 no longer receives traffic'''
| |
| | |
| === State at end of Q1 2013 ===
| |
| * SCL2 is dark
| |
| * Production is running off of 1 AWS region and 1 physical datacenter
| |
| * runbooks for AWS deployments & core troubleshooting have been developed
| |
| * The staging environment has been moved to AWS
| |
| ** key differences between production and staging AWS areas: server localization & access
| |
| * monitoring: existing monitoring minus some cepmon rate-of-change monitors has been moved into a new nagios deployment in AWS
| |
| * alerting: existing minus cepmon-triggered stuff has been migrated
| |
| | |
| == Q2 2013 roadmap ==
| |
| In Q2 DevOps will be bringing up the second AWS region and executing remaining tasks to get us to a truly highly available architecture, ready to graduate from beta
| |
| | |
| 4/8
| |
| * 3/25 - 4/8 : spun up region 2 AWS
| |
| * 3/25 - 4/8 : determined how to do log processing (logstash?) and pump data into zenoss
| |
| | |
| 4/15
| |
| * 4/8 - 4/15 : load tested/validated that region 2 is ready for prod traffic
| |
| * 4/1 - 4/15 : written auto provisioning logic to call AWS and spin up instances, assign them roles, and pass them to chef for provisioning
| |
| * dynect is changed to balance between AWS region 1 and AWS region 2
| |
| * '''milestone : persona is fully hosted in AWS multi-region'''
| |
| | |
| 4/30
| |
| * Final day to turn down servers at SCL2
| |
|
| |
|
| 5/13 | | 5/13 |