MozillaWiki

CloudServices/SimplePushServer: Difference between revisions

Page Discussion

CloudServices/SimplePushServer (view source)

Revision as of 00:22, 27 January 2016

6,487 bytes added ,  27 January 2016
no edit summary
No edit summary
 
(38 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<div style="border:1px solid red; background-color:#ECC;padding:2em;font-size:100%;"><b>Obsolete!</b>
Please refer to the [https://mozilla-push-service.readthedocs.org/en/latest/ Push Service] docs.
</div>
<h1><i>SimplePush Server</i></h1>
<h1><i>SimplePush Server</i></h1>
==Overview==
==Overview==
Line 5: Line 9:
==Project Contacts==
==Project Contacts==
''Principal Point of Contact'' - <i>Doug Turner</i> <i>dougt@mozilla</i>
''Principal Point of Contact'' - <i>Doug Turner</i> <i>dougt@mozilla</i>
''IRC'' - #<i>channel</i>
 
''IRC'' - #<i>push</i>
 
''Group Email'' - TBD
''Group Email'' - TBD


==Goals==
==Goals==
<i>How will you know things are working?</i>
Provide a scalable, fast server for the SimplePush protocol as defined by https://wiki.mozilla.org/WebAPI/SimplePush.
<i>What problems does this solve?</i>
 
In brief, SimplePush is a near dataless method to remotely wake a client application so that it can call "home" and determine what actions are needed. It solves the power and wasted bandwidth concerns of having dozens of applications constantly needing to be connected back with no action required.
 
This will provide endpoints for both websocket clients and PUTs from third party servers.
SimplePush
 
==Use Cases==
==Use Cases==
<i>How will this be used?</i>
Use cases are [https://wiki.mozilla.org/WebAPI/SimplePush#Use_Cases defined here]
 
==Definitions==
==Definitions==


Line 21: Line 33:
* PUSH CLIENT shall send UAID, APPID and any additional information required for proprietary KICK to the PUSH SERVER
* PUSH CLIENT shall send UAID, APPID and any additional information required for proprietary KICK to the PUSH SERVER
* PUSH SERVER shall create an ENDPOINT for the UAID and APPID and return it to the PUSH CLIENT.
* PUSH SERVER shall create an ENDPOINT for the UAID and APPID and return it to the PUSH CLIENT.
* If a KICK driver is present, PUSH SERVER shall relay appropriate information.  
* If a KICK driver is present, PUSH SERVER shall relay appropriate PUSH CLIENT provided information to the KICK driver.  
* PUSH CLIENT tenders the ENDPOINT to APP via callback.
* PUSH CLIENT tenders the ENDPOINT to APP via callback.
* APP sends ENDPOINT to the APP SERVER
* APP sends ENDPOINT to the APP SERVER
Line 47: Line 59:
=Design=
=Design=
==Points of Contact==
==Points of Contact==
Server Engineer - <i>"trink"</i> <i>trink@mozilla</i>
Server Engineer - <i>JR Conlin</i> <i>jrconlin@mozilla</i>
Server Engineer - <i>JR Conlin</i> <i>jrconlin@mozilla</i>
Client Engineer -


==API Reference/Documentation==
The protocol is [https://wiki.mozilla.org/WebAPI/SimplePush/Protocol defined here]
=== Data Schema ===
<i>How will data be stored</i>
=== API ===
<i>How to call the data</i>
==== <i>method</i> <i>URI</i> ====
<i>description</i>


Arguments <i>argument descriptions</i>
==Platform Requirements==
Returns: <i>returned contents</i>
This system runs on linux systems as a Go executable.


==Platform Requirements==
Go executables are mostly self contained, however the following external systems are strongly recommended:
<i>What are the things this needs (OS, language, databases, etc.)?
 
* a memcached server cluster
* heka logger


==Libraries Required==
It should also be noted that Go's SSL implementation is surprisingly CPU intensive as of 1.1.2. For our implementation, we decided that since PUTs require more setup/teardown than longer lived Websocket connections, we would use AWS ELB SSL termination to handle the secure PUTs. If peak user load is not expected to be higher than 100K or so, this may not be required.
<i> List of external project dependencies. (Stuff that's not pulled in via the installation script)</i>
== Proprietary Ping Requirements ==
* [[CloudServices/SimplePushServer/GCM|GCM]]
* APNS


==Code Repository==
==Code Repository==
<i>Links to the published code bases</i>
Previously (as referenced in other parts of this page), https://github.com/mozilla-services/pushgo/.
 
Currently, https://github.com/mozilla-services/autopush/ is used.
 
==Release Schedule==
==Release Schedule==
<i>Predicted code delivery dates</i>
=== 1.4 ===
* Target date: 20/11/2014
* Released: 20/11/2014
==== Common Changes: ====
<!-- in the future, point to CHANGES.log -->
* Fixes to critical etcd routing bug
* Fixes to Travis testing
* Convert to toml configuration system
* Integrated smoke tests
* Various optimizations.
==== Loop Push ====
* Create system that does not store data
==== Simple Push====
* No system specific changes made
=== 1.4.2 ===
==== Common Changes: ====
* https://github.com/mozilla-services/pushgo/issues?q=milestone%3Av1.4.2+is%3Aclosed
=== 1.5 ===
* Target Date: TBD
* Released: Unreleased
==== Common Changes:====
* Include support for "data" to connected devices only
 
=QA=
=QA=
==Points of Contact==
==Points of Contact==
Engineer - <i>Name</i> <i>contact@info</i>
* Primary - kthiessen@
* Backup - rpappalardo@
 
==Test Framework==
==Test Framework==
There are several test frameworks in place. Most systems are stand alone test suites so that they may be applied both to the current server and any externally created system.
https://github.com/mozilla-services/simplepush-testpod - provides an end-to-end stress test of the system.
https://github.com/jrconlin/simplepush_test - provides a quick "smoke test" as well as a thorough API test of bad or malicious tests.
== Notes ==
* [[/Testing Notes|Testing notes for Push]]
=Security and Privacy=
=Security and Privacy=
Fill out the security & privacy bug template: https://bugzilla.mozilla.org/form.moz-project-review
wiki page: https://wiki.mozilla.org/Security/Reviews/SimplePushSrv
(https://wiki.mozilla.org/Websites/Kick-Off_Form)
==Points of Contact==
==Points of Contact==
==Questionnaire Answers==
===1.1 Goal of Feature ===
===2. Potential Threat Vectors and Mitigation Points===
==Review Status==
==Review Status==
''Bugzilla Tracking #'' -
''Bugzilla Tracking #'' - https://bugzilla.mozilla.org/show_bug.cgi?id=897454
see https://wiki.mozilla.org/Security/Reviews
 
https://wiki.mozilla.org/Security/Reviews/SimplePushSrv
 
==Issues and Resolutions==
==Issues and Resolutions==


Line 91: Line 133:


==Points of Contact==
==Points of Contact==
Current Ops-Engineers are oremj@ and bwong@
==Deployment Architecture==
==Deployment Architecture==
''Bugzilla Tracking # '' -
''Bugzilla Tracking # '' -
https://mana.mozilla.org/wiki/display/SVCOPS/SimplePush#SimplePush-Deployments
==Deployment Request Template ==
'' Currently, deployments are created from Github Releases. Releases for Stable and Production must be signed off by QA before deployment. ''
Bugzilla Subject: Please deploy '' Tag name '' to '' Product '' '' Platform ''
Bug content:
  Version:
  '' Release Title (e.g. Simple Push Server 1.4.2 Release Candidate 1)''
  '' Release URL (e.g. https://github.com/mozilla-services/pushgo/releases/tag/1.4.2rc1) ''
 
  Product:
  '' What platform to deploy the target to (e.g. push-web, push-loop) ''
 
  Config Changes:
    +[''handlers'']: ''handlers'' section added.
    ~[''default'']: ''default'' section options changed.
    -[''propping'']: ''propping'' section removed.
    ...
   
  Deployment Notes:
  '' Please note any other modifications which may cause the server to fail to start. ''
==Escalation Paths==
==Escalation Paths==
==Lifespan Support Plans==
==Lifespan Support Plans==
=Logging and Metrics=
=Logging and Metrics=
Current logging and metrics are being filtered into the Heka system. Final logging and metrics are TBD; depending on the sorts of data that needs to be detected.
==Points of Contact==
==Points of Contact==
==Tracking Element Definitions==
==Tracking Element Definitions==
==Data Retention Plans==
==Data Retention Plans==
==Dashboard URL==
==Dashboard URL==
The following are mozilla private URLs.
* [https://kibana.shared.us-west-2.prod.mozaws.net/index.html#/dashboard/file/simplepush.json Kibana dashboard]
* [https://graphite.shared.us-west-2.prod.mozaws.net/ US-West-2 Push monitoring]
* Cloudwatch URL (pending?)
Usage metrics are viewable at:
https://graphite.shared.us-west-2.prod.mozaws.net/grafana/#/dashboard/file/default.json
and use the following view template:
https://dl.dropboxusercontent.com/u/361111/Push%20_%20SimplePush%20Stats-1421198257367
=Customer Support=
=Customer Support=
==Points of Contact==
== External Libraries & Users ==
==Sumo Tags==
 
==Review Meeting==
* [https://aerogear.org/push/ Aerogear Unified Push]
* [https://onesignal.com/ OneSignal]
 
== Deployments==
=== Loop ===
Hello (aka Loop) is a WebRTC based video chat program that is available to desktop and mobile devices. It uses a specialized version of SimplePush that does not have back end storage, since there is no need to alert connections that are offline.
==== Configuration ====
All systems are deployed to AWS and are set to autoscale within clusters. Clusters are not yet configured to scale, however ops is working to address this.
 
There are two non-production networks of systems:
<dl>
<dt>Stable</dt><dd>This is a non production environment which hosts a stable development version of Simplepush for development and integration tests. This version may be auto-updated from the "dev" branch of https://github.com/mozilla-services/pushgo. </dd>
<dt>QA</dt><dd>This is a non production environment that hosts the stable, pre-release version of SimplePush for load and QA testing. This version updates from explicit releases generated by https://github.com/mozilla-services/pushgo.</dd>
</dl>
 
It should be noted that effort is currently being made to ensure that the push service used by Hello is not significantly different (API wise) from the standard Push service.
==== Deployment Architectures ====
The <b>stable</b> environment is currently configured for push-loop-dev.stage.mozaws.net (for long lived socket connections) and updates-push-loop-dev.stage.mozaws.net (to receive the REST version PUTs)
 
While the client has the ability to retry a connection if a given machine is not responsive, it has been requested by Ops that Push Protocol Redirects be re-enabled and that a separate suite of machines be created to do connection load balancing. (so clients would first connect to the central server, and then be redirected to a machine with available resources.)
 
==== Monitoring And Metrics ====
Push currently provides metrics using logstash compatible reporting mechanisms (e.g. stackdriver). In addition, logs are scraped and information displayed via kibana.
Currently monitoring and metrics are being collected and displayed on to different systems. Effort will be made to simplify this.
 
TODO: Dev & Ops need to identify a list of key health metrics to monitor for this system.
 
* [https://metrics.services.mozilla.com/loop-server-dashboard/ Loop Server dashboard]
 
==== Points of Contact ====
In the event of significant events, operations notifies members of development teams that actions are required.
 
TODO: dev will provide contact information as well as a "jiggle list" of actions which may alleviate issues.
 
=== Find My Device ===
==== Configuration ====
==== Deployment Architectures ====
==== Monitoring And Metrics ====
==== Points of Contact ====
Jrconlin
Confirmed users
1,018

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/533331...1114411"