Switchboard is a service we use to segment users for A/B testing or staged rollout of features.
An experiment can be enabled for a specified percentage of users and limited to release channels, app versions, languages, countries (locale), device names and device manufacturers. An experiment can contain additional metadata to "configure" the experiment. This can be used to change parameters of an experiment without needing to ship a new app version. Which experiments a user is enrolled in is reported to our telemetry systems via the core ping.
History
We started with KeepSafe's Switchboard library and server component but since then the code and architecture has changed significantly. The original implementation used a special switchboard server that did decides which experiments a client is part of and then just returns a simplified list for the client to consume. This required the client to send data (including a unique id) to the server. To avoid this and make the administration of experiments simpler we moved to using Kinto as storage and server of the experiment configuration. Now clients load the whole configuration and decide independently what experiments they are enrolled it.
Summary
- The experiment configuration is stored in Kinto and served from a CDN: https://firefox.settings.services.mozilla.com/v1/buckets/fennec/collections/experiments/records
- The client regularly loads the configuration (or diffs, see Kinto API) and stores it locally.
- Based on a UUID for bucketing the client and using the experiment filters the client decide which experiments it is part of.
- The experiment configuration can be modified using the Kinto web admin (VPN + account whitelisting required)