Confirmed users
358
edits
Services/KeyValueStorage: Difference between revisions
→Planning Questions
mNo edit summary |
|||
| Line 1: | Line 1: | ||
== Planning Questions == | == Planning Questions == | ||
Scope | |||
*are we planning a full-stack solution, where we select a particular backend (e.g. riak/hbase/cassandra/membase/etc) then implement our API on top according to what it offers? | |||
*or are we planning to support multiple backends for apps with different tradeoffs? | |||
Use Cases | |||
* data storage for mozilla appstore | |||
* It would be good to find and isolate some use cases in the existing Services apps.<br>(see below for strawman implementation of SyncStorage backend using KVStore) | |||
How simple can we get away with while still providing useful functionality? | How simple can we get away with while still providing useful functionality? | ||
| Line 9: | Line 21: | ||
*keys in sorted order? (i.e. hash or btree? riak or hbase?) | *keys in sorted order? (i.e. hash or btree? riak or hbase?) | ||
*bulk insert, update or delete operations? | *bulk insert, update or delete operations? | ||
Consistency | Consistency | ||
| Line 14: | Line 27: | ||
*Always consistent, at cost of partition intolerance? (i.e. a "CA" system like membase) | *Always consistent, at cost of partition intolerance? (i.e. a "CA" system like membase) | ||
*Partition tolerant, at cost of eventual consistency? (i.e. a "AP" system like riak) | *Partition tolerant, at cost of eventual consistency? (i.e. a "AP" system like riak) | ||
*Is it possible to let the developer choose on a per-bucket level? On a per-request level? | *Is it possible to let the developer choose on a per-bucket level? On a per-request level?<br>(e.g. riak's per-request CAP tuning: http://wiki.basho.com/Tunable-CAP-Controls-in-Riak.html) | ||
Durability | Durability | ||
| Line 20: | Line 34: | ||
*flush-to-disk on each write? | *flush-to-disk on each write? | ||
*periodic flush-to-disk in background? | *periodic flush-to-disk in background? | ||
*Is it possible to let the developer choose on a per-bucket level? On a per-request level? | *Is it possible to let the developer choose on a per-bucket level? On a per-request level?<br>(e.g. set "time before write" on each request, tbw=0 for immediate write, tbw=30 for 30-second write-behind) | ||
| Line 27: | Line 41: | ||
*can it sensibly be done at this layer, or do we need to defer to the application? | *can it sensibly be done at this layer, or do we need to defer to the application? | ||
*encrypted keys would make key-ordering useless | *encrypted keys would make key-ordering useless | ||
Authentication to App | Authentication to App | ||
*can [[Services/AppKeys|AppKeys]] be used for authentication, e.g. some sort of request signing with the app key? | *can [[Services/AppKeys|AppKeys]] be used for authentication, e.g. some sort of request signing with the app key? | ||
We will probably want some sort of partitioning or "buckets". | We will probably want some sort of partitioning or "buckets". | ||
| Line 38: | Line 54: | ||
*can a bucket be shared between multiple apps? multiple users? | *can a bucket be shared between multiple apps? multiple users? | ||
Management features | Management features | ||
*explicit TTLs? | *explicit TTLs? | ||
*built-in quota system? | *built-in quota system? having it built-in will probably be more efficient than enforcing it at a higher level. | ||
== Strawman Proposal == | == Strawman Proposal == | ||