Confirmed users
595
edits
Performance/Avoid SQLite In Your Next Firefox Feature: Difference between revisions
Jump to navigation
Jump to search
m
Performance/Avoid SQLite In Your Next Firefox Feature (view source)
Revision as of 07:57, 8 April 2014
, 8 April 2014→How to Store Your Data
m (→I/O Patterns) |
|||
| Line 73: | Line 73: | ||
* If you need to store a small amount of data (less than 1MB), you should use JSON files, and do all your I/O off the main thread. Simple solutions with linear worst-case performance are ideal. | * If you need to store a small amount of data (less than 1MB), you should use JSON files, and do all your I/O off the main thread. Simple solutions with linear worst-case performance are ideal. | ||
** If you're working with larger amounts of data (roughly on the order of 1MB), you should compress the JSON data with Snappy before writing it to disk | ** If you're working with larger amounts of data (roughly on the order of 1MB), you should compress the JSON data with Snappy or lz4 before writing it to disk | ||
** You can use OS.File's [https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread#OS.File.writeAtomic%28%29 writeAtomic()] method. More information on writeAtomic [http://dutherenverseauborddelatable.wordpress.com/2014/02/05/is-my-data-on-the-disk-safety-properties-of-os-file-writeatomic/ in this blog post] | ** You can use OS.File's [https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread#OS.File.writeAtomic%28%29 writeAtomic()] method. More information on writeAtomic [http://dutherenverseauborddelatable.wordpress.com/2014/02/05/is-my-data-on-the-disk-safety-properties-of-os-file-writeatomic/ in this blog post] | ||
** If your workload involves a lot of strings, don't use SQLite. Store your data in external files. | ** If your workload involves a lot of strings, don't use SQLite. Store your data in external files. | ||
** If your workload involves a lot of blobs, don't use SQLite. Store your data in external files. | |||
* If you have a large dataset, but you don't need to run complex queries over it, evaluate a JSON files structure on disk | |||
* If you must read and write most of your data every time, evaluate using a JSON file | |||
* For larger datasets or when SQL is absolutely necessary, use SQLite. | * For larger datasets or when SQL is absolutely necessary, use SQLite. | ||
** Make sure you understand [https://sqlite.org/docs.html how SQLite works], carefully design your schemas and then profile your implementation. | ** Make sure you understand [https://sqlite.org/docs.html how SQLite works], carefully design your schemas and then profile your implementation. | ||