3
edits
Mozilla2:Unified Storage: Difference between revisions
→Kingsley Idehen
m (Mork -> Mork) |
|||
| (13 intermediate revisions by 8 users not shown) | |||
| Line 12: | Line 12: | ||
# Provide Mozilla core components and extension authors with tools to enable richer interaction with user data | # Provide Mozilla core components and extension authors with tools to enable richer interaction with user data | ||
# Eliminate the need for components to write their own file serialization/deserialization | # Eliminate the need for components to write their own file serialization/deserialization | ||
# Eliminate multiple file formats from Mozilla profiles (db1.85, | # Eliminate multiple file formats from Mozilla profiles (db1.85, [[Mork]], rdf/xml, xml, various text formats) | ||
== Requirements == | == Requirements == | ||
| Line 39: | Line 39: | ||
* History | * History | ||
: history.dat ([[Mork]]) | : history.dat ([[Mork]]) | ||
: Implements RDF data source on top of Mork. | : Implements RDF data source on top of [[Mork]]. | ||
* Bookmarks | * Bookmarks | ||
: bookmarks.html (NETSCAPE-Bookmark-file-1) | : bookmarks.html (NETSCAPE-Bookmark-file-1) | ||
| Line 111: | Line 111: | ||
object_is_literal BOOLEAN | object_is_literal BOOLEAN | ||
); | ); | ||
Perhaps best to consider this illustrative only. There are many RDF engines (Jena, Redland, ARC, ...) that already have schemas for storing RDF data in SQL tables. If a suitably similar structure is adopted here, it could save some debugging time w.r.t. spec compliance, as well as open up possibility of re-coding other useful work in Javascript. For example, http://arc.web-semantics.org/ offers a SPARQL implementation in PHP that works on top of MySQL, recoding this in Javascript could be a reasonably cheap way of providing SPARQL query capabilities over this Mozilla RDF data storage. --DanBri | |||
=== Shared profiles === | === Shared profiles === | ||
| Line 162: | Line 164: | ||
Is there any benefit to using the IPC daemon for managing interprocess transaction synchronization since it already has support for that? I guess I'm not clear on how much of this sqlite already does for us. Also, I envision including the IPC daemon as a core mozilla component since it enables support for distributed XPCOM, so we would potentially have it at our disposal. | Is there any benefit to using the IPC daemon for managing interprocess transaction synchronization since it already has support for that? I guess I'm not clear on how much of this sqlite already does for us. Also, I envision including the IPC daemon as a core mozilla component since it enables support for distributed XPCOM, so we would potentially have it at our disposal. | ||
<div style="border: 2px solid green; padding: 0.5em;"> | |||
'''Note from colmsmyth:''' | |||
I've added a comment to [[Mozilla2:Profile Sharing]] relating to enabling Mozilla data formats and protocols to be used by non-Mozilla applications, facilitating Mozilla's role as an platform for internet-aware clients. | |||
If the IPC daemon is to be used to enable distributed XPCOM, please a) consider using it only for service discovery and b) keep the protocol text-based (utf-8) to allow other applications to use the IPC daemon to synchronize access to Mozilla data like the cache, history and bookmarks. | |||
</div> | |||
<div style="border: 2px solid green; padding: 0.5em;"> | |||
'''Note from Standard8''' | |||
Not all applications that want to share profile data will necessarily have the .mozilla prefix on the path name, e.g. SeaMonkey uses .mozilla.org/seamonkey, I could foresee other apps (songbird?) wanting to be able to access data in a common profile area as well. | |||
</div> | |||
=== Comments from Vlad: === | === Comments from Vlad: === | ||
| Line 226: | Line 245: | ||
> Migration -from- Firefox (or any other xul/toolkit app) is not our problem, provided that we don't intentionally make it difficult. | > Migration -from- Firefox (or any other xul/toolkit app) is not our problem, provided that we don't intentionally make it difficult. | ||
sqlite is _worlds_ more usable for other apps than our current mix of ad-hoc formats and | sqlite is _worlds_ more usable for other apps than our current mix of ad-hoc formats and [[Mork]] chicanery. | ||
=== Comments from ago === | === Comments from ago === | ||
| Line 300: | Line 319: | ||
Any reason why you cannot use iODBC (http://www.iodbc.org) to keep your SQL Data Storage and Access DB Engine independent? This will not hinder your choice of SQL Lite as the default SQL Engine since ODBC Drivers already exist for SQL Lite. This is an important architectural decision with longterm ramifications. | Any reason why you cannot use iODBC (http://www.iodbc.org) to keep your SQL Data Storage and Access DB Engine independent? This will not hinder your choice of SQL Lite as the default SQL Engine since ODBC Drivers already exist for SQL Lite. This is an important architectural decision with longterm ramifications. | ||
===Follow-on comments from Kingsley Idehen === | |||
Since my earlier post (above) we have now released OpenLink Virtuoso in Open Source form. Note that Virtuoso is an Object-Relational DBMS Engine that includes support for; SPARQL, SQL-200n, XPath, and XQuery. It is a bonafide RDF Triple Store amongst other things. | |||
SPARQL protocol and Query Language support in Mozilla will open up alot of intriguing possibilities for Mozilla and Firefox (this would make Virtuoso based Data Spaces and other SPARQL Query Language Language & Procotol compliant Triple Stores directly accessible to the the browser (e.g. as an advanced search option feature) | |||
Technical details are available from the Virtuoso Wiki at: http://virtuoso.openlinksw.com/wiki/main/ | |||
Live SPARQL Demo is available at: http://demo.openlinksw.com/sparql_demo/ | |||
=== Comments from guanxi === | === Comments from guanxi === | ||
| Line 306: | Line 333: | ||
<blockquote>A good rule of thumb is that you should avoid using SQLite in situations where the same database will be accessed simultaneously from many computers over a network filesystem.</blockquote></em> | <blockquote>A good rule of thumb is that you should avoid using SQLite in situations where the same database will be accessed simultaneously from many computers over a network filesystem.</blockquote></em> | ||
Wouldn't that conflict with the goals of [[Mozilla2:Multi User Sharing]]? Also, users store profiles on network shares; whether that's officially supported or not (?), do we want to create additional issues for them? | Wouldn't that conflict with the goals of [[Mozilla2:Multi User Sharing]]? Also, users store profiles on network shares; whether that's officially supported or not (?), do we want to create additional issues for them? | ||
=== Comments from nekrad === | |||
Introducing an unified storage scheme for everything in one seems to be something like reinventing SQL or LDAP. But this can't be the issue. Instead SQL, LDAP or even flat text files should be the available datasource backends. | |||
There were many things mentioned here, ie. bookmarks, ssl certs, preferences, mail header caches, etc, etc. They're not all the same kind of data, instead fall apart into several different classes. | |||
For example, the bookmarks are an hierachical thing, history is an "flat" table, preferences are an propertylist. These are completely different type of data, so we should have different APIs for them. What they all have in common, that they're datasources which should be identified by an unique name or descriptor (URI). | |||
We also should consider that people don't want to have all kind of data in the same place. I personally like to have bookmarks stored in an postgresql db, share my mail aliases w/ mutt, etc. So each kind of data should get its own datasource. Each profile could have its root property list, where the URLs to the actual datasources are defined. | |||
You perhaps like to have a look at a related bug: | |||
* https://bugzilla.mozilla.org/show_bug.cgi?id=345055 | |||
---- | |||
SteveW: | |||
I think FireBird would be much preferable to SQLite, especially for Windows users as SQLite is flakey on Windows network drives. | |||
=== Comments from DanBri === | |||
The SPARQL query language (see notes in [http://esw.w3.org/topic/SparqlImplementations W3C ESW wiki) is a good candidate for such an abstract layer. Mozilla always had a notion of breaking things out into datasources, ... but the granularity (per-triple calls) was too fine-grained. SPARQL addresses this in a couple of ways. Firstly, you can wrap datasources by having them expose their data "as RDF" without doing it in terms of a triple-centric API. So for example, look at http://jena.sourceforge.net/SquirrelRDF/ which allows both LDAP and vanilla SQL to be mapped into RDF. Secondly, the query language itself has a notion of data provenance, as expressed through the GRAPH keyword, which has some potential for deealing with the 'free floating triples' issue. A query can include constructs that match against RDF graph structures (like a cleaned-up version of the old Mozilla RDF templates mechanism), ... but it can also include constraints that target triples from particular "named graphs". This seems a good match to Mozilla's needs and multi-datasource design, to me. | |||
The query language also comes with a simple XML representation of a result set, suitable for XSLT/Xpath/Xquery etc processing. This could replace a lot of what Templates used to do. | |||
I don't follow all the Mozilla forums but give me a shout if I can be useful somehow! | |||
---- | |||
=== Comments from Broofa (a pseudo-random user) === | |||
Would it be appropriate to have this Unified Storage system allow extension developers/XUL developers in general be able to persist arbitrary data structures? For example, if I wanted to do a "To Do List" extension for Firefox/T-Bird, it'd be nice to have a quick-n-convenient place to store that data w/out having to worry about writing my own XPCOM APIs. (... or maybe there's already a way to do that and I just don't know about it? That's actually what I was looking for when I stumbled across this page. :-) ) | |||
---- | |||
=== Comments from Kingsley Idehen === | |||
Unified Storage and Database Specific Storage simply do not mean the same thing. It was always my belief that "Unified Storage" would be an abstraction above SQL (Relational) and RDF (Graph) Data Models. What's going on? | |||
If you want to talk SQL, then why no do so via an ODBC abstraction? Why the DBMS specificity? Also, if you are going to SQL, does it means RDF has to be tossed out of Mozilla? How about simply leaving it alone for those who are RDF oriented to work with? | |||