User:Rkent/Folder Data Persistence: Difference between revisions

← Older edit

User:Rkent/Folder Data Persistence (view source)

Revision as of 21:16, 2 November 2014

1,776 bytes added , 2 November 2014

m

Conclusions for maildir folderSize

Rkent

Confirmed users

240

edits

@@ Line 85: / Line 85: @@
 As a general rule, changes to folder metadata is first written to dbFolderInfo without doing changes on the equivalent member variables in the msgFolder. The variables in the msgFolder are changed at the end of operations, reading from dbFolderInfo in ReadDBFolderInfo.
-=== Overloaded meaning and definition of folderSize ===
-One of the things that makes folderSize so complex is that its meaning is overloaded, with at least three different uses:
-* On local folders with mbox, folderSize is used to report to the UI the size of the message folder on disk, as well as used as an indicator of whether the summary file is valid. Although these values are the same, the timing issues on updates for these two issues may be different.
-* For maildir, calculating folderSize is slow directly from the disk, but it is not used as an indicator of validity of the message summary file.
-* On IMAP, the summary file is still used with offline folders, but the meaning of folderSize is changed to represent the server-side storage used for messages. For this reason, folderSize is unavailable for use with offline folders to represent the validity of the summary file.
 === ReadDBFolderInfo(force) ===
@@ Line 122: / Line 112: @@
 == folderSize and IMAP databases ==
-For IMAP databases, folderSize has a different meaning, namely it represents our estimate of the server-side storage for messages in a folder. This is really valuable, for example, when you are approaching the disk storage limits of your IMAP folder and need to know where that space is being used.
+=== Overloaded meaning and definition of folderSize ===
+One of the things that makes folderSize so complex is that its meaning is overloaded, with at least three different uses:
+* On local folders with mbox, folderSize is used to report to the UI the size of the message folder on disk, as well as used as an indicator of whether the summary file is valid. Although these values are the same, the timing issues on updates for these two issues may be different.
+* For maildir, calculating folderSize is slow directly from the disk, but it is not used as an indicator of validity of the message summary file.
+* On IMAP, the summary file is still used with offline folders, but the meaning of folderSize is changed to represent the server-side storage used for messages. For this reason, folderSize is unavailable for use with offline folders to represent the validity of the summary file.
+How does IMAP use the summary file and mbox storage, and avoid the paths that check for summary file valid?
+For a local folder with mbox, toggling a message as read results in marking the summary file valid through this stack:
+<pre>
+	xul.dll!nsMailDatabase::SetSummaryValid(bool aValid) Line 119
+ 	xul.dll!nsMailDatabase::EndBatch() Line 78
+ 	xul.dll!nsMsgDBFolder::EnableNotifications(int notificationType, bool enable, bool dbBatching) Line 5085
+ 	xul.dll!nsMsgDBView::ApplyCommandToIndices(int command, unsigned int * indices, int numIndices) Line 2932
+ 	xul.dll!nsMsgDBView::CycleCell(int row, nsITreeColumn * col) Line 2067
+</pre>
+The same operation on IMAP has an overridden EndBatch which is a no-op:
+<pre>
+	xul.dll!nsImapMailDatabase::EndBatch() Line 71
+ 	xul.dll!nsMsgDBFolder::EnableNotifications(int notificationType, bool enable, bool dbBatching) Line 5085
+ 	xul.dll!nsMsgDBView::ApplyCommandToIndices(int command, unsigned int * indices, int numIndices) Line 2932
+ 	xul.dll!nsMsgDBView::CycleCell(int row, nsITreeColumn * col) Line 2067
+</pre>
+But also, SetSummaryValid for IMAP is different, without the folderSize test:
+<pre>
+NS_IMETHODIMP	nsImapMailDatabase::SetSummaryValid(bool valid)
+{
+  if (m_dbFolderInfo)
+  {
+    m_dbFolderInfo->SetVersion(valid ? GetCurVersion() : 0);
+    Commit(nsMsgDBCommitType::kLargeCommit);
+  }
+  return NS_OK;
+}
+</pre>
+Where do we use summaryValid?
+* [http://mxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp#198 in nsMsgFolderCompactor to skip compacting folders that are being parsed]
+* [http://mxr.mozilla.org/comm-central/source/mailnews/db/msgdb/src/nsMsgDatabase.cpp#1267 on nsMsgDatabase::CheckForErrors, which is used in checking for errors when opening databases]
+* In MoveMail, LocalFolders, and POP3Server (all POP3/local)
+So for non-local folders, this is only used when opening the folder.
+== Conclusions for maildir folderSize ==
-But it creates havoc with the overloaded meaning of folderSize. In local folders,
+* maildir needs to send folderSize updates to the msgFolder, but IMAP needs to ignore those. That means that IMAP needs to override SetSizeOnDisk with a noop. Perhaps it needs instead SetSizeOnServer which it will use instead.