Confirmed users
240
edits
User:Rkent/Folder Data Persistence: Difference between revisions
User:Rkent/Folder Data Persistence (view source)
Revision as of 21:16, 2 November 2014
, 2 November 2014Conclusions for maildir folderSize
m (Analysis) |
m (Conclusions for maildir folderSize) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 85: | Line 85: | ||
As a general rule, changes to folder metadata is first written to dbFolderInfo without doing changes on the equivalent member variables in the msgFolder. The variables in the msgFolder are changed at the end of operations, reading from dbFolderInfo in ReadDBFolderInfo. | As a general rule, changes to folder metadata is first written to dbFolderInfo without doing changes on the equivalent member variables in the msgFolder. The variables in the msgFolder are changed at the end of operations, reading from dbFolderInfo in ReadDBFolderInfo. | ||
=== ReadDBFolderInfo(force) === | |||
nsMsgDBFolder::ReadDBFolderInfo(bool force) when force==false is a no-op except for the first time a folder is created, where the folder member objects for folder metadata are initialized from the folder cache. | |||
ReadDBFolderInfo(false) is used typically in a Get...() call, where you want to make sure that the variable has been initialized from the cache before returning it. | |||
ReadDBFolderInfo(true) is used typically after folder metadata has changed in dbFolderInfo, and you want to update the folder member variables, typically also doing any required notifications. See nsMsgDBFolder::UpdateSummaryTotals | |||
=== UpdateSummaryTotals(force) === | |||
In all cases, UpdateSummaryTotals will: | |||
* Initialize member variables using ReadDBFolderInfo(force) | |||
* Notify changes in kTotalMessagesAtom and kTotalUnreadMessagesAtom | |||
* call FlushToFolderCache to update the cache. | |||
When force==true, then the objects are read from dbFolderInfo, so this is the method used to update relevant member variables by reading from dbFolderInfo, doing folder-level notifications, and flushing the changes to the folderCache. | |||
When force==false the situation is trickier. Member variables are initialized from the folder cache, with notifications, without opening the database. This is done when the folder object is first initialized (in GetSubfolders), or in SummaryChanged(). | |||
=== SummaryChanged() === | |||
This is just a synonym for UpdateSummaryTotals(false), and is only used in IMAP. Its main purpose seems to be to write folder metadata to the folder cache. This assumes that the metadata was written to dbFolderInfo at the time it was changed. One common use seems to be after ChangeNumPending... which does changes in dbFolderInfo and notifications but not in folderCache. The call to UpdateSummaryTotals(false) only flushes the changes to the folder cache. | |||
== folderSize and IMAP databases == | |||
=== Overloaded meaning and definition of folderSize === | |||
One of the things that makes folderSize so complex is that its meaning is overloaded, with at least three different uses: | |||
* On local folders with mbox, folderSize is used to report to the UI the size of the message folder on disk, as well as used as an indicator of whether the summary file is valid. Although these values are the same, the timing issues on updates for these two issues may be different. | |||
* For maildir, calculating folderSize is slow directly from the disk, but it is not used as an indicator of validity of the message summary file. | |||
* On IMAP, the summary file is still used with offline folders, but the meaning of folderSize is changed to represent the server-side storage used for messages. For this reason, folderSize is unavailable for use with offline folders to represent the validity of the summary file. | |||
How does IMAP use the summary file and mbox storage, and avoid the paths that check for summary file valid? | |||
For a local folder with mbox, toggling a message as read results in marking the summary file valid through this stack: | |||
<pre> | |||
xul.dll!nsMailDatabase::SetSummaryValid(bool aValid) Line 119 | |||
xul.dll!nsMailDatabase::EndBatch() Line 78 | |||
xul.dll!nsMsgDBFolder::EnableNotifications(int notificationType, bool enable, bool dbBatching) Line 5085 | |||
xul.dll!nsMsgDBView::ApplyCommandToIndices(int command, unsigned int * indices, int numIndices) Line 2932 | |||
xul.dll!nsMsgDBView::CycleCell(int row, nsITreeColumn * col) Line 2067 | |||
</pre> | |||
The same operation on IMAP has an overridden EndBatch which is a no-op: | |||
<pre> | |||
xul.dll!nsImapMailDatabase::EndBatch() Line 71 | |||
xul.dll!nsMsgDBFolder::EnableNotifications(int notificationType, bool enable, bool dbBatching) Line 5085 | |||
xul.dll!nsMsgDBView::ApplyCommandToIndices(int command, unsigned int * indices, int numIndices) Line 2932 | |||
xul.dll!nsMsgDBView::CycleCell(int row, nsITreeColumn * col) Line 2067 | |||
</pre> | |||
But also, SetSummaryValid for IMAP is different, without the folderSize test: | |||
<pre> | |||
NS_IMETHODIMP nsImapMailDatabase::SetSummaryValid(bool valid) | |||
{ | |||
if (m_dbFolderInfo) | |||
{ | |||
m_dbFolderInfo->SetVersion(valid ? GetCurVersion() : 0); | |||
Commit(nsMsgDBCommitType::kLargeCommit); | |||
} | |||
return NS_OK; | |||
} | |||
</pre> | |||
Where do we use summaryValid? | |||
* [http://mxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp#198 in nsMsgFolderCompactor to skip compacting folders that are being parsed] | |||
* [http://mxr.mozilla.org/comm-central/source/mailnews/db/msgdb/src/nsMsgDatabase.cpp#1267 on nsMsgDatabase::CheckForErrors, which is used in checking for errors when opening databases] | |||
* In MoveMail, LocalFolders, and POP3Server (all POP3/local) | |||
So for non-local folders, this is only used when opening the folder. | |||
== Conclusions for maildir folderSize == | |||
* maildir needs to send folderSize updates to the msgFolder, but IMAP needs to ignore those. That means that IMAP needs to override SetSizeOnDisk with a noop. Perhaps it needs instead SetSizeOnServer which it will use instead. | |||