MailNews:Logging: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Update logging levels based on https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Gecko_Logging)
(bold)
 
(49 intermediate revisions by 4 users not shown)
Line 1: Line 1:
<small>[[MailNews:Home Page|<< Back to MailNews:Home Page]]</small>
<small>[[MailNews:Home Page|<< Back to MailNews:Home Page]]</small>


Mailnews logs help developers understand what is causing an issue. Please enable Thunderbird to write extra information to a log file, by setting two environment variables before running a MailNews application.
Logs provide data to help developers and triagers understand what is causing an issue. Please provide a log file when requested. If you have difficulty or questions, please ask for help.


__TOC__
__TOC__


== Logging prior to Thunderbird 55.0a1 / SeaMonkey 2.52a1 ==
== Introduction ==
 
Logging in recent years has evolved regarding:
* how logging is enabled - possibilities include command line (original method), and Thunderbird preferences (newer method)
* location of log data - possibilities include logging to a file (original method), and logging to the Error Console (alternative method)
 
Also, we have two types of logging:
* [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]], for calendar, chat, gloda indexing and search, ldap, nntp (news), pop, and smtp (sending mail)
* [[MailNews:Logging#gecko_logging|Gecko logging]], which uses case sensitive [[MailNews:Logging#Thunderbird_Module_Names|module names]], such as IMAP and IMAPAutoSync
 
== Thunderbird Logging ==
 
This is the modern method, where logging is specified by [[MailNews:Logging#Setting_a_Preference|setting a preference]] in Thunderbird.
 
Unless otherwise specified, logging output goes to the Error Console, at Tools > Developer > Error Console or use the hotkey Control+Shift+J (Command+Shift+J on macOS).
 
=== Thunderbird Logging Types ===
 
Log settings of type string are '''Case Sensitive'''.  For example, use <code>All</code>, not <code>ALL</code>.
 
* Calendar - set both <code>calendar.debug.log</code> and <code>calendar.debug.log.verbose</code> to <code>true</code>, and reload calendar pane.  Reference: [[Calendar:QA_CalDAV_Support#Diagnosis_hints|Calendar/Calendar logging]]
* Chat - <code>purple.debug.loglevel</code> set to [https://hg.mozilla.org/comm-central/file/tip/chat/chat-prefs.js#l94]
** 0  Show all libpurple messages (PURPLE_DEBUG_ALL)
** 1  Very verbose (PURPLE_DEBUG_MISC)
**  2 Verbose (PURPLE_DEBUG_INFO)
**  3  Show warnings (PURPLE_DEBUG_WARNING)
**  4  Show errors (PURPLE_DEBUG_ERROR)
**  5  Show only fatal errors (PURPLE_DEBUG_FATAL)
* Chat Off-The-Record (OTR) protocol for exchanging '''encrypted''' chat messages -  <code>chat.otr.trace</code> boolean preference
* [https://wiki.mozilla.org/Thunderbird:Debugging_Gloda Gloda Indexing (global search) activity to Error Console]
* imap (not available in version 115) - <code>mailnews.imap.loglevel</code> string, set to <code>All</code>, but only if you have set mailnews.imap.jsmodule=true for Beta and Daily channel builds only.
* ldap - <code>mailnews.ldap.loglevel</code>, string, set to <code>All</code>
* NNTP - <code>mailnews.nntp.loglevel</code>, string,  set to <code>All</code>
* OpenPGP/e2ee - logging is described at [[Thunderbird:OpenPGP#Debugging_.2F_Tracing|OpenPGP Debugging]]
* pop3 - <code>mailnews.pop3.loglevel</code>, string,  set to <code>All</code>
* smtp - <code>mailnews.smtp.loglevel</code>, string,  set to <code>All</code>
 
=== Setting a Preference ===
 
To configure logging via a Thunderbird Preference, you must create a hidden Thunderbird preference, not an environment variable. In Thunderbird :
# do Settings > General
# in the search field type <code>config editor</code>
# click <code>Config Editor</code> in the search results
# in the search field paste or type the preference name, for example <code>mailnews.pop3.loglevel</code>
# if you are presented with choices of Boolean, Number, String
## choose Boolean for true or false settings, otherwise choose String
## click the <code>+</code> (plus sign) to add the preference
# change the preferences' value to the desired logging value
#* for string preferences click the pencil icon, change the value, then click the check mark
#* for boolean preferences click the double headed arror to switch between <code>true</code> and <code>false</code>
 
Logging is permanent until you reset or delete a preference (by clicking the trash can).
 
It is not necessary to restart Thunderbird - logging starts immediately. 
 
You can view the output at Tools > Developer > Error Console. To expose timestamp, click the gear icon in top-right of the Error Console, turn on "Show Timestamps".


Around April 5, 2017, via {{bug|1222244}} and blockers of {{bug|1219461}}, changes were merged to the Mozilla trunk which impacted logging functionality. Prior to these changes, the variable names used to configure logging were '''NSPR_LOG_MODULES''' and '''NSPR_LOG_FILE''' rather than '''MOZ_LOG''' and '''MOZ_LOG_FILE'''. If you are working with an older version, use the old variable names.
== Gecko Logging ==


In addition, some log module names changed. The old names are documented below, but will eventually be removed (along with this section) when enough time has past that it is no longer necessary for most people to debug older versions of the applications. Of course, the obsolete documentation will remain accessible in the history of this page.
Gecko logging using module names is the classic method of getting log data. Gecko logging may be enabled via preferences (not discussed in this document), or command line, which is discussed below.  


Note that log module names are case-sensitive in the current logging implementation but were previously case-insensitive.
Thunderbird gets its module logging capabilities from Firefox core, also known as Gecko. Gecko logging capabilities are thoroughly documented in this [https://firefox-source-docs.mozilla.org/xpcom/logging.html excellent reference], including [https://searchfox.org/mozilla-central/search?q=LazyLogModule+.*%5C%28%22&path=&case=true&regexp=true module names].  


== Main module options within MailNews ==
=== Thunderbird Module Names ===


MailNews applications (SeaMonkey/Thunderbird) allow these '''case sensitive''' modules/protocols in both debug and release builds. In {{bug|1353919}} these options have been unified, the old options are in parenthesis:
All names are '''CASE SENSITIVE'''.  


# AbOutlookDirectory (before Thunderbird version 59: nsAbOutlookDirectoryLog)
# AbWinHelper (before Thunderbird version 59: nsAbWinHelperLog)
# BayesianFilter
# BayesianFilter
# IMAP
# CMS (for S/MIME)
# IMAPOffline (before Thunderbird version 59: IMAPOFFLINE)
# Filters
# LDAP (before Thunderbird version 59: ldap)
# IMAP
# IMAP_CS (for CONDSTORE)
# IMAP_KW (for keyword (tag) processing)
# IMAPAutoSync <small>(before Thunderbird version 59: ImapAutoSync)</small>
# IMAPCache (for IMAP caching of messages in no-sync folders)
# IMAPOffline <small>(before Thunderbird version 59: IMAPOFFLINE)</small>
# Import <small>(before Thunderbird version 59: IMPORT)</small>
# ldap <small>(After Thunderbird 90 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small> 
# Mailbox (before Thunderbird version 59: MAILBOX)
# Mailbox (before Thunderbird version 59: MAILBOX)
# MailDirStore
# MailDirStore
# MAPI
# MAPIAddressBook
# MIME
# MIME
# MsgBiff
# MsgBiff
# MsgCompose (before Thunderbird version 59: msgcompose)
# MsgCompose <small>(before Thunderbird version 59: msgcompose)</small>
# MsgCopyService <small>(3.3a4 nightly builds starting 5/2/2011)</small>
# MsgCopyService <small>(3.3a4 nightly builds starting 5/2/2011)</small>
# MsgDB <small>(level 1=opens/closes, level 5 lists open db's on close, number of msg hdrs in use)</small> (before Thunderbird version 59: MSGDB)
# MsgDB <small>(level 1=opens/closes, level 5 lists open db's on close, number of msg hdrs in use)</small> (before Thunderbird version 59: MSGDB)
# MsgPurge
# MsgPurge
# NNTP
# NNTP <small>(After Thunderbird 95 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>
# POP3
# pop3 <small>(After Thunderbird 98 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>
# SMTP
# smtp <small>(After Thunderbird 90 use "[[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>
 
Notes:
* '''The current, full list of Thunderbird module names is seen in [https://searchfox.org/comm-central/search?q=LazyLogModule.*%5C%28&path=mailnews&case=false&regexp=true this searchfox query].'''
* Gecko modules potentially useful to Thunderbird users:
** nsHostResolver (for DNS)
** GetAddrInfo (for DNS)
** nsSocketTransport (basic networking)
** negotiateauth (authentication)
* Version 59 {{bug|1353919}} changed these options to be mixed case. The old, inconsistent case options are in parenthesis above.
* Deprecated:
** Movemail
** AbOutlookDirFactory (before Thunderbird version 59: nsAbOutlookDirFactoryLog)
** WABAddressBook
 
=== Environment Variables to set ===
 
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].


([[MailNews:Logging#Other_Protocol_Logging_options_within_MailNews|Other options below]])
Two variables must be set, '''MOZ_LOG''' and '''MOZ_LOG_FILE''' as local environment variables in a batch file/script (described [[MailNews:Logging#Generating_a_Gecko_Log|below]]) or on a command line. (using global system environment variables is not recommended)  


== Environment Variables to set ==
==== MOZ_LOG_FILE ====


Two variables must be set. These can be as (global/system) environment variables. Or, what we document here, as variables in a batch file/script or on the command line:
'''MOZ_LOG_FILE''' is the path and file name to the log file. An extension of '''.moz_log''' will be added to the file name (since Thunderbird 70). If MOZ_LOG_FILE is not set, then the output will be logged to the console/terminal (stdout) where the application was launched.


# '''MOZ_LOG''' - '''Case sensitive''' module name and log level separated by colon, multiple modules separated by comma, and additional options:
Example:
#* Example module and loglevel with timestamp: "POP3:5,SMTP:3,timestamp"
<code>set MOZ_LOG_FILE=thunderbird-log-file</code>
#* Example module and loglevel: "IMAP:5"
#* recommended (optional) '''",timestamp"''' : a timestamp string is added to all log entries - recommended for all timeouts and issues which need to be correlated to an action at a given time of day <small>(as of 2009-05-13 3.0b3pre builds)</small>
#* optional ",sync" : Buffering in log writing is disabled, so latest log data can be seen by ad-hoc copy of log file
# '''MOZ_LOG_FILE''' - Path to the log file
#* If MOZ_LOG_FILE is not set, then the output will be logged to the console where the application was launched.


Note:  
Note:  
* The log is written over every time you re-launch the mozilla application.
* The log file is written over every time you re-launch the application.
* You must have write access to the directory of the log file.
* You must have write access to the directory of the log file. The user's desktop directory or system temp directory are often a good choices.
* If you run multiple applications at the same time, such as Firefox and Thunderbird, then might both write to the log file and possibly wipe out the other's log entries. Workarounds:
** Run only one application at a time
** Use the <code>append</code> option
** Force logging to the console by setting MOZ_LOG_FILE to null, <code>MOZ_LOG_FILE=''</code>.


=== Logging level ===
==== MOZ_LOG ====
 
'''MOZ_LOG''' syntax is a list of terms, separated by commas. There are two types of terms:
* A '''CASE SENSITIVE''' module name and its [[MailNews:Logging#Logging_Level|log level]], separated by a colon (:), such as '''example_module:5''' to enable the module example_module at logging level 5 (verbose).
* Optional special strings (switches) to configure the logging behaviour. Some configuration switches take an integer parameter, in which case the integer is separated from the string by a colon (:). Most switches only apply in a specific output context. Full documentation at [https://firefox-source-docs.mozilla.org/xpcom/logging.html#the-moz-log-syntax].
 
Example module and loglevel: '''"IMAP:4,IMAPAutoSync:5,timestamp"'''
 
''',timestamp''' adds a timestamp to all log lines - generally recommended, and required for all timeouts and issues which need to be correlated to an action at a given time of day
 
===== Logging Level =====
 
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].


The number in MOZ_LOG specifies the level of logging to be used. A lower number reduces the amount of information being logged. Use "5" unless requested otherwise.
The number in MOZ_LOG specifies the level of logging to be used. A lower number reduces the amount of information being logged. Use "5" unless requested otherwise.


* 0 = Disabled  /* Indicates logging is disabled. This should not be used directly in code. */
* 0 = Disabled  /* Indicates logging is disabled. Progammers should not use this directly in code. */
* 1 = Error  /* An error occurred, generally something you would consider asserting in a debug build.*/
* 1 = Error  /* An error occurred. Also generally something a programmer would consider asserting in a debug build. */
* 2 = Warning  /* A warning often indicates an unexpected state. */
* 2 = Warning  /* A warning often indicates an unexpected state. */
* 3 = Info  /* An informational message, often indicates the current program state. */
* 3 = Info  /* An informational message, which often indicates the current program state. */
* 4 = Debug  /* A debug message, useful for debugging but too verbose to be turned on normally. */
* 4 = Debug  /* A debug message, which is useful for debugging but too verbose to be turned on normally. */
* 5 = Verbose  /* A message that will be printed a lot, useful for debugging program flow and will probably impact performance. */
* 5 = Verbose  /* A message that will be printed a lot, useful for debugging program flow and will probably impact performance. */ '''Verbose is not recommended for IMAP.'''


<small>Some modules may not work exactly according to the log levels noted above. Very few modules actually have more than one logging level</small>
<small>Some modules may not work exactly according to the log levels noted above. Very few modules actually have more than one logging level.</small>


== Generating a Protocol Log ==
=== Generating a Gecko Log ===


=== Windows ===
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].


Create a batch file by copying the lines below and paste them into the notepad application, and save the file as "create_imap_log.bat".  Variables must not contain quotation marks.
For Gecko logging, use [[MailNews:Logging#Thunderbird_Module_Names|module names]].


  set MOZ_LOG=IMAP:5,timestamp
==== Windows ====
  set MOZ_LOG_FILE=%USERPROFILE%\Desktop\imap.log
 
Create a batch file by copying the lines below and paste them into the notepad application, and save the file as "create_log.bat".  Variables MOZ_LOG and MOZ_LOG_file must not contain quotation marks.
 
  set MOZ_LOG=moduleName:5,timestamp
  set MOZ_LOG_FILE=%USERPROFILE%\Desktop\log_file
  "%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"
  "%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"


* To log another protocol, replace "IMAP" with e.g. "SMTP" or "POP3" in the above instructions
* '''On versions with User Account Control (UAC), the batch file must be run with administrator privileges.''' Right click on the batch file and select "Run as administrator". Otherwise the log file will not be created and no UAC warnings will be generated.
* XP and Vista users should specify %ProgramFiles% instead of %ProgramFiles(x86)% (which is for Windows 7) - assuming Thunderbird was installed to it's default location.
* XP and Vista users should specify %ProgramFiles% instead of %ProgramFiles(x86)% (which is for Windows 7) - assuming Thunderbird was installed to it's default location.
* On versions with User Account Control such as Vista and Windows 7, the batch file must be run with administrator privileges. Right click on the batch file and select "Run as administrator". Otherwise the log file will not be created and no UAC warnings will be generated.


Now run the batch file. The example puts the log file on your desktop:
Now run the batch file. The example puts the log file on your desktop:
Line 84: Line 181:
Logging commands can also typed directly in the command prompt instead of running a batch file. On Windows 2000, XP, 2003 Server: Start>Programs>Accessories>Command Prompt. On later windows versions, you may type "cmd" and press enter in the launch dialog of the Start button.
Logging commands can also typed directly in the command prompt instead of running a batch file. On Windows 2000, XP, 2003 Server: Start>Programs>Accessories>Command Prompt. On later windows versions, you may type "cmd" and press enter in the launch dialog of the Start button.


=== Mac OS X ===
==== Mac OS X ====


To generate an IMAP protocol log, create a text file that contains the desired commands in a text editor such as BBEdit:  
To generate an log, create a text file that contains the desired commands in a text editor such as BBEdit:  


  #!/bin/sh
  #!/bin/sh
  export MOZ_LOG=IMAP:5,timestamp
  export MOZ_LOG=moduleName:5,timestamp
  export MOZ_LOG_FILE=$HOME/imap.log
  export MOZ_LOG_FILE=$HOME/Desktop/log_file
  /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;
  /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;
if on 10.5 change the above line to arch -arch i386 /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;
 
If on MacOS 10.5, change the last line to arch -arch i386 /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;


Save this file with a filename ending in .command, add execute permission by typing "<tt>chmod a+x ''filename''</tt>" in a Terminal window, then double-click it.  
Save this file with a filename ending in .command, add execute permission by typing "<tt>chmod a+x ''filename''</tt>" in a Terminal window, then double-click it.  


To get a log for another protocol, replace "imap" with e.g. "smtp" or "pop3" in the above instructions.
Alternatively, you can setup a file so that the protocol tracing will always be in effect. (Not recommended if you use both Firefox and Thunderbird) To do this open a Terminal window, cd into the .MacOSX directory under your login directory and create a file named <tt>environment.plist</tt> containing the following. Non-unix users can use the "pico" editor which I think is available on OS X by default:<br>  
 
Alternatively, you can setup a file so that the protocol tracing will always be in effect. To do this open a Terminal window, cd into the .MacOSX directory under your login directory and create a file named <tt>environment.plist</tt> containing the following. Non-unix users can use the "pico" editor which I think is available on OS X by default:<br>  
<pre>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
<pre>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
&lt;!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
Line 104: Line 200:
&lt;dict&gt;
&lt;dict&gt;
&lt;key&gt;MOZ_LOG_FILE&lt;/key&gt;
&lt;key&gt;MOZ_LOG_FILE&lt;/key&gt;
&lt;string&gt;/tmp/imaptrace.txt&lt;/string&gt;
&lt;string&gt;/tmp/log_file.txt&lt;/string&gt;
&lt;key&gt;MOZ_LOG&lt;/key&gt;
&lt;key&gt;MOZ_LOG&lt;/key&gt;
&lt;string&gt;IMAP:5&lt;/string&gt;
&lt;string&gt;moduleName:5,timestamp&lt;/string&gt;
&lt;/dict&gt;
&lt;/dict&gt;
&lt;/plist&gt;
&lt;/plist&gt;
</pre>  
</pre>  
Note, line starting with "&lt;!DOCTYPE" should be oneline all the way to "dtd", but I'm not sure if that makes a difference.<br><br>
Note, line starting with "&lt;!DOCTYPE" should be oneline all the way to "dtd", but I'm not sure if that makes a difference.


=== Linux/unix ===
==== Linux/unix ====


To generate an IMAP protocol log, run the following commands before running the application from the command line:
Variables set for MOZ_LOG and MOZ_LOG_file must be specified in quotation marks. MOZ_LOG_FILE must be fully specified without shortcuts, such as "~/". Run the following commands before running the application from the command line:


  # For bash shell (the default shell on most GNU/Linux systems):
  # For bash shell (the default shell on most GNU/Linux systems):
  export MOZ_LOG=IMAP:5,timestamp
  export MOZ_LOG="moduleName:5,timestamp"
  export MOZ_LOG_FILE=/tmp/imap.log
  export MOZ_LOG_FILE="/tmp/log_file"


  # For tcsh / csh (which is not as common):
  # For tcsh / csh (which is not as common):
  setenv MOZ_LOG IMAP:5
  setenv MOZ_LOG moduleName:5,timestamp
  setenv MOZ_LOG_FILE /tmp/imap.log
  setenv MOZ_LOG_FILE /tmp/log_file
 
To get a log for another protocol, replace "IMAP" with e.g. "SMTP" or "POP3" in the above instructions.


=== Missing Log File ===
=== Missing Log File ===


If your log file is empty or missing, review this document again, and check for these common problems:
If your log file is empty or missing, review this document again, and check for these common problems:
* Do you have write access to the directory specified?
* If logging ldap, nntp, pop3, or smtp, you '''must''' use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]], not Gecko module logging using environment variables.
* Did you shut down the mailnews application?  (the log file is buffered in memory)
* If logging imap, you '''must''' use [[MailNews:Logging#Gecko_Logging|Gecko module logging]] using environment variables.
* If linux, variables MOZ_LOG and MOZ_LOG_file must be in quotation marks.
* If linux, MOZ_LOG_file must be fully specified without shortcuts, such as "~/"
* Do you have write access to the directory specified in MOZ_LOG_FILE?
* Did you shut down the application?  (the log file is buffered in memory until the application is closed)
* Did you restart your mailnews application, and wipe out the log of the session you wanted to debug?  (every restart wipes out the previous log)
* Did you restart your mailnews application, and wipe out the log of the session you wanted to debug?  (every restart wipes out the previous log)
* If you used a batch file on Windows, is the batch file set to run with administrator privileges?
* If you used a batch file on Windows, is the batch file set to run with administrator privileges?
* Do your log variables contain quotation marks or other invalid characters?
* Do your log variables contain quotation marks or other invalid characters?
* Are your log module names correctly case-sensitive?  (not all upper case nor all lower case)
* Are your log module names correctly '''CASE SENSITIVE'''?  (meaning mixed case, or camel case which is not all upper case, nor all lower case)


== Reference Documents ==
== NSPR Logging Options ==


* [http://www.mozilla.org/projects/nspr/reference/html/prlog.html NSPR Reference (logging)]
NSPR also has logging.  Miscellaneous [http://www.mozilla.org/projects/nspr/reference/html/prlog.html NSPR Reference (logging)].


== Other Protocol Logging options within MailNews ==
== Enhancement Ideas ==  


The following modules may or may not be available in release builds, but are available in debug builds:
* {{Bug|492620}} - better, easier logging/diagnostics for Thunderbird [meta]
* {{Bug|193873}} - Add Mozilla logging to UI / [[Thunderbird:Logging_UI]]
* {{Bug|697522}} - precise logging of message filter runs and actions


# IMAPAutoSync (before Thunderbird version 59: ImapAutoSync)
== Logging prior to Thunderbird 55.0a1 / SeaMonkey 2.52a1 ==
# Import (before Thunderbird version 59: IMPORT)
# MAPI
# MAPIAddressBook
# Movemail
# AbOutlookDirectory (before Thunderbird version 59: nsAbOutlookDirectoryLog)
# AbOutlookDirFactory (before Thunderbird version 59: nsAbOutlookDirFactoryLog)
# AbWinHelper (before Thunderbird version 59: nsAbWinHelperLog)
# WABAddressBook
# negotiateauth (Mozilla core option)


== Other Important MailNews logging ==
Around April 5, 2017, via {{bug|1222244}} and blockers of {{bug|1219461}}, changes were merged to the Mozilla trunk which impacted logging functionality. Prior to these changes, the variable names used to configure logging were '''NSPR_LOG_MODULES''' and '''NSPR_LOG_FILE''' rather than '''MOZ_LOG''' and '''MOZ_LOG_FILE'''. If you are working with an older version, use the old variable names.


* Log Gloda activity to console (Thunderbird only) [https://wiki.mozilla.org/Thunderbird:Debugging_Gloda Debugging Gloda]
In addition, some log module names changed. The old names are documented below, but will eventually be removed (along with this section) when enough time has past that it is no longer necessary for most people to debug older versions of the applications. Of course, the obsolete documentation will remain accessible in the history of this page.
* [[Calendar:QA_CalDAV_Support#Diagnosis_hints|Calendar/Lightning logging]]
 
== Other Useful NSPR Logging Options ==
 
The following modules may or may not be available in release builds, but are available in debug builds:
 
* nsDragService
 
The following modules are not owned by Mail&News, but useful for problem analysis of Mail&News.
 
* timestamp [http://www.mozilla.org/projects/nspr/reference/html/prlog.html#25328 NSPR Reference, Chapter 26 Logging ] (Timestamp is added to NSPR log)
 
* nsHostResolver, nsSocketTransport [https://developer.mozilla.org/en/HTTP_Logging HTTP_Logging] (DNS lookup, Socket log)
 
* DOMLeak, DocumentLeak, nsDocShellLeak [https://wiki.mozilla.org/Performance:Leak_Tools#leak-gauge Leak Gauge] (Internal load of mail can be traced)
 
* pipnss - S/MIME (nsCMSMessage) debugging
 
== Logging Tools ==


* Log Parsing - Logs can be difficult to parse because of their size. Andrew Sutherland has written [http://hg.mozilla.org/users/bugmail_asutherland.org/tb-test-help/file/678da3990640/nspr_imap_log_parser.py a parsing script] in Python for analyzing these logs. The script may need to be tweaked to suit your needs. There is more information at this [https://bugzilla.mozilla.org/show_bug.cgi?id=517466#c4 bug comment].
Note that log module names are '''CASE SENSITIVE'''.
 
* [https://addons.mozilla.org/en-US/thunderbird/addon/tbtracer/ TBTracer addon] by Thundersomething author "derdoc"
 
* [https://addons.mozilla.org/en-US/thunderbird/addon/loghelper/ loghelper addon] by jtcranmer
 
== Enhancement Ideas ==
 
* {{Bug|492620}} - better, easier logging/diagnostics for Thunderbird [meta]
* {{Bug|193873}} - Add Mozilla logging to UI / [[Thunderbird:Logging_UI]]
* {{Bug|697522}} - precise logging of message filter runs and actions

Latest revision as of 15:11, 8 June 2024

<< Back to MailNews:Home Page

Logs provide data to help developers and triagers understand what is causing an issue. Please provide a log file when requested. If you have difficulty or questions, please ask for help.

Introduction

Logging in recent years has evolved regarding:

  • how logging is enabled - possibilities include command line (original method), and Thunderbird preferences (newer method)
  • location of log data - possibilities include logging to a file (original method), and logging to the Error Console (alternative method)

Also, we have two types of logging:

Thunderbird Logging

This is the modern method, where logging is specified by setting a preference in Thunderbird.

Unless otherwise specified, logging output goes to the Error Console, at Tools > Developer > Error Console or use the hotkey Control+Shift+J (Command+Shift+J on macOS).

Thunderbird Logging Types

Log settings of type string are Case Sensitive. For example, use All, not ALL.

  • Calendar - set both calendar.debug.log and calendar.debug.log.verbose to true, and reload calendar pane. Reference: Calendar/Calendar logging
  • Chat - purple.debug.loglevel set to [1]
    • 0 Show all libpurple messages (PURPLE_DEBUG_ALL)
    • 1 Very verbose (PURPLE_DEBUG_MISC)
    • 2 Verbose (PURPLE_DEBUG_INFO)
    • 3 Show warnings (PURPLE_DEBUG_WARNING)
    • 4 Show errors (PURPLE_DEBUG_ERROR)
    • 5 Show only fatal errors (PURPLE_DEBUG_FATAL)
  • Chat Off-The-Record (OTR) protocol for exchanging encrypted chat messages - chat.otr.trace boolean preference
  • Gloda Indexing (global search) activity to Error Console
  • imap (not available in version 115) - mailnews.imap.loglevel string, set to All, but only if you have set mailnews.imap.jsmodule=true for Beta and Daily channel builds only.
  • ldap - mailnews.ldap.loglevel, string, set to All
  • NNTP - mailnews.nntp.loglevel, string, set to All
  • OpenPGP/e2ee - logging is described at OpenPGP Debugging
  • pop3 - mailnews.pop3.loglevel, string, set to All
  • smtp - mailnews.smtp.loglevel, string, set to All

Setting a Preference

To configure logging via a Thunderbird Preference, you must create a hidden Thunderbird preference, not an environment variable. In Thunderbird :

  1. do Settings > General
  2. in the search field type config editor
  3. click Config Editor in the search results
  4. in the search field paste or type the preference name, for example mailnews.pop3.loglevel
  5. if you are presented with choices of Boolean, Number, String
    1. choose Boolean for true or false settings, otherwise choose String
    2. click the + (plus sign) to add the preference
  6. change the preferences' value to the desired logging value
    • for string preferences click the pencil icon, change the value, then click the check mark
    • for boolean preferences click the double headed arror to switch between true and false

Logging is permanent until you reset or delete a preference (by clicking the trash can).

It is not necessary to restart Thunderbird - logging starts immediately.

You can view the output at Tools > Developer > Error Console. To expose timestamp, click the gear icon in top-right of the Error Console, turn on "Show Timestamps".

Gecko Logging

Gecko logging using module names is the classic method of getting log data. Gecko logging may be enabled via preferences (not discussed in this document), or command line, which is discussed below.

Thunderbird gets its module logging capabilities from Firefox core, also known as Gecko. Gecko logging capabilities are thoroughly documented in this excellent reference, including module names.

Thunderbird Module Names

All names are CASE SENSITIVE.

  1. AbOutlookDirectory (before Thunderbird version 59: nsAbOutlookDirectoryLog)
  2. AbWinHelper (before Thunderbird version 59: nsAbWinHelperLog)
  3. BayesianFilter
  4. CMS (for S/MIME)
  5. Filters
  6. IMAP
  7. IMAP_CS (for CONDSTORE)
  8. IMAP_KW (for keyword (tag) processing)
  9. IMAPAutoSync (before Thunderbird version 59: ImapAutoSync)
  10. IMAPCache (for IMAP caching of messages in no-sync folders)
  11. IMAPOffline (before Thunderbird version 59: IMAPOFFLINE)
  12. Import (before Thunderbird version 59: IMPORT)
  13. ldap (After Thunderbird 90 use Thunderbird logging)
  14. Mailbox (before Thunderbird version 59: MAILBOX)
  15. MailDirStore
  16. MAPI
  17. MAPIAddressBook
  18. MIME
  19. MsgBiff
  20. MsgCompose (before Thunderbird version 59: msgcompose)
  21. MsgCopyService (3.3a4 nightly builds starting 5/2/2011)
  22. MsgDB (level 1=opens/closes, level 5 lists open db's on close, number of msg hdrs in use) (before Thunderbird version 59: MSGDB)
  23. MsgPurge
  24. NNTP (After Thunderbird 95 use Thunderbird logging)
  25. pop3 (After Thunderbird 98 use Thunderbird logging)
  26. smtp (After Thunderbird 90 use "Thunderbird logging)

Notes:

  • The current, full list of Thunderbird module names is seen in this searchfox query.
  • Gecko modules potentially useful to Thunderbird users:
    • nsHostResolver (for DNS)
    • GetAddrInfo (for DNS)
    • nsSocketTransport (basic networking)
    • negotiateauth (authentication)
  • Version 59 bug 1353919 changed these options to be mixed case. The old, inconsistent case options are in parenthesis above.
  • Deprecated:
    • Movemail
    • AbOutlookDirFactory (before Thunderbird version 59: nsAbOutlookDirFactoryLog)
    • WABAddressBook

Environment Variables to set

NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per Thunderbird logging.

Two variables must be set, MOZ_LOG and MOZ_LOG_FILE as local environment variables in a batch file/script (described below) or on a command line. (using global system environment variables is not recommended)

MOZ_LOG_FILE

MOZ_LOG_FILE is the path and file name to the log file. An extension of .moz_log will be added to the file name (since Thunderbird 70). If MOZ_LOG_FILE is not set, then the output will be logged to the console/terminal (stdout) where the application was launched.

Example: set MOZ_LOG_FILE=thunderbird-log-file

Note:

  • The log file is written over every time you re-launch the application.
  • You must have write access to the directory of the log file. The user's desktop directory or system temp directory are often a good choices.
  • If you run multiple applications at the same time, such as Firefox and Thunderbird, then might both write to the log file and possibly wipe out the other's log entries. Workarounds:
    • Run only one application at a time
    • Use the append option
    • Force logging to the console by setting MOZ_LOG_FILE to null, MOZ_LOG_FILE=.

MOZ_LOG

MOZ_LOG syntax is a list of terms, separated by commas. There are two types of terms:

  • A CASE SENSITIVE module name and its log level, separated by a colon (:), such as example_module:5 to enable the module example_module at logging level 5 (verbose).
  • Optional special strings (switches) to configure the logging behaviour. Some configuration switches take an integer parameter, in which case the integer is separated from the string by a colon (:). Most switches only apply in a specific output context. Full documentation at [2].

Example module and loglevel: "IMAP:4,IMAPAutoSync:5,timestamp"

,timestamp adds a timestamp to all log lines - generally recommended, and required for all timeouts and issues which need to be correlated to an action at a given time of day

Logging Level

NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per Thunderbird logging.

The number in MOZ_LOG specifies the level of logging to be used. A lower number reduces the amount of information being logged. Use "5" unless requested otherwise.

  • 0 = Disabled /* Indicates logging is disabled. Progammers should not use this directly in code. */
  • 1 = Error /* An error occurred. Also generally something a programmer would consider asserting in a debug build. */
  • 2 = Warning /* A warning often indicates an unexpected state. */
  • 3 = Info /* An informational message, which often indicates the current program state. */
  • 4 = Debug /* A debug message, which is useful for debugging but too verbose to be turned on normally. */
  • 5 = Verbose /* A message that will be printed a lot, useful for debugging program flow and will probably impact performance. */ Verbose is not recommended for IMAP.

Some modules may not work exactly according to the log levels noted above. Very few modules actually have more than one logging level.

Generating a Gecko Log

NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per Thunderbird logging.

For Gecko logging, use module names.

Windows

Create a batch file by copying the lines below and paste them into the notepad application, and save the file as "create_log.bat". Variables MOZ_LOG and MOZ_LOG_file must not contain quotation marks.

set MOZ_LOG=moduleName:5,timestamp
set MOZ_LOG_FILE=%USERPROFILE%\Desktop\log_file
"%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"
  • On versions with User Account Control (UAC), the batch file must be run with administrator privileges. Right click on the batch file and select "Run as administrator". Otherwise the log file will not be created and no UAC warnings will be generated.
  • XP and Vista users should specify %ProgramFiles% instead of %ProgramFiles(x86)% (which is for Windows 7) - assuming Thunderbird was installed to it's default location.

Now run the batch file. The example puts the log file on your desktop:

You can set up the batch file to be run using of these methods: double click the batch file, create a shortcut, add it to start menu, from a command window prompt, or from the start>run dialog.

Logging commands can also typed directly in the command prompt instead of running a batch file. On Windows 2000, XP, 2003 Server: Start>Programs>Accessories>Command Prompt. On later windows versions, you may type "cmd" and press enter in the launch dialog of the Start button.

Mac OS X

To generate an log, create a text file that contains the desired commands in a text editor such as BBEdit:

#!/bin/sh
export MOZ_LOG=moduleName:5,timestamp
export MOZ_LOG_FILE=$HOME/Desktop/log_file
/Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &

If on MacOS 10.5, change the last line to arch -arch i386 /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &

Save this file with a filename ending in .command, add execute permission by typing "chmod a+x filename" in a Terminal window, then double-click it.

Alternatively, you can setup a file so that the protocol tracing will always be in effect. (Not recommended if you use both Firefox and Thunderbird) To do this open a Terminal window, cd into the .MacOSX directory under your login directory and create a file named environment.plist containing the following. Non-unix users can use the "pico" editor which I think is available on OS X by default:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>MOZ_LOG_FILE</key>
<string>/tmp/log_file.txt</string>
<key>MOZ_LOG</key>
<string>moduleName:5,timestamp</string>
</dict>
</plist>

Note, line starting with "<!DOCTYPE" should be oneline all the way to "dtd", but I'm not sure if that makes a difference.

Linux/unix

Variables set for MOZ_LOG and MOZ_LOG_file must be specified in quotation marks. MOZ_LOG_FILE must be fully specified without shortcuts, such as "~/". Run the following commands before running the application from the command line:

# For bash shell (the default shell on most GNU/Linux systems):
export MOZ_LOG="moduleName:5,timestamp"
export MOZ_LOG_FILE="/tmp/log_file"
# For tcsh / csh (which is not as common):
setenv MOZ_LOG moduleName:5,timestamp 
setenv MOZ_LOG_FILE /tmp/log_file

Missing Log File

If your log file is empty or missing, review this document again, and check for these common problems:

  • If logging ldap, nntp, pop3, or smtp, you must use Thunderbird logging, not Gecko module logging using environment variables.
  • If logging imap, you must use Gecko module logging using environment variables.
  • If linux, variables MOZ_LOG and MOZ_LOG_file must be in quotation marks.
  • If linux, MOZ_LOG_file must be fully specified without shortcuts, such as "~/"
  • Do you have write access to the directory specified in MOZ_LOG_FILE?
  • Did you shut down the application? (the log file is buffered in memory until the application is closed)
  • Did you restart your mailnews application, and wipe out the log of the session you wanted to debug? (every restart wipes out the previous log)
  • If you used a batch file on Windows, is the batch file set to run with administrator privileges?
  • Do your log variables contain quotation marks or other invalid characters?
  • Are your log module names correctly CASE SENSITIVE? (meaning mixed case, or camel case which is not all upper case, nor all lower case)

NSPR Logging Options

NSPR also has logging. Miscellaneous NSPR Reference (logging).

Enhancement Ideas

Logging prior to Thunderbird 55.0a1 / SeaMonkey 2.52a1

Around April 5, 2017, via bug 1222244 and blockers of bug 1219461, changes were merged to the Mozilla trunk which impacted logging functionality. Prior to these changes, the variable names used to configure logging were NSPR_LOG_MODULES and NSPR_LOG_FILE rather than MOZ_LOG and MOZ_LOG_FILE. If you are working with an older version, use the old variable names.

In addition, some log module names changed. The old names are documented below, but will eventually be removed (along with this section) when enough time has past that it is no longer necessary for most people to debug older versions of the applications. Of course, the obsolete documentation will remain accessible in the history of this page.

Note that log module names are CASE SENSITIVE.