MailNews:Logging: Difference between revisions

bold
(bold)
 
(98 intermediate revisions by 12 users not shown)
Line 1: Line 1:
<small>[[MailNews:Home Page|<< Back to MailNews:Home Page]]</small>
<small>[[MailNews:Home Page|<< Back to MailNews:Home Page]]</small>
Logs provide data to help developers and triagers understand what is causing an issue. Please provide a log file when requested. If you have difficulty or questions, please ask for help.


__TOC__
__TOC__


The MailNews logging mechanism is activated by setting two environment variables before running a MailNews application.
== Introduction ==
 
Logging in recent years has evolved regarding:
* how logging is enabled - possibilities include command line (original method), and Thunderbird preferences (newer method)
* location of log data - possibilities include logging to a file (original method), and logging to the Error Console (alternative method)
 
Also, we have two types of logging:
* [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]], for calendar, chat, gloda indexing and search, ldap, nntp (news), pop, and smtp (sending mail)
* [[MailNews:Logging#gecko_logging|Gecko logging]], which uses case sensitive [[MailNews:Logging#Thunderbird_Module_Names|module names]], such as IMAP and IMAPAutoSync
 
== Thunderbird Logging ==
 
This is the modern method, where logging is specified by [[MailNews:Logging#Setting_a_Preference|setting a preference]] in Thunderbird.
 
Unless otherwise specified, logging output goes to the Error Console, at Tools > Developer > Error Console or use the hotkey Control+Shift+J (Command+Shift+J on macOS).
 
=== Thunderbird Logging Types ===
 
Log settings of type string are '''Case Sensitive'''.  For example, use <code>All</code>, not <code>ALL</code>.
 
* Calendar - set both <code>calendar.debug.log</code> and <code>calendar.debug.log.verbose</code> to <code>true</code>, and reload calendar pane.  Reference: [[Calendar:QA_CalDAV_Support#Diagnosis_hints|Calendar/Calendar logging]]
* Chat - <code>purple.debug.loglevel</code> set to [https://hg.mozilla.org/comm-central/file/tip/chat/chat-prefs.js#l94]
** 0  Show all libpurple messages (PURPLE_DEBUG_ALL)
** 1  Very verbose (PURPLE_DEBUG_MISC)
**  2  Verbose (PURPLE_DEBUG_INFO)
**  3  Show warnings (PURPLE_DEBUG_WARNING)
**  4  Show errors (PURPLE_DEBUG_ERROR)
**  5  Show only fatal errors (PURPLE_DEBUG_FATAL)
* Chat Off-The-Record (OTR) protocol for exchanging '''encrypted''' chat messages -  <code>chat.otr.trace</code> boolean preference
* [https://wiki.mozilla.org/Thunderbird:Debugging_Gloda Gloda Indexing (global search) activity to Error Console]
* imap (not available in version 115) - <code>mailnews.imap.loglevel</code> string, set to <code>All</code>, but only if you have set mailnews.imap.jsmodule=true for Beta and Daily channel builds only.
* ldap - <code>mailnews.ldap.loglevel</code>, string, set to <code>All</code>
* NNTP - <code>mailnews.nntp.loglevel</code>, string,  set to <code>All</code>
* OpenPGP/e2ee - logging is described at [[Thunderbird:OpenPGP#Debugging_.2F_Tracing|OpenPGP Debugging]]
* pop3 - <code>mailnews.pop3.loglevel</code>, string,  set to <code>All</code>
* smtp - <code>mailnews.smtp.loglevel</code>, string,  set to <code>All</code>
 
=== Setting a Preference ===
 
To configure logging via a Thunderbird Preference, you must create a hidden Thunderbird preference, not an environment variable. In Thunderbird :
# do Settings > General
# in the search field type <code>config editor</code>
# click <code>Config Editor</code> in the search results
# in the search field paste or type the preference name, for example <code>mailnews.pop3.loglevel</code>
# if you are presented with choices of Boolean, Number, String
## choose Boolean for true or false settings, otherwise choose String
## click the <code>+</code> (plus sign) to add the preference
# change the preferences' value to the desired logging value
#* for string preferences click the pencil icon, change the value, then click the check mark
#* for boolean preferences click the double headed arror to switch between <code>true</code> and <code>false</code>
 
Logging is permanent until you reset or delete a preference (by clicking the trash can).
 
It is not necessary to restart Thunderbird - logging starts immediately. 
 
You can view the output at Tools > Developer > Error Console. To expose timestamp, click the gear icon in top-right of the Error Console, turn on "Show Timestamps".
 
== Gecko Logging ==


== Main Logging options within MailNews ==
Gecko logging using module names is the classic method of getting log data. Gecko logging may be enabled via preferences (not discussed in this document), or command line, which is discussed below.


The following modules/protocols may be logged within MailNews applications (SeaMonkey/Thunderbird) in both debug and release builds:
Thunderbird gets its module logging capabilities from Firefox core, also known as Gecko. Gecko logging capabilities are thoroughly documented in this [https://firefox-source-docs.mozilla.org/xpcom/logging.html excellent reference], including [https://searchfox.org/mozilla-central/search?q=LazyLogModule+.*%5C%28%22&path=&case=true&regexp=true module names].


* mime
=== Thunderbird Module Names ===
* imap
* nntp
* smtp
* pop3
* ldap
* ImapAutoSync
* MSGDB (level 1 is just open and close logging, level 5 lists the open db's on every db close, along with the number of msg hdrs in use)
([[MailNews:Logging#Other_Protocol_Logging_options_within_MailNews|Other options below]])


== Environment Variables to set ==
All names are '''CASE SENSITIVE'''.


Two environment variables should set:
# AbOutlookDirectory (before Thunderbird version 59: nsAbOutlookDirectoryLog)
# AbWinHelper (before Thunderbird version 59: nsAbWinHelperLog)
# BayesianFilter
# CMS (for S/MIME)
# Filters
# IMAP
# IMAP_CS (for CONDSTORE)
# IMAP_KW (for keyword (tag) processing)
# IMAPAutoSync <small>(before Thunderbird version 59: ImapAutoSync)</small>
# IMAPCache (for IMAP caching of messages in no-sync folders)
# IMAPOffline <small>(before Thunderbird version 59: IMAPOFFLINE)</small>
# Import <small>(before Thunderbird version 59: IMPORT)</small>
# ldap <small>(After Thunderbird 90 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small> 
# Mailbox (before Thunderbird version 59: MAILBOX)
# MailDirStore
# MAPI
# MAPIAddressBook
# MIME
# MsgBiff
# MsgCompose <small>(before Thunderbird version 59: msgcompose)</small>
# MsgCopyService <small>(3.3a4 nightly builds starting 5/2/2011)</small>
# MsgDB <small>(level 1=opens/closes, level 5 lists open db's on close, number of msg hdrs in use)</small> (before Thunderbird version 59: MSGDB)
# MsgPurge
# NNTP <small>(After Thunderbird 95 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>
# pop3 <small>(After Thunderbird 98 use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>
# smtp <small>(After Thunderbird 90 use "[[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]])</small>


# NSPR_LOG_MODULES - Module name and level separated by colon
Notes:
#* Example: "imap:5"
* '''The current, full list of Thunderbird module names is seen in [https://searchfox.org/comm-central/search?q=LazyLogModule.*%5C%28&path=mailnews&case=false&regexp=true this searchfox query].'''
#* Example: "ldap:5,smtp:3"
* Gecko modules potentially useful to Thunderbird users:  
#* adding optional ",timestamp" time stamps all log entries <small>(as of 2009-05-13 3.0b3pre builds)</small><p>
** nsHostResolver (for DNS)
# NSPR_LOG_FILE - Path to the log file
** GetAddrInfo (for DNS)
#* If NSPR_LOG_FILE is not set, then the output will be logged to the console where the application was launched.
** nsSocketTransport (basic networking)
** negotiateauth (authentication)
* Version 59 {{bug|1353919}} changed these options to be mixed case. The old, inconsistent case options are in parenthesis above.
* Deprecated:
** Movemail
** AbOutlookDirFactory (before Thunderbird version 59: nsAbOutlookDirFactoryLog)
** WABAddressBook


On all platforms, the log is written over every time you re-launch the mozilla application.
=== Environment Variables to set ===


=== Logging level ===
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].


The number in the NSPR_LOG_MODULES example above specifies the level of logging to be used. A lower number reduces the amount of information that is logged to the file:
Two variables must be set, '''MOZ_LOG''' and '''MOZ_LOG_FILE''' as local environment variables in a batch file/script (described [[MailNews:Logging#Generating_a_Gecko_Log|below]]) or on a command line. (using global system environment variables is not recommended)


* PR_LOG_NONE = 0, /* nothing */
==== MOZ_LOG_FILE ====
* PR_LOG_ALWAYS = 1, /* always printed */
* PR_LOG_ERROR = 2, /* error messages */
* PR_LOG_WARNING = 3, /* warning messages */
* PR_LOG_DEBUG = 4, /* debug messages */


<small>Use "5" unless requested otherwise. Some modules may not work exactly according to the log levels noted above. Very few of the modules actually have different logging levels</small>
'''MOZ_LOG_FILE''' is the path and file name to the log file. An extension of '''.moz_log''' will be added to the file name (since Thunderbird 70). If MOZ_LOG_FILE is not set, then the output will be logged to the console/terminal (stdout) where the application was launched.


== Generating a Protocol Log ==
Example:
<code>set MOZ_LOG_FILE=thunderbird-log-file</code>


=== Windows ===
Note:
* The log file is written over every time you re-launch the application.
* You must have write access to the directory of the log file. The user's desktop directory or system temp directory are often a good choices.
* If you run multiple applications at the same time, such as Firefox and Thunderbird, then might both write to the log file and possibly wipe out the other's log entries. Workarounds:
** Run only one application at a time
** Use the <code>append</code> option
** Force logging to the console by setting MOZ_LOG_FILE to null, <code>MOZ_LOG_FILE=''</code>.


There are various ways to set the environment variables for protocol logging. One is to create a batch file e.g. "create_imap_log.bat" with the following lines:
==== MOZ_LOG ====


set NSPR_LOG_MODULES=imap:5
'''MOZ_LOG''' syntax is a list of terms, separated by commas. There are two types of terms:
set NSPR_LOG_FILE=c:\imap.log
* A '''CASE SENSITIVE''' module name and its [[MailNews:Logging#Logging_Level|log level]], separated by a colon (:), such as '''example_module:5''' to enable the module example_module at logging level 5 (verbose).
"C:\Program Files\Mozilla Thunderbird\thunderbird.exe"
* Optional special strings (switches) to configure the logging behaviour. Some configuration switches take an integer parameter, in which case the integer is separated from the string by a colon (:). Most switches only apply in a specific output context. Full documentation at [https://firefox-source-docs.mozilla.org/xpcom/logging.html#the-moz-log-syntax].


To get a log for another protocol, replace "imap" with e.g. "smtp" or "pop3" in the above instructions.
Example module and loglevel: '''"IMAP:4,IMAPAutoSync:5,timestamp"'''


When you want to run Thunderbird/SeaMonkey with logging, just run the batch file. You can double click it, create a shortcut to it, move it into your start menu, or run it from a command prompt or the start>run dialog.
''',timestamp''' adds a timestamp to all log lines - generally recommended, and required for all timeouts and issues which need to be correlated to an action at a given time of day


The above commands can also be executed directly in the command prompt. (On Windows 2000, XP, 2003 Server: Start>Programs>Accessories>Command Prompt.)
===== Logging Level =====


=== Mac OS X ===
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].


To generate an IMAP protocol log, create a text file that contains the desired commands in a text editor such as BBEdit:
The number in MOZ_LOG specifies the level of logging to be used. A lower number reduces the amount of information being logged. Use "5" unless requested otherwise.
 
* 0 = Disabled  /* Indicates logging is disabled. Progammers should not use this directly in code. */
* 1 = Error  /* An error occurred. Also generally something a programmer would consider asserting in a debug build. */
* 2 = Warning  /* A warning often indicates an unexpected state. */
* 3 = Info  /* An informational message, which often indicates the current program state. */
* 4 = Debug  /* A debug message, which is useful for debugging but too verbose to be turned on normally. */
* 5 = Verbose  /* A message that will be printed a lot, useful for debugging program flow and will probably impact performance. */ '''Verbose is not recommended for IMAP.'''
 
<small>Some modules may not work exactly according to the log levels noted above. Very few modules actually have more than one logging level.</small>
 
=== Generating a Gecko Log ===
 
'''NOTE - do not use this section for ldap, nntp, pop3 and smtp, which are configured per [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]].
 
For Gecko logging, use [[MailNews:Logging#Thunderbird_Module_Names|module names]].
 
==== Windows ====
 
Create a batch file by copying the lines below and paste them into the notepad application, and save the file as "create_log.bat".  Variables MOZ_LOG and MOZ_LOG_file must not contain quotation marks.
 
set MOZ_LOG=moduleName:5,timestamp
set MOZ_LOG_FILE=%USERPROFILE%\Desktop\log_file
"%ProgramFiles(x86)%\Mozilla Thunderbird\thunderbird.exe"
 
* '''On versions with User Account Control (UAC), the batch file must be run with administrator privileges.''' Right click on the batch file and select "Run as administrator". Otherwise the log file will not be created and no UAC warnings will be generated.
* XP and Vista users should specify %ProgramFiles% instead of %ProgramFiles(x86)% (which is for Windows 7) - assuming Thunderbird was installed to it's default location.
 
Now run the batch file. The example puts the log file on your desktop:
 
You can set up the batch file to be run using of these methods: double click the batch file, create a shortcut, add it to start menu, from a command window prompt, or from the start>run dialog.
 
Logging commands can also typed directly in the command prompt instead of running a batch file. On Windows 2000, XP, 2003 Server: Start>Programs>Accessories>Command Prompt. On later windows versions, you may type "cmd" and press enter in the launch dialog of the Start button.
 
==== Mac OS X ====
 
To generate an log, create a text file that contains the desired commands in a text editor such as BBEdit:  


  #!/bin/sh
  #!/bin/sh
  export NSPR_LOG_MODULES=imap:5
  export MOZ_LOG=moduleName:5,timestamp
  export NSPR_LOG_FILE=imap.log
  export MOZ_LOG_FILE=$HOME/Desktop/log_file
  /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &
  /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;


Save this file with a filename ending in .command, add execute permission by typing "<tt>chmod a+x <i>filename</i></tt>" in a Terminal window, then double-click it.
If on MacOS 10.5, change the last line to arch -arch i386 /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &amp;


To get a log for another protocol, replace "imap" with e.g. "smtp" or "pop3" in the above instructions.
Save this file with a filename ending in .command, add execute permission by typing "<tt>chmod a+x ''filename''</tt>" in a Terminal window, then double-click it.  


=== Linux/unix ===
Alternatively, you can setup a file so that the protocol tracing will always be in effect. (Not recommended if you use both Firefox and Thunderbird) To do this open a Terminal window, cd into the .MacOSX directory under your login directory and create a file named <tt>environment.plist</tt> containing the following. Non-unix users can use the "pico" editor which I think is available on OS X by default:<br>
<pre>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
&lt;plist version="1.0"&gt;
&lt;dict&gt;
&lt;key&gt;MOZ_LOG_FILE&lt;/key&gt;
&lt;string&gt;/tmp/log_file.txt&lt;/string&gt;
&lt;key&gt;MOZ_LOG&lt;/key&gt;
&lt;string&gt;moduleName:5,timestamp&lt;/string&gt;
&lt;/dict&gt;
&lt;/plist&gt;
</pre>
Note, line starting with "&lt;!DOCTYPE" should be oneline all the way to "dtd", but I'm not sure if that makes a difference.


To generate an IMAP protocol log, run the following commands before running the application from the command line:
==== Linux/unix ====
 
Variables set for MOZ_LOG and MOZ_LOG_file must be specified in quotation marks. MOZ_LOG_FILE must be fully specified without shortcuts, such as "~/". Run the following commands before running the application from the command line:


  # For bash shell (the default shell on most GNU/Linux systems):
  # For bash shell (the default shell on most GNU/Linux systems):
  export NSPR_LOG_MODULES=imap:5
  export MOZ_LOG="moduleName:5,timestamp"
  export NSPR_LOG_FILE=/tmp/imap.log
  export MOZ_LOG_FILE="/tmp/log_file"


  # For tcsh / csh (which is not as common):
  # For tcsh / csh (which is not as common):
  setenv NSPR_LOG_MODULES imap:5
  setenv MOZ_LOG moduleName:5,timestamp
  setenv NSPR_LOG_FILE /tmp/imap.log
  setenv MOZ_LOG_FILE /tmp/log_file
 
To get a log for another protocol, replace "imap" with e.g. "smtp" or "pop3" in the above instructions.


== Reference Documents ==
=== Missing Log File ===


* [http://www.mozilla.org/projects/nspr/reference/html/prlog.html NSPR Reference (logging)]
If your log file is empty or missing, review this document again, and check for these common problems:
* [http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html MailNews troubleshooting]
* If logging ldap, nntp, pop3, or smtp, you '''must''' use [[MailNews:Logging#Thunderbird_Logging|Thunderbird logging]], not Gecko module logging using environment variables.
* [https://bugzilla.mozilla.org/show_bug.cgi?id=411258#c10 Comment on bug about how to add timestamps to logs on windows]
* If logging imap, you '''must''' use [[MailNews:Logging#Gecko_Logging|Gecko module logging]] using environment variables.
* If linux, variables MOZ_LOG and MOZ_LOG_file must be in quotation marks.
* If linux, MOZ_LOG_file must be fully specified without shortcuts, such as "~/"
* Do you have write access to the directory specified in MOZ_LOG_FILE?
* Did you shut down the application?  (the log file is buffered in memory until the application is closed)
* Did you restart your mailnews application, and wipe out the log of the session you wanted to debug?  (every restart wipes out the previous log)
* If you used a batch file on Windows, is the batch file set to run with administrator privileges?
* Do your log variables contain quotation marks or other invalid characters?
* Are your log module names correctly '''CASE SENSITIVE'''?  (meaning mixed case, or camel case which is not all upper case, nor all lower case)


== Other Protocol Logging options within MailNews ==
== NSPR Logging Options ==


The following modules may or may not be available in release builds, but are available in debug builds:
NSPR also has logging.  Miscellaneous [http://www.mozilla.org/projects/nspr/reference/html/prlog.html NSPR Reference (logging)].


* imapoffline
== Enhancement Ideas ==
* bayesianfilter
* msgcompose
* mapi
* nsaboutlookcardlog
* nsabwinhelperlog
* ldapautocomplete
* nswabaddressbook
* nsaboutlookdirectorylog
* movemail
* msgbiff
* msgpurge
* applemailimportlog
* import


== Other Logging Options ==
* {{Bug|492620}} - better, easier logging/diagnostics for Thunderbird [meta]
* {{Bug|193873}} - Add Mozilla logging to UI / [[Thunderbird:Logging_UI]]
* {{Bug|697522}} - precise logging of message filter runs and actions


The following modules may or may not be available in release builds, but are available in debug builds:
== Logging prior to Thunderbird 55.0a1 / SeaMonkey 2.52a1 ==


* nsDragService
Around April 5, 2017, via {{bug|1222244}} and blockers of {{bug|1219461}}, changes were merged to the Mozilla trunk which impacted logging functionality. Prior to these changes, the variable names used to configure logging were '''NSPR_LOG_MODULES''' and '''NSPR_LOG_FILE''' rather than '''MOZ_LOG''' and '''MOZ_LOG_FILE'''. If you are working with an older version, use the old variable names.


== Log Parsing Tools ==
In addition, some log module names changed. The old names are documented below, but will eventually be removed (along with this section) when enough time has past that it is no longer necessary for most people to debug older versions of the applications. Of course, the obsolete documentation will remain accessible in the history of this page.


Logs can be difficult to parse because of their size. Andrew Sutherland has written [http://hg.mozilla.org/users/bugmail_asutherland.org/tb-test-help/file/678da3990640/nspr_imap_log_parser.py a parsing script] in Python for analyzing these logs. The script may need to be tweaked to suit your needs. There is more information at this [https://bugzilla.mozilla.org/show_bug.cgi?id=517466#c4 bug comment].
Note that log module names are '''CASE SENSITIVE'''.
canmove, Confirmed users
2,081

edits