# QueueLogger and Python JSON Logger

#### First posted on: 2017/03/01

Categories: Python

# Using QueueLogger with Python JSON Logger

When logging from multiple processes (via multiprocessing module), using QueueHandler is one approach with Python 2.

QueueHandler however sets exc_info attribute of a LogRecord to None since it is not "pickleable" (more on this later). This becomes a problem when you use python-json-logger to format your logs as JSON since it relies on exc_info being set. The result is you don't get exc_info in your logs. This is no however no longer true since this PR was merged. Sample Code.

## Solution #1: Modify Python JSON logger

The first solution is to look for exc_text which is set by the standard Formatter class as per my PR and set that as the value of exc_info which means at least we get the string representation of the traceback. This PR is now merged.

## Solution #2: Subclass QueueHandler

The second solution is it to subclass logutils.queue.QueueHandler as follows and add pickling support for exc_info via tblib:

# Bring in support for serializing/deserializing tracebacks
# needed by QueueHandler
from tblib import pickling_support
pickling_support.install()

class QueueHandlerWithTraceback(logutils.queue.QueueHandler):
""" QueueHandler with support for pickling/unpickling
Tracebacks via tblib. We only override the prepare()
method to *not* set exc_info=None
"""
def __init__(self, *args, **kwargs):
logutils.queue.QueueHandler.__init__(self, *args, **kwargs)

def prepare(self, record):
self.format(record)
record.msg = record.message
record.args = None
return record


Instead of logutils.queue.QueueHandler, we will then use QueueHandlerWithTraceback instead above (Sample Code).