Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 25 additions & 46 deletions Doc/howto/logging-cookbook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -332,11 +332,10 @@ Suppose you configure logging with the following JSON:
}
}

This configuration does *almost* what we want, except that ``sys.stdout`` would show messages
of severity ``ERROR`` and only events of this severity and higher will be tracked
as well as ``INFO`` and ``WARNING`` messages. To prevent this, we can set up a filter which
excludes those messages and add it to the relevant handler. This can be configured by
adding a ``filters`` section parallel to ``formatters`` and ``handlers``:
This configuration does *almost* what we want, except that ``sys.stdout`` would
also show messages of severity ``ERROR`` and above. To exclude those messages,
we can add a filter to the stdout handler. This can be configured by adding a
``filters`` section parallel to ``formatters`` and ``handlers``:

.. code-block:: json

Expand Down Expand Up @@ -564,9 +563,9 @@ One solution is to use a two-part approach. For the first part, attach only a
:class:`QueueHandler` to those loggers which are accessed from
performance-critical threads. They simply write to their queue, which can be
sized to a large enough capacity or initialized with no upper bound to their
size. The write to the queue will typically be accepted quickly, though you
will probably need to catch the :exc:`queue.Full` exception as a precaution
in your code. If you are a library developer who has performance-critical
size. The write to the queue will typically be accepted quickly. If a bounded
queue is full, :class:`QueueHandler` calls :meth:`~logging.Handler.handleError`
with the record. If you are a library developer who has performance-critical
threads in their code, be sure to document this (together with a suggestion to
attach only ``QueueHandlers`` to your loggers) for the benefit of other
developers who will use your code.
Expand Down Expand Up @@ -1613,8 +1612,8 @@ application:

The most current file is always :file:`logging_rotatingfile_example.out`,
and each time it reaches the size limit it is renamed with the suffix
``.1``. Each of the existing backup files is renamed to increment the suffix
(``.1`` becomes ``.2``, etc.) and the ``.6`` file is erased.
``.1``. Before this happens, the oldest backup (``.5``), if present, is
deleted and the remaining backups are renamed (``.4`` becomes ``.5``, etc.).

Obviously this example sets the log length much too small as an extreme
example. You would want to set *maxBytes* to an appropriate value.
Expand Down Expand Up @@ -1806,7 +1805,7 @@ Before Python 3.2, there were only two places where this creation was done:
* :func:`makeLogRecord`, which is called with a dictionary containing
attributes to be added to the LogRecord. This is typically invoked when a
suitable dictionary has been received over the network (e.g. in pickle form
via a :class:`~handlers.SocketHandler`, or in JSON form via an
via a :class:`~handlers.SocketHandler`, or as URL-encoded data via an
:class:`~handlers.HTTPHandler`).

This has usually meant that if you need to do anything special with a
Expand All @@ -1832,11 +1831,7 @@ do simply by adding new packages or modules and doing ::

logger = logging.getLogger(__name__)

at module level). It's probably one too many things to think about. Developers
could also add the filter to a :class:`~logging.NullHandler` attached to their
top-level logger, but this would not be invoked if an application developer
attached a handler to a lower-level library logger --- so output from that
handler would not reflect the intentions of the library developer.
at module level). It's probably one too many things to think about.

In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a
factory, which you can specify. The factory is just a callable you can set with
Expand Down Expand Up @@ -1918,15 +1913,15 @@ You can also subclass :class:`QueueListener` to get messages from other kinds
of queues, for example a ZeroMQ 'subscribe' socket. Here's an example::

class ZeroMQSocketListener(QueueListener):
def __init__(self, uri, /, *handlers, **kwargs):
self.ctx = kwargs.get('ctx') or zmq.Context()
def __init__(self, uri, /, *handlers, ctx=None, **kwargs):
self.ctx = ctx or zmq.Context()
socket = zmq.Socket(self.ctx, zmq.SUB)
socket.setsockopt_string(zmq.SUBSCRIBE, '') # subscribe to everything
socket.connect(uri)
super().__init__(socket, *handlers, **kwargs)

def dequeue(self):
msg = self.queue.recv_json()
def dequeue(self, block):
msg = self.queue.recv_json(flags=0 if block else zmq.NOBLOCK)
return logging.makeLogRecord(msg)

.. _pynng-handlers:
Expand Down Expand Up @@ -2250,7 +2245,7 @@ works::
import logging.config
import logging.handlers
from multiprocessing import Process, Queue, Event, current_process
import os
from multiprocessing import get_start_method
import random
import time

Expand Down Expand Up @@ -2286,13 +2281,8 @@ works::
logging.config.dictConfig(config)
listener = logging.handlers.QueueListener(q, MyHandler())
listener.start()
if os.name == 'posix':
# On POSIX, the setup logger will have been configured in the
# parent process, but should have been disabled following the
# dictConfig call.
# On Windows, since fork isn't used, the setup logger won't
# exist in the child, so it would be created and the message
# would appear - hence the "if posix" clause.
if get_start_method() == 'fork':
# dictConfig disables the setup logger inherited from the parent.
logger = logging.getLogger('setup')
logger.critical('Should not appear, because of disabled logger ...')
stop_event.wait()
Expand All @@ -2317,13 +2307,8 @@ works::
logging.CRITICAL]
loggers = ['foo', 'foo.bar', 'foo.bar.baz',
'spam', 'spam.ham', 'spam.ham.eggs']
if os.name == 'posix':
# On POSIX, the setup logger will have been configured in the
# parent process, but should have been disabled following the
# dictConfig call.
# On Windows, since fork isn't used, the setup logger won't
# exist in the child, so it would be created and the message
# would appear - hence the "if posix" clause.
if get_start_method() == 'fork':
# dictConfig disables the setup logger inherited from the parent.
logger = logging.getLogger('setup')
logger.critical('Should not appear, because of disabled logger ...')
for i in range(100):
Expand All @@ -2350,9 +2335,7 @@ works::
}
# The worker process configuration is just a QueueHandler attached to the
# root logger, which allows all messages to be sent to the queue.
# We disable existing loggers to disable the "setup" logger used in the
# parent process. This is needed on POSIX because the logger will
# be there in the child following a fork().
# Disable the parent's "setup" logger if it was inherited via fork.
config_worker = {
'version': 1,
'disable_existing_loggers': True,
Expand All @@ -2370,9 +2353,7 @@ works::
# The listener process configuration shows that the full flexibility of
# logging configuration is available to dispatch events to handlers however
# you want.
# We disable existing loggers to disable the "setup" logger used in the
# parent process. This is needed on POSIX because the logger will
# be there in the child following a fork().
# Disable the parent's "setup" logger if it was inherited via fork.
config_listener = {
'version': 1,
'disable_existing_loggers': True,
Expand Down Expand Up @@ -4082,11 +4063,9 @@ How to uniformly handle newlines in logging output

Usually, messages that are logged (say to console or file) consist of a single
line of text. However, sometimes there is a need to handle messages with
multiple lines - whether because a logging format string contains newlines, or
logged data contains newlines. If you want to handle such messages uniformly, so
that each line in the logged message appears uniformly formatted as if it was
logged separately, you can do this using a handler mixin, as in the following
snippet:
multiple lines because the message or its arguments contain newlines.
To format each line as if it were logged separately, you can use a handler
mixin, as in the following snippet:

.. code-block:: python

Expand Down
25 changes: 7 additions & 18 deletions Doc/howto/logging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -148,13 +148,6 @@ interpreter, and don't just continue from the session described above::
logger.warning('And this, too')
logger.error('And non-ASCII stuff, too, like Øresund and Malmö')

.. versionchanged:: 3.9
The *encoding* argument was added. In earlier Python versions, or if not
specified, the encoding used is the default value used by :func:`open`. While
not shown in the above example, an *errors* argument can also now be passed,
which determines how encoding errors are handled. For available values and
the default, see the documentation for :func:`open`.

And now if we open the file and look at what we have, we should find the log
messages:

Expand Down Expand Up @@ -467,10 +460,9 @@ With the logger object configured, the following methods create log messages:
a message and a level that corresponds to their respective method names. The
message is actually a format string, which may contain the standard string
substitution syntax of ``%s``, ``%d``, ``%f``, and so on. The
rest of their arguments is a list of objects that correspond with the
substitution fields in the message. With regard to ``**kwargs``, the
logging methods care only about a keyword of ``exc_info`` and use it to
determine whether to log exception information.
remaining positional arguments correspond to the substitution fields in the
message. These methods also accept the keyword arguments *exc_info*, *extra*,
*stack_info*, and *stacklevel*, described in :meth:`Logger.debug`.

* :meth:`Logger.exception` creates a log message similar to
:meth:`Logger.error`. The difference is that :meth:`Logger.exception` dumps a
Expand Down Expand Up @@ -1008,9 +1000,9 @@ Logged messages are formatted for presentation through instances of the
use with the % operator and a dictionary.

For formatting multiple messages in a batch, instances of
:class:`BufferingFormatter` can be used. In addition to the format
string (which is applied to each message in the batch), there is provision for
header and trailer format strings.
:class:`BufferingFormatter` can be used. A line formatter formats each record
in the batch. To add a header or footer, override
:meth:`~BufferingFormatter.formatHeader` or :meth:`~BufferingFormatter.formatFooter`.

When filtering based on logger level and/or handler level is not enough,
instances of :class:`Filter` can be added to both :class:`Logger` and
Expand Down Expand Up @@ -1061,10 +1053,7 @@ In the preceding sections and examples, it has been assumed that the message
passed when logging the event is a string. However, this is not the only
possibility. You can pass an arbitrary object as a message, and its
:meth:`~object.__str__` method will be called when the logging system needs to
convert it to a string representation. In fact, if you want to, you can avoid
computing a string representation altogether - for example, the
:class:`~handlers.SocketHandler` emits an event by pickling it and sending it
over the wire.
convert it to a string representation.


Optimization
Expand Down
Loading