Skip to content

Commit 64be119

Browse files
Doc: Correct logging HOWTO and cookbook (GH-162198)
(cherry picked from commit b0ccebc) Co-authored-by: Inada Naoki <songofacandy@gmail.com>
1 parent 59b1570 commit 64be119

2 files changed

Lines changed: 32 additions & 64 deletions

File tree

‎Doc/howto/logging-cookbook.rst‎

Lines changed: 25 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -332,11 +332,10 @@ Suppose you configure logging with the following JSON:
332332
}
333333
}
334334
335-
This configuration does *almost* what we want, except that ``sys.stdout`` would show messages
336-
of severity ``ERROR`` and only events of this severity and higher will be tracked
337-
as well as ``INFO`` and ``WARNING`` messages. To prevent this, we can set up a filter which
338-
excludes those messages and add it to the relevant handler. This can be configured by
339-
adding a ``filters`` section parallel to ``formatters`` and ``handlers``:
335+
This configuration does *almost* what we want, except that ``sys.stdout`` would
336+
also show messages of severity ``ERROR`` and above. To exclude those messages,
337+
we can add a filter to the stdout handler. This can be configured by adding a
338+
``filters`` section parallel to ``formatters`` and ``handlers``:
340339

341340
.. code-block:: json
342341
@@ -564,9 +563,9 @@ One solution is to use a two-part approach. For the first part, attach only a
564563
:class:`QueueHandler` to those loggers which are accessed from
565564
performance-critical threads. They simply write to their queue, which can be
566565
sized to a large enough capacity or initialized with no upper bound to their
567-
size. The write to the queue will typically be accepted quickly, though you
568-
will probably need to catch the :exc:`queue.Full` exception as a precaution
569-
in your code. If you are a library developer who has performance-critical
566+
size. The write to the queue will typically be accepted quickly. If a bounded
567+
queue is full, :class:`QueueHandler` calls :meth:`~logging.Handler.handleError`
568+
with the record. If you are a library developer who has performance-critical
570569
threads in their code, be sure to document this (together with a suggestion to
571570
attach only ``QueueHandlers`` to your loggers) for the benefit of other
572571
developers who will use your code.
@@ -1613,8 +1612,8 @@ application:
16131612
16141613
The most current file is always :file:`logging_rotatingfile_example.out`,
16151614
and each time it reaches the size limit it is renamed with the suffix
1616-
``.1``. Each of the existing backup files is renamed to increment the suffix
1617-
(``.1`` becomes ``.2``, etc.) and the ``.6`` file is erased.
1615+
``.1``. Before this happens, the oldest backup (``.5``), if present, is
1616+
deleted and the remaining backups are renamed (``.4`` becomes ``.5``, etc.).
16181617

16191618
Obviously this example sets the log length much too small as an extreme
16201619
example. You would want to set *maxBytes* to an appropriate value.
@@ -1806,7 +1805,7 @@ Before Python 3.2, there were only two places where this creation was done:
18061805
* :func:`makeLogRecord`, which is called with a dictionary containing
18071806
attributes to be added to the LogRecord. This is typically invoked when a
18081807
suitable dictionary has been received over the network (e.g. in pickle form
1809-
via a :class:`~handlers.SocketHandler`, or in JSON form via an
1808+
via a :class:`~handlers.SocketHandler`, or as URL-encoded data via an
18101809
:class:`~handlers.HTTPHandler`).
18111810

18121811
This has usually meant that if you need to do anything special with a
@@ -1832,11 +1831,7 @@ do simply by adding new packages or modules and doing ::
18321831

18331832
logger = logging.getLogger(__name__)
18341833

1835-
at module level). It's probably one too many things to think about. Developers
1836-
could also add the filter to a :class:`~logging.NullHandler` attached to their
1837-
top-level logger, but this would not be invoked if an application developer
1838-
attached a handler to a lower-level library logger --- so output from that
1839-
handler would not reflect the intentions of the library developer.
1834+
at module level). It's probably one too many things to think about.
18401835

18411836
In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a
18421837
factory, which you can specify. The factory is just a callable you can set with
@@ -1918,15 +1913,15 @@ You can also subclass :class:`QueueListener` to get messages from other kinds
19181913
of queues, for example a ZeroMQ 'subscribe' socket. Here's an example::
19191914

19201915
class ZeroMQSocketListener(QueueListener):
1921-
def __init__(self, uri, /, *handlers, **kwargs):
1922-
self.ctx = kwargs.get('ctx') or zmq.Context()
1916+
def __init__(self, uri, /, *handlers, ctx=None, **kwargs):
1917+
self.ctx = ctx or zmq.Context()
19231918
socket = zmq.Socket(self.ctx, zmq.SUB)
19241919
socket.setsockopt_string(zmq.SUBSCRIBE, '') # subscribe to everything
19251920
socket.connect(uri)
19261921
super().__init__(socket, *handlers, **kwargs)
19271922

1928-
def dequeue(self):
1929-
msg = self.queue.recv_json()
1923+
def dequeue(self, block):
1924+
msg = self.queue.recv_json(flags=0 if block else zmq.NOBLOCK)
19301925
return logging.makeLogRecord(msg)
19311926

19321927
.. _pynng-handlers:
@@ -2250,7 +2245,7 @@ works::
22502245
import logging.config
22512246
import logging.handlers
22522247
from multiprocessing import Process, Queue, Event, current_process
2253-
import os
2248+
from multiprocessing import get_start_method
22542249
import random
22552250
import time
22562251

@@ -2286,13 +2281,8 @@ works::
22862281
logging.config.dictConfig(config)
22872282
listener = logging.handlers.QueueListener(q, MyHandler())
22882283
listener.start()
2289-
if os.name == 'posix':
2290-
# On POSIX, the setup logger will have been configured in the
2291-
# parent process, but should have been disabled following the
2292-
# dictConfig call.
2293-
# On Windows, since fork isn't used, the setup logger won't
2294-
# exist in the child, so it would be created and the message
2295-
# would appear - hence the "if posix" clause.
2284+
if get_start_method() == 'fork':
2285+
# dictConfig disables the setup logger inherited from the parent.
22962286
logger = logging.getLogger('setup')
22972287
logger.critical('Should not appear, because of disabled logger ...')
22982288
stop_event.wait()
@@ -2317,13 +2307,8 @@ works::
23172307
logging.CRITICAL]
23182308
loggers = ['foo', 'foo.bar', 'foo.bar.baz',
23192309
'spam', 'spam.ham', 'spam.ham.eggs']
2320-
if os.name == 'posix':
2321-
# On POSIX, the setup logger will have been configured in the
2322-
# parent process, but should have been disabled following the
2323-
# dictConfig call.
2324-
# On Windows, since fork isn't used, the setup logger won't
2325-
# exist in the child, so it would be created and the message
2326-
# would appear - hence the "if posix" clause.
2310+
if get_start_method() == 'fork':
2311+
# dictConfig disables the setup logger inherited from the parent.
23272312
logger = logging.getLogger('setup')
23282313
logger.critical('Should not appear, because of disabled logger ...')
23292314
for i in range(100):
@@ -2350,9 +2335,7 @@ works::
23502335
}
23512336
# The worker process configuration is just a QueueHandler attached to the
23522337
# root logger, which allows all messages to be sent to the queue.
2353-
# We disable existing loggers to disable the "setup" logger used in the
2354-
# parent process. This is needed on POSIX because the logger will
2355-
# be there in the child following a fork().
2338+
# Disable the parent's "setup" logger if it was inherited via fork.
23562339
config_worker = {
23572340
'version': 1,
23582341
'disable_existing_loggers': True,
@@ -2370,9 +2353,7 @@ works::
23702353
# The listener process configuration shows that the full flexibility of
23712354
# logging configuration is available to dispatch events to handlers however
23722355
# you want.
2373-
# We disable existing loggers to disable the "setup" logger used in the
2374-
# parent process. This is needed on POSIX because the logger will
2375-
# be there in the child following a fork().
2356+
# Disable the parent's "setup" logger if it was inherited via fork.
23762357
config_listener = {
23772358
'version': 1,
23782359
'disable_existing_loggers': True,
@@ -4082,11 +4063,9 @@ How to uniformly handle newlines in logging output
40824063

40834064
Usually, messages that are logged (say to console or file) consist of a single
40844065
line of text. However, sometimes there is a need to handle messages with
4085-
multiple lines - whether because a logging format string contains newlines, or
4086-
logged data contains newlines. If you want to handle such messages uniformly, so
4087-
that each line in the logged message appears uniformly formatted as if it was
4088-
logged separately, you can do this using a handler mixin, as in the following
4089-
snippet:
4066+
multiple lines because the message or its arguments contain newlines.
4067+
To format each line as if it were logged separately, you can use a handler
4068+
mixin, as in the following snippet:
40904069

40914070
.. code-block:: python
40924071

‎Doc/howto/logging.rst‎

Lines changed: 7 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -148,13 +148,6 @@ interpreter, and don't just continue from the session described above::
148148
logger.warning('And this, too')
149149
logger.error('And non-ASCII stuff, too, like Øresund and Malmö')
150150

151-
.. versionchanged:: 3.9
152-
The *encoding* argument was added. In earlier Python versions, or if not
153-
specified, the encoding used is the default value used by :func:`open`. While
154-
not shown in the above example, an *errors* argument can also now be passed,
155-
which determines how encoding errors are handled. For available values and
156-
the default, see the documentation for :func:`open`.
157-
158151
And now if we open the file and look at what we have, we should find the log
159152
messages:
160153

@@ -467,10 +460,9 @@ With the logger object configured, the following methods create log messages:
467460
a message and a level that corresponds to their respective method names. The
468461
message is actually a format string, which may contain the standard string
469462
substitution syntax of ``%s``, ``%d``, ``%f``, and so on. The
470-
rest of their arguments is a list of objects that correspond with the
471-
substitution fields in the message. With regard to ``**kwargs``, the
472-
logging methods care only about a keyword of ``exc_info`` and use it to
473-
determine whether to log exception information.
463+
remaining positional arguments correspond to the substitution fields in the
464+
message. These methods also accept the keyword arguments *exc_info*, *extra*,
465+
*stack_info*, and *stacklevel*, described in :meth:`Logger.debug`.
474466

475467
* :meth:`Logger.exception` creates a log message similar to
476468
:meth:`Logger.error`. The difference is that :meth:`Logger.exception` dumps a
@@ -1008,9 +1000,9 @@ Logged messages are formatted for presentation through instances of the
10081000
use with the % operator and a dictionary.
10091001

10101002
For formatting multiple messages in a batch, instances of
1011-
:class:`BufferingFormatter` can be used. In addition to the format
1012-
string (which is applied to each message in the batch), there is provision for
1013-
header and trailer format strings.
1003+
:class:`BufferingFormatter` can be used. A line formatter formats each record
1004+
in the batch. To add a header or footer, override
1005+
:meth:`~BufferingFormatter.formatHeader` or :meth:`~BufferingFormatter.formatFooter`.
10141006

10151007
When filtering based on logger level and/or handler level is not enough,
10161008
instances of :class:`Filter` can be added to both :class:`Logger` and
@@ -1061,10 +1053,7 @@ In the preceding sections and examples, it has been assumed that the message
10611053
passed when logging the event is a string. However, this is not the only
10621054
possibility. You can pass an arbitrary object as a message, and its
10631055
:meth:`~object.__str__` method will be called when the logging system needs to
1064-
convert it to a string representation. In fact, if you want to, you can avoid
1065-
computing a string representation altogether - for example, the
1066-
:class:`~handlers.SocketHandler` emits an event by pickling it and sending it
1067-
over the wire.
1056+
convert it to a string representation.
10681057

10691058

10701059
Optimization

0 commit comments

Comments
 (0)