diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index c105620fc3c04a..d8a177020b921b 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -330,11 +330,10 @@ Suppose you configure logging with the following JSON: } } -This configuration does *almost* what we want, except that ``sys.stdout`` would show messages -of severity ``ERROR`` and only events of this severity and higher will be tracked -as well as ``INFO`` and ``WARNING`` messages. To prevent this, we can set up a filter which -excludes those messages and add it to the relevant handler. This can be configured by -adding a ``filters`` section parallel to ``formatters`` and ``handlers``: +This configuration does *almost* what we want, except that ``sys.stdout`` would +also show messages of severity ``ERROR`` and above. To exclude those messages, +we can add a filter to the stdout handler. This can be configured by adding a +``filters`` section parallel to ``formatters`` and ``handlers``: .. code-block:: json @@ -562,9 +561,9 @@ One solution is to use a two-part approach. For the first part, attach only a :class:`QueueHandler` to those loggers which are accessed from performance-critical threads. They simply write to their queue, which can be sized to a large enough capacity or initialized with no upper bound to their -size. The write to the queue will typically be accepted quickly, though you -will probably need to catch the :exc:`queue.Full` exception as a precaution -in your code. If you are a library developer who has performance-critical +size. The write to the queue will typically be accepted quickly. If a bounded +queue is full, :class:`QueueHandler` calls :meth:`~logging.Handler.handleError` +with the record. If you are a library developer who has performance-critical threads in their code, be sure to document this (together with a suggestion to attach only ``QueueHandlers`` to your loggers) for the benefit of other developers who will use your code. @@ -1608,8 +1607,8 @@ application: The most current file is always :file:`logging_rotatingfile_example.out`, and each time it reaches the size limit it is renamed with the suffix -``.1``. Each of the existing backup files is renamed to increment the suffix -(``.1`` becomes ``.2``, etc.) and the ``.6`` file is erased. +``.1``. Before this happens, the oldest backup (``.5``), if present, is +deleted and the remaining backups are renamed (``.4`` becomes ``.5``, etc.). Obviously this example sets the log length much too small as an extreme example. You would want to set *maxBytes* to an appropriate value. @@ -1801,7 +1800,7 @@ Before Python 3.2, there were only two places where this creation was done: * :func:`makeLogRecord`, which is called with a dictionary containing attributes to be added to the LogRecord. This is typically invoked when a suitable dictionary has been received over the network (e.g. in pickle form - via a :class:`~handlers.SocketHandler`, or in JSON form via an + via a :class:`~handlers.SocketHandler`, or as URL-encoded data via an :class:`~handlers.HTTPHandler`). This has usually meant that if you need to do anything special with a @@ -1827,11 +1826,7 @@ do simply by adding new packages or modules and doing :: logger = logging.getLogger(__name__) -at module level). It's probably one too many things to think about. Developers -could also add the filter to a :class:`~logging.NullHandler` attached to their -top-level logger, but this would not be invoked if an application developer -attached a handler to a lower-level library logger --- so output from that -handler would not reflect the intentions of the library developer. +at module level). It's probably one too many things to think about. In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a factory, which you can specify. The factory is just a callable you can set with @@ -1913,15 +1908,15 @@ You can also subclass :class:`QueueListener` to get messages from other kinds of queues, for example a ZeroMQ 'subscribe' socket. Here's an example:: class ZeroMQSocketListener(QueueListener): - def __init__(self, uri, /, *handlers, **kwargs): - self.ctx = kwargs.get('ctx') or zmq.Context() + def __init__(self, uri, /, *handlers, ctx=None, **kwargs): + self.ctx = ctx or zmq.Context() socket = zmq.Socket(self.ctx, zmq.SUB) socket.setsockopt_string(zmq.SUBSCRIBE, '') # subscribe to everything socket.connect(uri) super().__init__(socket, *handlers, **kwargs) - def dequeue(self): - msg = self.queue.recv_json() + def dequeue(self, block): + msg = self.queue.recv_json(flags=0 if block else zmq.NOBLOCK) return logging.makeLogRecord(msg) .. _pynng-handlers: @@ -2245,7 +2240,7 @@ works:: import logging.config import logging.handlers from multiprocessing import Process, Queue, Event, current_process - import os + from multiprocessing import get_start_method import random import time @@ -2281,13 +2276,8 @@ works:: logging.config.dictConfig(config) listener = logging.handlers.QueueListener(q, MyHandler()) listener.start() - if os.name == 'posix': - # On POSIX, the setup logger will have been configured in the - # parent process, but should have been disabled following the - # dictConfig call. - # On Windows, since fork isn't used, the setup logger won't - # exist in the child, so it would be created and the message - # would appear - hence the "if posix" clause. + if get_start_method() == 'fork': + # dictConfig disables the setup logger inherited from the parent. logger = logging.getLogger('setup') logger.critical('Should not appear, because of disabled logger ...') stop_event.wait() @@ -2312,13 +2302,8 @@ works:: logging.CRITICAL] loggers = ['foo', 'foo.bar', 'foo.bar.baz', 'spam', 'spam.ham', 'spam.ham.eggs'] - if os.name == 'posix': - # On POSIX, the setup logger will have been configured in the - # parent process, but should have been disabled following the - # dictConfig call. - # On Windows, since fork isn't used, the setup logger won't - # exist in the child, so it would be created and the message - # would appear - hence the "if posix" clause. + if get_start_method() == 'fork': + # dictConfig disables the setup logger inherited from the parent. logger = logging.getLogger('setup') logger.critical('Should not appear, because of disabled logger ...') for i in range(100): @@ -2345,9 +2330,7 @@ works:: } # The worker process configuration is just a QueueHandler attached to the # root logger, which allows all messages to be sent to the queue. - # We disable existing loggers to disable the "setup" logger used in the - # parent process. This is needed on POSIX because the logger will - # be there in the child following a fork(). + # Disable the parent's "setup" logger if it was inherited via fork. config_worker = { 'version': 1, 'disable_existing_loggers': True, @@ -2365,9 +2348,7 @@ works:: # The listener process configuration shows that the full flexibility of # logging configuration is available to dispatch events to handlers however # you want. - # We disable existing loggers to disable the "setup" logger used in the - # parent process. This is needed on POSIX because the logger will - # be there in the child following a fork(). + # Disable the parent's "setup" logger if it was inherited via fork. config_listener = { 'version': 1, 'disable_existing_loggers': True, @@ -4077,11 +4058,9 @@ How to uniformly handle newlines in logging output Usually, messages that are logged (say to console or file) consist of a single line of text. However, sometimes there is a need to handle messages with -multiple lines - whether because a logging format string contains newlines, or -logged data contains newlines. If you want to handle such messages uniformly, so -that each line in the logged message appears uniformly formatted as if it was -logged separately, you can do this using a handler mixin, as in the following -snippet: +multiple lines because the message or its arguments contain newlines. +To format each line as if it were logged separately, you can use a handler +mixin, as in the following snippet: .. code-block:: python diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index c8ce0df9e937f8..584f0901b8a69f 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -146,13 +146,6 @@ interpreter, and don't just continue from the session described above:: logger.warning('And this, too') logger.error('And non-ASCII stuff, too, like Øresund and Malmö') -.. versionchanged:: 3.9 - The *encoding* argument was added. In earlier Python versions, or if not - specified, the encoding used is the default value used by :func:`open`. While - not shown in the above example, an *errors* argument can also now be passed, - which determines how encoding errors are handled. For available values and - the default, see the documentation for :func:`open`. - And now if we open the file and look at what we have, we should find the log messages: @@ -465,10 +458,9 @@ With the logger object configured, the following methods create log messages: a message and a level that corresponds to their respective method names. The message is actually a format string, which may contain the standard string substitution syntax of ``%s``, ``%d``, ``%f``, and so on. The - rest of their arguments is a list of objects that correspond with the - substitution fields in the message. With regard to ``**kwargs``, the - logging methods care only about a keyword of ``exc_info`` and use it to - determine whether to log exception information. + remaining positional arguments correspond to the substitution fields in the + message. These methods also accept the keyword arguments *exc_info*, *extra*, + *stack_info*, and *stacklevel*, described in :meth:`Logger.debug`. * :meth:`Logger.exception` creates a log message similar to :meth:`Logger.error`. The difference is that :meth:`Logger.exception` dumps a @@ -1006,9 +998,9 @@ Logged messages are formatted for presentation through instances of the use with the % operator and a dictionary. For formatting multiple messages in a batch, instances of -:class:`BufferingFormatter` can be used. In addition to the format -string (which is applied to each message in the batch), there is provision for -header and trailer format strings. +:class:`BufferingFormatter` can be used. A line formatter formats each record +in the batch. To add a header or footer, override +:meth:`~BufferingFormatter.formatHeader` or :meth:`~BufferingFormatter.formatFooter`. When filtering based on logger level and/or handler level is not enough, instances of :class:`Filter` can be added to both :class:`Logger` and @@ -1059,10 +1051,7 @@ In the preceding sections and examples, it has been assumed that the message passed when logging the event is a string. However, this is not the only possibility. You can pass an arbitrary object as a message, and its :meth:`~object.__str__` method will be called when the logging system needs to -convert it to a string representation. In fact, if you want to, you can avoid -computing a string representation altogether - for example, the -:class:`~handlers.SocketHandler` emits an event by pickling it and sending it -over the wire. +convert it to a string representation. Optimization