gluetool.log module¶
Logging support.
Sets up logging environment for use by gluetool
and modules. Based
on standard library’s logging
module, augmented a bit to
support features loke colorized messages and stackable context information.
Example usage:
# initialize logger as soon as possible
Logging.setup_logger()
logger = Logging.get_logger()
# now it's possible to use it for logging:
logger.debug('foo!')
# find out what your logging should look like, e.g. by parsing command-line options
...
# tell logger about the final setup
logger = Logging.setup_logger(output_file='/tmp/foo.log', level=...)
# when you want to make logging methods easily accessible in your class, throw in
# LoggerMixin:
class Foo(LoggerMixin, ...):
def __init__(self, some_logger):
super(Foo, self).__init__(logger)
self.debug('foo!')
-
class
gluetool.log.
BlobLogger
(intro, outro=None, on_finally=None, writer=None)[source]¶ Bases:
object
Context manager to help with “real time” logging - some code may produce output continuously, e.g. when running a command and streaming its output to our stdout, and yet we still want to wrap it with boundaries and add a header.
This code:
with BlobLogger('ls of root', outro='end of ls'): subprocess.call(['ls', '/'])
will lead to the output similar to this:
[20:30:50] [+] ---v---v---v---v---v--- ls of root bin boot data dev ... [20:30:50] [+] ---^---^---^---^---^--- end of ls
Note
When you already hold the data you wish to log, please use
gluetool.log.log_blob()
orgluetool.log.log_dict()
. The example above could be rewritten usinglog_blob
by usingsubprocess.check_output()
and passing its return value tolog_blob
.BlobLogger
is designed to wrap output whose creation caller don’t want to (or cannot) control.Parameters: - intro (str) – Label to show what is the meaning of the logged data.
- outro (str) – Label to show by the final boundary to mark the end of logging.
- on_finally (callable) – When set, it will be called in
__exit__
method. User of this context manager might need to flush used streams or close resources even in case the exception was raised while inside the context manager.on_finally
is called with all arguments the__exit__
was called, and its return value is returned by__exit__
itself, therefore it can examine possible exceptions, and override them. - writer (callable) – A function which is used to actually log the text. Usually a one of some logger methods.
-
class
gluetool.log.
ContextAdapter
(logger, extra=None)[source]¶ Bases:
logging.LoggerAdapter
Generic logger adapter that collects “contexts”, and prepends them to the message.
“context” is any key in
extra
dictionary starting withctx_
, whose value is expected to be tuple of(priority, value)
. Contexts are then sorted by their priorities before inserting them into the message (lower priority means context will be placed closer to the beggining of the line - highest priority comes last.ContextAdapter
is a corner stone of our logging infrastructure, everything is supposed to, one way or another, to use this class (or one of its children) for logging. We should avoid using bareLogger
instances because they lack context propagation, message routing to debug and verbose files,verbose
method &sentry
parameter.Parent class,
logging.LoggerAdapter
, provides all common logging methods. We overload them with our own implementations because we want to 1) handle type annotations on our side, 2) let users submit messages to Sentry.Parameters: - logger – parent logger this adapter modifies.
- extras (dict) – additional extra keys passed to the parent class.
The dictionary is then used to update messages’
extra
key with the information about context.
-
process
(msg, kwargs)[source]¶ Original
process
overwriteskwargs['extra']
which doesn’t work for us - we want to chain adapters, getting more and more contexts on the way. Thereforeupdate
instead of assignment.
-
warn
(msg, exc_info=None, extra=None, sentry=False)¶
-
class
gluetool.log.
JSONLoggingFormatter
(**kwargs)[source]¶ Bases:
logging.Formatter
Custom logging formatter producing a JSON dictionary describing the log record.
-
static
_format_exception_chain
(serialized, exc_info)[source]¶ “Format” exception chain - transform it into a bunch of JSON structures describing exceptions, stack frames, local variables and so on.
Serves the same purpose as
LoggingFormatter._format_exception_chain
but that one produces a string, textual representation suitable for printing. This method produces JSON structures, suitable for, hm, JSON log.
-
static
-
class
gluetool.log.
LoggerMixin
(logger, *args, **kwargs)[source]¶ Bases:
object
Use as a parent class (or one of them) when you want to “attach” methods of a given logger to class’ instances.
Parameters: logger (ContextAdapter) – logger to propagate.
-
class
gluetool.log.
Logging
[source]¶ Bases:
object
Container wrapping configuration and access to
logging
infrastructuregluetool
uses for logging.-
OUR_LOGGERS
= (<logging.Logger object>, <logging.Logger object>, <logging.Logger object>)¶
-
static
_setup_log_file
(filepath, level, limit_level=False, formatter_class=<class 'gluetool.log.LoggingFormatter'>)[source]¶
-
adapted_logger
= None¶ Logger singleton - if anyone asks for a logger, they will get this one. Needs to be properly initialized by calling
setup_logger()
.
-
static
configure_logger
(logger)[source]¶ Configure given logger to conform with Gluetool’s idea of logging. The logger is set to
VERBOSE
level, shared stderr handler is added, and Sentry integration status is propagated as well.After this method, the logger will behave like Gluetool’s main logger.
-
debug_file_handler
= None¶
-
static
get_logger
()[source]¶ Returns a logger-like object suitable for logging stuff.
Return type: ContextAdapter Returns: an instance of ContextAdapter wrapping root logger, or None
when there’s no logger yet.
-
json_file_handler
= None¶
-
logger
= None¶ Bare root logger we’re hiding inside
adapted_logger
-
sentry
= None¶
-
static
setup_logger
(level=20, debug_file=None, verbose_file=None, json_file=None, sentry=None, show_traceback=False)[source]¶ Create and setup logger.
This method is called at least twice:
- when
gluetool.glue.Glue
is instantiated: only astderr
handler is set up, with loglevel beingINFO
; - when all arguments and options are processed, and Glue instance can determine desired log level, whether it’s expected to stream debugging messages into a file, etc. This time, method only modifies propagates necessary updates to already existing logger.
Parameters: - debug_file (str) – if set, new handler will be attached to the logger, streaming
messages of at least
DEBUG
level into this this file. - verbose_file (str) – if set, new handler will be attached to the logger, streaming
messages of
VERBOSE
log levels into this this file. - json_file (str) – if set, all logging messages are sent to this file in a form of JSON structures.
- level (int) – desired log level. One of constants defined in
logging
module, e.g.logging.DEBUG
orlogging.ERROR
. - sentry (bool) – if set, logger will be augmented to send every log message to the Sentry server.
- show_traceback (bool) – if set, exception tracebacks would be sent to
stderr
handler as well as to the debug file.
Return type: Returns: a
ContextAdapter
instance, set up for logging.- when
-
stderr_handler
= None¶ Stream handler printing out to stderr.
-
verbose_file_handler
= None¶
-
-
class
gluetool.log.
LoggingFormatter
(colors=True, log_tracebacks=False)[source]¶ Bases:
logging.Formatter
Custom log record formatter. Produces output in form of:
[stamp] [level] [ctx1] [ctx2] ... message
Parameters: - colors (bool) – if set, colorize output. Enabled by default but when used with file-backed destinations, colors are disabled by logging subsystem.
- log_tracebacks (bool) – if set, add tracebacks to the message. By default, we don’t need tracebacks on the terminal, unless its loglevel is verbose enough, but we want them in the debugging file.
-
static
_format_exception_chain
(exc_info)[source]¶ Format exception chain. Start with the one we’re given, and follow its caused_by property until we ran out of exceptions to format.
-
_level_color
= {40: <function <lambda>>, 50: <function <lambda>>, 20: <function <lambda>>, 30: <function <lambda>>}¶ Colorizers assigned to loglevels
Tags used to express loglevel.
-
format
(record)[source]¶ Format a logging record. It puts together pieces like time stamp, log level, possibly also different contexts if there are any stored in the record, and finally applies colors if asked to do so.
Parameters: record (logging.LogRecord) – record describing the event. Return type: str Returns: string representation of the event record.
-
class
gluetool.log.
ModuleAdapter
(logger, module)[source]¶ Bases:
gluetool.log.ContextAdapter
Custom logger adapter, adding module name as a context.
Parameters: - logger – parent logger this adapter modifies.
- module (gluetool.glue.Module) – module whose name is added as a context.
-
class
gluetool.log.
PackageAdapter
(logger, name)[source]¶ Bases:
gluetool.log.ContextAdapter
Custom logger dapter, adding a package name as a context. Intended to taint log records produced by a 3rd party packages.
Parameters: - logger – parent logger this adapter modifies.
- name (str) – name of the library.
-
class
gluetool.log.
SingleLogLevelFileHandler
(level, *args, **kwargs)[source]¶ Bases:
logging.FileHandler
-
class
gluetool.log.
StreamToLogger
(log_fn)[source]¶ Bases:
object
Fake
file
-like stream object that redirects writes to a given logging method.
-
gluetool.log.
_extract_stack
(tb)[source]¶ Construct a “stack” by merging two sources of data:
- what’s provided by
traceback.extract_tb()
, i.e.(filename, lineno, fnname, text)
tuple for each frame; - stack frame objects, hidden inside traceback object and available via following links from one frame to another.
Return type: list(list(str, int, str, str, frame)) - what’s provided by
-
gluetool.log.
format_blob
(blob)[source]¶ Format a blob of text for printing. Wraps the text with boundaries to mark its borders.
-
gluetool.log.
format_dict
(dictionary)[source]¶ Format a Python data structure for printing. Uses
json.dumps()
formatting capabilities to present readable representation of a given structure.
-
gluetool.log.
format_table
(table, **kwargs)[source]¶ Format a table, represented by an iterable of rows, represented by iterables.
Internally,
tabulate
is used to do the formatting. All keyword arguments are passed totabulate
call.Parameters: table (list(list())) – table to format. Returns: formatted table.
-
gluetool.log.
format_xml
(element)[source]¶ Format an XML element, e.g. Beaker job description, for printing.
Parameters: element – XML element to format.
-
gluetool.log.
log_blob
(writer, intro, blob)[source]¶ Log “blob” of characters of unknown structure, e.g. output of a command or response of a HTTP request. The blob is preceded by a header and followed by a footer to mark exactly the blob boundaries.
Note
For logging structured data, e.g. JSON or Python structures, use
gluetool.log.log_dict()
. It will make structure of the data more visible, resulting in better readability of the log.Parameters:
-
gluetool.log.
log_dict
(writer, intro, data)[source]¶ Log structured data, e.g. JSON responses or a Python
list
.Note
For logging unstructured “blobs” of text, use
gluetool.log.log_blob()
. It does not attempt to format the output, and wraps it by header and footer to mark its boundaries.Note
Using
gluetool.log.format_dict()
directly might be shorter, depending on your your code. For example, this code:self.debug('Some data:\n{}'.format(format_dict(data)))
is equivalent to:
log_dict(self.debug, 'Some data', data)
If you need more formatting, or you wish to fit more information into a single message, using logger methods with
format_dict
is a way to go, while for logging a single structurelog_dict
is more suitable.Parameters:
-
gluetool.log.
log_table
(writer, intro, table, **kwargs)[source]¶ Log a formatted table.
All keyword arguments are passed to
format_table()
call which does the actual formatting.Parameters:
-
gluetool.log.
log_xml
(writer, intro, element)[source]¶ Log an XML element, e.g. Beaker job description.
Parameters:
-
gluetool.log.
print_wrapper
(*args, **kwds)[source]¶ While active, replaces
sys.stdout
andsys.stderr
streams with fakefile
-like streams which send all outgoing data to a given logging method.Parameters: - log_fn (call) – A callback that is called for every line, produced by
print
. If not set, ainfo
method ofgluetool
main logger is used. - label (text) – short description, presented in the intro and outro headers, wrapping captured output.
- log_fn (call) – A callback that is called for every line, produced by