logilab.common package

Subpackages

• logilab.common.ureports package
  • Submodules
  • logilab.common.ureports.docbook_writer module
  • logilab.common.ureports.html_writer module
  • logilab.common.ureports.nodes module
  • logilab.common.ureports.text_writer module
  • Module contents

Submodules

logilab.common.cache module

Cache module, with a least recently used algorithm for the management of the deletion of entries.

class logilab.common.cache.Cache(size: int = 100)

Bases: dict

A dictionary like cache.

inv:

len(self._usage) <= self.size len(self.data) <= self.size

clear() → None. Remove all items from D.

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, default is returned if given, otherwise KeyError is raised

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

logilab.common.changelog module

Manipulation of upstream change log files.

The upstream change log files format handled is simpler than the one often used such as those generated by the default Emacs changelog mode.

Sample ChangeLog format:

Change log for project Yoo
==========================

 --
    * add a new functionality

2002-02-01 -- 0.1.1
    * fix bug #435454
    * fix bug #434356

2002-01-01 -- 0.1
    * initial release

There is 3 entries in this change log, one for each released version and one for the next version (i.e. the current entry). Each entry contains a set of messages corresponding to changes done in this release. All the non empty lines before the first entry are considered as the change log title.

class logilab.common.changelog.ChangeLog(changelog_file: str, title: str = '')

Bases: object

object representation of a whole ChangeLog file

add(msg, create=None)

add a new message to the latest opened entry

add_entry(entry: logilab.common.changelog.ChangeLogEntry) → None

add a new entry to the change log

entry_class

alias of logilab.common.changelog.ChangeLogEntry

format_title() → str

get_entry(version='', create=None)

return a given changelog entry if version is omitted, return the current entry

load() → None

read a logilab’s ChangeLog from file

save()

write back change log

write(stream: _io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

write changelog to stream

class logilab.common.changelog.ChangeLogEntry(date: Optional[str] = None, version: Optional[str] = None, **kwargs: Any)

Bases: object

a change log entry, i.e. a set of messages associated to a version and its release date

add_message(msg: str) → None

add a new message

add_sub_message(sub_msg: str, key: Optional[Any] = None) → None

complete_latest_message(msg_suite: str) → None

complete the latest added message

version_class

alias of logilab.common.changelog.Version

write(stream: _io.StringIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>) → None

write the entry to file

exception logilab.common.changelog.EntryNotFound

Bases: Exception

raised when we are unable to find a given entry

exception logilab.common.changelog.NoEntry

Bases: Exception

raised when we are unable to find an entry

class logilab.common.changelog.Version(versionstr)

Bases: tuple

simple class to handle soft version number has a tuple while correctly printing it as X.Y.Z

classmethod parse(versionstr: str) → List[int]

logilab.common.clcommands module

Helper functions to support command line tools providing more than one command.

e.g called as “tool command [options] args…” where <options> and <args> are command’specific

exception logilab.common.clcommands.BadCommandUsage

Bases: Exception

Raised when an unknown command is used or when a command is not correctly used (bad options, too much / missing arguments…).

Trigger display of command usage.

class logilab.common.clcommands.Command(logger)

Bases: logilab.common.configuration.Configuration

Base class for command line commands.

Class attributes:

• name, the name of the command

• min_args, minimum number of arguments, None if unspecified

• max_args, maximum number of arguments, None if unspecified

• arguments, string describing arguments, used in command usage

• hidden, boolean flag telling if the command should be hidden, e.g. does not appear in help’s commands list

• options, options list, as allowed by :mod:configuration

arguments = ''

check_args(args)

check command’s arguments are provided

classmethod description()

hidden = False

main_run(args, rcfile=None)

Run the command and return status 0 if everything went fine.

If CommandError is raised by the underlying command, simply log the error and return status 2.

Any other exceptions, including BadCommandUsage will be propagated.

max_args = None

min_args = None

name = ''

run(args)

run the command with its specific arguments

classmethod short_description()

exception logilab.common.clcommands.CommandError

Bases: Exception

Raised when a command can’t be processed and we want to display it and exit, without traceback nor usage displayed.

class logilab.common.clcommands.CommandLine(pgm=None, doc=None, copyright=None, version=None, rcfile=None, logthreshold=40, check_duplicated_command=True)

Bases: dict

Usage:

>>> LDI = cli.CommandLine('ldi', doc='Logilab debian installer',
                          version=version, rcfile=RCFILE)
>>> LDI.register(MyCommandClass)
>>> LDI.register(MyOtherCommandClass)
>>> LDI.run(sys.argv[1:])

Arguments:

• pgm, the program name, default to basename(sys.argv[0])

• doc, a short description of the command line tool

• copyright, additional doc string that will be appended to the generated doc

• version, version number of string of the tool. If specified, global –version option will be available.

• rcfile, path to a configuration file. If specified, global –C/–rc-file option will be available? self.rcfile = rcfile

• logger, logger to propagate to commands, default to logging.getLogger(self.pgm))

create_logger(handler, logthreshold=None)

get_command(cmd, logger=None)

register(cls, force=False)

register the given Command subclass

run(args)

main command line access point: * init logging * handle global options (-h/–help, –version, -C/–rc-file) * check command * run command

Terminate by SystemExit

usage()

display usage for the main program (i.e. when no command supplied) and exit

usage_and_exit(status)

class logilab.common.clcommands.ListCommandsCommand(logger)

Bases: logilab.common.clcommands.Command

list available commands, useful for bash completion.

arguments = '[command]'

hidden = True

name = 'listcommands'

option_groups: List[Tuple[Any, str]]

options_providers: List[ConfigurationMixIn]

run(args)

run the command with its specific arguments

logilab.common.compat module

Wrappers around some builtins introduced in python 2.3, 2.4 and 2.5, making them available in for earlier versions of python.

See another compatibility snippets from other projects:

lib2to3.fixes coverage.backward unittest2.compatibility

logilab.common.compat.method_type(callable, instance, klass)

logilab.common.compat.str_encode(string: Union[int, str], encoding: str) → str

logilab.common.compat.str_to_bytes(string)

logilab.common.configuration module

Classes to handle advanced configuration in simple to complex applications.

Allows to load the configuration from a file or from command line options, to generate a sample configuration file or to display program’s usage. Fills the gap between optik/optparse and ConfigParser by adding data types (which are also available as a standalone optik extension in the optik_ext module).

Quick start: simplest usage

class logilab.common.configuration.Configuration(config_file=None, options=None, name=None, usage=None, doc=None, version=None)

Bases: logilab.common.configuration.ConfigurationMixIn

class for simple configurations which don’t need the manager / providers model and prefer delegation to inheritance

configuration values are accessible through a dict like interface

class logilab.common.configuration.ConfigurationMixIn(*args: Any, **kwargs: Any)

Bases: logilab.common.configuration.OptionsManagerMixIn, logilab.common.configuration.OptionsProviderMixIn

basic mixin for simple configurations which don’t need the manager / providers model

get(key, default=None)

load_defaults()

initialize the provider using default values

register_options(options)

add some options to the configuration

class logilab.common.configuration.OptionsManager2ConfigurationAdapter(provider)

Bases: object

Adapt an option manager to behave like a logilab.common.configuration.Configuration instance

get(key, default=None)

class logilab.common.configuration.OptionsManagerMixIn(usage: Optional[str], config_file: Optional[Any] = None, version: Optional[Any] = None, quiet: int = 0)

Bases: object

MixIn to handle a configuration from both a configuration file and command line options

add_help_section(title: str, description: str, level: int = 0) → None

add a dummy option section for help purpose

add_optik_option(provider: logilab.common.configuration.ConfigurationMixIn, optikcontainer: Union[OptionParser, optparse.OptionGroup], opt: str, optdict: Dict[str, Any]) → None

add_option_group(group_name: str, doc: Optional[str], options: Union[List[Tuple[str, Dict[str, Any]]], List[Tuple[str, Dict[str, str]]]], provider: logilab.common.configuration.ConfigurationMixIn) → None

add an option group including the listed options

cb_set_provider_option(option: Option, opt: str, value: Union[List[str], int, str], parser: OptionParser) → None

optik callback for option setting

generate_config(stream: Optional[Union[_io.StringIO, _io.TextIOWrapper]] = None, skipsections: Tuple[()] = (), encoding: Optional[Any] = None) → None

write a configuration file according to the current configuration into the given stream or stdout

generate_manpage(pkginfo: attrdict, section: int = 1, stream: Optional[_io.StringIO] = None) → None

write a man page for the current configuration into the given stream or stdout

global_set_option(opt: str, value: Union[List[str], int, str]) → None

set option on the correct option provider

help(level: int = 0) → str

return the usage string for available options

input_config(onlysection=None, inputlevel=0, stream=None)

interactively get configuration values by asking to the user and generate a configuration file

load_command_line_configuration(args: Optional[List[str]] = None) → List[str]

override configuration according to command line parameters

return additional arguments

load_config_file() → None

dispatch values previously read from a configuration file to each options provider)

load_configuration(**kwargs: Any) → None

override configuration according to given parameters

load_file_configuration(config_file: Optional[str] = None) → None

load the configuration from file

load_provider_defaults() → None

initialize configuration using default values

optik_option(provider: logilab.common.configuration.ConfigurationMixIn, opt: str, optdict: Dict[str, Any]) → Tuple[List[str], Dict[str, Any]]

get our personal option definition and return a suitable form for use with optik/optparse

read_config_file(config_file: Optional[str] = None) → None

read the configuration file but do not load it (i.e. dispatching values to each options provider)

register_options_provider(provider: logilab.common.configuration.ConfigurationMixIn, own_group: bool = True) → None

register an options provider

reset_parsers(usage: Optional[str] = '', version: Optional[Any] = None) → None

class logilab.common.configuration.OptionsProviderMixIn

Bases: object

Mixin to provide options to an OptionsManager

all_options()

return an iterator on available options for this provider option are actually described by a 3-uple: (section, option name, option dictionary)

get_option_def(opt)

return the dictionary defining an option given it’s name

input_option(option, optdict, inputlevel=99)

level = 0

load_defaults() → None

initialize the provider using default values

name = 'default'

option_attrname(opt, optdict=None)

get the config attribute corresponding to opt

option_default(opt, optdict=None)

return the default value for an option

option_name(opt, optdict=None) → Callable

option_value(opt)

get the current value for the given option

options: Tuple = ()

options_and_values(options=None)

options_by_section() → Iterator[Any]

return an iterator on options grouped by section

(section, [list of (optname, optdict, optvalue)])

priority = -1

set_option(opt, value, action=None, optdict=None)

method called to set an option (registered in the options list)

logilab.common.daemon module

A daemonize function (for Unices)

logilab.common.daemon.daemonize(pidfile=None, uid=None, umask=63)

daemonize a Unix process. Set paranoid umask by default.

Return 1 in the original process, 2 in the first fork, and None for the second fork (eg daemon process).

logilab.common.daemon.setugid(user)

Change process user and group ID

Argument is a numeric user id or a user name

logilab.common.date module

Date manipulation helper functions.

logilab.common.date.add_days_worked(start: datetime.date, days: int) → datetime.date

adds date but try to only take days worked into account

logilab.common.date.date_range(begin: datetime.date, end: datetime.date, incday: Optional[Any] = None, incmonth: Optional[bool] = None) → Generator[datetime.date, Any, None]

yields each date between begin and end

Parameters

• begin – the start date

• end – the end date

• incr – the step to use to iterate over dates. Default is one day.

• include – None (means no exclusion) or a function taking a date as parameter, and returning True if the date should be included.

When using mx datetime, you should NOT use incmonth argument, use instead oneDay, oneHour, oneMinute, oneSecond, oneWeek or endOfMonth (to enumerate months) as incday argument

logilab.common.date.datefactory(year: int, month: int, day: int, sampledate: Union[datetime.date, datetime.datetime]) → Union[datetime.date, datetime.datetime]

logilab.common.date.datetime2ticks(somedate: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.datetime_to_seconds(date)

return the number of seconds since the begining of the day for that date

logilab.common.date.days_between(start: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.days_in_month(somedate: datetime.date) → int

logilab.common.date.days_in_year(somedate)

logilab.common.date.first_day(somedate)

logilab.common.date.get_national_holidays(begin: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → Union[List[datetime.date], List[datetime.datetime]]

return french national days off between begin and end

logilab.common.date.get_step(dateobj: Union[datetime.date, datetime.datetime], nbdays: int = 1) → datetime.timedelta

logilab.common.date.last_day(somedate: datetime.date) → datetime.date

logilab.common.date.nb_open_days(start: Union[datetime.date, datetime.datetime], end: Union[datetime.date, datetime.datetime]) → int

logilab.common.date.next_month(somedate: datetime.date, nbmonth: int = 1) → datetime.date

logilab.common.date.previous_month(somedate, nbmonth=1)

logilab.common.date.str2date(datestr: str, sampledate: Union[datetime.date, datetime.datetime]) → Union[datetime.date, datetime.datetime]

logilab.common.date.strptime_time(value, format='%H:%M')

logilab.common.date.ticks2datetime(ticks: int) → datetime.datetime

logilab.common.date.timedelta_to_days(delta)

return the time delta as a number of seconds

logilab.common.date.timedelta_to_seconds(delta)

return the time delta as a fraction of days

logilab.common.date.todate(somedate: datetime.date) → datetime.date

return a date from a date (leaving unchanged) or a datetime

logilab.common.date.todatetime(somedate)

return a date from a date (leaving unchanged) or a datetime

logilab.common.date.totime(somedate)

return a time from a time (leaving unchanged), date or datetime

logilab.common.date.ustrftime(somedate: datetime.datetime, fmt: str = '%Y-%m-%d') → str

like strftime, but returns a unicode string instead of an encoded string which may be problematic with localized date.

logilab.common.date.utcdatetime(dt: datetime.datetime) → datetime.datetime

logilab.common.date.utctime(dt)

logilab.common.date.weekday(dateobj: Union[datetime.date, datetime.datetime]) → int

logilab.common.debugger module

Customized version of pdb’s default debugger.

• sets up a history file

• uses ipython if available to colorize lines of code

• overrides list command to search for current block instead of using 5 lines of context

class logilab.common.debugger.Debugger(tcbk=None)

Bases: pdb.Pdb

custom debugger

• sets up a history file

• uses ipython if available to colorize lines of code

• overrides list command to search for current block instead of using 5 lines of context

attr_matches(text, namespace)

implementation coming from rlcompleter.Completer.attr_matches Compute matches when text contains a dot.

Assuming the text is of the form NAME.NAME….[NAME], and is evaluatable in self.namespace, it will be evaluated and its attributes (as revealed by dir()) are used as possible completions. (For class instances, class members are also considered.)

WARNING: this can still invoke arbitrary C code, if an object with a __getattr__ hook is evaluated.

complete_p(text, line, begin_idx, end_idx)

provide variable names completion for the p command

do_l(arg)

overrides default list command to display the surrounding block instead of 5 lines of context

do_list(arg)

overrides default list command to display the surrounding block instead of 5 lines of context

do_o(arg)

opens source file corresponding to the current stack level

do_open(arg)

opens source file corresponding to the current stack level

get_class_members(klass)

implementation coming from rlcompleter.get_class_members

set_quit()

quit hook: save commands in the history file

setup(frame, tcbk)

setup hook: set up history file

setup_history_file()

if readline is available, read pdb history file

start()

starts the interactive mode

logilab.common.debugger.colorize(source, start_lineno, curlineno)

fallback colorize function

logilab.common.debugger.colorize_source(source)

logilab.common.debugger.getsource(obj)

Return the text of the source code for an object.

The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a single string. An IOError is raised if the source code cannot be retrieved.

logilab.common.debugger.pm()

use our custom debugger

logilab.common.debugger.set_trace()

logilab.common.decorators module

A few useful function/method decorators.

logilab.common.decorators.cached(callableobj: Optional[Callable] = None, keyarg: Optional[int] = None, **kwargs: Any) → Union[Callable, logilab.common.decorators.cached_decorator]

Simple decorator to cache result of method call.

class logilab.common.decorators.cached_decorator(cacheattr: Optional[str] = None, keyarg: Optional[int] = None)

Bases: object

class logilab.common.decorators.cachedproperty(wrapped)

Bases: object

Provides a cached property equivalent to the stacking of @cached and @property, but more efficient.

After first usage, the <property_name> becomes part of the object’s __dict__. Doing:

del obj.<property_name> empties the cache.

Idea taken from the pyramid framework and the mercurial project.

wrapped

class logilab.common.decorators.classproperty(get)

Bases: object

this is a simple property-like class but for class attributes.

logilab.common.decorators.clear_cache(obj, funcname)

Clear a cache handled by the cached() decorator. If ‘x’ class has @cached on its method foo, type

>>> clear_cache(x, 'foo')

to purge this method’s cache on the instance.

logilab.common.decorators.copy_cache(obj, funcname, cacheobj)

Copy cache for <funcname> from cacheobj to obj.

logilab.common.decorators.get_cache_impl(obj, funcname)

class logilab.common.decorators.iclassmethod(func)

Bases: object

Descriptor for method which should be available as class method if called on the class or instance method if called on an instance.

logilab.common.decorators.locked(acquire, release)

Decorator taking two methods to acquire/release a lock as argument, returning a decorator function which will call the inner method after having called acquire(self) et will call release(self) afterwards.

logilab.common.decorators.monkeypatch(klass: type, methodname: Optional[str] = None) → Callable

Decorator extending class with the decorated callable. This is basically a syntactic sugar vs class assignment.

>>> class A:
...     pass
>>> @monkeypatch(A)
... def meth(self):
...     return 12
...
>>> a = A()
>>> a.meth()
12
>>> @monkeypatch(A, 'foo')
... def meth(self):
...     return 12
...
>>> a.foo()
12

logilab.common.decorators.timed(f)

class logilab.common.decorators.wproperty(setfunc)

Bases: object

Simple descriptor expecting to take a modifier function as first argument and looking for a _<function name> to retrieve the attribute.

logilab.common.deprecation module

Deprecation utilities.

class logilab.common.deprecation.CallableDeprecatedCallable(*args, **kwargs)

Bases: Protocol

class logilab.common.deprecation.DeprecationWarningKind(value)

Bases: enum.Enum

An enumeration.

ARGUMENT = 'argument'

ATTRIBUTE = 'attribute'

CALLABLE = 'callable'

CLASS = 'class'

MODULE = 'module'

class logilab.common.deprecation.DeprecationWarningOperation(value)

Bases: enum.Enum

An enumeration.

DEPRECATED = 'deprecated'

MOVED = 'moved'

REMOVED = 'removed'

RENAMED = 'renamed'

class logilab.common.deprecation.DeprecationWrapper(proxied: Any, msg: Optional[str] = None, version: Optional[str] = None)

Bases: object

proxy to print a warning on access to any attribute of the wrapped object

exception logilab.common.deprecation.StructuredDeprecationWarning(reason: str)

Bases: DeprecationWarning

Base class for all structured DeprecationWarning Mostly used with isinstance

exception logilab.common.deprecation.TargetDeprecatedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind)

Bases: logilab.common.deprecation.StructuredDeprecationWarning

exception logilab.common.deprecation.TargetMovedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, old_name: str, new_name: str, old_module: str, new_module: str)

Bases: logilab.common.deprecation.StructuredDeprecationWarning

exception logilab.common.deprecation.TargetRemovedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, name: str)

Bases: logilab.common.deprecation.StructuredDeprecationWarning

exception logilab.common.deprecation.TargetRenamedDeprecationWarning(reason: str, kind: logilab.common.deprecation.DeprecationWarningKind, old_name: str, new_name: str)

Bases: logilab.common.deprecation.StructuredDeprecationWarning

logilab.common.deprecation.argument_removed(old_argument_name: str, version: Optional[str] = None) → Callable

callable decorator to allow getting backward compatibility for renamed keyword arguments.

>>> @argument_removed("old")
... def some_function(new):
...     return new
>>> some_function(old=42)
sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
is deprecated, use keyword argument new instead some_function(old=42)
42

logilab.common.deprecation.argument_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

callable decorator to allow getting backward compatibility for renamed keyword arguments.

>>> @argument_renamed(old_name="old", new_name="new")
... def some_function(new):
...     return new
>>> some_function(old=42)
sample.py:15: DeprecationWarning: argument old of callable some_function has been renamed and
is deprecated, use keyword argument new instead
  some_function(old=42)
42

logilab.common.deprecation.attribute_renamed(old_name: str, new_name: str, version: Optional[str] = None) → Callable

class decorator to allow getting backward compatibility for renamed attributes.

>>> @attribute_renamed(old_name="old", new_name="new")
... class SomeClass:
...     def __init__(self):
...         self.new = 42

>>> some_class = SomeClass()
>>> print(some_class.old)
sample.py:15: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
SomeClass.new instead
  print(some_class.old)
42
>>> some_class.old = 43
sample.py:16: DeprecationWarning: SomeClass.old has been renamed and is deprecated, use
SomeClass.new instead
  some_class.old = 43
>>> some_class.old == some_class.new
True

logilab.common.deprecation.callable_deprecated(reason: Optional[str] = None, version: Optional[str] = None, stacklevel: int = 2) → Callable

logilab.common.deprecation.callable_moved(module_name: str, object_name: str, version: Optional[str] = None, stacklevel: int = 2, new_name: Optional[str] = None) → Callable

logilab.common.deprecation.callable_renamed(old_name: str, new_function: Callable, version: Optional[str] = None) → Callable

use to tell that a callable has been renamed.

It returns a callable wrapper, so that when its called a warning is printed telling what is the object new name.

>>> old_function = renamed('old_function', new_function)
>>> old_function()
sample.py:57: DeprecationWarning: old_function has been renamed and is deprecated, uses
new_function instead old_function()
>>>

logilab.common.deprecation.class_deprecated

alias of logilab.common.deprecation._generate_class_deprecated.<locals>._class_deprecated

logilab.common.deprecation.class_moved(new_class: type, old_name: Optional[str] = None, message: Optional[str] = None, version: Optional[str] = None) → type

nice wrapper around class_renamed when a class has been moved into another module

logilab.common.deprecation.class_renamed(old_name: str, new_class: type, message: Optional[str] = None, version: Optional[str] = None, module_name: Optional[str] = None, deprecated_warning_class=<class 'logilab.common.deprecation.TargetRenamedDeprecationWarning'>, deprecated_warning_kwargs=None) → type

automatically creates a class which fires a DeprecationWarning when instantiated.

>>> Set = class_renamed('Set', set, 'Set is now replaced by set')
>>> s = Set()
sample.py:57: DeprecationWarning: Set is now replaced by set
s = Set()
>>>

logilab.common.deprecation.deprecated(reason: Optional[str] = None, version: Optional[str] = None, stacklevel: int = 2) → Callable

logilab.common.deprecation.get_real__name__(some_callable: Callable) → str

This is another super edge magic case which is needed because we uses lazy_wraps because of logilab.common.modutils.LazyObject and because __name__ has special behavior and doesn’t work like a normal attribute and that __getattribute__ of lazy_wraps is bypassed.

Therefor, to get the real callable name when several lazy_wrapped decorator are used we need to travers the __wrapped__ attributes chain.

logilab.common.deprecation.lazy_wraps(wrapped: Callable) → Callable

This is the equivalent of the @wraps decorator of functools except it won’t try to grabs attributes of the targeted function on decoration but on access.

This is needed because of logilab.common.modutils.LazyObject.

Indeed: if you try to decorate a LazyObject with @wraps, wraps will try to access attributes of LazyObject and this will trigger the attempt to import the module decorated by LazyObject which you don’t want to do when you just want to mark this LazyObject has been a deprecated objet that you only wants to trigger if the user try to use it.

Usage: like @wraps()

>>> @lazy_wraps(function)
>>> def wrapper(*args, **kwargs): ...

logilab.common.deprecation.moved(module_name: str, object_name: str, version: Optional[str] = None, stacklevel: int = 2, new_name: Optional[str] = None) → Callable

logilab.common.deprecation.renamed(old_name: str, new_function: Callable, version: Optional[str] = None) → Callable

logilab.common.deprecation.send_warning(reason: str, deprecation_class: Type[DeprecationWarning], deprecation_class_kwargs: Dict[str, Any], version: Optional[str] = None, stacklevel: int = 2, module_name: Optional[str] = None) → None

Display a deprecation message only if the version is older than the compatible version.

logilab.common.fileutils module

File and file-path manipulation utilities.

group path manipulation

first_level_directory, relative_path, is_binary,get_by_ext, remove_dead_links

group file manipulation

norm_read, norm_open, lines, stream_lines, lines,write_open_mode, ensure_fs_mode, export

sort

path manipulation, file manipulation

class logilab.common.fileutils.ProtectedFile(filepath: str, mode: str)

Bases: _io.FileIO

A special file-object class that automatically does a ‘chmod +w’ when needed.

XXX: for now, the way it is done allows ‘normal file-objects’ to be created during the ProtectedFile object lifetime. One way to circumvent this would be to chmod / unchmod on each write operation.

One other way would be to :

• catch the IOError in the __init__

• if IOError, then create a StringIO object

• each write operation writes in this StringIO object

• on close()/del(), write/append the StringIO content to the file and do the chmod only once

close() → None

restore mode before closing

exception logilab.common.fileutils.UnresolvableError

Bases: Exception

Exception raised by relative path when it’s unable to compute relative path between two paths.

logilab.common.fileutils.abspath_listdir(path)

Lists path’s content using absolute paths.

logilab.common.fileutils.ensure_fs_mode(filepath, desired_mode=128)

Check that the given file has the given mode(s) set, else try to set it.

Parameters

• filepath (str) – path of the file

• desired_mode (int) – ORed flags describing the desired mode. Use constants from the stat module for file permission’s modes

logilab.common.fileutils.export(from_dir: str, to_dir: str, blacklist: Tuple[str, str, str, str, str, str, str, str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build'), ignore_ext: Tuple[str, str, str, str, str, str] = ('.pyc', '.pyo', '.elc', '~', '.swp', '.orig'), verbose: int = 0) → None

Make a mirror of from_dir in to_dir, omitting directories and files listed in the black list or ending with one of the given extensions.

Parameters

• from_dir (str) – directory to export

• to_dir (str) – destination directory

• blacklist (list or tuple) – list of files or directories to ignore, default to the content of BASE_BLACKLIST

• ignore_ext (list or tuple) – list of extensions to ignore, default to the content of IGNORED_EXTENSIONS

• verbose (bool) – flag indicating whether information about exported files should be printed to stderr, default to False

logilab.common.fileutils.first_level_directory(path: str) → str

Return the first level directory of a path.

>>> first_level_directory('home/syt/work')
'home'
>>> first_level_directory('/home/syt/work')
'/'
>>> first_level_directory('work')
'work'
>>>

Parameters

path (str) – the path for which we want the first level directory

Return type

str

Returns

the first level directory appearing in path

logilab.common.fileutils.is_binary(filename: str) → int

Return true if filename may be a binary file, according to it’s extension.

Parameters

filename (str) – the name of the file

Return type

bool

Returns

true if the file is a binary file (actually if it’s mime type isn’t beginning by text/)

logilab.common.fileutils.lines(path: str, comments: Optional[str] = None) → List[str]

Return a list of non empty lines in the file located at path.

Parameters

• path (str) – path to the file

• comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

Return type

list

Returns

a list of stripped line in the file, without empty and commented lines

Warning

at some point this function will probably return an iterator

logilab.common.fileutils.relative_path(from_file, to_file)

Try to get a relative path from from_file to to_file (path will be absolute if to_file is an absolute file). This function is useful to create link in from_file to to_file. This typical use case is used in this function description.

If both files are relative, they’re expected to be relative to the same directory.

>>> relative_path( from_file='toto/index.html', to_file='index.html')
'../index.html'
>>> relative_path( from_file='index.html', to_file='toto/index.html')
'toto/index.html'
>>> relative_path( from_file='tutu/index.html', to_file='toto/index.html')
'../toto/index.html'
>>> relative_path( from_file='toto/index.html', to_file='/index.html')
'/index.html'
>>> relative_path( from_file='/toto/index.html', to_file='/index.html')
'../index.html'
>>> relative_path( from_file='/toto/index.html', to_file='/toto/summary.html')
'summary.html'
>>> relative_path( from_file='index.html', to_file='index.html')
''
>>> relative_path( from_file='/index.html', to_file='toto/index.html')
Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "<stdin>", line 37, in relative_path
UnresolvableError
>>> relative_path( from_file='/index.html', to_file='/index.html')
''
>>>

Parameters

• from_file (str) – source file (where links will be inserted)

• to_file (str) – target file (on which links point)

Raises

UnresolvableError – if it has been unable to guess a correct path

Return type

str

Returns

the relative path of to_file from from_file

logilab.common.fileutils.remove_dead_links(directory, verbose=0)

Recursively traverse directory and remove all dead links.

Parameters

• directory (str) – directory to cleanup

• verbose (bool) – flag indicating whether information about deleted links should be printed to stderr, default to False

logilab.common.fileutils.stream_lines(stream: _io.TextIOWrapper, comments: Optional[str] = None) → List[str]

Return a list of non empty lines in the given stream.

Parameters

• stream (object implementing 'xreadlines' or 'readlines') – file like object

• comments (str or None) – optional string which can be used to comment a line in the file (i.e. lines starting with this string won’t be returned)

Return type

list

Returns

a list of stripped line in the file, without empty and commented lines

Warning

at some point this function will probably return an iterator

logilab.common.fileutils.write_open_mode(filename: str) → str

Return the write mode that should used to open file.

Parameters

filename (str) – the name of the file

Return type

str

Returns

the mode that should be use to open the file (‘w’ or ‘wb’)

logilab.common.graph module

Graph manipulation utilities.

(dot generation adapted from pypy/translator/tool/make_dot.py)

class logilab.common.graph.DotBackend(graphname, rankdir=None, size=None, ratio=None, charset='utf-8', renderer='dot', additionnal_param={})

Bases: object

Dot File backend.

emit(line)

Adds <line> to final output.

emit_edge(name1, name2, **props)

emit an edge from <name1> to <name2>. edge properties: see http://www.graphviz.org/doc/info/attrs.html

emit_node(name, **props)

emit a node with given properties. node properties: see http://www.graphviz.org/doc/info/attrs.html

generate(outputfile=None, dotfile=None, mapfile=None)

Generates a graph file.

Parameters

• outputfile – filename and path [defaults to graphname.png]

• dotfile – filename and path [defaults to graphname.dot]

Return type

str

Returns

a path to the generated file

get_source()

returns self._source

property source

returns self._source

class logilab.common.graph.GraphGenerator(backend)

Bases: object

generate(visitor, propshdlr, outputfile=None, mapfile=None)

exception logilab.common.graph.UnorderableGraph

Bases: Exception

logilab.common.graph.escape(value)

Make <value> usable in a dot file.

logilab.common.graph.get_cycles(graph_dict: Dict[V, List[V]], vertices: Optional[Iterable] = None) → List[List]

given a dictionary representing an ordered graph (i.e. key are vertices and values is a list of destination vertices representing edges), return a list of detected cycles

logilab.common.graph.has_path(graph_dict: Dict[str, List[str]], fromnode: str, tonode: str, path: Optional[List[str]] = None) → Optional[List[str]]

generic function taking a simple graph definition as a dictionary, with node has key associated to a list of nodes directly reachable from it.

Return None if no path exists to go from fromnode to tonode, else the first path found (as a list including the destination node at last)

logilab.common.graph.normalize_node_id(nid)

Returns a suitable DOT node id for nid.

logilab.common.graph.ordered_nodes(graph: Dict[V, List[V]]) → Tuple[V, …]

takes a dependency graph dict as arguments and return an ordered tuple of nodes starting with nodes without dependencies and up to the outermost node.

If there is some cycle in the graph, UnorderableGraph will be raised.

Also the given graph dict will be emptied.

logilab.common.graph.target_info_from_filename(filename)

Transforms /some/path/foo.png into (‘/some/path’, ‘foo.png’, ‘png’).

logilab.common.interface module

Bases class for interfaces to provide ‘light’ interface handling.

TODO:

_ implements a check method which check that an object implements the interface _ Attribute objects

This module requires at least python 2.2

class logilab.common.interface.Interface

Bases: object

Base class for interfaces.

classmethod is_implemented_by(instance: type) → bool

logilab.common.interface.extend(klass: type, interface: type, _recurs: bool = False) → None

Add interface to klass’__implements__ if not already implemented in.

If klass is subclassed, ensure subclasses __implements__ it as well.

NOTE: klass should be e new class.

logilab.common.interface.implements(obj: type, interface: type) → bool

Return true if the give object (maybe an instance or class) implements the interface.

logilab.common.logging_ext module

Extends the logging module from the standard library.

class logilab.common.logging_ext.ColorFormatter(fmt=None, datefmt=None, colors=None)

Bases: logging.Formatter

A color Formatter for the logging standard module.

By default, colorize CRITICAL and ERROR in red, WARNING in orange, INFO in green and DEBUG in yellow.

self.colors is customizable via the ‘color’ constructor argument (dictionary).

self.colorfilters is a list of functions that get the LogRecord and return a color name or None.

format(record)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

logilab.common.logging_ext.get_formatter(logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S')

logilab.common.logging_ext.get_handler(debug=False, syslog=False, logfile=None, rotation_parameters=None)

get an apropriate handler according to given parameters

logilab.common.logging_ext.get_threshold(debug=False, logthreshold=None)

logilab.common.logging_ext.init_log(debug=False, syslog=False, logthreshold=None, logfile=None, logformat='%(asctime)s - (%(name)s) %(levelname)s: %(message)s', logdateformat='%Y-%m-%d %H:%M:%S', fmt=None, rotation_parameters=None, handler=None)

init the log service

logilab.common.logging_ext.set_color_formatter(logger=None, **kw)

Install a color formatter on the ‘logger’. If not given, it will defaults to the default logger.

Any additional keyword will be passed as-is to the ColorFormatter constructor.

logilab.common.logging_ext.set_log_methods(cls, logger)

bind standard logger’s methods as methods on the class

logilab.common.logging_ext.xxx_cyan(record)

logilab.common.modutils module

Python modules manipulation utility functions.

type PY_SOURCE_EXTS

tuple(str)

var PY_SOURCE_EXTS

list of possible python source file extension

type STD_LIB_DIR

str

var STD_LIB_DIR

directory where standard modules are located

type BUILTIN_MODULES

dict

var BUILTIN_MODULES

dictionary with builtin module names as key

class logilab.common.modutils.LazyObject(module, obj)

Bases: object

This class allows to lazyly declare a object (most likely only a callable according to the code) from a module without importing it.

The import will be triggered when the user tries to access attributes of the object/callable or call it.

Trying to set or delete attributes of the wrapped object/callable will not works as expected.

exception logilab.common.modutils.NoSourceFile

Bases: Exception

exception raised when we are not able to get a python source file for a precompiled file

logilab.common.modutils.clean_sys_modules(names)

remove submodules starting with name from names from sys.modules

logilab.common.modutils.cleanup_sys_modules(directories)

remove submodules of directories from sys.modules

logilab.common.modutils.file_from_modpath(modpath: List[str], path: Optional[Any] = None, context_file: Optional[str] = None) → Optional[str]

given a mod path (i.e. splitted module / package name), return the corresponding file, giving priority to source file over precompiled file if it exists

Parameters

• modpath (list or tuple) – splitted module’s name (i.e name of a module or package splitted on ‘.’) (this means explicit relative imports that start with dots have empty strings in this list!)

• path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

• context_file (str or None) – context file to consider, necessary if the identifier has been introduced using a relative import unresolvable in the actual context (i.e. modutils)

Raises

ImportError – if there is no such module in the directory

Return type

str or None

Returns

the path to the module’s file or None if it’s an integrated builtin module such as ‘sys’

logilab.common.modutils.get_module_files(src_directory: str, blacklist: Sequence[str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

given a package directory return a list of all available python module’s files in the package and its subpackages

Parameters

• src_directory (str) – path of the directory corresponding to the package

• blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

Return type

list

Returns

the list of all available python module’s files in the package and its subpackages

logilab.common.modutils.get_module_part(dotted_name: str, context_file: Optional[str] = None) → str

given a dotted name return the module part of the name :

>>> get_module_part('logilab.common.modutils.get_module_part')
'logilab.common.modutils'

Parameters

• dotted_name (str) – full name of the identifier we are interested in

• context_file (str or None) – context file to consider, necessary if the identifier has been introduced using a relative import unresolvable in the actual context (i.e. modutils)

Raises

ImportError – if there is no such module in the directory

Return type

str or None

Returns

the module part of the name or None if we have not been able at all to import the given name

XXX: deprecated, since it doesn’t handle package precedence over module (see #10066)

logilab.common.modutils.get_modules(package: str, src_directory: str, blacklist: Sequence[str] = ('CVS', '.svn', '.hg', '.git', '.tox', 'debian', 'dist', 'build')) → List[str]

given a package directory return a list of all available python modules in the package and its subpackages

Parameters

• package (str) – the python name for the package

• src_directory (str) – path of the directory corresponding to the package

• blacklist (list or tuple) – optional list of files or directory to ignore, default to the value of logilab.common.STD_BLACKLIST

Return type

list

Returns

the list of all available python modules in the package and its subpackages

logilab.common.modutils.get_source_file(filename: str, include_no_ext: bool = False) → str

given a python module’s file name return the matching source file name (the filename will be returned identically if it’s a already an absolute path to a python source file…)

Parameters

filename (str) – python module’s file name

Raises

NoSourceFile – if no source file exists on the file system

Return type

str

Returns

the absolute path of the source file if it exists

logilab.common.modutils.is_python_source(filename)

rtype: bool return: True if the filename is a python source file

logilab.common.modutils.is_relative(modname: str, from_file: str) → bool

return true if the given module name is relative to the given file name

Parameters

• modname (str) – name of the module we are interested in

• from_file (str) – path of the module from which modname has been imported

Return type

bool

Returns

true if the module has been imported relatively to from_file

logilab.common.modutils.is_standard_module(modname: str, std_path: Union[List[str], Tuple[str]] = ('/usr/lib/python3.9')) → bool

try to guess if a module is a standard python module (by default, see std_path parameter’s description)

Parameters

• modname (str) – name of the module we are interested in

• std_path (list(str) or tuple(str)) – list of path considered as standard

Return type

bool

Returns

true if the module: - is located on the path listed in one of the directory in std_path - is a built-in module

Note: this function is known to return wrong values when inside virtualenv. See https://www.logilab.org/ticket/294756.

logilab.common.modutils.load_module_from_file(filepath, path=None, use_sys=True, extrapath=None)

Load a Python module from it’s path.

Parameters

• filepath (str) – path to the python module or package

• path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

• use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

Raises

ImportError – if the module or package is not found

Return type

module

Returns

the loaded module

logilab.common.modutils.load_module_from_modpath(parts: List[str], path: Optional[Any] = None, use_sys: int = True) → Optional[module]

Load a python module from its splitted name.

Parameters

• parts (list(str) or tuple(str)) – python name of a module or package splitted on ‘.’

• path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

• use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

Raises

ImportError – if the module or package is not found

Return type

module

Returns

the loaded module

logilab.common.modutils.load_module_from_name(dotted_name: str, path: Optional[Any] = None, use_sys: int = True) → module

Load a Python module from its name.

Parameters

• dotted_name (str) – python name of a module or package

• path (list or None) – optional list of path where the module or package should be searched (use sys.path if nothing or None is given)

• use_sys (bool) – boolean indicating whether the sys.modules dictionary should be used or not

Raises

ImportError – if the module or package is not found

Return type

module

Returns

the loaded module

logilab.common.optik_ext module

Add an abstraction level to transparently import optik classes from optparse (python >= 2.3) or the optik package.

It also defines three new types for optik/optparse command line parser :

• regexp argument of this type will be converted using re.compile

• csv argument of this type will be converted using split(‘,’)

• yn argument of this type will be true if ‘y’ or ‘yes’, false if ‘n’ or ‘no’

• named argument of this type are in the form <NAME>=<VALUE> or <NAME>:<VALUE>

• password argument of this type wont be converted but this is used by other tools such as interactive prompt for configuration to double check value and use an invisible field

• multiple_choice same as default “choice” type but multiple choices allowed

• file argument of this type wont be converted but checked that the given file exists

• color argument of this type wont be converted but checked its either a named color or a color specified using hexadecimal notation (preceded by a #)

• time argument of this type will be converted to a float value in seconds according to time units (ms, s, min, h, d)

• bytes argument of this type will be converted to a float value in bytes according to byte units (b, kb, mb, gb, tb)

class logilab.common.optik_ext.Option(*opts: str, **attrs: Any)

Bases: optparse.Option

override optik.Option to add some new option types

ATTRS = ['action', 'type', 'dest', 'default', 'nargs', 'const', 'choices', 'callback', 'callback_args', 'callback_kwargs', 'help', 'metavar', 'hide', 'level']

TYPES = ('string', 'int', 'long', 'float', 'complex', 'choice', 'regexp', 'csv', 'yn', 'named', 'password', 'multiple_choice', 'file', 'color', 'time', 'bytes')

TYPE_CHECKER = {'bytes': <function check_bytes>, 'choice': <function check_choice>, 'color': <function check_color>, 'complex': <function check_builtin>, 'csv': <function check_csv>, 'file': <function check_file>, 'float': <function check_builtin>, 'int': <function check_builtin>, 'long': <function check_builtin>, 'multiple_choice': <function check_csv>, 'named': <function check_named>, 'password': <function check_password>, 'regexp': <function check_regexp>, 'time': <function check_time>, 'yn': <function check_yn>}

process(opt: str, value: str, values: optparse.Values, parser: optparse.OptionParser) → int

class logilab.common.optik_ext.OptionGroup(parser, title, description=None)

Bases: optparse.OptionContainer

destroy()

see OptionParser.destroy().

format_help(formatter)

level = 0

set_title(title)

class logilab.common.optik_ext.OptionParser(option_class: type = <class 'logilab.common.optik_ext.Option'>, *args: Any, **kwargs: Any)

Bases: optparse.OptionParser

override optik.OptionParser to use our Option class

format_option_help(formatter: Optional[optparse.HelpFormatter] = None) → str

exception logilab.common.optik_ext.OptionValueError(msg)

Bases: optparse.OptParseError

Raised if an invalid option value is encountered on the command line.

class logilab.common.optik_ext.Values(defaults=None)

Bases: object

ensure_value(attr, value)

read_file(filename, mode='careful')

read_module(modname, mode='careful')

logilab.common.optparser module

Extend OptionParser with commands.

Example:

>>> parser = OptionParser()
>>> parser.usage = '%prog COMMAND [options] <arg> ...'
>>> parser.add_command('build', 'mymod.build')
>>> parser.add_command('clean', run_clean, add_opt_clean)
>>> run, options, args = parser.parse_command(sys.argv[1:])
>>> return run(options, args[1:])

With mymod.build that defines two functions run and add_options

class logilab.common.optparser.