Modules¶
Flexible Metadata Format
- class fmf.Tree(data, name=None, parent=None)¶
Metadata Tree
- adjust(context, key='adjust', undecided='skip')¶
Adjust tree data based on provided context and rules
The ‘context’ should be an instance of the fmf.context.Context class describing the environment context. By default, the key ‘adjust’ of each node is inspected for possible rules that should be applied. Provide ‘key’ to use a custom key instead.
Optional ‘undecided’ parameter can be used to specify what should happen when a rule condition cannot be decided because context dimension is not defined. By default, such rules are skipped. In order to raise the fmf.context.CannotDecide exception in such cases use undecided=’raise’.
- child(name, data, source=None)¶
Create or update child with given data
- climb(whole=False)¶
Climb through the tree (iterate leaf/all nodes)
- property commit¶
Commit hash if tree grows under a git repo, False otherwise
Return current commit hash if the metadata tree root is located under a git repository. For metadata initialized from a dict or local directory with no git repo ‘False’ is returned instead.
- copy()¶
Create and return a deep copy of the node and its subtree
It is possible to call copy() on any node in the tree, not only on the tree root node. Note that in that case, parent node and the rest of the tree attached to it is not copied in order to save memory.
- find(name)¶
Find node with given name
- get(name=None, default=None)¶
Get attribute value or return default
Whole data dictionary is returned when no attribute provided. Supports direct values retrieval from deep dictionaries as well. Dictionary path should be provided as list. The following two examples are equal:
tree.data[‘hardware’][‘memory’][‘size’] tree.get([‘hardware’, ‘memory’, ‘size’])
However the latter approach will also correctly handle providing default value when any of the dictionary keys does not exist.
- grow(path)¶
Grow the metadata tree for the given directory path
Note: For each path, grow() should be run only once. Growing the tree from the same path multiple times with attribute adding using the “+” sign leads to adding the value more than once!
- inherit()¶
Apply inheritance
- static init(path)¶
Create metadata tree root under given path
- merge(parent=None)¶
Merge parent data
- static node(reference)¶
Return Tree node referenced by the fmf identifier
Keys supported in the reference:
url …. git repository url (optional) ref …. branch, tag or commit (default branch if not provided) path … metadata tree root (‘.’ by default) name … tree node name (‘/’ by default)
See the documentation for the full fmf id specification: https://fmf.readthedocs.io/en/latest/concept.html#identifiers Raises ReferenceError if referenced node does not exist.
- prune(whole=False, keys=None, names=None, filters=None, conditions=None)¶
Filter tree nodes based on given criteria
- show(brief=False, formatting=None, values=None)¶
Show metadata
- update(data)¶
Update metadata, handle virtual hierarchy
- fmf.filter(filter, data, sensitive=True, regexp=False)¶
Return true if provided filter matches given dictionary of values
Filter supports disjunctive normal form with ‘|’ used for OR, ‘&’ for AND and ‘-’ for negation. Individual values are prefixed with ‘value:’, leading/trailing white-space is stripped. For example:
tag: Tier1 | tag: Tier2 | tag: Tier3 category: Sanity, Security & tag: -destructive
Note that multiple comma-separated values can be used as a syntactic sugar to shorten the filter notation:
tag: A, B, C ---> tag: A | tag: B | tag: C
Values should be provided as a dictionary of lists each describing the values against which the filter is to be matched. For example:
data = {tag: ["Tier1", "TIPpass"], category: ["Sanity"]}
Other types of dictionary values are converted into a string. A FilterError exception is raised when a dimension parsed from the filter is not found in the data dictionary. Set option ‘sensitive’ to False to enable case-insensitive matching. If ‘regexp’ option is True, regular expressions can be used in the filter values as well.
base¶
Base Metadata Classes
- class fmf.base.Tree(data, name=None, parent=None)¶
Metadata Tree
- adjust(context, key='adjust', undecided='skip')¶
Adjust tree data based on provided context and rules
The ‘context’ should be an instance of the fmf.context.Context class describing the environment context. By default, the key ‘adjust’ of each node is inspected for possible rules that should be applied. Provide ‘key’ to use a custom key instead.
Optional ‘undecided’ parameter can be used to specify what should happen when a rule condition cannot be decided because context dimension is not defined. By default, such rules are skipped. In order to raise the fmf.context.CannotDecide exception in such cases use undecided=’raise’.
- child(name, data, source=None)¶
Create or update child with given data
- climb(whole=False)¶
Climb through the tree (iterate leaf/all nodes)
- property commit¶
Commit hash if tree grows under a git repo, False otherwise
Return current commit hash if the metadata tree root is located under a git repository. For metadata initialized from a dict or local directory with no git repo ‘False’ is returned instead.
- copy()¶
Create and return a deep copy of the node and its subtree
It is possible to call copy() on any node in the tree, not only on the tree root node. Note that in that case, parent node and the rest of the tree attached to it is not copied in order to save memory.
- find(name)¶
Find node with given name
- get(name=None, default=None)¶
Get attribute value or return default
Whole data dictionary is returned when no attribute provided. Supports direct values retrieval from deep dictionaries as well. Dictionary path should be provided as list. The following two examples are equal:
tree.data[‘hardware’][‘memory’][‘size’] tree.get([‘hardware’, ‘memory’, ‘size’])
However the latter approach will also correctly handle providing default value when any of the dictionary keys does not exist.
- grow(path)¶
Grow the metadata tree for the given directory path
Note: For each path, grow() should be run only once. Growing the tree from the same path multiple times with attribute adding using the “+” sign leads to adding the value more than once!
- inherit()¶
Apply inheritance
- static init(path)¶
Create metadata tree root under given path
- merge(parent=None)¶
Merge parent data
- static node(reference)¶
Return Tree node referenced by the fmf identifier
Keys supported in the reference:
url …. git repository url (optional) ref …. branch, tag or commit (default branch if not provided) path … metadata tree root (‘.’ by default) name … tree node name (‘/’ by default)
See the documentation for the full fmf id specification: https://fmf.readthedocs.io/en/latest/concept.html#identifiers Raises ReferenceError if referenced node does not exist.
- prune(whole=False, keys=None, names=None, filters=None, conditions=None)¶
Filter tree nodes based on given criteria
- show(brief=False, formatting=None, values=None)¶
Show metadata
- update(data)¶
Update metadata, handle virtual hierarchy
utils¶
Logging, config, constants & utilities
- class fmf.utils.Coloring(*args, **kwargs)¶
Coloring configuration
- MODES = ['COLOR_OFF', 'COLOR_ON', 'COLOR_AUTO']¶
- enabled()¶
True if coloring is currently enabled
- get()¶
Get the current color mode
- set(mode=None)¶
Set the coloring mode
If enabled, some objects (like case run Status) are printed in color to easily spot failures, errors and so on. By default the feature is enabled when script is attached to a terminal. Possible values are:
COLOR=0 ... COLOR_OFF .... coloring disabled COLOR=1 ... COLOR_ON ..... coloring enabled COLOR=2 ... COLOR_AUTO ... if terminal attached (default)
Environment variable COLOR can be used to set up the coloring to the desired mode without modifying code.
- exception fmf.utils.FetchError¶
Fatal error in helper command while fetching
- exception fmf.utils.FileError¶
File reading error
- exception fmf.utils.FilterError¶
Missing data when filtering
- exception fmf.utils.FormatError¶
Metadata format error
- exception fmf.utils.GeneralError¶
General error
- class fmf.utils.Logging(name='fmf')¶
Logging Configuration
- COLORS = {4: 'magenta', 7: 'cyan', 10: 'green', 20: 'blue', 30: 'yellow', 40: 'red'}¶
- class ColoredFormatter(fmt=None, datefmt=None, style='%')¶
Custom color formatter for logging
- format(record)¶
Format the specified record as text.
The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.
- LEVELS = ['CRITICAL', 'DEBUG', 'ERROR', 'FATAL', 'INFO', 'NOTSET', 'WARN', 'WARNING']¶
- MAPPING = {0: 30, 1: 20, 2: 10, 3: 7, 4: 4, 5: 1}¶
- get()¶
Get the current log level
- set(level=None)¶
Set the default log level
If the level is not specified environment variable DEBUG is used with the following meaning:
DEBUG=0 ... LOG_WARN (default) DEBUG=1 ... LOG_INFO DEBUG=2 ... LOG_DEBUG DEBUG=3 ... LOG_CACHE DEBUG=4 ... LOG_DATA DEBUG=5 ... LOG_ALL (log all messages)
- exception fmf.utils.MergeError¶
Unable to merge data between parent and child
- exception fmf.utils.ReferenceError¶
Referenced tree node cannot be found
- exception fmf.utils.RootError¶
Metadata tree root missing
- fmf.utils.clean_cache_directory()¶
Delete used cache directory if it exists
- fmf.utils.color(text, color=None, background=None, light=False, enabled='auto')¶
Return text in desired color if coloring enabled
Available colors: black red green yellow blue magenta cyan white. Alternatively color can be prefixed with “light”, e.g. lightgreen.
- fmf.utils.dict_to_yaml(data, width=None, sort=False)¶
Convert dictionary into yaml
- fmf.utils.evaluate(expression, data, _node=None)¶
Evaluate arbitrary Python expression against given data
Expects data dictionary which will be used to populate local namespace. Used to provide flexible conditions for filtering.
- fmf.utils.fetch(url, ref=None, destination=None, env=None)¶
Deprecated: Use
fetch_repo()
instead
- fmf.utils.fetch_repo(url, ref=None, destination=None, env=None)¶
Fetch remote git repository and return local directory
Fetch git repository from provided url into a local cache directory, checkout requested ref and return path to the repo. If no ref is provided, the default branch from the origin is used. If destination directory is provided, it should not exist or needs to be empty. Use dictionary env to set environment variables for git calls.
Raises FetchError upon failure with the original exception included.
- fmf.utils.fetch_tree(url, ref=None, path='.')¶
Get initialized Tree from a remote git repository
url …. git repository url (required) ref …. branch, tag or commit (default branch if None) path … metadata tree root (default to ‘.’)
See
fmf.base.Tree.node()
to canonical default values.Remote repository is cached locally (see
get_cache_directory()
), local directory with cache is locked during reading.Raises GeneralError when lock couldn’t be acquired.
- fmf.utils.filter(filter, data, sensitive=True, regexp=False)¶
Return true if provided filter matches given dictionary of values
Filter supports disjunctive normal form with ‘|’ used for OR, ‘&’ for AND and ‘-’ for negation. Individual values are prefixed with ‘value:’, leading/trailing white-space is stripped. For example:
tag: Tier1 | tag: Tier2 | tag: Tier3 category: Sanity, Security & tag: -destructive
Note that multiple comma-separated values can be used as a syntactic sugar to shorten the filter notation:
tag: A, B, C ---> tag: A | tag: B | tag: C
Values should be provided as a dictionary of lists each describing the values against which the filter is to be matched. For example:
data = {tag: ["Tier1", "TIPpass"], category: ["Sanity"]}
Other types of dictionary values are converted into a string. A FilterError exception is raised when a dimension parsed from the filter is not found in the data dictionary. Set option ‘sensitive’ to False to enable case-insensitive matching. If ‘regexp’ option is True, regular expressions can be used in the filter values as well.
- fmf.utils.get_cache_directory(create=True)¶
Return cache directory, created by this call if necessary
Cache directory is (first existing): - Value of FMF_CACHE_DIRECTORY environment variable - Value set by the last call of set_cache_directory() - $XDG_CACHE_HOME/fmf - ~/.cache/fmf
Raise GeneralError if it is not possible to create it.
- fmf.utils.info(message, newline=True)¶
Log provided info message to the standard error output
- fmf.utils.invalidate_cache()¶
Force fetch next time cache is used regardless its age
- fmf.utils.listed(items, singular=None, plural=None, max=None, quote='', join='and')¶
Convert an iterable into a nice, human readable list or description:
listed(range(1)) .................... 0 listed(range(2)) .................... 0 and 1 listed(range(3), join='or') ......... 0, 1 or 2 listed(range(3), quote='"') ......... "0", "1" and "2" listed(range(4), max=3) ............. 0, 1, 2 and 1 more listed(range(5), 'number', max=3) ... 0, 1, 2 and 2 more numbers listed(range(6), 'category') ........ 6 categories listed(7, "leaf", "leaves") ......... 7 leaves
If singular form is provided but max not set the description-only mode is activated as shown in the last two examples. Also, an int can be used in this case to get a simple inflection functionality.
- fmf.utils.pluralize(singular=None)¶
Naively pluralize words
- fmf.utils.run(command, cwd=None, check_exit_code=True, env=None)¶
Run command and return a (stdout, stderr) tuple
:command as list (name, arg1, arg2…) :cwd path to directory where to run the command :check_exit_code raise CalledProcessError if exit code is non-zero :env dictionary of the environment variables for the command
- fmf.utils.set_cache_directory(cache_directory)¶
Set preferred cache directory
- fmf.utils.set_cache_expiration(seconds)¶
Seconds until cache expires
- fmf.utils.split(values, separator=re.compile('[ ,]+'))¶
Convert space-or-comma-separated values into a single list
Common use case for this is merging content of options with multiple values allowed into a single list of strings thus allowing any of the formats below and converts them into [‘a’, ‘b’, ‘c’]:
--option a --option b --option c ... ['a', 'b', 'c'] --option a,b --option c ............ ['a,b', 'c'] --option 'a b c' ................... ['a b c']
Accepts both string and list. By default space and comma are used as value separators. Use any regular expression for custom separator.
cli¶
This is command line interface for the Flexible Metadata Format.
Available commands are:
fmf ls List identifiers of available objects
fmf show Show metadata of available objects
fmf init Initialize a new metadata tree
fmf clean Remove cache directory and its content
See online documentation for more details and examples:
Check also help message of individual commands for the full list of available options.
- class fmf.cli.Parser(arguments=None, path=None)¶
Command line options parser
- clean()¶
Remove cache directory
- command_clean()¶
Clean cache
- command_init()¶
Initialize tree
- command_ls()¶
List names
- command_show()¶
Show metadata
- options_formatting()¶
Formating options
- options_select()¶
Select by name, filter
- options_utils()¶
Utilities
- show(brief=False)¶
Show metadata for each path given
- fmf.cli.main(arguments=None, path=None)¶
Parse options, do what is requested