argschema package

Submodules

argschema.argschema_parser module

Module that contains the base class ArgSchemaParser which should be subclassed when using this library

class argschema.argschema_parser.ArgSchemaParser(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')

Bases: object

The main class you should sub-class to write your own argschema module. Takes input_data, reference to a input_json and the command line inputs and parses out the parameters and validates them against the schema_type specified.

To subclass this and make a new schema be default, simply override the default_schema and default_output_schema attributes of this class.

Parameters:
  • input_data (dict or None) – dictionary parameters to fall back on if not source is given or configured via command line
  • schema_type (schemas.ArgSchema) – the schema to use to validate the parameters
  • output_schema_type (marshmallow.Schema) – the schema to use to validate the output, used by self.output
  • input_source (argschema.sources.source.Source) – a generic source of a dictionary
  • output_sink (argschema.sources.source.Source) – a generic sink to write output dictionary to
  • args (list or None) – command line arguments passed to the module, if None use argparse to parse the command line, set to [] if you want to bypass command line parsing
  • logger_name (str) – name of logger from the logging module you want to instantiate default (‘argschema’)
Raises:

marshmallow.ValidationError – If the combination of input_json, input_data and command line arguments do not pass the validation of the schema

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type ArgSchema

default_configurable_sinks = [<class 'argschema.sources.json_source.JsonSink'>]
default_configurable_sources = [<class 'argschema.sources.json_source.JsonSource'>]
default_output_schema = None
default_schema

alias of argschema.schemas.ArgSchema

get_output_json(d)

method for getting the output_json pushed through validation if validation exists :param d: output dictionary to output :type d: dict

Returns:validated and serialized version of the dictionary
Return type:dict
Raises:marshmallow.ValidationError – If any of the output dictionary doesn’t meet the output schema
static initialize_logger(name, log_level)

initializes the logger to a level with a name logger = initialize_logger(name, log_level)

Parameters:
  • name (str) – name of the logger
  • log_level
Returns:

a logger set with the name and level specified

Return type:

logging.Logger

load_schema_with_defaults(schema, args)

method for deserializing the arguments dictionary (args) given the schema (schema) making sure that the default values have been filled in.

Parameters:
  • args (dict) – a dictionary of input arguments
  • schema
Returns:

a deserialized dictionary of the parameters converted through marshmallow

Return type:

dict

Raises:

marshmallow.ValidationError – If this schema contains nested schemas that don’t subclass argschema.DefaultSchema because these won’t work with loading defaults.

output(d, sink=None)

method for outputing dictionary to the output_json file path after validating it through the output_schema_type

Parameters:
  • d (dict) – output dictionary to output
  • sink (argschema.sources.source.ArgSink) – output_sink to output to (optional default to self.output_source)
Raises:

marshmallow.ValidationError – If any of the output dictionary doesn’t meet the output schema

class argschema.argschema_parser.ArgSchemaYamlParser(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')

Bases: argschema.argschema_parser.ArgSchemaParser

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type ArgSchema

default_configurable_sinks = [<class 'argschema.sources.yaml_source.YamlSink'>]
default_configurable_sources = [<class 'argschema.sources.yaml_source.YamlSource'>]

argschema.schemas module

class argschema.schemas.ArgSchema(only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: argschema.schemas.DefaultSchema

The base marshmallow schema used by ArgSchemaParser to identify input_json and output_json files and the log_level

This schema is designed to be a schema_type for an ArgSchemaParser object
ArgSchema
key description default field_type json_type
log_level set the logging level of the module ERROR LogLevel str
opts = <marshmallow.schema.SchemaOpts object>
class argschema.schemas.DefaultSchema(only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Bases: marshmallow.schema.Schema

mm.Schema class with support for making fields default to values defined by that field’s arguments.

make_object(in_data)

marshmallow.pre_load decorated function for applying defaults on deserialation

Parameters:in_data
Returns:a dictionary with default values applied
Return type:dict
opts = <marshmallow.schema.SchemaOpts object>

argschema.utils module

module that contains argschema functions for converting marshmallow schemas to argparse and merging dictionaries from both systems

argschema.utils.args_to_dict(argsobj, schemas=None)

function to convert namespace returned by argsparse into a nested dictionary

Parameters:
  • argsobj (argparse.Namespace) – Namespace object returned by standard argparse.parse function
  • schemas (list[marshmallow.Schema]) – Optional list of schemas which will be used to cast fields via FIELD_TYPE_MAP
Returns:

dictionary of namespace values where nesting elements uses ‘.’ to denote nesting of keys

Return type:

dict

argschema.utils.build_schema_arguments(schema, arguments=None, path=None, description=None)

given a jsonschema, create a dictionary of argparse arguments, by navigating down the Nested schema tree. (recursive function)

Parameters:
  • schema (marshmallow.Schema) – schema with field.description filled in with help values
  • arguments (list or None) – list of argument group dictionaries to add to (see Returns) (Default value = None)
  • path (list or None) – list of strings denoted where you are in the tree (Default value = None)
  • description (str or None) – description for the argument group at this level of the tree
Returns:

List of argument group dictionaries, with keys [‘title’,’description’,’args’] which contain the arguments for argparse. ‘args’ is an OrderedDict of dictionaries with keys of the argument names with kwargs to build an argparse argument

Return type:

list

argschema.utils.cli_error_dict(arg_path, field_type, index=0)

Constuct a nested dictionary containing a casting error message

Matches the format of errors generated by schema.dump.

Parameters:
  • arg_path (string) – List of nested keys
  • field_type (string) – Name of the marshmallow.Field type
  • index (int) – Index into arg_path for recursion
Returns:

Dictionary representing argument path, containing error.

Return type:

dict

argschema.utils.dump(schema, d)
function to wrap marshmallow dump to smooth
differences from marshmallow 2 to 3
Parameters:
  • schema (marshmallow.Schema) – schema that you want to use to validate and dump
  • d (dict) – dictionary to validate and dump
Returns:

serialized and validated dictionary

Return type:

dict

argschema.utils.get_description_from_field(field)

get the description for this marshmallow field

Parameters:field (marshmallow.fields.field) – field to get description
Returns:description string (or None)
Return type:str
argschema.utils.get_field_def_from_schema(parts, schema)

function to get a field_definition from a particular key, specified by it’s parts list

Parameters:
  • parts (list[str]) – the list of keys to get this schema
  • schema (marshmallow.Schema) – the marshmallow schema to look up this key
Returns:

returns the field in the schema if it exists, otherwise returns None

Return type:

marshmallow.Field or None

argschema.utils.get_type_from_field(field)

Get type casting for command line argument from marshmallow.Field

Parameters:field (marshmallow.Field) – Field class from input schema
Returns:Function to call to cast argument to
Return type:callable
argschema.utils.load(schema, d)
function to wrap marshmallow load to smooth
differences from marshmallow 2 to 3
Parameters:
  • schema (marshmallow.Schema) – schema that you want to use to validate
  • d (dict) – dictionary to validate and load
Returns:

deserialized and validated dictionary

Return type:

dict

argschema.utils.merge_value(a, b, key, func=<built-in function add>)

attempt to merge these dictionaries using function defined by func (default to add) raise an exception if this fails

Parameters:
  • a (dict) – one dictionary
  • b (dict) – second dictionary
  • key (key) – key to merge dictionary values on
  • func (a[key]) – function that merges two values of this key Returns (Default value = add)
  • func – merged version of values (Default value = add)
Returns:

Return type:

value

argschema.utils.prune_dict_with_none(d)

function to remove all dictionaries from a nested dictionary when all the values of a particular dictionary are None

Parameters:d (dictionary to prune) –
Returns:pruned dictionary
Return type:dict
argschema.utils.schema_argparser(schema, additional_schemas=None)

given a jsonschema, build an argparse.ArgumentParser

Parameters:
  • schema (argschema.schemas.ArgSchema) – schema to build an argparser from
  • additional_schemas (list[marshmallow.schema]) – list of additional schemas to add to the command line arguments
Returns:

that represents the schemas

Return type:

argparse.ArgumentParser

argschema.utils.smart_merge(a, b, path=None, merge_keys=None, overwrite_with_none=False)

updates dictionary a with values in dictionary b being careful not to write things with None, and performing a merge on merge_keys

Parameters:
  • a (dict) – dictionary to perform update on
  • b (dict) – dictionary to perform update with
  • path (list) – list of nested keys traversed so far (used for recursion) (Default value = None)
  • merge_keys (list) – list of keys to do merging on (default None)
  • overwrite_with_none – (Default value = False)
Returns:

a dictionary that is a updated with b’s values

Return type:

dict

argschema.validate module

module for custom marshmallow validators

class argschema.validate.Shape(shape=None)

Bases: marshmallow.validate.Validator

Validator which succeeds if value.shape matches shape

Parameters:

shape (tuple) – Tuple specifying the required shape. If a value in the tuple is None, any length in that dimension is valid.

Raises:
  • ValueError – If the provided shape is not a valid tuple of integers and/or None types
  • marshmallow.ValidationError – If the value being validated does not have a shape attribute

argschema.autodoc module

argschema.autodoc.process_schemas(app, what, name, obj, options, lines)

function designed to process a sphinx.ext.autodoc event as autodoc hook to alter docstring lines of argschema related classes, providing a table of parameters for schemas and links to the default schemas for ArgSchemaParser derived elements

use in sphnix conf.py as follows

from argschema.autodoc import process_schemas
def setup(app):
    app.connect('autodoc-process-docstring',process_schemas)

Module contents

argschema: flexible definition, validation and setting of parameters

argschema.main()