argschema documentation¶

This python module enables python programs to specify and validate their input parameters via a schema, while allowing those parameters to be passed into it in different ways in different contexts.

In particular it will allow you to

Specify an input_json file which contains the parameters via the command line
OR pass a dictionary directly into the module with the parameters defined
AND/OR pass individual parameters via the command line, in a way that will override the input_json or the dictionary given.

In all cases, it will merge these different parameters into a single dictionary and then validate the parameters against your schema.

The User Guide¶

This is where you should start to understand how to use argschema

User Guide¶

Installation¶

install via source code

$ python setup.py install

or pip

$ pip install argschema

Your First Module¶

mymodule.py¶

import argschema

class MySchema(argschema.ArgSchema):
    a = argschema.fields.Int(default = 42, description= 'my first parameter')
                            
if __name__ == '__main__':
    mod = argschema.ArgSchemaParser(schema_type=MySchema)
    print(mod.args)
    

running this code produces

$ python mymodule.py
{'a': 42, 'log_level': u'ERROR'}
$ python mymodule.py --a 2
{'a': 2, 'log_level': u'ERROR'}
$ python mymodule.py --a 2 --log_level WARNING
{'a': 2, 'log_level': u'WARNING'}
WARNING:argschema.argschema_parser:this program does nothing useful
$ python mymodule.py -h
usage: mymodule.py [-h] [--a A] [--output_json OUTPUT_JSON]
                [--log_level LOG_LEVEL] [--input_json INPUT_JSON]

optional arguments:
-h, --help            show this help message and exit
--a A                 my first parameter
--output_json OUTPUT_JSON
                        file path to output json file
--log_level LOG_LEVEL
                        set the logging level of the module
--input_json INPUT_JSON
                        file path of input json file

Great you are thinking, that is basically argparse, congratulations!

But there is more.. you can also give your module a dictionary in an interactive session

>>> from argschema import ArgSchemaParser
>>> from mymodule import MySchema
>>> d = {'a':5}
>>> mod = ArgSchemaParser(input_data=d,schema_type=MySchema)
>>> print(mod.args)
{'a': 5, 'log_level': u'ERROR'}

or you write out a json file and pass it the path on the command line

myinput.json¶

{
    "a":99
}

$ python mymodule.py --input_json myinput.json
{'a': 99, 'log_level': u'ERROR', 'input_json': u'myinput.json'}

or override a parameter if you want

$ python mymodule.py --input_json myinput.json --a 100
{'a': 100, 'log_level': u'ERROR', 'input_json': u'myinput.json'}

plus, no matter how you give it parameters, they will always be validated, before any of your code runs.

Whether from the command line

$ python mymodule.py --input_json myinput.json --a 5!
usage: mymodule.py [-h] [--a A] [--output_json OUTPUT_JSON]
                [--log_level LOG_LEVEL] [--input_json INPUT_JSON]
mymodule.py: error: argument --a: invalid int value: '5!'

or from a dictionary

>>> from argschema import ArgSchemaParser
>>> from mymodule import MySchema
>>> d={'a':'hello'}
>>> mod = ArgSchemaParser(input_data=d,schema_type=MySchema)
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/Users/forrestcollman/argschema/argschema/argschema_parser.py", line 159, in __init__
        raise mm.ValidationError(json.dumps(result.errors, indent=2))
    marshmallow.exceptions.ValidationError: {
    "a": [
        "Not a valid integer."
    ]
    }

Fields¶

argschema uses marshmallow (http://marshmallow.readthedocs.io/) under the hood to define the parameters schemas. It comes with a basic set of fields that you can use to define your schemas. One powerful feature of Marshmallow is that you can define custom fields that do arbitrary validation. argschema.fields contains all the built-in marshmallow fields, but also some useful custom ones, such as argschema.fields.InputFile, argschema.fields.OutputFile, argschema.fields.InputDir that validate that the paths exist and have the proper permissions to allow files to be read or written.

Other fields, such as argschema.fields.NumpyArray will deserialize ordered lists of lists directly into a numpy array of your choosing.

Finally, an important Field to know is argschema.fields.Nested, which allows you to define heirarchical nested structures. Note, that if you use Nested schemas, your Nested schemas should subclass argschema.schemas.DefaultSchema in order that they properly fill in default values, as marshmallow.Schema does not do that by itself.

The template_module example shows how you might combine these features to define a more complex parameter structure.

template_module.py¶

from argschema import ArgSchemaParser, ArgSchema
from argschema.fields import OutputFile, NumpyArray, Boolean, Int, Str, Nested
from argschema.schemas import DefaultSchema
import numpy as np
import json

# these are the core parameters for my module
class MyNestedParameters(DefaultSchema):
    name = Str(required=True, description='name of vector')
    increment = Int(required=True, description='value to increment')
    array = NumpyArray(dtype=np.float, required=True, description='array to increment')
    write_output = Boolean(required=False, default=True)

# but i'm going to nest them inside a subsection called inc
class MyParameters(ArgSchema):
    inc = Nested(MyNestedParameters)

#this is another schema we will use to validate and deserialize our output
class MyOutputParams(DefaultSchema):
    name = Str(required=True, description='name of vector')
    inc_array = NumpyArray(dtype=np.float, required=True, description='incremented array')

if __name__ == '__main__':
    
    # this defines a default dictionary that will be used if input_json is not specified
    example_input = {
        "inc": {
            "name": "from_dictionary",
            "increment": 5,
            "array": [0, 2, 5],

            "write_output": True
        },
        "output_json": "output_dictionary.json"
    }

    # here is my ArgSchemaParser that processes my inputs
    mod = ArgSchemaParser(input_data=example_input,
                          schema_type=MyParameters,
                          output_schema_type=MyOutputParams)
                          
    # pull out the inc section of the parameters
    inc_params = mod.args['inc']

    # do my simple addition of the parameters
    inc_array = inc_params['array'] + inc_params['increment']

    # define the output dictionary
    output = {
        'name': inc_params['name'],
        'inc_array': inc_array
    }

    # if the parameters are set as such write the output
    if inc_params['write_output']:
        mod.output(output)

so now if run the example commands found in run_template.sh

input.json¶

  {
      "inc": {
            "name": "from_json",
            "increment": 1,
            "array": [3, 2, 1],
            "write_output": true
       }
  }

$ python template_module.py \
    --output_json output_command.json \
    --inc.name from_command \
    --inc.increment 2
{u'name': u'from_command', u'inc_array': [2.0, 4.0, 7.0]}
$ python template_module.py \
    --input_json input.json \
    --output_json output_fromjson.json
{u'name': u'from_json', u'inc_array': [4.0, 3.0, 2.0]}
$ python template_module.py
{u'name': u'from_dictionary', u'inc_array': [5.0, 7.0, 10.0]}

Command-Line Specification¶

As mentioned in the section Your First Module, argschema supports setting arguments at the command line, along with providing arguments either in an input json or directly passing a dictionary as input_data. Values passed at the command line will take precedence over those passed to the parser or in the input json.

Arguments are specified with –argument_name <value>, where value is passed by the shell. If there are spaces in the value, it will need to be wrapped in quotes, and any special characters will need to be escaped with . Booleans are set with True or 1 for true and False or 0 for false.

An exception to this rule is list formatting. If a schema contains a List and does not set the cli_as_single_argument keyword argument to True, lists will be parsed as –list_name <value1> <value2> …. In argschema 2.0 lists will be parsed in the same way as other arguments, as it allows more flexibility in list types and more clearly represents the intended data structure.

An example script showing old and new list settings:

deprecated_example.py¶

from argschema import ArgSchema, ArgSchemaParser
from argschema.fields import List, Float


class MySchema(ArgSchema):
    list_old = List(Float, default=[1.1, 2.2, 3.3],
                    description="float list with deprecated cli")
    list_new = List(Float, default=[4.4, 5.5, 6.6],
                    cli_as_single_argument=True,
                    description="float list with supported cli")


if __name__ == '__main__':
    mod = ArgSchemaParser(schema_type=MySchema)
    print(mod.args)

Running this code can demonstrate the differences in command-line usage:

$ python deprecated_example.py --help
FutureWarning: '--list_old' is using old-style command-line syntax
with each element as a separate argument. This will not be supported
in argschema after 2.0. See http://argschema.readthedocs.io/en/master/user/intro.html#command-line-specification
for details.
warnings.warn(warn_msg, FutureWarning)
usage: deprecated_example.py [-h] [--input_json INPUT_JSON]
                             [--output_json OUTPUT_JSON]
                             [--log_level LOG_LEVEL]
                             [--list_old [LIST_OLD [LIST_OLD ...]]]
                             [--list_new LIST_NEW]

optional arguments:
  -h, --help            show this help message and exit

MySchema:
  --input_json INPUT_JSON
                        file path of input json file
  --output_json OUTPUT_JSON
                        file path to output json file
  --log_level LOG_LEVEL
                        set the logging level of the module (default=ERROR)
  --list_old [LIST_OLD [LIST_OLD ...]]
                        float list with deprecated cli (default=[1.1, 2.2,
                        3.3])
  --list_new LIST_NEW   float list with supported cli (default=[4.4, 5.5,
                        6.6])
$ python deprecated_example.py --list_old 9.1 8.2 7.3 --list_new [6.4,5.5,4.6]
FutureWarning: '--list_old' is using old-style command-line syntax
with each element as a separate argument. This will not be supported
in argschema after 2.0. See http://argschema.readthedocs.io/en/master/user/intro.html#command-line-specification
for details.
warnings.warn(warn_msg, FutureWarning)
{'log_level': 'ERROR', 'list_new': [6.4, 5.5, 4.6], 'list_old': [9.1, 8.2, 7.3]}

We can explore some typical examples of command line usage with the following script:

cli_example.py¶

from argschema import ArgSchema, ArgSchemaParser
from argschema.fields import List, NumpyArray, Bool, Int, Nested, Str
from argschema.schemas import DefaultSchema


class MyNestedSchema(DefaultSchema):
    a = Int(default=42, description= "my first parameter")
    b = Bool(default=True, description="my boolean")


class MySchema(ArgSchema):
    array = NumpyArray(default=[[1, 2, 3],[4, 5, 6]], dtype="uint8",
                       description="my example array")
    string_list = List(List(Str),
                       default=[["hello", "world"], ["lists!"]],
                       cli_as_single_argument=True,
                       description="list of lists of strings")
    int_list = List(Int, default=[1, 2, 3],
                    cli_as_single_argument=True,
                    description="list of ints")
    nested = Nested(MyNestedSchema, required=True)


if __name__ == '__main__':
    mod = ArgSchemaParser(schema_type=MySchema)
    print(mod.args)

$ python cli_example.py --help
usage: cli_example.py [-h] [--input_json INPUT_JSON]
                      [--output_json OUTPUT_JSON] [--log_level LOG_LEVEL]
                      [--array ARRAY] [--string_list STRING_LIST]
                      [--int_list INT_LIST] [--nested.a NESTED.A]
                      [--nested.b NESTED.B]

optional arguments:
  -h, --help            show this help message and exit

MySchema:
  --input_json INPUT_JSON
                        file path of input json file
  --output_json OUTPUT_JSON
                        file path to output json file
  --log_level LOG_LEVEL
                        set the logging level of the module (default=ERROR)
  --array ARRAY         my example array (default=[[1, 2, 3], [4, 5, 6]])
  --string_list STRING_LIST
                        list of lists of strings (default=[['hello', 'world'],
                        ['lists!']])
  --int_list INT_LIST   list of ints (default=[1, 2, 3])

nested:
  --nested.a NESTED.A   my first parameter (default=42)
  --nested.b NESTED.B   my boolean (default=True)

We can set some values and observe the output:

$ python cli_example.py --nested.b 0 --string_list "[['foo','bar'],['baz','buz']]"
{'int_list': [1, 2, 3], 'string_list': [['foo', 'bar'], ['baz', 'buz']], 'array': array([[1, 2, 3],
   [4, 5, 6]], dtype=uint8), 'log_level': 'ERROR', 'nested': {'a': 42, 'b': False}}

If we try to set a field in a way the parser can’t cast the variable (for example, having an invalid literal) we will see a casting validation error:

$ python cli_example.py --array [1,foo,3]
Traceback (most recent call last):
  File "cli_example.py", line 25, in <module>
    mod = ArgSchemaParser(schema_type=MySchema)
  ...
marshmallow.exceptions.ValidationError: {
  "array": [
    "Command-line argument can't cast to NumpyArray"
  ]
}

argschema does not support setting Dict at the command line.

Alternate Sources/Sinks¶

Json files are just one way that you might decide to serialize module parameters or outputs. Argschema by default provides json support because that is what we use most frequently at the Allen Institute, however we have generalized the concept to allow argschema.ArgSchemaParser to plugin alternative “sources” and “sinks” of dictionary inputs and outputs.

For example, yaml is another reasonable choice for storing nested key-value stores. argschema.argschema_parser.ArgSchemaYamlParser demonstrates just that functionality. So now input_yaml and output_yaml can be specified instead.

Furthermore, you can pass an ArgSchemaParser an argschema.sources.ArgSource object which implements a get_dict method, and any argschema.ArgSchemaParser will get its input parameters from that dictionary. Importantly, this is true even when the original module author didn’t explicitly support passing parameters from that mechanism, and the parameters will still be deserialized and validated in a uniform manner.

Similarly you can pass an argschema.sources.ArgSink object which implements a put_dict method, and argschema.ArgSchemaParser.output will output the dictionary however that argschema.sources.ArgSink specifies it should.

Finally, both argschema.sources.ArgSource and argschema.sources.ArgSink have a property called ConfigSchema, which is a marshmallow.Schema for how to deserialize the kwargs to it’s init class.

For example, the default argschema.sources.json_source.JsonSource has one string field of ‘input_json’. This is how argschema.ArgSchemaParser is told what keys and values should be read to initialize a argschema.sources.ArgSource or

So for example, if you wanted to define a argschema.sources.ArgSource which loaded a dictionary from a particular host, port and url, and a module which had a command line interface for setting that host port and url you could do so like this.

from argschema.sources import ArgSource, ArgSink
from argschema.schemas import DefaultSchema
from argschema.fields import Str,Int
from argschema import ArgSchemaParser
from test_classes import MySchema
import requests
try:
    from urllib.parse import urlunparse 
except:
    from urlparse import urlunparse

class UrlSourceConfig(DefaultSchema):
    input_host = Str(required=True, description="host of url")
    input_port = Int(required=False, default=None, description="port of url")
    input_url = Str(required=True, description="location on host of input")
    input_protocol = Str(required=False, default='http', description="url protocol to use")

class UrlSource(ArgSource):
    ConfigSchema = UrlSourceConfig

    def get_dict(self):
        if self.input_port is None:
            netloc = self.input_host
        else:
            netloc = "{}:{}".format(self.input_host,self.input_port)
        url = urlunparse((self.input_protocol,netloc,self.input_url,None,None,None))                             
        response = requests.get(url)
        return response.json()


class UrlArgSchemaParser(ArgSchemaParser):
    default_configurable_sources = [UrlSource]
    default_schema = MySchema

so now a UrlArgSchemaParser would expect command line flags of ‘–input_host’ and ‘–input_url’, and optionally ‘–input_port’,’–input_protocol’ (or look for them in input_data) and will look to download the json from that http location via requests. In addition, an existing argschema.ArgSchemaParser module could be simply passed a configured UrlSource via input_source, and it would get its parameters from there.

Sphinx Documentation¶

argschema comes with a autodocumentation feature for Sphnix which will help you automatically add documentation of your Schemas and argschema.ArgSchemaParser classes in your project. This is how the documentation of the test suite included here was generated.

To configure sphnix to use this function, you must be using the sphnix autodoc module and add the following to your conf.py file

from argschema.autodoc import process_schemas

def setup(app):
    app.connect('autodoc-process-docstring',process_schemas)

Indices and tables¶

API¶

This contains the complete documentation of the api

argschema package¶

Subpackages¶

argschema.fields package¶

Submodules¶

argschema.fields.files module¶

marshmallow fields related to validating input and output file paths

class argschema.fields.files.InputDir(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

InputDir is marshmallow.fields.Str subclass which is a path to a a directory that exists and that the user can access (presently checked with os.access)

class argschema.fields.files.InputFile(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

InputDile is a marshmallow.fields.Str subclass which is a path to a file location which can be read by the user (presently passes os.path.isfile and os.access = R_OK)

class argschema.fields.files.OutputDir(mode=None, *args, **kwargs)[source]¶

Bases: marshmallow.fields.String

OutputDir is a marshmallow.fields.Str subclass which is a path to a location where this module will write files. Validation will check that the directory exists and create the directory if it is not present, and will fail validation if the directory cannot be created or cannot be written to.

Parameters:	mode (str) – mode to create directory args – smae as passed to marshmallow.fields.Str *kwargs – same as passed to marshmallow.fields.Str

class argschema.fields.files.OutputFile(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

OutputFile marshmallow.fields.Str subclass which is a path to a: file location that can be written to by the current user (presently tested by opening a temporary file to that location)

argschema.fields.files.validate_outpath(path)[source]¶

argschema.fields.loglevel module¶

marshmallow fields related to setting logging levels

class argschema.fields.loglevel.LogLevel(**kwargs)[source]¶

Bases: marshmallow.fields.String

LogLevel is a field type that provides a setting for the loglevel of python.logging. This class will both validate the input and also set the input globally. In simple scenarios, a module will not have to do any manipulation of loglevel.

options = ['FATAL', 'CRITICAL', 'ERROR', 'WARN', 'WARNING', 'INFO', 'DEBUG']¶

argschema.fields.numpyarrays module¶

marshmallow fields related to reading in numpy arrays

class argschema.fields.numpyarrays.NumpyArray(dtype=None, *args, **kwargs)[source]¶

Bases: marshmallow.fields.List

NumpyArray is a marshmallow.fields.List subclass which will convert any numpy compatible set of lists into a numpy array after deserialization and convert it back to a list when serializing,

Parameters:	dtype (numpy.Dtype) – dtype specifying the desired data type. if dtype is given the array will be converted to the type, otherwise numpy will decide what type it should be. (Default=None)

argschema.fields.slice module¶

class argschema.fields.slice.Slice(**kwargs)[source]¶

Bases: marshmallow.fields.String

Slice is a :class:’marshmallow.fields.Str’ field that supports a range or slice argument for selecting some subset of a larger dataset. The syntax is identical to numpy slicing. Examples: “10:20”, “40”, “:30”, “10:2:40”

Parameters:	kwargs – the same as any `Str` receive

Module contents¶

sub-module for custom marshmallow fields of general utility

class argschema.fields.Field(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.base.FieldABC

Basic field from which other fields should extend. It applies no formatting by default, and should only be used in cases where data does not need to be formatted before being serialized or deserialized. On error, the name of the field will be returned.

Parameters:

default – If set, this value will be used during serialization if the input value is missing. If not set, the field will be excluded from the serialized output if the input value is missing. May be a value or a callable.
attribute (str) – The name of the attribute to get the value from. If None, assumes the attribute has the same name as the field.
load_from (str) – Additional key to look for when deserializing. Will only be checked if the field’s name is not found on the input dictionary. If checked, it will return this parameter on error.
dump_to (str) – Field name to use as a key when serializing.
validate (callable) – Validator or collection of validators that are called during deserialization. Validator takes a field’s input value as its only parameter and returns a boolean. If it returns False, an ValidationError is raised.
required – Raise a ValidationError if the field value is not supplied during deserialization.
allow_none – Set this to True if None should be considered a valid value during validation/deserialization. If missing=None and allow_none is unset, will default to True. Otherwise, the default is False.
load_only (bool) – If True skip this field during serialization, otherwise its value will be present in the serialized data.
dump_only (bool) – If True skip this field during deserialization, otherwise its value will be present in the deserialized object. In the context of an HTTP API, this effectively marks the field as “read-only”.
missing – Default deserialization value for the field if the field is not found in the input data. May be a value or a callable.
error_messages (dict) – Overrides for Field.default_error_messages.
metadata – Extra arguments to be stored as metadata.

Changed in version 2.0.0: Removed error parameter. Use error_messages instead.

Changed in version 2.0.0: Added allow_none parameter, which makes validation/deserialization of None consistent across fields.

Changed in version 2.0.0: Added load_only and dump_only parameters, which allow field skipping during the (de)serialization process.

Changed in version 2.0.0: Added missing parameter, which indicates the value for a field if the field is not found during deserialization.

Changed in version 2.0.0: default value is only used if explicitly set. Otherwise, missing values inputs are excluded from serialized output.

context¶: The context dictionary for the parent Schema.

default_error_messages = {u'null': u'Field may not be null.', u'validator_failed': u'Invalid value.', u'required': u'Missing data for required field.', u'type': u'Invalid input type.'}¶

deserialize(value, attr=None, data=None)[source]¶

Deserialize value.

Raises:	ValidationError – If an invalid value is passed or if a required value is missing.

fail(key, **kwargs)[source]¶: A helper method that simply raises a ValidationError.

get_value(attr, obj, accessor=None, default=<marshmallow.missing>)[source]¶: Return the value for a given key from an object.

root¶: Reference to the Schema that this field belongs to even if it is buried in a List. Return None for unbound fields.

serialize(attr, obj, accessor=None)[source]¶

Pulls the value for the given key from the object, applies the field’s formatting and returns the result.

Parameters:	attr (str) – The attibute or key to get from the object. obj (str) – The object to pull the key from. accessor (callable) – Function used to pull values from `obj`.
Raises:	ValidationError – In case of formatting problem

class argschema.fields.Raw(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

Field that applies no formatting or validation.

class argschema.fields.Nested(nested, default=<marshmallow.missing>, exclude=(), only=None, **kwargs)[source]¶

Bases: marshmallow.fields.Field

Allows you to nest a Schema inside a field.

Examples:

user = fields.Nested(UserSchema)
user2 = fields.Nested('UserSchema')  # Equivalent to above
collaborators = fields.Nested(UserSchema, many=True, only='id')
parent = fields.Nested('self')

When passing a Schema <marshmallow.Schema> instance as the first argument, the instance’s exclude, only, and many attributes will be respected.

Therefore, when passing the exclude, only, or many arguments to fields.Nested, you should pass a Schema <marshmallow.Schema> class (not an instance) as the first argument.

# Yes
author = fields.Nested(UserSchema, only=('id', 'name'))

# No
author = fields.Nested(UserSchema(), only=('id', 'name'))

Parameters:

nested (Schema) – The Schema class or class name (string) to nest, or "self" to nest the Schema within itself.
default – Default value to if attribute is missing or None
exclude (tuple) – A list or tuple of fields to exclude.
required – Raise an ValidationError during deserialization if the field, and any required field values specified in the nested schema, are not found in the data. If not a bool (e.g. a str), the provided value will be used as the message of the ValidationError instead of the default message.
only – A tuple or string of the field(s) to marshal. If None, all fields will be marshalled. If a field name (string) is given, only a single value will be returned as output instead of a dictionary. This parameter takes precedence over exclude.
many (bool) – Whether the field is a collection of objects.
kwargs – The same keyword arguments that Field receives.

default_error_messages = {u'type': u'Invalid type.'}¶

schema¶: The nested Schema object.

Changed in version 1.0.0: Renamed from serializer to schema

class argschema.fields.Dict(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

A dict field. Supports dicts and dict-like objects.

Note

This field is only appropriate when the structure of nested data is not known. For structured data, use Nested.

New in version 2.1.0.

default_error_messages = {u'invalid': u'Not a valid mapping type.'}¶

class argschema.fields.List(cls_or_instance, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A list field, composed with another Field class or instance.

Example:

numbers = fields.List(fields.Float())

Parameters:	cls_or_instance (Field) – A field class or instance. default (bool) – Default value for serialization. kwargs – The same keyword arguments that `Field` receives.

Changed in version 2.0.0: The allow_none parameter now applies to deserialization and has the same semantics as the other fields.

default_error_messages = {u'invalid': u'Not a valid list.'}¶

get_value(attr, obj, accessor=None)[source]¶: Return the value for a given key from an object.

class argschema.fields.String(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

A string field.

Parameters:	kwargs – The same keyword arguments that `Field` receives.

default_error_messages = {u'invalid_utf8': u'Not a valid utf-8 string.', u'invalid': u'Not a valid string.'}¶

class argschema.fields.UUID(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

A UUID field.

default_error_messages = {u'invalid_uuid': u'Not a valid UUID.', u'invalid_guid': u'Not a valid UUID.'}¶

class argschema.fields.Number(as_string=False, **kwargs)[source]¶

Bases: marshmallow.fields.Field

Base class for number fields.

Parameters:	as_string (bool) – If True, format the serialized value as a string. kwargs – The same keyword arguments that `Field` receives.

default_error_messages = {u'invalid': u'Not a valid number.'}¶

num_type¶: alias of float

serialize(attr, obj, accessor=None)[source]¶: Pulls the value for the given key from the object and returns the serialized number representation. Return a string if self.as_string=True, othewise return this field’s num_type. Receives the same args and kwargs as Field.

class argschema.fields.Integer(as_string=False, **kwargs)[source]¶

Bases: marshmallow.fields.Number

An integer field.

Parameters:	kwargs – The same keyword arguments that `Number` receives.

default_error_messages = {u'invalid': u'Not a valid integer.'}¶

num_type¶: alias of int

class argschema.fields.Decimal(places=None, rounding=None, allow_nan=False, as_string=False, **kwargs)[source]¶

Bases: marshmallow.fields.Number

A field that (de)serializes to the Python decimal.Decimal type. It’s safe to use when dealing with money values, percentages, ratios or other numbers where precision is critical.

Warning

This field serializes to a decimal.Decimal object by default. If you need to render your data as JSON, keep in mind that the json module from the standard library does not encode decimal.Decimal. Therefore, you must use a JSON library that can handle decimals, such as simplejson, or serialize to a string by passing as_string=True.

Warning

If a JSON float value is passed to this field for deserialization it will first be cast to its corresponding string value before being deserialized to a decimal.Decimal object. The default __str__ implementation of the built-in Python float type may apply a destructive transformation upon its input data and therefore cannot be relied upon to preserve precision. To avoid this, you can instead pass a JSON string to be deserialized directly.

Parameters:

places (int) – How many decimal places to quantize the value. If None, does not quantize the value.
rounding – How to round the value during quantize, for example decimal.ROUND_UP. If None, uses the rounding value from the current thread’s context.
allow_nan (bool) – If True, NaN, Infinity and -Infinity are allowed, even though they are illegal according to the JSON specification.
as_string (bool) – If True, serialize to a string instead of a Python decimal.Decimal type.
kwargs – The same keyword arguments that Number receives.

New in version 1.2.0.

default_error_messages = {u'special': u'Special numeric values are not permitted.'}¶

num_type¶: alias of Decimal

class argschema.fields.Boolean(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

A boolean field.

Parameters:	kwargs – The same keyword arguments that `Field` receives.

default_error_messages = {u'invalid': u'Not a valid boolean.'}¶

falsy = set([0, u'False', u'F', u'f', u'FALSE', u'0', u'false'])¶

truthy = set([1, u'true', u'1', u't', u'True', u'TRUE', u'T'])¶

class argschema.fields.FormattedString(src_str, *args, **kwargs)[source]¶

Bases: marshmallow.fields.Field

Interpolate other values from the object into this field. The syntax for the source string is the same as the string str.format method from the python stdlib.

class UserSchema(Schema):
    name = fields.String()
    greeting = fields.FormattedString('Hello {name}')

ser = UserSchema()
res = ser.dump(user)
res.data  # => {'name': 'Monty', 'greeting': 'Hello Monty'}

default_error_messages = {u'format': u'Cannot format string with given data.'}¶

class argschema.fields.Float(as_string=False, **kwargs)[source]¶

Bases: marshmallow.fields.Number

A double as IEEE-754 double precision string.

Parameters:	as_string (bool) – If True, format the value as a string. kwargs – The same keyword arguments that `Number` receives.

num_type¶: alias of float

class argschema.fields.DateTime(format=None, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A formatted datetime string in UTC.

Example: '2014-12-22T03:12:58.019077+00:00'

Timezone-naive datetime objects are converted to UTC (+00:00) by Schema.dump. Schema.load returns datetime objects that are timezone-aware.

Parameters:	format (str) – Either `"rfc"` (for RFC822), `"iso"` (for ISO8601), or a date format string. If None, defaults to “iso”. kwargs – The same keyword arguments that `Field` receives.

DATEFORMAT_DESERIALIZATION_FUNCS = {u'rfc': <function from_rfc>, u'rfc822': <function from_rfc>, u'iso': <function from_iso>, u'iso8601': <function from_iso>}¶

DATEFORMAT_SERIALIZATION_FUNCS = {u'rfc': <function rfcformat>, u'rfc822': <function rfcformat>, u'iso': <function isoformat>, u'iso8601': <function isoformat>}¶

DEFAULT_FORMAT = u'iso'¶

default_error_messages = {u'invalid': u'Not a valid datetime.', u'format': u'"{input}" cannot be formatted as a datetime.'}¶

localtime = False¶

class argschema.fields.LocalDateTime(format=None, **kwargs)[source]¶

Bases: marshmallow.fields.DateTime

A formatted datetime string in localized time, relative to UTC.

Takes the same arguments as DateTime.

localtime = True¶

class argschema.fields.Time(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

ISO8601-formatted time string.

Parameters:	kwargs – The same keyword arguments that `Field` receives.

default_error_messages = {u'invalid': u'Not a valid time.', u'format': u'"{input}" cannot be formatted as a time.'}¶

class argschema.fields.Date(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.Field

ISO8601-formatted date string.

Parameters:	kwargs – The same keyword arguments that `Field` receives.

default_error_messages = {u'invalid': u'Not a valid date.', u'format': u'"{input}" cannot be formatted as a date.'}¶

class argschema.fields.TimeDelta(precision=u'seconds', error=None, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A field that (de)serializes a datetime.timedelta object to an integer and vice versa. The integer can represent the number of days, seconds or microseconds.

Parameters:	precision (str) – Influences how the integer is interpreted during (de)serialization. Must be ‘days’, ‘seconds’ or ‘microseconds’. error (str) – Error message stored upon validation failure. kwargs – The same keyword arguments that `Field` receives.

Changed in version 2.0.0: Always serializes to an integer value to avoid rounding errors. Add precision parameter.

DAYS = u'days'¶

MICROSECONDS = u'microseconds'¶

SECONDS = u'seconds'¶

default_error_messages = {u'invalid': u'Not a valid period of time.', u'format': u'{input!r} cannot be formatted as a timedelta.'}¶

class argschema.fields.Url(relative=False, schemes=None, **kwargs)[source]¶

Bases: marshmallow.fields.ValidatedField, marshmallow.fields.String

A validated URL field. Validation occurs during both serialization and deserialization.

Parameters:	default – Default value for the field if the attribute is not set. attribute (str) – The name of the attribute to get the value from. If None, assumes the attribute has the same name as the field. relative (bool) – Allow relative URLs. kwargs – The same keyword arguments that `String` receives.

default_error_messages = {u'invalid': u'Not a valid URL.'}¶

argschema.fields.URL¶: alias of Url

class argschema.fields.Email(*args, **kwargs)[source]¶

Bases: marshmallow.fields.ValidatedField, marshmallow.fields.String

A validated email field. Validation occurs during both serialization and deserialization.

Parameters:	args – The same positional arguments that `String` receives. kwargs – The same keyword arguments that `String` receives.

default_error_messages = {u'invalid': u'Not a valid email address.'}¶

class argschema.fields.Method(serialize=None, deserialize=None, method_name=None, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A field that takes the value returned by a Schema method.

Parameters:	method_name (str) – The name of the Schema method from which to retrieve the value. The method must take an argument `obj` (in addition to self) that is the object to be serialized. deserialize (str) – Optional name of the Schema method for deserializing a value The method must take a single argument `value`, which is the value to deserialize.

Changed in version 2.0.0: Removed optional context parameter on methods. Use self.context instead.

Changed in version 2.3.0: Deprecated method_name parameter in favor of serialize and allow serialize to not be passed at all.

class argschema.fields.Function(serialize=None, deserialize=None, func=None, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A field that takes the value returned by a function.

Parameters:

serialize (callable) – A callable from which to retrieve the value. The function must take a single argument obj which is the object to be serialized. It can also optionally take a context argument, which is a dictionary of context variables passed to the serializer. If no callable is provided then the `load_only` flag will be set to True.
deserialize (callable) – A callable from which to retrieve the value. The function must take a single argument value which is the value to be deserialized. It can also optionally take a context argument, which is a dictionary of context variables passed to the deserializer. If no callable is provided then `value` will be passed through unchanged.
func (callable) – This argument is to be deprecated. It exists for backwards compatiblity. Use serialize instead.

Changed in version 2.3.0: Deprecated func parameter in favor of serialize.

argschema.fields.Str¶: alias of String

argschema.fields.Bool¶: alias of Boolean

argschema.fields.Int¶: alias of Integer

class argschema.fields.Constant(constant, **kwargs)[source]¶

Bases: marshmallow.fields.Field

A field that (de)serializes to a preset constant. If you only want the constant added for serialization or deserialization, you should use dump_only=True or load_only=True respectively.

Parameters:	constant – The constant to return for the field attribute.

New in version 2.0.0.

class argschema.fields.OutputFile(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

OutputFile marshmallow.fields.Str subclass which is a path to a: file location that can be written to by the current user (presently tested by opening a temporary file to that location)

class argschema.fields.InputDir(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

InputDir is marshmallow.fields.Str subclass which is a path to a a directory that exists and that the user can access (presently checked with os.access)

class argschema.fields.InputFile(default=<marshmallow.missing>, attribute=None, load_from=None, dump_to=None, error=None, validate=None, required=False, allow_none=None, load_only=False, dump_only=False, missing=<marshmallow.missing>, error_messages=None, **metadata)[source]¶

Bases: marshmallow.fields.String

InputDile is a marshmallow.fields.Str subclass which is a path to a file location which can be read by the user (presently passes os.path.isfile and os.access = R_OK)

class argschema.fields.OutputDir(mode=None, *args, **kwargs)[source]¶

Bases: marshmallow.fields.String

OutputDir is a marshmallow.fields.Str subclass which is a path to a location where this module will write files. Validation will check that the directory exists and create the directory if it is not present, and will fail validation if the directory cannot be created or cannot be written to.

Parameters:	mode (str) – mode to create directory args – smae as passed to marshmallow.fields.Str *kwargs – same as passed to marshmallow.fields.Str

class argschema.fields.NumpyArray(dtype=None, *args, **kwargs)[source]¶

Bases: marshmallow.fields.List

NumpyArray is a marshmallow.fields.List subclass which will convert any numpy compatible set of lists into a numpy array after deserialization and convert it back to a list when serializing,

Parameters:	dtype (numpy.Dtype) – dtype specifying the desired data type. if dtype is given the array will be converted to the type, otherwise numpy will decide what type it should be. (Default=None)

class argschema.fields.OptionList(options, **kwargs)[source]¶

Bases: marshmallow.fields.Field

OptionList is a marshmallow field which enforces that this field: is one of a finite set of options. OptionList(options,*args,**kwargs) where options is a list of json compatible options which this option will be enforced to belong

Parameters:	options (list) – A list of python objects of which this field must be one of kwargs (dict) – the same as any `Field` receives

class argschema.fields.LogLevel(**kwargs)[source]¶

Bases: marshmallow.fields.String

LogLevel is a field type that provides a setting for the loglevel of python.logging. This class will both validate the input and also set the input globally. In simple scenarios, a module will not have to do any manipulation of loglevel.

options = ['FATAL', 'CRITICAL', 'ERROR', 'WARN', 'WARNING', 'INFO', 'DEBUG']¶

class argschema.fields.Slice(**kwargs)[source]¶

Bases: marshmallow.fields.String

Slice is a :class:’marshmallow.fields.Str’ field that supports a range or slice argument for selecting some subset of a larger dataset. The syntax is identical to numpy slicing. Examples: “10:20”, “40”, “:30”, “10:2:40”

Parameters:	kwargs – the same as any `Str` receive

Submodules¶

argschema.argschema_parser module¶

Module that contains the base class ArgSchemaParser which should be subclassed when using this library

class argschema.argschema_parser.ArgSchemaParser(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')[source]¶

Bases: object

The main class you should sub-class to write your own argschema module. Takes input_data, reference to a input_json and the command line inputs and parses out the parameters and validates them against the schema_type specified.

To subclass this and make a new schema be default, simply override the default_schema and default_output_schema attributes of this class.

Parameters:

input_data (dict or None) – dictionary parameters to fall back on if not source is given or configured via command line
schema_type (schemas.ArgSchema) – the schema to use to validate the parameters
output_schema_type (marshmallow.Schema) – the schema to use to validate the output, used by self.output
input_source (argschema.sources.source.Source) – a generic source of a dictionary
output_sink (argschema.sources.source.Source) – a generic sink to write output dictionary to
args (list or None) – command line arguments passed to the module, if None use argparse to parse the command line, set to [] if you want to bypass command line parsing
logger_name (str) – name of logger from the logging module you want to instantiate default (‘argschema’)

Raises:

marshmallow.ValidationError – If the combination of input_json, input_data and command line arguments do not pass the validation of the schema

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type ArgSchema

default_configurable_sinks = [<class 'argschema.sources.json_source.JsonSink'>]¶

default_configurable_sources = [<class 'argschema.sources.json_source.JsonSource'>]¶

default_output_schema = None¶

default_schema¶: alias of ArgSchema

get_output_json(d)[source]¶

method for getting the output_json pushed through validation if validation exists :param d: output dictionary to output :type d: dict

Returns:	validated and serialized version of the dictionary
Return type:	dict
Raises:	`marshmallow.ValidationError` – If any of the output dictionary doesn’t meet the output schema

static initialize_logger(name, log_level)[source]¶

initializes the logger to a level with a name logger = initialize_logger(name, log_level)

Parameters:	name (str) – name of the logger log_level –
Returns:	a logger set with the name and level specified
Return type:	logging.Logger

load_schema_with_defaults(schema, args)[source]¶

method for deserializing the arguments dictionary (args) given the schema (schema) making sure that the default values have been filled in.

Parameters:	args (dict) – a dictionary of input arguments schema –
Returns:	a deserialized dictionary of the parameters converted through marshmallow
Return type:	dict
Raises:	`marshmallow.ValidationError` – If this schema contains nested schemas that don’t subclass argschema.DefaultSchema because these won’t work with loading defaults.

output(d, output_path=None, sink=None, **sink_options)[source]¶

method for outputing dictionary to the output_json file path after validating it through the output_schema_type

Parameters:

d (dict) – output dictionary to output
sink (argschema.sources.source.ArgSink) – output_sink to output to (optional default to self.output_source)
output_path (str) – path to save to output file, optional (with default to self.mod[‘output_json’] location) (DEPRECATED path to save to output file, optional (with default to self.mod[‘output_json’] location)
**sink_options – will be passed through to sink.put_dict

Raises:

marshmallow.ValidationError – If any of the output dictionary doesn’t meet the output schema

class argschema.argschema_parser.ArgSchemaYamlParser(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')[source]¶

Bases: argschema.argschema_parser.ArgSchemaParser

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type ArgSchema

default_configurable_sinks = [<class 'argschema.sources.yaml_source.YamlSink'>]¶

default_configurable_sources = [<class 'argschema.sources.yaml_source.YamlSource'>]¶

argschema.argschema_parser.contains_non_default_schemas(schema, schema_list=[])[source]¶

returns True if this schema contains a schema which was not an instance of DefaultSchema

Parameters:	schema (marshmallow.Schema) – schema to check schema_list – (Default value = [])
Returns:	does this schema only contain schemas which are subclassed from schemas.DefaultSchema
Return type:	bool

argschema.argschema_parser.fill_defaults(schema, args)[source]¶

DEPRECATED, function to fill in default values from schema into args bug: goes into an infinite loop when there is a recursively defined schema

Parameters:	schema (marshmallow.Schema) – schema to get defaults from args –
Returns:	dictionary with missing default values filled in
Return type:	dict

argschema.argschema_parser.is_recursive_schema(schema, schema_list=[])[source]¶

returns true if this schema contains recursive elements

Parameters:	schema (marshmallow.Schema) – schema to check schema_list – (Default value = [])
Returns:	does this schema contain any recursively defined schemas
Return type:	bool

argschema.deprecated module¶

class argschema.deprecated.JsonModule(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')[source]¶

Bases: argschema.argschema_parser.ArgSchemaParser

deprecated name of ArgSchemaParser

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type ArgSchema

class argschema.deprecated.ModuleParameters(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

deprecated name of ArgSchema

ModuleParameters¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

argschema.schemas module¶

class argschema.schemas.ArgSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

The base marshmallow schema used by ArgSchemaParser to identify input_json and output_json files and the log_level

ArgSchema¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class argschema.schemas.DefaultSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: marshmallow.schema.Schema

mm.Schema class with support for making fields default to values defined by that field’s arguments.

make_object(in_data)[source]¶

marshmallow.pre_load decorated function for applying defaults on deserialation

Parameters:	in_data –
Returns:	a dictionary with default values applied
Return type:	dict

opts = <marshmallow.schema.SchemaOpts object>¶

argschema.utils module¶

module that contains argschema functions for converting marshmallow schemas to argparse and merging dictionaries from both systems

argschema.utils.args_to_dict(argsobj, schemas=None)[source]¶

function to convert namespace returned by argsparse into a nested dictionary

Parameters:	argsobj (argparse.Namespace) – Namespace object returned by standard argparse.parse function schemas (list[marshmallow.Schema]) – Optional list of schemas which will be used to cast fields via FIELD_TYPE_MAP
Returns:	dictionary of namespace values where nesting elements uses ‘.’ to denote nesting of keys
Return type:	dict

argschema.utils.build_schema_arguments(schema, arguments=None, path=None, description=None)[source]¶

given a jsonschema, create a dictionary of argparse arguments, by navigating down the Nested schema tree. (recursive function)

Parameters:	schema (marshmallow.Schema) – schema with field.description filled in with help values arguments (list or None) – list of argument group dictionaries to add to (see Returns) (Default value = None) path (list or None) – list of strings denoted where you are in the tree (Default value = None) description (str or None) – description for the argument group at this level of the tree
Returns:	List of argument group dictionaries, with keys [‘title’,’description’,’args’] which contain the arguments for argparse. ‘args’ is an OrderedDict of dictionaries with keys of the argument names with kwargs to build an argparse argument
Return type:	list

argschema.utils.cli_error_dict(arg_path, field_type, index=0)[source]¶

Constuct a nested dictionary containing a casting error message

Matches the format of errors generated by schema.dump.

Parameters:	arg_path (string) – List of nested keys field_type (string) – Name of the marshmallow.Field type index (int) – Index into arg_path for recursion
Returns:	Dictionary representing argument path, containing error.
Return type:	dict

argschema.utils.get_description_from_field(field)[source]¶

get the description for this marshmallow field

Parameters:	field (marshmallow.fields.field) – field to get description
Returns:	description string (or None)
Return type:	str

argschema.utils.get_field_def_from_schema(parts, schema)[source]¶

function to get a field_definition from a particular key, specified by it’s parts list

Parameters:	parts (list[str]) – the list of keys to get this schema schema (marshmallow.Schema) – the marshmallow schema to look up this key
Returns:	returns the field in the schema if it exists, otherwise returns None
Return type:	marshmallow.Field or None

argschema.utils.get_type_from_field(field)[source]¶

Get type casting for command line argument from marshmallow.Field

Parameters:	field (marshmallow.Field) – Field class from input schema
Returns:	Function to call to cast argument to
Return type:	callable

argschema.utils.merge_value(a, b, key, func=<built-in function add>)[source]¶

attempt to merge these dictionaries using function defined by func (default to add) raise an exception if this fails

Parameters:	a (dict) – one dictionary b (dict) – second dictionary key (key) – key to merge dictionary values on func (a[key]) – function that merges two values of this key Returns (Default value = add) func – merged version of values (Default value = add)

argschema.utils.prune_dict_with_none(d)[source]¶

function to remove all dictionaries from a nested dictionary when all the values of a particular dictionary are None

Parameters:	d (dictionary to prune) –
Returns:	pruned dictionary
Return type:	dict

argschema.utils.schema_argparser(schema, additional_schemas=None)[source]¶

given a jsonschema, build an argparse.ArgumentParser

Parameters:	schema (argschema.schemas.ArgSchema) – schema to build an argparser from additional_schemas (list[marshmallow.schema]) – list of additional schemas to add to the command line arguments
Returns:	that represents the schemas
Return type:	argparse.ArgumentParser

argschema.utils.smart_merge(a, b, path=None, merge_keys=None, overwrite_with_none=False)[source]¶

updates dictionary a with values in dictionary b being careful not to write things with None, and performing a merge on merge_keys

Parameters:	a (dict) – dictionary to perform update on b (dict) – dictionary to perform update with path (list) – list of nested keys traversed so far (used for recursion) (Default value = None) merge_keys (list) – list of keys to do merging on (default None) overwrite_with_none – (Default value = False)
Returns:	a dictionary that is a updated with b’s values
Return type:	dict

argschema.validate module¶

module for custom marshmallow validators

class argschema.validate.Shape(shape=None)[source]¶

Bases: marshmallow.validate.Validator

Validator which succeeds if value.shape matches shape

Parameters:	shape (tuple) – Tuple specifying the required shape. If a value in the tuple is None, any length in that dimension is valid.
Raises:	`ValueError` – If the provided shape is not a valid tuple of integers and/or None types `marshmallow.ValidationError` – If the value being validated does not have a shape attribute

argschema.autodoc module¶

argschema.autodoc.process_schemas(app, what, name, obj, options, lines)[source]¶

function designed to process a sphinx.ext.autodoc event as autodoc hook to alter docstring lines of argschema related classes, providing a table of parameters for schemas and links to the default schemas for ArgSchemaParser derived elements

use in sphnix conf.py as follows

from argschema.autodoc import process_schemas
def setup(app):
    app.connect('autodoc-process-docstring',process_schemas)

Module contents¶

argschema: flexible definition, validation and setting of parameters

argschema.main()[source]¶

TESTS¶

This contains the tests

test¶

test_first_test module¶

class test_first_test.BadExampleRecursiveSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

BadExampleRecursiveSchema¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
tree	no description	(REQUIRED)	`BadRecursiveSchema`	dict

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.BadRecursiveSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: marshmallow.schema.Schema

BadRecursiveSchema¶
key	description	default	field_type	json_type
children	children of this node	NA	`BadRecursiveSchema`	list
name	name of this node	anonymous	`String`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.ExampleRecursiveSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

ExampleRecursiveSchema¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
tree	no description	(REQUIRED)	`RecursiveSchema`	dict

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.ModelFit(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

ModelFit¶
key	description	default	field_type	json_type
fit_type	no description	NA	`String`	unicode
hof	no description	NA	`InputFile`	unicode
hof_fit	no description	NA	`InputFile`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.MyExtension(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

MyExtension¶
key	description	default	field_type	json_type
a	a string	(REQUIRED)	`String`	unicode
c	an integer	10	`Integer`	int
b	an integer	NA	`Integer`	int
d	a list of integers	NA	`List`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.MyExtensionOld(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: marshmallow.schema.Schema

MyExtensionOld¶
key	description	default	field_type	json_type
a	a string	NA	`String`	unicode
c	an integer	10	`Integer`	int
b	an integer	NA	`Integer`	int
d	a list of integers	NA	`List`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.MyShorterExtension(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

MyShorterExtension¶
key	description	default	field_type	json_type
a	a string	NA	`String`	unicode
c	an integer	10	`Integer`	int
b	an integer	NA	`Integer`	int
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
d	a list of integers	NA	`List`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.PopulationSelectionParameters(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

PopulationSelectionParameters¶
key	description	default	field_type	json_type
paths	no description	NA	`PopulationSelectionPaths`	dict
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.PopulationSelectionPaths(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

PopulationSelectionPaths¶
key	description	default	field_type	json_type
fits	no description	NA	`ModelFit`	list

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.RecursiveSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

RecursiveSchema¶
key	description	default	field_type	json_type
children	children of this node	NA	`RecursiveSchema`	list
name	name of this node	anonymous	`String`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.SimpleExtension(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

SimpleExtension¶
key	description	default	field_type	json_type
test	no description	(REQUIRED)	`MyExtension`	dict
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_first_test.SimpleExtensionOld(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

SimpleExtensionOld¶
key	description	default	field_type	json_type
test	no description	None	`MyExtensionOld`	dict
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

test_first_test.bad_test_recursive_schema()[source]¶

test_first_test.simple_extension_file(tmpdir_factory)[source]¶

test_first_test.test_bad_input_json_argparse()[source]¶

test_first_test.test_bad_path()[source]¶

test_first_test.test_david_example(tmpdir_factory)[source]¶

test_first_test.test_log_catch()[source]¶

test_first_test.test_recursive_schema()[source]¶

test_first_test.test_simple_description()[source]¶

test_first_test.test_simple_example()[source]¶

test_first_test.test_simple_extension_fail()[source]¶

test_first_test.test_simple_extension_old_pass()[source]¶

test_first_test.test_simple_extension_pass()[source]¶

test_first_test.test_simple_extension_required()[source]¶

test_first_test.test_simple_extension_write_debug_level(simple_extension_file)[source]¶

test_first_test.test_simple_extension_write_overwrite(simple_extension_file)[source]¶

test_first_test.test_simple_extension_write_overwrite_list(simple_extension_file)[source]¶

test_first_test.test_simple_extension_write_pass(simple_extension_file)[source]¶

test_output module¶

class test_output.MyOutputSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

MyOutputSchema¶
key	description	default	field_type	json_type
a	a simple string	(REQUIRED)	`String`	unicode
b	a default integer	5	`Integer`	int
M	a numpy array of answers	(REQUIRED)	`NumpyArray`	?

opts = <marshmallow.schema.SchemaOpts object>¶

test_output.test_alt_output(tmpdir)[source]¶

test_output.test_bad_output(tmpdir)[source]¶

test_output.test_output(tmpdir)[source]¶

test_output.test_output_unvalidated(tmpdir)[source]¶

test_output.test_tmp_output_cleanput(tmpdir)[source]¶

test_argschema_parser module¶

class test_argschema_parser.MyNestedSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

MyNestedSchema¶
key	description	default	field_type	json_type
two	a nested boolean	(REQUIRED)	`Boolean`	bool
one	nested integer	(REQUIRED)	`Integer`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_argschema_parser.MyNestedSchemaWithDefaults(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

MyNestedSchemaWithDefaults¶
key	description	default	field_type	json_type
two	a nested boolean	True	`Boolean`	bool
one	nested integer	1	`Integer`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_argschema_parser.MyParser(input_data=None, schema_type=None, output_schema_type=None, args=None, input_source=None, output_sink=None, logger_name='argschema.argschema_parser')[source]¶

Bases: argschema.argschema_parser.ArgSchemaParser

Note

This class takes a ArgSchema as an input to parse inputs , with a default schema of type MySchema

default_schema¶: alias of MySchema

class test_argschema_parser.MySchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

MySchema¶
key	description	default	field_type	json_type
a	parameter a	(REQUIRED)	`Integer`	int
nest	a nested schema	NA	`MyNestedSchema`	dict
b	optional b string parameter	my value	`String`	unicode
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class test_argschema_parser.MySchema2(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

MySchema2¶
key	description	default	field_type	json_type
a	parameter a	(REQUIRED)	`Integer`	int
nest	a nested schema	NA	`MyNestedSchemaWithDefaults`	dict
b	optional b string parameter	my value	`String`	unicode
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

test_argschema_parser.test_boolean_command_line(default, args, expected)[source]¶

test_argschema_parser.test_my_default_nested_parser()[source]¶

test_argschema_parser.test_my_parser()[source]¶

test_argschema_parser.test_parser_output(tmpdir_factory)[source]¶

test_utils module¶

class test_utils.BaseballSituation(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

A description of a baseball situation

BaseballSituation¶
key	description	default	field_type	json_type
inning	inning (1-9)	(REQUIRED)	`Integer`	int
outs	number of outs (0-2)	(REQUIRED)	`Integer`	int
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
bottom	is it the bottom of the inning	(REQUIRED)	`Boolean`	bool
score_away	away team score (non-negative)	(REQUIRED)	`Integer`	int
pitcher	who is pitching	(REQUIRED)	`Player`	dict
strikes	how many strikes (0-2)	(REQUIRED)	`Integer`	int
balls	number of balls (0-4)	0	`Integer`	int
batter	who is batting	(REQUIRED)	`Player`	dict
bases_occupied	which bases are occupied	NA	`List`	int
score_home	home team score (non-negative)	(REQUIRED)	`Integer`	int

opts = <marshmallow.schema.SchemaOpts object>¶

class test_utils.Player(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.DefaultSchema

player information

Player¶
key	description	default	field_type	json_type
number	player’s number (must be >0)	(REQUIRED)	`Integer`	int
name	players name	(REQUIRED)	`String`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

test_utils.test_merge_value_add()[source]¶

test_utils.test_merge_value_fail()[source]¶

test_utils.test_merge_value_subtract()[source]¶

test_utils.test_schema_argparser_with_baseball()[source]¶

test_utils.test_smart_merge()[source]¶

test_utils.test_smart_merge_add()[source]¶

test_utils.test_smart_merge_nested()[source]¶

test_utils.test_smart_merge_none()[source]¶

test_utils.test_smart_merge_not_none()[source]¶

test_utils.test_smart_merge_same()[source]¶

test_autodoc module¶

fields package¶

Submodules¶

fields.test_deprecated module¶

class fields.test_deprecated.OptionSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

OptionSchema¶
key	description	default	field_type	json_type
a	one of 1,2,3	(REQUIRED)	`OptionList`	?
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

fields.test_deprecated.test_bad_option()[source]¶

fields.test_deprecated.test_option_list()[source]¶

fields.test_files module¶

class fields.test_files.BasicInputDir(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

BasicInputDir¶
key	description	default	field_type	json_type
input_dir	a simple file	(REQUIRED)	`InputDir`	unicode
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class fields.test_files.BasicInputFile(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

BasicInputFile¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
input_file	a simple file	(REQUIRED)	`InputFile`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class fields.test_files.BasicOutputDir(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

BasicOutputDir¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
output_dir	basic output dir	(REQUIRED)	`OutputDir`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class fields.test_files.BasicOutputFile(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

BasicOutputFile¶
key	description	default	field_type	json_type
output_file	a simple output file	(REQUIRED)	`OutputFile`	unicode
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

class fields.test_files.ModeOutputDirSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

ModeOutputDirSchema¶
key	description	default	field_type	json_type
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode
output_dir	775 output directory	(REQUIRED)	`OutputDir`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

fields.test_files.test_access_inputfile_failed()[source]¶

fields.test_files.test_bad_inputdir()[source]¶

fields.test_files.test_basic_inputdir(tmpdir)[source]¶

fields.test_files.test_enoent_outputfile_failed()[source]¶

fields.test_files.test_failed_mode(tmpdir)[source]¶

fields.test_files.test_inputdir_no_access(tmpdir)[source]¶

fields.test_files.test_mode_output_osdir(tmpdir)[source]¶

fields.test_files.test_output_dir_bad_location()[source]¶

fields.test_files.test_output_dir_bad_permission()[source]¶

fields.test_files.test_output_dir_basic()[source]¶

fields.test_files.test_output_file_relative()[source]¶

fields.test_files.test_output_path(tmpdir)[source]¶

fields.test_files.test_output_path_cannot_write()[source]¶

fields.test_files.test_output_path_noapath()[source]¶

fields.test_files.test_outputfile_no_write(tmpdir)[source]¶

fields.test_files.test_outputfile_not_a_path()[source]¶

fields.test_files.test_relative_file_input()[source]¶

fields.test_files.test_relative_file_input_failed()[source]¶

fields.test_loglevel module¶

fields.test_loglevel.test_bad_option()[source]¶

fields.test_loglevel.test_option_list()[source]¶

fields.test_numpyarray module¶

class fields.test_numpyarray.NumpyFileuint16(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

NumpyFileuint16¶
key	description	default	field_type	json_type
a	list of lists representing a uint16 numpy array	(REQUIRED)	`NumpyArray`	?
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

fields.test_numpyarray.test_bad_data()[source]¶

fields.test_numpyarray.test_bad_shape()[source]¶

fields.test_numpyarray.test_numpy()[source]¶

fields.test_numpyarray.test_serialize()[source]¶

fields.test_slice module¶

class fields.test_slice.SliceSchema(extra=None, only=(), exclude=(), prefix=u'', strict=None, many=False, context=None, load_only=(), dump_only=(), partial=False)[source]¶

Bases: argschema.schemas.ArgSchema

SliceSchema¶
key	description	default	field_type	json_type
a	slice the dataset	slice(None, None, None)	`Slice`	unicode
log_level	set the logging level of the module	ERROR	`LogLevel`	unicode

opts = <marshmallow.schema.SchemaOpts object>¶

fields.test_slice.test_bad_slice()[source]¶

fields.test_slice.test_slice()[source]¶

Module contents¶

Indices and tables¶

Support/Contribute¶

We are planning on occasional updating this tool with no fixed schedule. Community involvement is encouraged through both issues and pull requests. Please make pull requests against the dev branch, as we will test changes there before merging into master.

Issue Tracker: https://github.com/AllenInstitute/argschema/issues
Source Code: https://github.com/AllenInstitute/argschema

License¶

The project is licensed under the BSD Clause 2 license with a non-commercial use clause.