hatto

The hatto is CLI for SBOM policy evaluation.

$ hatto help
heriet <heriet@heriet.info>
CLI for software license policy check.

USAGE:
    hatto <SUBCOMMAND>

OPTIONS:
    -h, --help       Print help information
    -V, --version    Print version information

SUBCOMMANDS:
    evaluate    evaluate policy
    help        Print this message or the help of the given subcommand(s)

Mean

The hatto is derived from Japanese word 法度(HATTO). HATTO means rule of prohibition.

evaluate

hatto evaluate is evaluate license policy.

$ hatto evaluate --help
hatto-evaluate
evaluate policy

USAGE:
    hatto evaluate [OPTIONS] <FILE>

ARGS:
    <FILE>

OPTIONS:
    -c, --curation <FILE>
    -h, --help                         Print help information
    -o, --output <OUTPUT_FORMAT>       [default: human] [possible values: human, json]
    -p, --policy <FILE>
    -t, --source-type <SOURCE_TYPE>    [possible values: tsv, spdx-tag, spdx-json, spdx-yaml,
                                       cyclone-dx-json, cyclone-dx-xml]

The evaluate ARGS file is SBOM or tsv. SBOM supports SPDX or CycloneDX.

Yet another hatto supports tsv. This tsv file must contain header.

example example.tsv

name	version	licenses	annotations
foo	1.0.1	MIT,Apache-2.0	usage=service
bar	1.1.2	UNKNOWN

These files can generate with any license collection tool. If the license collection tool does not support SBOM, you shoud convert to tsv or SBOM.

And you can configure --policy and --curation.

The --policy file defines license policy that written in python. The policy file must implements def evaluate(material, result).

example polocy.py

#!/usr/bin/python

allowed_licenses = [
    "Apache-2.0",
    "BSD-3-Clause",
    "MIT",
    "Unlicense",
]

def evaluate(material, result):
    for license in material.licenses:
        if license not in allowed_licenses:
           result.add_error(f"{license} is not allowed")

$ hatto evaluate --policy policy.py example.tsv
OK foo 1.0.1 licenses:["MIT", "Apache-2.0"] annotations:{"usage": "service"}
NG bar 1.1.2 licenses:["UNKNOWN"] annotations:{}
  ERROR UNKNOWN is not allowed
Failure: evaluate failed

UNKNOWN is not allowed on policy.py. Therefore hatto evaluate is failed.

You may know bar true license is BSD-3-Clause. In such a case you can patch license information by --curation file. The curation file must implement def curate_material(material).

example curation.py

#!/usr/bin/python

def curate_material(material):
    if material.name == "bar":
      material.licenses = ["BSD-3-Clause"]

$ hatto evaluate --policy policy.py --curation curation.py example.tsv
OK foo 1.0.1 licenses:["MIT", "Apache-2.0"] annotations:{"usage": "service"}
OK bar 1.1.2 licenses:["BSD-3-Clause"] annotations:{}

These allow hatto to perform flexible license policy evaluation on your teams or organizations.

policy

The policy is intended to be written by someone on your team or organization who is considering available licenses.

The policy file must implements def evaluate(material, result) written by python.

evaluate

The def evaluate(material, result) is evaluate policy. If the evaluation fails, you call result.add_error(message) in this function.

example polocy.py

#!/usr/bin/python

allowed_licenses = [
    "Apache-2.0",
    "BSD-3-Clause",
    "MIT",
    "Unlicense",
]

def evaluate(material, result):
    for license in material.licenses:
        if license not in allowed_licenses:
           result.add_error(f"{license} is not allowed")

This policy allows Apache-2.0, BSD-3-Clause, MIT and Unlicense.

The evaluate argment material is curated Material by curation, and result is EvaluateResult.

evaluate using annotations

The Material set some kind of annotation, you can evaluate the policy more flexibly. For example, there may be cases where you want to ignore a license that is normally deny due to some special circumstances.

def evaluate(material, result):
    if "ignore" in material.annotations:
        return

    # your policy check

In the above, when ignore is set on annotation, the evaluation is always ignored.

The rules for granting annotations are free, so you will need to decide the person who created the policy. For example, the above rule of ignore by ignore annotation should be in your policy document.

As another example, let's take a policy evaluate of the AGPL 3.0 license.

Suppose your team or organization may use AGPL 3.0 software in a service and you want to check whether the composed software is source distributed according to the AGPL 3.0 license.

For example, you can write the following.

def evaluate(material, result):
    for license in material.licenses:
        if license.startswith("AGPL"):
            usage = material.annotations.get("usage", "unknown")

            if usage == "unknown":
                result.add_error(f"you must set usage for {license} software")
            elif usage == "service" and "project-source-distributed" not in material.annotations:
                result.add_error(f"you must project-source-distribute on {license} software")

WARNING: that the above policy does not check for compliance with all AGPL 3.0 license terms by legal perspective. For example, AGPL 3.0 also requires source code distribution when distributing a composed software even if it is not for use in a service.

The method of compliance with the terms of a particular license must be determined by your team or organization.

Even in the above example, your team or organization should decide under what conditions usage annotation and project-source-distribute will be granted and with what values.

EvaluateResult

EvaluateResult is result of evaluate. If EvaluateResult contains any errors, hatto evaluate is failed. Conversely, EvaluateResult not contains errors, hatto evaluate is success.

It can also contain warnings. If the EvaluateResult contains any warnings, hatto evaluate is not failed.

Methods

EvaluateResult.add_errors(message)

The add_errors is add error to result.

def evaluate(material, result):
    result.add_error("Any licences is not allowed")

$ hatto evaluate --policy=policy.py example.tsv
NG foo 1.0.0 licenses:["MIT"] annotations:{}
  ERROR Any licences is not allowed
Failure: evaluate failed

EvaluateResult.add_warnings(message)

The add_warnings is add warning to result.

def evaluate(material, result):
    result.add_warning("Any licences is warning")

$ hatto evaluate --policy=policy.py example.tsv
OK foo 1.0.0 licenses:["MIT"] annotations:{}
    WARNING Any licences is warning

default policy

If the --policy option is not set, the following default policy is executed.

The default policy allows at least what is commonly used in some Permissive License. This policy may change in future versions. So, we recommend that it be not use as possible.

The policy for your team or organization should be written by your team or organization.

#!/usr/bin/python

allowed_licenses = [
    "Apache-2.0",
    "MIT",
    "BSD-3-Clause",
    "Unlicense",
]

def evaluate(material, result):
    for license in material.licenses:
        if license not in allowed_licenses:
           result.add_error(f"{license} is not allowed")

tests

The policy is python code. So, it can be test in common way of python tests.

example test_polocy.py

from policy import evaluate

import pytest


class Material:
    def __init__(self, name="", version="", licenses=[], annotations={}):
        self.name = name
        self.version = version
        self.licenses = licenses
        self.annotations = annotations

    def update_annotation(self, key, value):
        self.annotations[key] = value


class EvaluateResult:
    def __init__(self):
        self.errors = []
        self.warnings = []

    def add_error(self, message):
        self.errors.append(message)

    def add_warning(self, message):
        self.warnings.append(message)


@pytest.mark.parameterize("license_name", ["Apache-2.0", "BSD-3-Clause", "MIT", "Unlicense"])
def test_evaluate_allow(license_name):
    result = EvaluateResult()
    material = Material("foo", "v0.1.0", [license_name])
    evaluate()

    assert len(result.errors) == 0

@pytest.mark.parameterize("license_name", ["UNKNOWN"])
def test_evaluate_deny(license_name):
    result = EvaluateResult()
    material = Material("foo", "v0.1.0", [license_name])
    evaluate()

    assert len(result.errors) == 1

curation

The role of curation is to correct license information. License information collected by license information collection tools is rarely inaccurate and may need to be corrected manually.

The curation is intended to be written by the project owner. The project owner and the person that determining the organization's licensing policy may be different.

The curation file must implements def curate_material(material) written by python.

curate_material

The def curate_material(material) is curate material. In this context, curete means to modify the material and license information.

For example, suppose the license of foo package detected MIT by license collection tool. However, there was an error in the collection process and the true license is Apache 2.0.

You can correct this error using curate_material.

#!/usr/bin/python

def curate_material(material):
    if material.name == "foo":
      material.licenses = ["Apache-2.0"]

In another case, suppose you had many package that dual licensed MIT and Apache 2.0 in your project.

For reasons of your organizational policy, you may want to specify which license to use.

#!/usr/bin/python

def curate_material(material):
    if set(material.licenses) == set(["MIT", "Apache-2.0"]):
      material.licenses = ["MIT"]

Material

Instance Variables

nametypeexplain
namestringname of material
versionstringversion of material
licenseslistlist of license name(string). In most cases, license name is expected to specify SPDX license identifier
annotationsdictdict of annotation key(string) to value(string)

Methods

Material.update_annotation(key, value)

The update_annotation is update annotation by key-value pair.

WARNING: Currently, it is possible to update annotations dict, but cannot update by annotations dict key. Therefore, this method is temporary provided. This method may be removed in the future.

#!/usr/bin/python

def curate_material(material):
    material.annotations["foo"] = "bar" # not updated
    material.annotations |= {"hoge": "fuga"} # python 3.9 later
    material.update_annotation("x", "y")

    print(material.annotations) # {"hoge": "fuga", "x": "y"}