fastiot.cli.model package

Data model for configurations of fastIoT based projects

Submodules

fastiot.cli.model.compose_info module

pydantic model fastiot.cli.model.compose_info.ServiceComposeInfo[source]
field name: str [Required]
field image: str [Required]
field environment: Dict[str, str] [Required]
field ports: List[str] [Required]
field volumes: List[str] [Required]
field devices: List[str] = []
field tmpfs: List[str] = []
field privileged: bool = False
field extras: str = ''
field labels: List = []
field extra_networks: List[str] = []

fastiot.cli.model.deployment module

Data models for deployment configurations

pydantic model fastiot.cli.model.deployment.InfrastructureServiceConfig[source]
field external: bool = False

Allows to mention services running on external servers and configured manually. This will avoid warnings in the setup process if services specified by the services as dependency could not be found.

Attention: You need to manage the environment variables like host and port yourself if using external = True.

pydantic model fastiot.cli.model.deployment.ServiceConfig[source]

The config for a service

field image: str [Required]

The name defines which service is taken. Must contain possible namespace identifiers as a prefix, e.g. fastiot/time_series

field docker_registry: str = ''

The specified docker registry. If given it will override the docker_registry for the service, otherwise the locally configured docker registry will be used.

field tag: str = ''

The specified tag. If given it will override the tag for the service

field environment: Dict[str, str] = {}

Includes all environment variables specifically for this service

field labels: list = []

Provide any docker labels like needed for Traefik

field extra_networks: list = []

Hook this service to an external docker network by e.g. another deployment on the same host.

pydantic model fastiot.cli.model.deployment.AnsibleHost[source]

Represents a host for Ansible based deployments, used by fastiot.cli.models.deployment.DeploymentTargetSetup

field ip: str [Required]

IP-Address (or DNS-resolvable hostname) where to find the host.

You may also add additional parameters to be placed in the inventory (hosts) file for ansible like

ansible_user=some_special_user directly following the string.

pydantic model fastiot.cli.model.deployment.DeploymentTargetSetup[source]

Configuration options to generate ansible playbooks on the fly to deploy your project

field hosts: Dict[str, AnsibleHost] [Required]

A list with ansible hosts to deploy the setup to

field remote_user: Optional[str] = 'ubuntu'

The remote user to use to logins for all hosts, defaults to ubuntu

Set to True to enable automatically link the Prometheus-Client configuration copied by Ansible to host to the current project. Only works if you do not have docker-compose.override.yaml in your deployment already.

pydantic model fastiot.cli.model.deployment.DeploymentLogging[source]

Adjust docker-compose logging configuration if needed. The defaults should fit well

field log_driver: str = 'local'

Set a logging driver. The docker default is json-file with recommandation for local (used as default here)

field max_size: Optional[str] = '10m'

Maximum size for a single logfile, defaults to ‘10m’ (10 MB)

field max_file: Optional[int] = 5

Maximum number of files to keep

field additional_options: Dict[str, str] = {}

Any additional options you may want to set for your selected driver

pydantic model fastiot.cli.model.deployment.DeploymentConfig[source]

Represents an imported config. All fields are already overwritten specified command line parameters, currently including environment, docker_registry and tag

field name: str [Required]
Validated by:
field version: int = 1
Validated by:
field services: Dict[str, Optional[ServiceConfig]] = {}

List of services for the deployment

Validated by:
field infrastructure_services: Dict[str, InfrastructureServiceConfig] = {}

List of infrastructure services for the deployment

Validated by:
field deployment_target: Optional[DeploymentTargetSetup] = None

A deployment configuration to auto-generate Ansible Playbooks

Validated by:
field docker_registry: str = ''

Specify a docker registry which acts as a default registry for all services (not infrastructure services). Overrides any docker registry specified by CLI.

Validated by:
field tag: str = ''

Specify a docker tag which acts as a default tag for all services (not infrastructure services). Overrides any docker tag specified by CLI.

Validated by:
field config_dir: str = './config_dir'

Specify a config dir. The config dir will get mounted to /etc/fastiot

It defaults to config_dir

Validated by:
field logging_config: DeploymentLogging = DeploymentLogging(log_driver='local', max_size='10m', max_file=5, additional_options={})
Validated by:
validator check_services  »  all fields[source]
static from_yaml_file(filename)[source]
Return type:

DeploymentConfig

to_yaml_file(filename)[source]

fastiot.cli.model.docker_template module

pydantic model fastiot.cli.model.docker_template.DockerTemplate[source]

Docker template base class. Inherit from this class to define dockerfile templates. Via python’s __subclasses__ mechanism they get imported and can be used in the service’s manifest.

field name: str [Required]

The template name used for referencing

field dir: str [Required]

The template dir where this template is located

field filename: str = 'Dockerfile.j2'

The filename of the template. Must reference a file inside template dir.

class property all: Dict[str, DockerTemplate]

Method to get a dict of all available services as instantiated fastiot.cli.model.infrastructure_service.InfrastructureService.

To append own services you simply have to inherit from this class and put them into your project. Then import those parts using fastiot.cli.model.project.ProjectContext.extensions. This method will try to import anything from there and for services.

classmethod get(name)[source]
Return type:

DockerTemplate

fastiot.cli.model.infrastructure_service module

Some helpers to work with external services: importing, port handling, …

pydantic model fastiot.cli.model.infrastructure_service.InfrastructureServiceEnvVar[source]
field name: str [Required]

The name of the infrastructure service env var

field default: str [Required]

The default value for the infrastructure service env var

field env_var: str = ''

The env var which can be used for modification. If empty it cannot be modified, therefore is static for the infrastructure service

pydantic model fastiot.cli.model.infrastructure_service.InfrastructureServicePort[source]
field container_port: int [Required]

The port inside the container

field default_port_mount: int [Required]

The default port for mounting

field env_var: str = ''

The env var which can be used for port mount modification. If given, this env var will also be provided to all services with the given container port so they can connect to the service

pydantic model fastiot.cli.model.infrastructure_service.InfrastructureServiceVolume[source]

An infrastructure service volume.

field container_volume: str [Required]

The volume inside the container. Per default it gets mounted to tmpfs if the deployment is a test integration deployment and otherwise to volume dir -> project namespace -> deployment name -> service name e.g. /var/fastiot/fastiot/core/mariadb

field env_var: str [Required]

The env var which can be used for volume mount modification. You may simply use your .env to set this. If the env var is set to ‘tmpfs’, it will mount the volume to a temporary file system inside the RAM. If the env var is set to ‘’, the volume isn’t mounted. If env var is set to a relative path (not starting with ‘/’) it is interpreted relative to volume dir. If env var starts with a / the absolute path will be used.

field default_volume_mount: Optional[str] = None

Set an default for the volume mount if env_var is not set. If it uses the default None it will use FASTIOT_VOLUME_DIR as root_volume and the following path root_volume/project_namespace/deployment/infrastructure_service_name Otherwise the options from env_var will apply.

field tmpfs_for_tests: bool = True

By default all volumes will be mounted to tmpfs for the integration test deployment. Set to False to skip this. Be aware that you have to take care of cleaning yourself!

pydantic model fastiot.cli.model.infrastructure_service.InfrastructureServiceComposeExtras[source]

Options to be added to the docker-compose file for the infrastructure service.

field option_name: str [Required]

The name of the option for container

field env_var: str = ''

The env var which can be used for option modification. Add this variable like FASTIOT_MONGODB_MEMLIMIT to your .env and the value will be used.

If the option only should have a fixed value, e.g. setting some extra permissions, you may as well leave this empty.

field default_value: str = ''

The default value to set if no env_var is set

pydantic model fastiot.cli.model.infrastructure_service.InfrastructureService[source]

Class to describe external services to be integrated in the deployments.

Please refer to Adding custom infrastructure services to your project for more information on adding your own infrastructure to your project.

Validators:
field name: str = ''

Name of the external service, e.g. mariadb. Per convention these names should be in lower case

Validated by:
field image: str = ''

Name of the image

field environment: List[InfrastructureServiceEnvVar] = []
field host_name_env_var: str = ''

The environment variable setting the hostname. This variable needs special attention: When working in a local development environment this defaults to localhost. Within a docker network the value needs to be the docker-internal hostname.

field password_env_vars: List[str] = []

List of environment variables containing a password. Those will be filled with random passwords for new projects automatically.

field ports: List[InfrastructureServicePort] = []

List of ports to assign to the service

field volumes: List[InfrastructureServiceVolume] = []

List of volumes to mount inside the service

field compose_extras: List[InfrastructureServiceComposeExtras] = []

Add additional infos to be added to the docker-compose file.

Caution: These parameters will not be checked! Make sure, that those do not make the docker-compose file invalid!

validator check_name  »  name[source]

Make sure there are now spaces and - in the name.

class property all: Dict[str, InfrastructureService]

Method to get a dict of all available services as instantiated fastiot.cli.model.infrastructure_service.InfrastructureService.

To append own services you simply have to inherit from this class and put them into your project. Then import those parts using fastiot.cli.model.project.ProjectContext.extensions. This method will try to import anything from there and for services.

get_default_env(name)[source]

Will return the default set for the given FastIoT environment variable.

Return type:

str

get_default_port(name='')[source]

Will return the default value for the given FastIoT port by environment variable. If you leave out the name the first port configured will be returned.

Return type:

int

fastiot.cli.model.manifest module

Data model for fastiot service manifests

pydantic model fastiot.cli.model.manifest.Port[source]

A port entry represents one port used by the service which should be mounted outside the container.

field port: int [Required]

The default port location.

field env_variable: str [Required]

The environment variable which is passed to the container to change the port, e.g. for automated testing.

pydantic model fastiot.cli.model.manifest.Volume[source]

A volume entry represents one directory used by the service which should be mounted from your host into the container.

field location: str [Required]

Provide the location inside the container. This value will also be used in your local environment if the defined environment variable is not set.

field env_variable: str [Required]

Define the variable name, that is used to overwrite the default location specified in location. If you put e.g. MYSERVICE_DATA_VOLUME you may add something like to following to your :file:.env` file: MYSERVICE_DATA_VOLUME=./data or MYSERVICE_DATA_VOLUME=/opt/myproject/data. The first one will be used relative to your deployments dir and the docker-compose file in there. The second one will be used as absolute path in your filesystem. If undefined the path specified in location will also be used on the host. If the environment variable is set to an empty string ("") the volume will not be mounted and no persistence will happen for this volume.

To find the volume inside your service please read the specified environment variable. Inside the container this will be the same as defined in location. For debugging purposes the env in the build-directory will be adjusted to point to your deployments directory.

pydantic model fastiot.cli.model.manifest.Device[source]

A device entry represents one device used by the service which should be mounted outside the container.

field location: str [Required]

The default device location, e.g. /dev/ttyS0 for a serial port

field env_variable: str [Required]

See attribute env_variable in fastiot.cli.model.manifest.Port.

class fastiot.cli.model.manifest.MountConfigDirEnum(value)[source]

Set if the configuration dir needs to be mounted in the container

required = 'required'
optional = 'optional'
no = 'no'
class fastiot.cli.model.manifest.CPUPlatform(value)[source]

Definition of the CPU platform the container will be built for

amd64 = 'amd64'
amd64_2 = 'amd64_2'
arm64 = 'arm64'
armv7 = 'armv7'
as_docker_platform()[source]

Returns a member (accessed by self in this case!) as docker-style platform. This usually means e.g. linux/amd64

as_qemu_platform()[source]

Returns the platform according the qemu emulator

pydantic model fastiot.cli.model.manifest.Healthcheck[source]

Configuration options for Docker Healtcheck. This allows for automatic restarting the service if the service does not provide the configured life sign.

field cmd: str = ''

Command to run. This could e.g. be a curl request to your API or check wether a logfile gets updated every n seconds.

field interval: str = '30s'

Interval to check the service

field timeout: str = '30s'

Timeout for the command

field start_period: str = '1s'
field retries: int = 3
pydantic model fastiot.cli.model.manifest.NPM[source]

Use this part if your project contains a frontend created with npm, e.g. a vue.js application

field src: str [Required]
field dist: str = 'dist'

Destination where npm will place its files for distribution. If not changed npm will have save its files in the <src>/dist which is also the default here. If you have something like:

service.exports = {
  outputDir:"../flask_server/static",
  assetsDir: "static"
}

in your npm config or in case of vue js application in the file vue.config.js use the outputDir variable as relative path here, e.g. dist: ../static

field node_base_image: str = 'node:16-bookworm-slim'

If you need a different image for building your vue.js app (or whatever) you may define it here. The default is currently set to node:16-buster-slim.

pydantic model fastiot.cli.model.manifest.ServiceManifest[source]

Every service needs a manifest.yaml to describe the service.

The following options may be used. The file always starts with a fastiot_service: in the first level. Then options from the following may (name must) follow.

fastiot.cli.model.manifest.ServiceManifest.name is needed, others are mostly optional!

field name: str [Required]

Name needs to comply with the services name (Mandatory)

field ports: List[Port] = []

Provide a list with some name for the service and a port that this container will open, e.g. when operating as a webserver.`

field template: str = 'python3'

Specify the template to use. You can provide your custom Templates by overwriting fastiot.cli.model.docker_template.DockerTemplate. Make sure that you imported it e.g. via configure.py’s extensions property.

field base_image: str = 'python:3.10-slim-bookworm'

Use this to provide an alternative base image, otherwise fastiot.cli.constants.DEFAULT_BASE_IMAGE will be used.

Be aware, that the further Dockerfile will be unchanged, thus your base image should be based on some Debian-style. If this does not work for you, you may also provide a Dockerfile in your service which will automatically be used.

field volumes: List[Volume] = []

Volumes to be mounted in the container

field devices: List[Device] = []

Devices, e.g. serial devices, to be mounted in the container

field depends_on: List[str] = []

List of infrastructure services that need to be deployed

field mount_config_dir: MountConfigDirEnum = MountConfigDirEnum.optional

Specify, if a config dir must be mounted.

field privileged: bool = False

Enable if this service needs privileged permissions inside docker, e.g. for hardware access

field platforms: List[CPUPlatform] = [<CPUPlatform.amd64: 'amd64'>]

Define the cpu platforms to build the container for. It defaults to amd64. If doing local builds the first one specified (or amd64 if none) will be used to build the image.

field healthcheck: Healthcheck = Healthcheck(cmd='', interval='30s', timeout='30s', start_period='1s', retries=3)

Configure healthcheck for the container

field copy_dirs_to_container: List[str] = []

Directories which shall be copied to container. They must be specified relative to manifest.yaml.

field npm: Optional[NPM] = None

If your project contains a vue.js application you can automatically build it here. For required configuration see fastiot.cli.model.manifest.Vue

field additional_requirements: List[str] = []

If a specific service needs more packages installed than the others, you may add these here. It will look in the requirements directory for the specified additional requirements and copy them inside the container. The install.sh must be handle to install these.

Please have a look at Managing dependencies for more information about dependency management in FastIoT.

field compile: bool = True

If your service should not be compiled can change to False. Per default your service will be compiled using Nuitka to have some obfuscation in the code and potentially speed up the program.

field compose_extras: Optional[Dict] = {}

ATTENTION: Use at your own risk! Adding options not known or duplicating entries in docker-compose.yaml will make the file invalid and the services will deny to start.

If you want to set some more options for your container, like mem_limit, you may list them here. You need to know which option accepts docker-compose. Please refer to https://docs.docker.com/compose/compose-file/compose-file-v2/#service-configuration-reference .

For configuring this you can refer to src/fastiot_sample_services/producer/manifest.yaml. It shows, how you can add more compose extras in manifest.yaml.

static from_yaml_file(filename, check_service_name='')[source]

Does the magic of import yaml to pydantic model

Return type:

ServiceManifest

classmethod from_docker_image(docker_image_name, pull_always=False)[source]
Return type:

ServiceManifest

fastiot.cli.model.project module

data model for project configuration

class fastiot.cli.model.project.CompileSettingsEnum(value)[source]

Different settings to define how to compile the library.

only_compiled = 'only_compiled'
only_source = 'only_source'
all_variants = 'all_variants'
none = 'none'
disabled = 'disabled'
pydantic model fastiot.cli.model.project.ProjectContext[source]

This class holds all variables reade from configure.py in the project root directory. Use this as for hints to do custom adjustments to your configure.py. We try to set sensible defaults so that changes should only be needed in rare cases.

Validators:
field project_namespace: str = ''

Namespace of the project used e.g. on the docker registry. This should be your project name.

field project_root_dir: str = '.'

Project root directory. As the cli is intended to be run from the project root directory the default using the current working directory should be fine.

Validated by:
field library_package: str = ''

Define a python package within your src directory containing library methods to be shared between your services. If not specified no library will be built.

field library_setup_py_dir: str = '.'

Path where to find the setup.py to build your library for exporting. The default with the current working directory should be fine, if you put your setup.py at the toplevel of your project (common).

Validated by:
field services: List[Service] = []

Define a list of fastiot.cli.model.service.ServiceConfig. If not defined the cli will look for python packages containing a manifest.yaml and a run.py with the pattern src/*/*/manifest.yaml and call this package a FastIoT Service. Be aware of this when creating a manifest.yaml somewhere, e.g. in your tests.

Hint: If you want to override this configuration you may use fastiot.cli.helper_fn.find_services() to create a list of services to build by package.

field deployments: Dict[str, DeploymentConfig] = {}

Manually define a list of deployments as fastiot.cli.model.DeploymentConfig to actually build using the command fastiot.cli.commands.config.config(). If left empty all deployment configurations in the path deployments will be used.

Hint: You may use fastiot.cli.helper_fn.find_deployments() to create a list of deployments according to your special needs.

field integration_test_deployment: str = ''

If you need any services to be started for automatic testing your project you may define the name of this special deployment found within the fastiot.cli.model.project.ProjectContext.deployments.

field test_package: str = ''

Name of the package in the src directory where automated tests are stored. Common is to use something like myproject_tests.

field npm_test_dir: str = ''
field build_dir: str = 'build'

The dir where generated build files (Dockerfiles, Docker-compose files, etc.) are stored. This dir is relative to your project_root_dir.

field extensions: List[str] = []

Use to add own extensions to the FastIoT CLI. The CLI will try to import your custom infrastructure services as described in Adding custom infrastructure services to your project.

Make sure importing this service will import further commands and fastiot.cli.model.infrastructure_service.InfrastructureService. Most of the times this is done filling the __init__.py correspondingly.

field lib_compilation_mode: CompileSettingsEnum = CompileSettingsEnum.disabled

Adjust, if you want your library to be build as wheel or compiled (and obfuscated). Use options from fastiot.cli.model.project.CompileSettingsEnum

class property default: ProjectContext[source]

Use this method to retrieve the singleton fastiot.cli.model.project.ProjectContext

validator default_to_abspath  »  project_root_dir, library_setup_py_dir[source]
property deployment_names: List[str]

Returns a list of all deployment names configured by configuration (fastiot.cli.model.project.ProjectContext.deployments) or by convention in deployments dir.

deployment_by_name(name)[source]

Returns a specific deployment by its name.

Return type:

DeploymentConfig

deployment_dir(name)[source]

Returns the deployment build dir for specific deployment

Return type:

str

deployment_build_dir(name)[source]

Returns the deployment build dir for specific deployment

Return type:

str

env_file_for_deployment(name)[source]
Return type:

str

env_for_deployment(name)[source]
Return type:

Dict[str, str]

build_env_file_for_deployment(name)[source]
Return type:

str

build_env_for_deployment(name)[source]
Return type:

Dict[str, str]

build_env_for_internal_services_deployment(name)[source]
Return type:

Dict[str, str]

get_service_by_name(name)[source]

Get a configured service from the project by name

Return type:

Service

get_all_service_names()[source]

Returns a list of all configured services

Return type:

List[str]

fastiot.cli.model.project.parse_env_file(env_filename)[source]
Return type:

Dict[str, str]

fastiot.cli.model.service module

data models for FastIoT and infrastructure services

pydantic model fastiot.cli.model.service.Service[source]

The model class for a service

field name: str [Required]
field build_date: str = '2023-06-30T12:21:43.302776+00:00'
field version: str = '1.0.1'
field package: str [Required]
field cache: str = ''

The name to use as the cache on the set docker cache registry. The tag will be appended automatically (in case not empty), please do not specify it. Example: mypackage-cache

field extra_caches: List[str] = []

A list of extra caches used to speed up building which will all be read-only. It is intended if you want to read from other caches or different tags. Each extra cache must match a cache name extended by ‘:’ followed by the tag for the cache. It is useful to always include the cache name followed by tag latest to always use latest cache for feature branches. Examples: mypackage-cache:latest, mypackage-cache:my-feature

field maintainer: str = ''

Used for docker labels

field git_revision: str = ''

Used for docker labels

read_manifest(check_service_name='')[source]

Reads out the service manifest, if not run before using the yaml-file otherwise just self.manifest

Return type:

ServiceManifest