fastiot_core_services.object_storage package

FastIoT Service to save “Object” into mongodb, see fastiot.db.mongodb_helper_fn.get_mongodb_client_from_env()

This Service is intended for saving “Object” inheriting the fastiot.core.data_models.FastIoTData, and requesting of historical object data. This Service is also designed to save one data type, for demanding saving multiple data types, you can instance multiple Services, using config file.

Your Object will be saved in Dictionary in such format, ‘yyy’ stands for the unpacked Attributes of Object:

{'_id': 'xxx', '_subject': 'xxx', '_timestamp': 'xxx', 'yyy': 'xxx', 'yyy': 'xxx}

In Order to use this Service, you must set a config file named ObjectStorageService.yaml or ObjectStorageService_1.yaml in my_deployment/config_dir, for reading ObjectStorageService_1.yaml FASTIOT_SERVICE_ID must be set.

The configuration is based on the configuration model fastiot_core_services.object_storage.config_model.ObjectStorageConfig. Please refer to the model for a description of the single fields.

The following example should provide you with a working configuration:

search_index:
  thing:
    - "_subject" , "_timestamp"
    - "_id"
subscriptions:
  'thing.*':
    collection: 'thing'
    reply_subject_name: 'objects'
    enable_overwriting: false
    identify_object_with:
      - "measurement_id"
      - "_timestamp"

Remarks: * search_index defines the MongoDB index to speed up the query.

_subject and _timestamp are defined as compound index in the above example.

subject_name is the subject, where you send your data.
enable_overwriting is a boolean flag for object overwriting
identify_object_with defines object fields, which define the whole object with its uniqueness The other fields will be overwritten respectively. Needed only if enable_overwriting set on true

Changed in version 0.2.101: Using subject_name to define the subject this service will respond is deprecated. Use the new field reply_subject_name instead. The old style will continue to work but throw a warning till it will be removed completely.

New in version 0.8.10:

Added mode for overwriting objects.
Added option for compound indices

Changed in version 0.9.33: Migrating to model based configuration with fastiot_core_services.object_storage.config_model.ObjectStorageConfig

You can assign the subject like following MyDataType or my_data_type, both will return you a right subject name format: v1.my_data_type.

CAUTION!: Thing will be instanced with different names. So by subscribing thing please always with a *. For understanding, you may reference to https://docs.nats.io/nats-concepts/subjects.

The other functionality for Object Storage Service is, you can make a request of the historical data stored in the MongoDB. To request the historical data, you need to instantiate fastiot.msg.hist.HistObjectReq, with subject_name ‘reply_object’. The code will look like:

hist_req_msg = HistObjectReq(dt_start=datetime(), dt_end=datetime(), limit=10, subject_name='logged_subject')
subject = hist_req_msg.get_reply_subject(name='my_set_reply_subject_name')

CAUTION! This subject_name should be the same as, which you have defined in ObjectStorageService.yaml. This request will reply to you a list of dictionaries. Then you can convert it to your own data type using fastiot.util.object_helper.parse_object_list().

Submodules

fastiot_core_services.object_storage.config_model module

pydantic model fastiot_core_services.object_storage.config_model.SubscriptionConfig[source]

Configuration for each subscription

field collection: str [Required]: MongoDB collection to store the data in

field reply_subject_name: str = '': The subject this services listens for Historic Object Requests (fastiot.msg.hist.HistObjectReq)

field enable_overwriting: bool = False: Overwrite data in the MongoDB if it matches with fastiot_core_services.object_storage.config_model.SubscriptionConfig.identify_object_with

field identify_object_with: List[str] = []: Set the fields identifying an object to overwrite. All values coming with the same fields set will be overwritten.

pydantic model fastiot_core_services.object_storage.config_model.ObjectStorageConfig[source]

Base configuration for an object storage service

field search_index: Dict[str, List[Union[str, List[str]]]] = {}: Define search indices to be added to the database here to speed up the query. Use a dictionary with the collection as key. You may provide a single entry (string type) to create a single index. To create a multi-index simply provide a list of strings with the corresponding fields in the MongoDB here.

field subscriptions: Dict[str, SubscriptionConfig] = {}: Add subscriptions here, the key is the subscription like thing.*.

fastiot_core_services.object_storage.env module

class fastiot_core_services.object_storage.env.ObjectStorageConstants[source]

property subject

fastiot_core_services.object_storage.mongodb_handler module

class fastiot_core_services.object_storage.mongodb_handler.MongoDBHandler[source]

__init__()[source]

health_check()[source]

Return type:: bool

get_database(name)[source]

drop_database(name)[source]

fsync()[source]

static create_index(collection, index, index_name)[source]

Creates the defined index in the defined collection if the index does not exist. Otherwise, no changes will be done to the database to save time-consuming rebuilding of the index

Parameters:

collection (Collection) – Collection (instance, not name) to create index in
index (List[Tuple[str, Union[int, Any]]]) – Define index as wanted by pymongo create_index, eg. [(column_name, pymongo.ASCENDING)] Instead of pymongo.ASCENDING you can also write 1, DESCENDING is -1
index_name (str) – Define a unique name for the index. This name will be used to check if index has already been created

Return type:

bool

Returns:

True if index has been created, False if index has been created already

close()[source]

fastiot_core_services.object_storage.object_storage_helper_fn module

fastiot_core_services.object_storage.object_storage_helper_fn.build_query_dict(hist_object_req)[source]

This function parses the HistObjectReq instance, and build the query dict to search data in database

Return type:: Dict

fastiot_core_services.object_storage.object_storage_service module

class fastiot_core_services.object_storage.object_storage_service.ObjectStorageService(**kwargs)[source]

__init__(**kwargs)[source]