tracdap.rt.api¶

TRAC model API for Python

Classes¶

`ArrayValue`	An array value holds an array of other Values.
`BasicType`	Basic types provide the set of core types available in the TRAC type system.
`CommonFileTypes`
`CopyStatus`	Status of an individual copy of a data storage item
`CustomDefinition`	Define a custom object that can be stored and managed in the TRAC metadata store
`DataDefinition`	Define a dataset that can be stored and managed in the TRAC platform
`DateValue`	Represent a date value.
`DatetimeValue`	Represent a date-time value.
`DecimalValue`	Represent a decimal value.
`ExportDataJob`	Specification for an EXPORT_DATA job
`FieldSchema`	Schema for an individual field in a tabular dataset
`FileDefinition`	Describes a file object stored in the TRAC platform
`FileType`
`FlowDefinition`	A flow defines an execution graph as a set of connections between models and data
`FlowEdge`	A connection between two nodes in a flow
`FlowNode`	Describes an individual node in a flow
`FlowNodeType`	Specify the type of an individual flow node
`FlowSocket`	A socket is a point of connection for wiring up the edges in a flow
`ImportDataJob`	Specification for an IMPORT_DATA job
`ImportModelJob`	Specification for an IMPORT_MODEL job
`IncarnationStatus`	Status of an individual incarnation of a data storage item
`JobDefinition`	Define a job to run on the TRAC platform
`JobGroup`	Specification for a JOB_GROUP job, which runs a collection of other jobs
`JobGroupType`	Specify the group type for a JOB_GROUP job
`JobStatusCode`	Indicate the status of a job in the TRAC platform
`JobType`	Specify the type of an individual TRAC job
`LogicalExpression`	Logical expression for a search of the TRAC metadata store.
`LogicalOperator`	Metadata search logical operator, used as part of a LogicalExpression.
`MapValue`	A map value holds a map of string keys to other Values.
`MetadataFormat`	Available formats for representing the TRAC metadata.
`MetadataVersion`	Explicit versioning of the metadata schema.
`ModelDefinition`	Define a model for execution on the TRAC platform
`ModelInputSchema`(schema[, label, optional, dynamic, ...])
`ModelOutputSchema`(schema[, label, optional, dynamic, ...])
`ModelParameter`	Describes an individual parameter of a model
`ModelType`	Identify specialized model types for specific tasks
`ObjectDefinition`	Object definitions are the core structural element of TRAC's metadata model
`ObjectType`	Enumeration of TRAC's core object types.
`ParallelJobGroup`	A job group where all jobs runs in parallel
`PartKey`	Partition key for tabular datasets
`PartType`	Partitioning scheme applied to a dataset
`ResourceType`	Enumeration of infrastructure resources that can be added to a tenant
`ResultDefinition`	Define the result of a job after it has completed
`RunFlowJob`	Specification for a RUN_FLOW job
`RunModelJob`	Specification for a RuN_MODEL job
`SchemaDefinition`	A schema definition describes the schema of a dataset
`SchemaType`	Enumeration of the available types of data schema
`SearchExpression`	Search expression for a search of the TRAC metadata store.
`SearchOperator`	Metadata search term operator, used as part of a SearchTerm
`SearchParameters`	Parameters to define a metadata search.
`SearchTerm`	Individual search term for a search of the TRAC metadata store.
`SequentialJobGroup`	A job group where each job runs in sequence
`StorageCopy`	Define physical storage for an individual copy of a data item
`StorageDefinition`	Defines the physical storage for a file or dataset object
`StorageIncarnation`	Define physical storage for an individual incarnation of a data item
`StorageItem`	Define physical storage for an individual data item
`TableSchema`	Schema for a tabular dataset
`Tag`	Tags are the core informational element of TRAC's metadata model.
`TagHeader`	A tag header describes the identity and version of an object.
`TagOperation`	Enumeration of available TagUpdate operations.
`TagSelector`	A tag selector describes the selection of a unique object at a point in time.
`TagUpdate`	A tag update is a request for a single update operation on a tag.
`TenantInfo`	Information about a tenant that is set up on the TRAC platform.
`TracContext`	Interface that allows model components to interact with the platform at runtime
`TracModel`	Base class that model components inherit from to be recognised by the platform
`TypeDescriptor`	A type descriptor describes a data type used in the TRAC platform.
`Value`	A value expressed in the TRAC type system.

Functions¶

`A`(attr_name, attr_value[, attr_type, categorical])	Shorthand alias for `define_attribute()`
`F`(field_name, field_type, label[, business_key, ...])	Shorthand alias for `define_field()`
`P`(param_name, param_type, label[, default_value, ...])	Shorthand alias for `define_parameter()`
`define_attribute`(attr_name, attr_value[, attr_type, ...])	Define an individual model attribute
`define_attributes`(*attributes)	Define a set of attributes to catalogue and describe a model
`define_field`(field_name, field_type, label[, ...])	Define an individual field for use in a schema
`define_file_type`(extension, mime_type)
`define_input`(requirement, *[, label, optional, ...])
`define_input_file`(extension, mime_type, *[, label, ...])
`define_input_table`(*fields[, label, optional, ...])	Define a model input with a table schema
`define_output`(requirement, *[, label, optional, ...])
`define_output_file`(extension, mime_type, *[, label, ...])
`define_output_table`(*fields[, label, optional, ...])	Define a model output with a table schema
`define_parameter`(param_name, param_type, label[, ...])	Define an individual model parameter
`define_parameters`(*parameters)	Defined the set of parameters used by a model
`define_schema`(*fields[, schema_type, dynamic])	Create a `SchemaDefinition` from a list of fields
`load_schema`(package, schema_file[, schema_type])	Load a `SchemaDefinition` from a CSV file in the model package

Attributes¶

`BOOLEAN`	Synonym for `BasicType.BOOLEAN`
`DATE`	Synonym for `BasicType.DATE`
`DATETIME`	Synonym for `BasicType.DATETIME`
`DECIMAL`	Synonym for `BasicType.DECIMAL`
`FLOAT`	Synonym for `BasicType.FLOAT`
`INTEGER`	Synonym for `BasicType.INTEGER`
`STRING`	Synonym for `BasicType.STRING`

Package Contents¶

tracdap.rt.api.A(attr_name, attr_value, attr_type=None, categorical=False)¶

Shorthand alias for define_attribute()

Note

This is an experimental API that is not yet stabilised, expect changes in future versions of TRAC

Return type:

TagUpdate

Parameters:

attr_name (str)
attr_value (Any)
attr_type (BasicType | None)
categorical (bool)

tracdap.rt.api.F(field_name, field_type, label, business_key=False, categorical=False, not_null=None, format_code=None, field_order=None)¶

Shorthand alias for define_field()

Return type:

FieldSchema

Parameters:

field_name (str)
field_type (BasicType)
label (str)
business_key (bool)
categorical (bool)
not_null (bool | None)
format_code (str | None)
field_order (int | None)

tracdap.rt.api.P(param_name, param_type, label, default_value=None, *, param_props=None)¶

Shorthand alias for define_parameter()

Return type:

_Named[ModelParameter]

Parameters:

param_name (str)
param_type (BasicType | TypeDescriptor)
label (str)
default_value (Any | None)
param_props (Dict[str, Any] | None)

tracdap.rt.api.define_attribute(attr_name, attr_value, attr_type=None, categorical=False)¶

Define an individual model attribute

Note

This is an experimental API that is not yet stabilised, expect changes in future versions of TRAC

Model attributes can be defined using this function or the shorthand alias A(). A name and value are always required to define an attribute. Attribute type is required for multivalued attributes but is optional otherwise. The categorical flag can be applied to STRING attributes to mark them as categorical.

Model attributes can be passed to define_attributes(), either as individual arguments or as a list, to create the set of attributes for a model.

Parameters:

attr_name (str) – The attribute name
attr_value (Any) – The attribute value (as a raw Python value)
attr_type (BasicType | None) – The TRAC type for this attribute (optional, except for multivalued attributes)
categorical (bool) – A flag to indicate whether this attribute is categorical

Returns:

A model attribute, in the format understood by the TRAC platform

Return type:

TagUpdate

tracdap.rt.api.define_attributes(*attributes)¶

Define a set of attributes to catalogue and describe a model

Note

This is an experimental API that is not yet stabilised, expect changes in future versions of TRAC

Model attributes can be defined using define_attribute() or the shorthand alias A(). This function takes a number of model attributes, either as individual arguments or as a list, and arranges them in the format required by TracModel.define_attributes().

Parameters:: attributes (TagUpdate | List[TagUpdate]) – The attributes that will be defined, either as individual arguments or as a list
Returns:: A set of model attributes, in the correct format to return from TracModel.define_attributes()
Return type:: List[TagUpdate]

tracdap.rt.api.define_field(field_name, field_type, label, business_key=False, categorical=False, not_null=None, format_code=None, field_order=None)¶

Define an individual field for use in a schema

Individual fields in a schema can be defined using this method or the shorthand alias F(). The name, type and label of a field are always required. The business_key and categorical flags are false by default. The not_null flag is false by default unless the field is a business key, in which case it is true by default. Explicitly specifying not_null=False for a business key will cause a validation error. Format code is optional.

So long as field order is not specified for any field in a schema, field ordering will be assigned automatically. If field orders are specified explicitly, the fields in a schema must have a contiguous ordering starting at 0. When updating a model it is good practice to leave existing fields in order and add any new fields to the end of the list.

Schema fields can be passed to define_schema(), either as individual arguments or as a list, to create a SchemaDefinition. To define the inputs or outputs of a TracModel, fields can also be passed directly to define_input_table() or define_output_table().

Parameters:

field_name (str) – The field’s name, used as the field identifier in code and queries (must be a valid identifier)
field_type (BasicType) – The data type of the field, only primitive types are allowed
label (str) – A descriptive label for the field (required)
business_key (bool) – Flag indicating whether this field is a business key for its dataset (default: False)
categorical (bool) – Flag indicating whether this is a categorical field (default: False)
not_null (bool | None) – Whether this field has a not null constraint (default: True for business keys, false otherwise)
format_code (str | None) – A code that can be interpreted by client applications to format the field (optional)
field_order (int | None) – Explicit field ordering (optional)

Returns:

A field schema, suitable for use in a schema definition

Return type:

FieldSchema

tracdap.rt.api.define_file_type(extension, mime_type)¶

Parameters:

extension (str)
mime_type (str)

Return type:

FileType

tracdap.rt.api.define_input(requirement, *, label=None, optional=False, dynamic=False, input_props=None)¶

Parameters:

requirement (Union[SchemaDefinition, FileType])
label (Optional[str])
optional (bool)
dynamic (bool)
input_props (Optional[Dict[str, Any]])

tracdap.rt.api.define_input_file(extension, mime_type, *, label=None, optional=False, input_props=None)¶

Parameters:

extension (str)
mime_type (str)
label (Optional[str])
optional (bool)
input_props (Optional[Dict[str, Any]])

Return type:

ModelInputSchema

tracdap.rt.api.define_input_table(*fields, label=None, optional=False, dynamic=False, input_props=None)¶

Define a model input with a table schema

Individual fields can be defined using define_field() or the shorthand alias F(). This function takes a number of fields, either as individual arguments or as a list, and uses them to create a ModelInputSchema.

Use the label property to add a descriptive label to a model input. Inputs can be marked as optional in which case they are not required when running a job, use TracContext.has_dataset() to determine whether an optional input has been provided. Inputs can be marked as dynamic in which case the schema is not defined until the model runs, use TracContext.get_schema() to get the schema of a dynamic input.

You can use input_props to associate arbitrary key-value properties with this model input. These properties are not used by the TRAC engine, but are stored in the model metadata for the input and can be used as needed in 3rd-party applications.

Parameters:

fields (FieldSchema | List[FieldSchema]) – A set of fields to make up a TableSchema
label (str | None) – An optional label (of type str) for a model input schema. Default value: None.
optional (bool) – Mark this input as an optional model input
dynamic (bool) – Mark this input as a dynamic model input (the list of fields must be empty)
input_props (Dict[str, Any] | None) – Associate key-value properties with this input (not used by the TRAC engine)

Returns:

A model input schema, suitable for returning from TracModel.define_inputs()

Return type:

ModelInputSchema

tracdap.rt.api.define_output(requirement, *, label=None, optional=False, dynamic=False, output_props=None)¶

Parameters:

requirement (Union[SchemaDefinition, FileType])
label (Optional[str])
optional (bool)
dynamic (bool)
output_props (Optional[Dict[str, Any]])

tracdap.rt.api.define_output_file(extension, mime_type, *, label=None, optional=False, output_props=None)¶

Parameters:

extension (str)
mime_type (str)
label (Optional[str])
optional (bool)
output_props (Optional[Dict[str, Any]])

Return type:

ModelOutputSchema

tracdap.rt.api.define_output_table(*fields, label=None, optional=False, dynamic=False, output_props=None)¶

Define a model output with a table schema

Use the label property to add a descriptive label to a model output. Outputs can be marked as optional, a model can decide not to provide an optional output without causing an error. Outputs can be marked as dynamic in which case the schema is not defined until the model runs, use TracContext.put_schema() to set the schema of a dynamic output before saving it.

You can use output_props to associate arbitrary key-value properties with this model output. These properties are not used by the TRAC engine, but are stored in the model metadata for the output and can be used as needed in 3rd-party applications.

Parameters:

fields (FieldSchema | List[FieldSchema]) – A set of fields to make up a TableSchema
label (str | None) – An optional label (of type str) for a model output schema. Default value: None.
optional (bool) – Mark this output as an optional model output
dynamic (bool) – Mark this output as a dynamic model output (the list of fields must be empty)
output_props (Dict[str, Any] | None) – Associate key-value properties with this output (not used by the TRAC engine)

Returns:

A model output schema, suitable for returning from TracModel.define_outputs()

Return type:

ModelOutputSchema

tracdap.rt.api.define_parameter(param_name, param_type, label, default_value=None, *, param_props=None)¶

Define an individual model parameter

Model parameters can be defined using this method or the shorthand alias P(). Name, type and label are always required to define a parameter. The parameter name is used to set up parameters in a job and to access parameter values at runtime using TracContext.get_parameter().

Use the label property to add a descriptive label to a model parameter. If a default value is specified, the model parameter becomes optional. It is ok to omit optional parameters when running models or setting up jobs, in which case the default value will be used. If no default is specified then the model parameter becomes mandatory, a value must always be supplied in order to execute the model. TRAC will apply type coercion where possible to ensure the default value matches the parameter type, if the default value cannot be coerced to match the parameter type then model validation will fail.

You can use param_props to associate arbitrary key-value properties with this model parameter. These properties are not used by the TRAC engine, but are stored in the model metadata for the parameter and can be used as needed in 3rd-party applications.

Model parameters can be passed to define_parameters(), either as individual arguments or as a list, to create the set of parameters for a model.

Parameters:

param_name (str) – The parameter name, used to identify the parameter in code (must be a valid identifier)
param_type (BasicType | TypeDescriptor) – The parameter type, expressed in the TRAC type system
label (str) – A descriptive label for the parameter (required)
default_value (Any | None) – A default value to use if no explicit value is supplied (optional)
param_props (Dict[str, Any] | None) – Associate key-value properties with this parameter (not used by the TRAC engine)

Returns:

A named model parameter, suitable for passing to define_parameters()

Return type:

_Named[ModelParameter]

tracdap.rt.api.define_parameters(*parameters)¶

Defined the set of parameters used by a model

Model parameters can be defined using define_parameter() or the shorthand alias P(). This function takes a number of parameters, either as individual arguments or as a list, and arranges them in the format required by TracModel.define_parameters()

Parameters:: parameters (_Named[ModelParameter] | List[_Named[ModelParameter]]) – The parameters that will be defined, either as individual arguments or as a list
Returns:: A set of model parameters, in the correct format to return from TracModel.define_parameters()
Return type:: Dict[str, ModelParameter]

tracdap.rt.api.define_schema(*fields, schema_type=SchemaType.TABLE, dynamic=False)¶

Create a SchemaDefinition from a list of fields

Individual fields can be defined using define_field() or the shorthand alias F(). This function takes a number of fields, either as individual arguments or as a list, and arranges them into a SchemaDefinition.

A schema type can be specified explicitly using the schema_type parameter, currently only TABLE is supported and this is also the default.

Note

To define the inputs or outputs of a TracModel, use define_input_table() or define_output_table() instead.

Parameters:

fields (FieldSchema | List[FieldSchema]) – The list of fields to include in the schema
schema_type (SchemaType) – The type of schema to create (currently only TABLE schemas are supported)
dynamic (bool) – Define a dynamic schema (fields list should be empty)

Returns:

A schema definition built from the supplied fields

Return type:

SchemaDefinition

tracdap.rt.api.load_schema(package, schema_file, schema_type=SchemaType.TABLE)¶

Load a SchemaDefinition from a CSV file in the model package

The schema CSV file must contain the following columns:

field_name (string, required)
field_type (BasicType, required)
label (string, required)
business_key (boolean, optional)
categorical (boolean, optional)
format_code (string, optional)

Field ordering is assigned by the order the fields are listed in the CSV file. A schema type can be specified explicitly using the schema_type parameter, currently only TABLE is supported and this is also the default.

Note

To define the inputs or outputs of a TracModel, a schema can be loaded with this function and used to construct a ModelInputSchema or ModelOutputSchema.

Parameters:

package (ModuleType | str) – Package (or package name) in the model repository that contains the schema file
schema_file (str) – Name of the schema file to load, which must be in the specified package
schema_type (SchemaType) – The type of schema to create (currently only TABLE schemas are supported)

Returns:

A schema definition loaded from the schema file

Return type:

SchemaDefinition

tracdap.rt.api.BOOLEAN¶: Synonym for BasicType.BOOLEAN

tracdap.rt.api.DATE¶: Synonym for BasicType.DATE

tracdap.rt.api.DATETIME¶: Synonym for BasicType.DATETIME

tracdap.rt.api.DECIMAL¶: Synonym for BasicType.DECIMAL

tracdap.rt.api.FLOAT¶: Synonym for BasicType.FLOAT

tracdap.rt.api.INTEGER¶: Synonym for BasicType.INTEGER

tracdap.rt.api.STRING¶: Synonym for BasicType.STRING