TracMetadataApi¶

class tracdap.api.TracMetadataApi¶

Public API for creating, updating, reading and searching for objects in the TRAC metadata store.

The TRAC metadata API is expressed in terms of Objects and Tags. Objects are structural items (nouns and verbs) such as data, models and jobs that describe resources and operations on the TRAC platform. Tags contain descriptive and informational data and provide a means to classify, catalogue and control objects. Tags can also be used to describe the structure of applications built on the TRAC platform. For more information on the TRAC metadata model, refer to the documentation for ObjectDefinition and Tag.

The write API allows objects to be created directly, by supplying an object definition and a set of tags. These definitions and tags are validated, assigned an ID and version (in a tag header) then saved as they are. Only certain types of objects can be created / updated directly via the public API (currently FLOW and CUSTOM objects). Other types of objects are created by TRAC as a result of operations on the platform, for example a model import job will create a model definition.

Creating a new version of an object is very similar to creating a new object, in that the entire metadata definition must be supplied. TRAC will perform validation to make sure the new definition is a valid version update from the previous version - the semantics of versioning depend on the particular object type.

The metadata write API also provides a means for updating tags (i.e. creating a new tag version); tag updates are allowed for all object types via the public API. The current API requires a whole new tag to be supplied, this will be replaced in a future version with an API that supplies attributes changes as a set of add/modify/remove operations.

The read API is for reading metadata from the metadata store when the IDs of the metadata items are known. The easiest way to read metadata is using TagSelectors, which may be present in other metadata items (e.g. job definitions record tag selectors to indicate which versions of resources were used in a job) or can be easily constructed from an object ID or tag header. HTTP GET calls are also available for REST clients (REST clients can still use selectors with POST requests).

Metadata searches are performed using tag attributes, as well as an object type and a few top-level parameters to handle versioning and temporality. See the SearchParameters object for a more detailed description. The result of a search call is a list of matching tags, which are always arranged with the most recent tags first. A search that matches no results will return an empty list.

This API is a multi-tenant API. For gRPC requests every request includes a tenant code, for REST requests the tenant code is the first element of every path.

createObject(request)¶

Create a new object in the TRAC metadata store.

To create an object, supply a metadata write request with the tenant code, object type and a definition for the item you want to save. The tag should contain all the attributes you want to set and the object definition, without an object header. TRAC will create a header when it assigns and ID for the new object. Only FLOW and CUSTOM objects can be created by this API call.

Validation is performed on new objects before they are saved. Objects can fail validation because they are semantically invalid (e.g. missing required fields or inconsistencies within the object definition) or due to consistency errors (referring to other metadata items that don’t exist or don’t meet certain requirements).

NOTE: Validation is only partially implemented in the current release.

The call returns an ID response to indicate the ID of the newly created object, as well as the object and tag version (which will always be 1).

Error conditions include: Invalid request, unknown tenant, object type does not match the supplied definition, validation failure.

See also

ObjectDefinition, TagUpdate

Parameters:: request (MetadataWriteRequest)
Return type:: tracdap.metadata.TagHeader

getLatestObject(request)¶

Get the latest version of a metadata object using an HTTP GET request.

This call is intended for use by REST clients to provide TRAC metadata objects as RESTFUL resources. For gRPC clients it is recommended to use readObject instead and supply a TagSelector.

See also

readObject()

To use this call, supply a tenant code, object type and ID. The call will return the latest version of the object with its latest tag, including the full object definition.

Error conditions include: Invalid request, unknown tenant, unknown object ID, object type does not match ID.

Parameters:: request (MetadataGetRequest)
Return type:: tracdap.metadata.Tag

getLatestTag(request)¶

Get the latest tag for a metadata object using an HTTP GET request.

This call is intended for use by REST clients to provide TRAC metadata objects as RESTFUL resources. For gRPC clients it is recommended to use readObject instead and supply a TagSelector.

See also

readObject()

To use this call, supply a tenant code, object type, ID and version. The call will return the latest tag for the specified version of the object, including the full object definition.

Error conditions include: Invalid request, unknown tenant, unknown object ID, object type does not match ID, unknown object version.

Parameters:: request (MetadataGetRequest)
Return type:: tracdap.metadata.Tag

getObject(request)¶

Get a single metadata object using an HTTP GET request.

This call is intended for use by REST clients to provide TRAC metadata objects as RESTFUL resources. For gRPC clients it is recommended to use readObject instead and supply a TagSelector.

See also

readObject()

To use this call, supply a tenant code, object type, ID, object version and tag version. The call will return a tag for the exact object version and tag version specified, including the full object definition.

Error conditions include: Invalid request, unknown tenant, unknown object ID, object type does not match ID, unknown object version, unknown tag version.

Parameters:: request (MetadataGetRequest)
Return type:: tracdap.metadata.Tag

listResources(request)¶

Get a list of infrastructure resources of a given type.

For a given tenant and resource type, obtain a list of all the resources accessible to the client. Client must have sufficient permissions within the tenant to make this call and the list returned will be restricted by the permissions of the logged in user. The returned list includes all the same information that can be obtained by calling resourceInfo(), so there is no need to make a second API call.

As an example, calling listResources() for resource type MODEL_REPOSITORY would provide a list of repositories that are accessible to the current user to import models from. The resource key of the selected repository can be used in an IMPORT_MODEL job as the repository key.

Error conditions include permission denied, invalid request, unknown tenant and unknown resource type.

See also

ResourceType

Parameters:: request (ListResourcesRequest)
Return type:: ListResourcesResponse

listTenants(request)¶

Get a list of tenants that the current user has access to.

Information returned includes the available tenant codes, as well as human readable descriptions. Client tools that support multiple tenants and tenant switching will need to fetch this list.

Only tenants accessible to the current user are returned.

Parameters:: request (ListTenantsRequest)
Return type:: ListTenantsResponse

platformInfo(request)¶

Get some general information about the TRAC platform.

Information returned includes what environment you are connected to and what version of TRAC is running. Useful for displaying in “about” boxes in a UI or client tool, and for submitting error reports.

Parameters:: request (PlatformInfoRequest)
Return type:: PlatformInfoResponse

readBatch(request)¶

Read multiple objects from the TRAC metadata store using tag selectors.

Tag selectors are used throughout the TRAC metadata structures and API calls to refer to objects. For example, a job definition includes selectors to define the data sets and models that were selected for the job. Those selectors can be passed directly into this call.

Tag selectors can refer to object and tag versions either using explicit version numbers, with as-of times or by referring to the current latest version. Combinations are also possible (e.g. object version x, latest tag).

This call takes multiple selectors and returns multiple Tags in the same order, including their object definitions. Selectors for different object types and with different criteria for selecting versions can be mixed freely in a single request. Error conditions include: Invalid request, unknown tenant, unknown object ID, object type does not match ID, unknown object version, unknown tag version. If there is an error for any of the selectors the whole request will fail.

See also

TagSelector

Parameters:: request (MetadataBatchRequest)
Return type:: MetadataBatchResponse

readObject(request)¶

Read a single object from the TRAC metadata store using a tag selector.

This call takes a single selector and returns a single Tag, including the object definition. Error conditions include: Invalid request, unknown tenant, unknown object ID, object type does not match ID, unknown object version, unknown tag version.

See also

TagSelector

Parameters:: request (MetadataReadRequest)
Return type:: tracdap.metadata.Tag

resourceInfo(request)¶

Get information about a specific infrastructure resource.

Use this method to get the public properties associated with a specific infrastructure resource, if you already know the resource type and key. Client must have sufficient permissions within the tenant to make this call and the result will only be returned if the user has permission to access the specified resource.

This method is intended to allow integrations between client applications and infrastructure resources. For example, if a model repository or data location is available in TRAC as a resource, client applications can allow users to browse or interact with those resources either by directing them to an external URL or by making API calls to an external service. The resource must be configured with the necessary public properties to make this work. Typically the protocol and address of a resource can be public properties but the user must supply their own credentials, which will be different from the system credentials used by the TRAC platform. It is likely that some resources available in the platform will not be accessible to all users, and some resources accessible to a user will not be available on the platform.

Error conditions include permission denied, invalid request, unknown tenant, unknown resource type and resource not found.

See also

ResourceType

Parameters:: request (ResourceInfoRequest)
Return type:: ResourceInfoResponse

search(request)¶

Perform a search against the TRAC metadata store.

Searches are always for a specific tenant and object type. For details of the available search parameters, see SearchParameters.

In gRPC calls, a MetadataSearchRequest specifies the tenant and includes the SearchParameters, where object type and the search expression are set. For REST calls the tenant is passed in the URL and searchParameters are supplied as the request body.

See also

SearchParameters

Parameters:: request (MetadataSearchRequest)
Return type:: MetadataSearchResponse

updateObject(request)¶

Update an existing object in the TRAC metadata store.

To save a new version, supply a metadata write request with the tenant code, object type and a tag for the item you want to save. The tag should contain all the attributes you want to set and the updated object definition, which must be supplied with the header from the previous version of the object. TRAC will apply the new version number when it creates the new version of the object. Only the latest version of an object can be updated and only CUSTOM objects can be updated by this API call.

NOTE: Validation is only partially implemented in the current release.

The call returns an ID response with the ID object and the newly assigned version number, as well as the tag version (which will always be 1).

Error conditions include: Invalid request, unknown tenant, unknown object ID or version, wrong object type (not the same as the prior version), object version is superseded.

Parameters:: request (MetadataWriteRequest)
Return type:: tracdap.metadata.TagHeader

updateTag(request)¶

Update the tag for an existing object in the TRAC metadata store.

To save a new tag, supply a metadata write request with the tenant code, object type and a tag for the item you want to save. The tag should contain all the attributes you want to include in the new tag, even if they have not been changed. The object definition must be supplied with a header only, supplying an object body is an invalid request. Tags for any type of object can be updated by this API call. Only the latest version of a tag can be updated.

The call returns an ID response with the ID and version of the object and the newly assigned tag number.

Error conditions include: Invalid request, unknown tenant, unknown object ID, object version or tag version, wrong object type (does not match what is stored in TRAC), tag version is superseded.

See also

TagSelector, TagUpdate

Parameters:: request (MetadataWriteRequest)
Return type:: tracdap.metadata.TagHeader

writeBatch(request)¶

Perform a batch of write operations on the TRAC metadata store.

A writeBatch request allows multiple createObject, updateObject and updateTag requests to be combined into a single API call. It can result in considerable performance gains when a large number of objects are created or updated together. Using writeBatch() also gives a transactional guarantee on the metadata store, so the whole batch will either succeed or fail. This can be useful for maintaining consistency when updating multiple objects that refer to each other.

Each object ID can only occur once for updateObject, while each object version can only occur once for updateTag. It is allowed to update tags for multiple object versions in the same request. It is also allowed to use updateObject and updateTag on the same object, which may be used e.g. to update an object and also update the tag on the prior version. It is not possible to create a new object version and update the tag of that new version in the same request.

Validation follows mostly the same rules as for individual write requests. The tenant code must be specified in the batch and is optional in the individual request objects, if specified it must match what is in the batch request. Validation also checks for duplicate object IDs in updateObject and duplicates of the same object version in updateTag.

The call returns tag headers for all the requested write operations in order. For example if a request creates two objects and updates 3, then the response will contain two tag headers for createObject and three for updateObject, in the same order as the request.

Error conditions include all the error conditions that can occur for individual write operations, as well as errors relating to duplicate updates for the same object ID.

Parameters:: request (MetadataWriteBatchRequest)
Return type:: MetadataWriteBatchResponse