6. Using Polars¶

This tutorial is based on example code which can be found in the TRAC GitHub Repository under examples/models/python.

The runtime supports Polars as an alternative to Pandas for reading and writing tabular data. This tutorial builds the same PnL aggregation model from Wrapping Models & Using Data, using Polars instead of Pandas, to show how the two compare.

Reading and writing Polars tables¶

To work with Polars instead of Pandas, use get_polars_table() and put_polars_table() in place of the Pandas equivalents. Everything else about defining inputs and outputs is unchanged - the schema, parameters and job config are all identical to the Pandas version of this model.

src/tutorial/using_polars.py¶

    def run_model(self, ctx: trac.TracContext):

        eur_usd_rate = ctx.get_parameter("eur_usd_rate")
        default_weighting = ctx.get_parameter("default_weighting")
        filter_defaults = ctx.get_parameter("filter_defaults")

        customer_loans = ctx.get_polars_table("customer_loans")

        profit_by_region = calculate_profit_by_region_polars(
            customer_loans, eur_usd_rate,
            default_weighting, filter_defaults)

        ctx.put_polars_table("profit_by_region", profit_by_region)

The calculation itself uses Polars’ lazy API. Building the query with .lazy() defers execution until the result is actually needed, which allows Polars to optimise the whole sequence of operations rather than running each step eagerly. Calling .collect() evaluates the lazy query and returns a materialised dataframe, which is what the runtime expects when saving an output.

def calculate_profit_by_region_polars(
        customer_loans: "polars.DataFrame",
        eur_usd_rate: float,
        default_weighting: float,
        filter_defaults: bool):

    if filter_defaults:
        customer_loans = customer_loans.filter(polars.col("loan_condition_cat") == 0)

    # Build a weighting vector, use default_weighting for bad loans and 1.0 for good loans
    condition_weighting = customer_loans \
            .get_column("loan_condition_cat") \
            .map_elements(lambda c: default_weighting if c > 0 else 1.0) \
            .cast(polars.Decimal(38, 10))

    # Use lazy processing
    customer_loans = customer_loans.lazy() \
            .with_columns(gross_profit_unweighted = (polars.col("total_pymnt") - polars.col("loan_amount"))) \
            .with_columns(gross_profit_weighted = (polars.col("gross_profit_unweighted") * condition_weighting)) \
            .with_columns(gross_profit = (polars.col("gross_profit_weighted") * eur_usd_rate))

    profit_by_region = customer_loans \
        .group_by("region") \
        .agg(polars.col("gross_profit").sum())

    # Evaluate lazy result before giving the result to TRAC
    return profit_by_region.collect()

Model attributes¶

Note

Model attributes are an experimental API that is not yet stabilised, expect changes in future versions of TRAC D.A.P.

Models can define a set of attributes to catalogue and describe themselves, by implementing define_attributes(). Attributes are defined using define_attributes(), which takes a number of individual attributes built with A() (a shorthand alias for define_attribute()).

    def define_attributes(self) -> tp.List[trac.TagUpdate]:

        return trac.define_attributes(
            trac.A("model_description", "A example model, for testing purposes"),
            trac.A("business_segment", "retail_products", categorical=True),
            trac.A("classifiers", ["loans", "uk", "examples"], attr_type=trac.STRING)
        )

A name and value are always required for each attribute. Attribute type is optional for single-valued attributes but required for multivalued attributes, such as the classifiers attribute in this example. The categorical flag can be applied to STRING attributes, to mark them for use in categorical searches and filters.