Skip to content

Milk Production 2019

s3://trase-storage/brazil/flow_constraints/production/MILK_PRODUCTION_2019.csv

Dbt path: trase_production.main_brazil.milk_production_2019

Explore on Metabase: Full table; summary statistics

Containing yaml file link: trase/data_pipeline/models/brazil/flow_constraints/production/_schema.yml

Model file link: trase/data_pipeline/models/brazil/flow_constraints/production/milk_production_2019.py

Calls script: trase/data/brazil/flow_constraints/production/MILK_PRODUCTION_2019.py

Dbt test runs & lineage: Test results ยท Lineage

Full dbt_docs page: Open in dbt docs (includes lineage graph -at the bottom right-, tests, and downstream dependencies)

Tags: mock_model, brazil, flow_constraints, production

milk_production_2019

Description

This model was auto-generated based off .yml 'lineage' files in S3. The DBT model just raises an error; the actual script that created the data lives elsewhere. The script is located at trase/data/brazil/flow_constraints/production/MILK_PRODUCTION_2019.py [permalink]. It was last run by Harry Biddle.

Details

Column Type Description

Models / Seeds

  • model.trase_duckdb.milk_eggs_honey_wool_production_2019
from trase.tools.aws.aws_helpers_cached import get_pandas_df_once

from trase.tools.aws.metadata import write_csv_for_upload


df = get_pandas_df_once(
    "brazil/production/statistics/ibge/milk_eggs_honey_wool/MILK_EGGS_HONEY_WOOL_PRODUCTION_2019.csv",
    sep=";",
    converters={"GEOCODMUN": str, "MILK_THOUSAND_LITRES": int},
)

# rename columns
df["LITERS"] = 1_000 * df["MILK_THOUSAND_LITRES"]
df = df[["GEOCODMUN", "LITERS"]]
df["TYPE"] = "MILK"
df["YEAR"] = 2019

# some quick QA
assert all(df["GEOCODMUN"].str.len() == 7)
assert df["GEOCODMUN"].is_unique
assert all(df["LITERS"] >= 0)

write_csv_for_upload(df, "brazil/flow_constraints/production/MILK_PRODUCTION_2019.csv")

import pandas as pd


def model(dbt, cursor):
    dbt.ref("milk_eggs_honey_wool_production_2019")

    raise NotImplementedError()
    return pd.DataFrame({"hello": ["world"]})