Skip to content

DBT: Brazil Mdic Disaggregated 2021 Beef

File location: s3://trase-storage/brazil/trade/mdic/disaggregated/brazil_mdic_disaggregated_2021_beef.parquet

DBT model name: brazil_mdic_disaggregated_2021_beef

Explore on Metabase: Full table; summary statistics

DBT details

Description

Brazil MDIC/SECOMEX Disaggregation

This dataset attempts to reconstruct the MDIC/SECOMEX trade export dataset to the level of information that was provided before 2018. To do so, it combines the post-2018 MDIC/SECOMEX trade export datasets with bills of lading.

This reconstruction is run as a model written in using the supply chain mapping tool: see trase/models/brazil/customs_2019/README.md.

History

It was last run by Nanxu Su for the update to the Brazil beef model to add the year 2020. Since we only needed the model output for beef, we filtered to beef commodity codes and wrote the file brazil_mdic_disaggregated_2020_beef.csv.

Details

Column Type Description
month VARCHAR
country_of_destination.name VARCHAR
hs4 VARCHAR
via VARCHAR
port.name VARCHAR
state.trase_id VARCHAR
hs6 VARCHAR
hs8 VARCHAR
exporter.municipality.trase_id VARCHAR
success BOOLEAN
message VARCHAR
vol DOUBLE

Models / Seeds

  • source.trase_duckdb.trase-storage-raw.brazil_close_ports
  • source.trase_duckdb.trase-storage-raw.commodities
  • model.trase_duckdb.brazil_mdic_port_2021
  • model.trase_duckdb.brazil_mdic_municipality_2021
  • model.trase_duckdb.brazil_bol_2021

Sources

  • ['trase-storage-raw', 'brazil_close_ports']
  • ['trase-storage-raw', 'commodities']

No called script or script source not found.

from unittest.mock import patch

from trase.models.brazil.customs_2019.main import register_mosex_license
from trase.tools.etl_internal.processors import S3Mixin
from trase.tools.sps import SupplyChain


def model(dbt, cursor):
    dbt.config(materialized="external")

    register_mosex_license()

    # provide the SupplyChain class with data from DBT rather than allowing it to
    # read from S3. The keys in the dictionary relate to the "outname" properties of the
    # classes in preparation.py. Reference years are hard-coded as dbt functions only allows
    # string literals.
    data = {
        "port_renames": dbt.source("trase-storage-raw", "brazil_close_ports").df(),
        "commodity": dbt.source("trase-storage-raw", "commodities").df(),
        "mdic_port": dbt.ref("brazil_mdic_port_2021").df(),
        "mdic_municipality": dbt.ref("brazil_mdic_municipality_2021").df(),
        "bill_of_lading": dbt.ref("brazil_bol_2021").df(),
    }

    supply_chain = SupplyChain(
        "brazil/customs_2019", year=2021, pre_extracted_data=data
    )

    with patch.object(
        S3Mixin,
        "extract",
        side_effect=RuntimeError("Preparation class should not be reading from S3"),
    ):
        supply_chain.preparation()

    supply_chain.load()
    supply_chain.run()

    return supply_chain.df_export()