Skip to content

DBT: Brazil Mdic Disaggregated 2020 Beef

File location: s3://trase-storage/brazil/trade/mdic/disaggregated/brazil_mdic_disaggregated_2020_beef.parquet

DBT model name: brazil_mdic_disaggregated_2020_beef

Explore on Metabase: Full table; summary statistics

DBT details

Description

Brazil MDIC/SECOMEX Disaggregation

This dataset attempts to reconstruct the MDIC/SECOMEX trade export dataset to the level of information that was provided before 2018. To do so, it combines the post-2018 MDIC/SECOMEX trade export datasets with bills of lading.

This reconstruction is run as a model written in using the supply chain mapping tool: see trase/models/brazil/customs_2019/README.md.

History

It was last run by Nanxu Su for the update to the Brazil beef model to add the year 2020. Since we only needed the model output for beef, we filtered to beef commodity codes and wrote the file brazil_mdic_disaggregated_2020_beef.csv.

Details

Column Type Description
month INTEGER
country_of_destination.name INTEGER
hs4 INTEGER
via INTEGER
port.name INTEGER
state.trase_id INTEGER
hs6 INTEGER
hs8 INTEGER
exporter.municipality.trase_id INTEGER
success BOOLEAN
message INTEGER
vol INTEGER

Models / Seeds

  • source.trase_duckdb.trase-storage-raw.commodities
  • model.trase_duckdb.empresas_cadastro_2020
  • model.trase_duckdb.brazil_mdic_port_2020
  • model.trase_duckdb.brazil_mdic_municipality_2020
  • model.trase_duckdb.brazil_bol_2020

Sources

  • ['trase-storage-raw', 'commodities']

No called script or script source not found.

from unittest.mock import patch

from trase.models.brazil.customs_2019.main import register_mosex_license
from trase.tools.etl_internal.processors import S3Mixin
from trase.tools.sps import SupplyChain


def model(dbt, cursor):
    dbt.config(materialized="external")

    register_mosex_license()

    # provide the SupplyChain class with data from DBT rather than allowing it to
    # read from S3. The keys in the dictionary relate to the "outname" properties of the
    # classes in preparation.py
    data = {
        "secex": dbt.ref("empresas_cadastro_2020").df(),
        "commodity": dbt.source("trase-storage-raw", "commodities").df(),
        "mdic_port": dbt.ref("brazil_mdic_port_2020").df(),
        "mdic_municipality": dbt.ref("brazil_mdic_municipality_2020").df(),
        "bill_of_lading": dbt.ref("brazil_bol_2020").df(),
    }

    supply_chain = SupplyChain(
        "brazil/customs_2019", year=2020, pre_extracted_data=data
    )

    with patch.object(
        S3Mixin,
        "extract",
        side_effect=RuntimeError("Preparation class should not be reading from S3"),
    ):
        supply_chain.preparation()

    supply_chain.load()
    supply_chain.run()

    return supply_chain.df_export()