DBT: Silver Ethiopia Coffee 2020

File location: s3://trase-storage/ethiopia/trade/bol/2020/silver/silver_ethiopia_coffee-2020.parquet

DBT model name: silver_ethiopia_coffee_2020

Explore on Metabase: Full table; summary statistics

DBT details

Lineage
Test results
Dbt path: trase_production.main_ethiopia_coffee.silver_ethiopia_coffee_2020
Containing yaml link: trase/data_pipeline/models/ethiopia/trade/bol/2020/_schema_ethiopia_coffee.yml
Model file: trase/data_pipeline/models/ethiopia/trade/bol/2020/silver_ethiopia_coffee_2020.py
Calls script: trase/data/ethiopia/trade/bol/2020/silver/silver_ethiopia_coffee-2020.R
Original dbt_docs
Tags: silver, ethiopia, coffee, trade, 2020, diet-trase-coffee

Description

....

Details

ColumnsDepends OnModel code

Column	Type	Description
`DATE`	`TIMESTAMP WITH TIME ZONE`
`OFFICE_CODE`	`VARCHAR`
`OFFICE_NAME`	`VARCHAR`
`ASSDATE`	`TIMESTAMP WITH TIME ZONE`
`EXPORTER_NAME`	`VARCHAR`
`BUYER_NAME`	`VARCHAR`
`HS_CODE`	`VARCHAR`
`HS_CODE_DESCRIPTION`	`VARCHAR`
`CATEGORY`	`VARCHAR`
`COMERCIAL_DESCRIPTION`	`VARCHAR`
`COMMERCIAL_DESCRIPTION_EN`	`VARCHAR`
`QUANTITY`	`DOUBLE`
`UNIT`	`VARCHAR`
`GROSS_WEIGHT`	`DOUBLE`
`NET_WEIGHT`	`DOUBLE`
`FOB_VALUE_IN_USD`	`DOUBLE`
`FOB_VALUE_IN_BIRR`	`DOUBLE`
`COUNTRY_DESTINATION`	`VARCHAR`
`chapter`	`VARCHAR`
`4_digit`	`VARCHAR`
`MONTH`	`VARCHAR`
`YEAR`	`VARCHAR`

Models / Seeds

source.trase_duckdb.source_ethiopia_trade.original_ethiopia_comtrade_2020
source.trase_duckdb.source_ethiopia_trade.original_ethiopia_bol_2020

Sources

['source_ethiopia_trade', 'original_ethiopia_comtrade_2020']
['source_ethiopia_trade', 'original_ethiopia_bol_2020']

No called script or script source not found.

import pyarrow.parquet as pq

from trase.tools.r.utilities import run_r_script_in_repository


def model(dbt, session):
    dbt.config(
        materialized="external",
        meta={
            "external_location": "s3://trase-storage/ethiopia/trade/bol/2020/silver/silver_ethiopia_coffee-2020.parquet",
            "model_maturity": "in production",
            "calls_script": "trase/data/ethiopia/trade/bol/2020/silver/silver_ethiopia_coffee-2020.R",
        },
        persist_docs={"relation": True, "columns": True},
    )

    # Declaring the sources so they appear in the documentation / lineage
    session.execute("INSTALL spatial; LOAD spatial;")  # Needed to reference excel files
    comtrade_data = dbt.source(
        "source_ethiopia_trade", "original_ethiopia_comtrade_2020"
    )
    trade_data = dbt.source("source_ethiopia_trade", "original_ethiopia_bol_2020")

    # Execute the script using the run_r_script_in_repository  wrapper
    script_path = (
        "trase/data/ethiopia/trade/bol/y2020/silver/silver_ethiopia_coffee-2020.R"
    )
    output_file_path = "/tmp/silver_ethiopia_coffee-2020.parquet"
    print("Executing R script:", script_path)
    run_r_script_in_repository(script_path, output_files=[output_file_path])

    export_data = pq.read_table(output_file_path)

    return export_data