Main

View or edit on GitHub

This page is synchronized from trase/models/brazil/soy_2023_2024_v27/main.ipynb. Last modified on 2026-05-06 16:54 CEST by Trase Admin. Please view or edit the original file there; changes should be reflected here after a midnight build (CET time), or manually triggering it with a GitHub action (link).

from trase.tools import sps
import warnings
from trase.tools.aws.metadata import write_csv_for_upload
import pandas as pd
from trase.tools.aws.aws_helpers import read_geojson
from quality_assurance import quality_assurance
from post_processing import export_flows_for_supply_shed, export_flows_for_visualization

warnings.filterwarnings("ignore")
pd.options.display.float_format = "{:,.2f}".format

for year in [2023]:
    try:
        print(f"Running {year}")
        supplychain = sps.SupplyChain("brazil/soy_2023_2024_v27", year=year)
        supplychain.preparation()
        supplychain.load()
        supplychain.run()
        supplychain.flow_report_by_attribute("vol", ["branch"], 8)

        quality_assurance(supplychain)

        # Export flows for 
        export_flows_for_supply_shed(supplychain)
        export_flows_for_visualization(supplychain)

    except Exception as e:
        print(f"Failed {year}: {e}")

Running 2023
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Municipality[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of IndustrialCapacity[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of IndustrialCapacityFacilities[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of CommodityRatios[0m
Extracting data from source ...[0m [97m took 0.3 seconds[0m
[97mSkipping re-process of ExporterSpecialCases[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of ExtraCnpjs[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Production[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Cost[0m
    [97mLoading data from disk took 15.7 seconds[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of SilosGeometry[0m
[1mRunning pre-processing for Silos[0m
Report: Before processing[0m
        | Row Count: 0[0m
Report: After processing[0m
        | Row Count: 9,376[0m
[32mWritten /Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/2023/prepared/silos.csv[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Hs[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Flows[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[1mRunning pre-processing for Cnpj[0m
Report: Before processing[0m
        | Row Count: 427,313[0m
Report: Drop rows with missing level or geocode[0m
        | Row Count: 427,313[0m
Report: Drop duplicates[0m
        | Row Count: 351,483[0m
Report: Add extra CNPJs[0m
        | Row Count: 352,127[0m
Report: After processing[0m
        | Row Count: 318,727[0m
[32mWritten /Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/2023/prepared/cnpj.csv[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of RoutesSpecialCases[0m
Extracting data from source ...[0m [97m took 0.2 seconds[0m
[97mSkipping re-process of Sicasq[0m


Use the validate option with pd.merge or pass validate=None to suppress this warning
/Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/bills_of_lading_decision_tree.py:32
Use the validate option with pd.merge or pass validate=None to suppress this warning
/Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/bills_of_lading_decision_tree.py:251
Use the validate option with pd.merge or pass validate=None to suppress this warning
/Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/bills_of_lading_decision_tree.py:267
Use the validate option with pd.merge or pass validate=None to suppress this warning
/Users/jailsonsoares/repos/TRASE/trase/models/brazil/soy_2023_2024_v27/bills_of_lading_decision_tree.py:275


                                                                     sum percentage
branch                                                                             
Unknown - missing information                                 25,400,122        21%
4.1.1 office is not located at a port and trades >20,000 t    25,363,635        21%
2.1. link to silo list and cadastro                           16,316,416        14%
3.1.2.2 link to crush with >1 silo in same municipality       11,071,994         9%
2.2. link to silo cadastro only                                9,786,426         8%
1.2 link to farm municipality no production                    8,921,831         8%
1.1 link to farm municipality with production                  5,969,094         5%
4.1.2 office is located at a port                              5,676,496         5%
3.1.1 link to crush with silo                                  3,404,382         3%
3.1.2.1 link to crush with 1 silo in same municipality         2,336,246         2%
Unknown - missing data                                         2,226,841         2%
3.2.2 link to crush no silos in municipality, use supply shed  1,934,186         2%
3.2.1 link to crush no silos in municipality, use SICASQ         185,513         0%
5. no soy economic activity                                          390         0%