Skip to content

View or edit on GitHub

This page is synchronized from trase/models/brazil/soy_2023_2024_v27/readme_unknown_allocation.md. Last modified on 2026-09-17 20:47 CEST by achandra. Please view or edit the original file there; changes should be reflected here after a midnight build (CET time), or manually triggering it with a GitHub action (link).

Allocation of Unknown Flows in the Soy Model

Use of SECEX/MDIC data to improve the allocation of unknown-origin soy shipments

The current soy supply chain model is able to allocate approximately 65% of total soy exports to known municipality logistics hubs (LHs). However, a significant share of export volumes (~40–45%, depending on the year) still has unknown origin due to incomplete information on upstream sourcing.

This exercise aims to improve the allocation of these unknown-origin flows by using SECEX/MDIC export data to derive probabilistic distributions of likely sourcing states and, subsequently, allocate volumes to representative logistics hub municipalities.

Why assign Logistic Hub Municipalities (LHM) / aggregators?

SECEX/MDIC data provides information about the state associated with export shipments, but this information represents a large geographical area rather than a precise production location.

For Brazil, state-level information is often too coarse to directly identify producing municipalities because:

  • Brazilian states cover very large areas.
  • Soy supply chains frequently involve intermediate aggregation points, such as warehouses, silos, and trading facilities, before reaching export terminals.
  • State of production from SECEX/MDIC does not necessarily represent production areas; they may represent the location of aggregation facilities.

The supply shed model considers that soy can move significant distances through internal logistics networks. Therefore, applying distance thresholds between production municipalities and logistics hubs (approximately 300–1000 km, depending on the state) is compatible with the spatial state's size.

Evidence that MDIC state information captures aggregation regions

When filtering SECEX/MDIC exports using soybean-related products:

  • At the HS4 level, exported volumes may include states with limited or no soybean production.
  • Even when applying more restrictive HS6 product classifications, states such as Rio de Janeiro and Espírito Santo, which have negligible soybean production, still appear as exporting origins.

This indicates that the state associated with export records can represent aggregation and logistics locations, rather than only areas where soybeans were cultivated.

Why allocate unknown flows using MDIC distribution?

The approach uses observed export patterns to answer:

"For shipments leaving through a specific port and going to a specific destination region, what is the historical distribution of associated states?"

For example:

MDIC data:

Port Destination State Export volume
Santos China Mato Grosso 700
Santos China Goiás 300
Santos China Paraná 100

The model converts this into probabilities:

State Weight
Mato Grosso 63.6%
Goiás 27.3%
Paraná 9.1%

An unknown shipment of 1,000 tons exported through Santos to China is therefore distributed as:

State Allocated volume
Mato Grosso 636 tons
Goiás 273 tons
Paraná 91 tons

The total volume remains unchanged: 

636 + 273 + 91 = 1000 tons

This creates probabilistic state attribution while conserving the original export volume.

Step-by-step unknown allocation

Step 1 — Generate state distribution from MDIC data

The MDIC dataset is aggregated by:

  • Export port group
  • Region of destination
  • State

For each (port_group, destination_region) combination, the model calculates the contribution of each state: 

state volume / total volume for port + destination

This creates state-level allocation weights.

Example: 

Santos + China

Mato Grosso: 700 / 1100 = 63.6%
Goiás: 300 / 1100 = 27.3%
Paraná: 100 / 1100 = 9.1%

Step 2 — Fan out unknown flows across candidate states

Unknown-origin shipments are matched against the MDIC state distribution using: 

(port_group, destination_region)

A single unknown flow becomes multiple candidate flows.

Before: 

Unknown origin
Santos → China
1000 tons

After applying MDIC distribution: 

Mato Grosso
Santos → China
636 tons

Goiás
Santos → China
273 tons

Paraná
Santos → China
91 tons

Step 3 — Allocate logistics hubs using known supply chain patterns

After assigning probable states, unknown flows are matched against known flows.

The model uses a hierarchical allocation strategy:

  1. State + port + destination + exporter + product
  2. State + port + destination + product
  3. Port + destination + product
  4. Port + destination
  5. Port only

The most specific available information is always prioritized.

Example:

If known flows indicate: 

Mato Grosso → Santos → China

Warehouse A: 80%
Warehouse B: 20%

An unknown flow allocated to Mato Grosso will inherit this distribution: 

Warehouse A: 80% of unknown volume
Warehouse B: 20% of unknown volume

Advantages of this approach

1. Reduces unknown supply chain volumes

Unknown export flows are no longer treated as completely unallocated. Instead, they receive a probability-based allocation based on observed export patterns. So with this uknown-origin is reduced to ~0%

2. Preserves model modularity

The existing supply shed model remains responsible for: 

Production areas → Logistics hubs

The MDIC allocation step only improves: 

Unknown export origin → Candidate logistics hubs

This keeps each component with a clear responsibility.

Allocation Results

The unknown flow allocation was evaluated for two model branches:

  • Branch 2.4 — Distribution of unknown flows
  • Branch 3.4 — Allocation of unknown flows

The allocation strategy prioritizes the most detailed matching criteria and progressively relaxes constraints when no equivalent known pattern is available.

Branch 2.4 — Distribution of unknown flows

The total unknown volume allocated through this step was distributed across the following allocation tiers:

Allocation tier Allocated volume (tons)
state + port group + destination region + exporter group + product type 11,734,845.72
state + port group + destination region + product type 19,458,249.09
port group + destination region + product type 63,000.00
port group only 350,804.30

The results show that the majority of unknown flows were allocated using state-level information combined with export characteristics:

  • State + port + destination + product-based allocations represent ~31.2 million tons, corresponding to the most informative allocation tiers.
  • Only ~350 thousand tons (~1%) required the coarsest allocation level using port group only.

This indicates that SECEX/MDIC state distributions provide strong additional information for resolving unknown origins while maintaining consistency with observed trade patterns.

Branch 3.4 — Allocation of unknown flows

A second allocation step was performed to assign unknown flows to logistics hubs using known supply chain patterns.

The allocated volumes were:

Allocation tier Allocated volume (tons)
state + port group + destination region + exporter group + product type 1,341,396.07
state + port group + destination region + product type 6,100,513.29
port group + destination region 294.62
port group only 294,801.71

Similar to Branch 2.4, most flows were allocated using detailed combinations including state, destination, and product information.

Approximately 7.4 million tons were allocated using state-informed tiers, while only ~295 thousand tons relied exclusively on port-level information.