Skip to content

View or edit on GitHub

This page is synchronized from trase/models/brazil/soy_2023_2024_v27/DECISIONS.md. Last modified on 2026-09-17 20:47 CEST by achandra. Please view or edit the original file there; changes should be reflected here after a midnight build (CET time), or manually triggering it with a GitHub action (link).

Brazil Soy model decisions

2026 09 — São Paulo, Mairinque and Porto Alegre are never office logistics hubs

Decision

In branch 4, an office/wholesale CNPJ registered in São Paulo (BR-3550308), Mairinque (BR-3528403) or Porto Alegre (BR-4314902) is not treated as a logistics hub. The flow is left unresolved so that it falls through to "Unknowns to be solved", where the existing unknown-sourcing allocation spreads it on the known sourcing pattern.

The rule is a plain municipality list. Two narrower screens were considered and deliberately not included:

  • A test that the CNPJ carries an office/wholesale CNAE. This is redundant — every row reaching branch 4 is an office registration by construction — and implementing it as a cadastro lookup would have been wrong: Bunge's São Paulo CNPJ 84046101033873 is not in SOY_CNPJ_2023.parquet at all, and reaches branch 4 through a hand-maintained level-4 row in brazil/dictionaries/extra_cnpj_dict.csv.
  • A test that the exporter's company group owns no storage or crushing capacity in the municipality. This would have spared Bunge's 1.82 Mt (2023) and 2.01 Mt (2024), because Bunge owns the single facility recorded in São Paulo — all 7,920 t of it, registered to a different establishment. A binary "owns something there" test lets a small shed immunise a very large flow.

A capacity-ratio rule (exclude any municipality assigned more than ~100× its recorded storage) was also considered. It lands in almost the same place on this data — 10.39 Mt in 2023 against 10.34 Mt for the three municipalities under the fuller screen — but on a different set of municipalities. The named list was preferred as easier to review and as avoiding a threshold in the model that we would have to defend; the trade-off is that it needs revisiting as registrations move.

Why

An office/wholesale address is a corporate registration, not a point of aggregation. QA of the v2.7.0 output found each of these three carrying far more volume than the storage recorded there could hold:

Municipality Logistics-hub volume Recorded storage Implied turnover
São Paulo 22.6 Mt (2023) 7,920 t ~2,850×
Porto Alegre 1.7–2.1 Mt (branch 4.1.1) 29,900 t 57–69×
Mairinque 1.4 Mt (branch 4.1.1, 2023) 13,970 t ~98×

The median hub municipality turns its recorded storage over about 2.1× a year and the 95th percentile is about 120×.

Routing to unknowns was preferred over reassigning to the exporter's nearest owned facility. ADM's 2023 São Paulo volume alone is roughly 3× its total national storage capacity across all 33 of its facilities, so reassignment would replace one artificial concentration with another, and constructing a spread across selected facilities would need distance, capacity and sourcing assumptions the data does not support.

Impact / implications

  • Moves 12.16 Mt in 2023 (10.5% of national volume) and 7.79 Mt in 2024 (6.7%) out of branch 4.1.1 and into the unknown-sourcing allocation: São Paulo 9.10 / 5.57 Mt, Porto Alegre 1.70 / 2.07 Mt, Mairinque 1.37 / 0.14 Mt.
  • The exporters affected in São Paulo are ADM, Bunge, Engelhart, Wilmar, Marubeni and Olam; at Porto Alegre COFCO (and Syngenta in 2024); at Mairinque Cargill.
  • This is a deliberate partial fix. It covers branch 4 only. São Paulo is still reached through branches 1.2 and 2.1 by head offices whose CNPJ also carries a warehousing (5211-7/01, 5211-7/99) or post-harvest (0163-6/00) CNAE, because the cadastro keeps the lowest level per CNPJ. That is 10.7 Mt in 2023 and 13.2 Mt in 2024, and it means São Paulo remains the largest single logistics hub afterwards — roughly 13.5 Mt (2023) and 15.6 Mt (2024), against Rondonópolis at 6.1 / 7.6 Mt. Anyone reporting this change should not describe São Paulo as fixed.
  • Branch 2.1 additionally only requires a silo in the municipality, belonging to anyone, so Bunge's 7,920 t qualifies São Paulo as a silo municipality for every exporter registered there. Any follow-up should address that as well as the CNAE precedence.
  • The set of affected CNPJs is not stable between years — Engelhart moved from branch 4.1.1 in 2023 to branch 2.1 in 2024 — so the volume this rule moves will vary year to year.

Where this sits in the code

  • OFFICE_ONLY_HUB_MUNICIPALITIES in trase/models/brazil/soy_2023_2024_v27/model.py, applied in branch_4_1_1_exports_linked_to_office and branch_4_1_2_exports_linked_to_office.
  • The logistics hub is also reset to the BR-XXXXXXX sentinel in both. supplychain.update writes logistics_hub.trase_id for every row a branch merges, not only the ones its mask resolved, so branch 4.1.2 re-resolves the rows 4.1.1 left unprocessed and would otherwise write the office municipality straight back onto them.
  • A silo_turnover_within_limit dbt test over the v2.7.0 outputs reports hub municipalities assigned more volume than the recorded storage could turn over. It runs as a warning, not an error: the silo map is a lower bound on real capacity, and it is expected to keep flagging São Paulo until the branch 2.1 and 1.2 routes are dealt with.

References

2026 09 — Supply-shed model relaxes the sourcing-distance cutoff for "starved" silos (retrospective record)

Decision

Keep the existing supply-shed rule by which the model identifies starved silos — silos/hubs whose assigned demand cannot be met from production lying within the applicable distance cutoff — and relaxes the maximum sourcing distance for them rather than leaving their demand unfilled. The larger a destination's demand, the further the model reaches for supply.

This rule was agreed in earlier supply-shed discussions but was never written into this log; it is recorded here retrospectively at Jailson's request, because QA of the 2023 flows kept re-discovering it as an apparent anomaly.

Why

The distance cutoff is derived from road distances (state-level CUTOFF_KM values), so it does not represent long-distance movements that in practice are made by rail. Applying it as a hard limit would leave demand at high-throughput destinations unfillable. Relaxing it for starved silos has, in Jailson's assessment, worked well precisely as a way of filling that rail-shaped gap.

Impact / implications

  • Volume assigned to a logistics hub from origin municipalities beyond the stated distance cutoff is expected behaviour, not a defect. The cutoff is a starting constraint, not a hard maximum.
  • The effect scales with demand: São Paulo, which carries very large assigned demand, therefore relaxes the maximum distance a long way. In the largest 2023 flow found in QA (ADM, export via Santos to China, 3.38 Mt, attributed almost entirely to São Paulo municipality as the logistics hub), 68.5% of the assigned volume comes from municipalities beyond the stated cutoff — this is the relaxation rule operating, not a separate bug.
  • QA of Brazil soy flows should not spend time on beyond-cutoff sourcing as such; the question worth investigating in cases like the ADM/São Paulo one is whether the logistics hub assignment is right (see the São Paulo entry above), not the sourcing distance.
  • Any user-facing description of a distance cutoff (methods notes, FAQ) must say that the cutoff can be relaxed for starved silos, otherwise readers will read the published cutoff as a hard limit and treat these flows as errors.

Where this sits in the code

In trase/models/brazil/soy_supply_sheds/model.py:

  • The farm → silo cost matrix used by the first-stage LP is filtered to pairs within the state-specific cutoff read from brazil/logistics/freight/distance_cutoff.csv (model.py:42-51; preprocessor DistanceCutoff in preparation.py:626).
  • That LP is then iterated: demand which the solution could not meet is treated as a deficit and redistributed by get_updated_silos_demand (model.py:162-181, model.py:625). Demand originating from the CNPJ-matched branches — branch 1.*, 2.* and 4.1.1, which is how São Paulo enters (model.py:78-85) — is deliberately never reduced by that redistribution.
  • The second-stage silo → hub/crushing LP is run against the unfiltered cost matrix all_costs, i.e. with no distance limit at all (model.py:310-350).

Whoever next writes this up for users should confirm with Jailson which of these mechanisms he means by "relaxing the cutoff", and what the effective maximum distance ends up being, before quoting a number publicly.

Whether São Paulo should be a logistics hub at all — since decided, see the entry above

This came out of the same discussion and was open when this entry was written. It has since been settled: São Paulo, Mairinque and Porto Alegre are excluded from branch 4 and routed to unknowns. The options as they stood were:

  • Office/wholesale CNAEs should not in general be interpreted as a point of aggregation of commodities. The single exception is branch 4.1.1 — an office that is not located at a port, where the exporter's combined exports from that municipality exceed 20,000 tonnes, and which therefore acts as a hub. São Paulo is not on the list of municipalities containing ports, so ADM's São Paulo corporate/wholesale registration falls into 4.1.1 and is treated as a hub.
  • Jailson's two options: add an exception for São Paulo and reassign the logistics hub to the company's nearest hub/facility; or keep São Paulo, on the grounds that it genuinely is a very large hub. Note that São Paulo is also reached as a logistics hub through branches 1.1 and 2.1 in other cases, so any exception would have to be scoped rather than a blanket exclusion of the municipality. (In the event, neither: the flows go to unknowns rather than to a nearest facility, and the exception is scoped to branch 4, leaving the branch 2.1 and 1.2 routes as follow-up work. Branch 1.1 turns out never to assign São Paulo, because it requires the farm municipality to produce soy and São Paulo produces none — those flows fall to 1.2 instead.)
  • Mike's view: São Paulo has been a known problem since older SEI-PCS versions because it is such a large hub, similar special rules have existed before, and reassigning specifically for São Paulo makes sense.
  • Supporting evidence from QA: ADM has 33 silo/storage facilities in the dataset, including large ones at Rondonópolis, Sinop, Primavera do Leste, Sorriso and Querência, which would be candidate physical destinations for the volume currently attributed to the São Paulo office address. (These were not used: the volume is roughly 3× ADM's total national storage capacity, so concentrating it on them would be no better supported than the office address.)

References

2026 08 — Use MDIC-derived known-pattern fallback to reduce unknown logistics hubs to ~3%

Decision

For the v2.7 Brazil Soy release, use MDIC state-level logistics-hub patterns to backfill the roughly 30% of 2023 volume for which the decision tree cannot otherwise locate a logistics-hub municipality. Upsample the state-level MDIC pattern to municipality level by extrapolating the pattern observed in the already-solved portion of the supply chain (the first of the two methods evaluated). Implement this as an additional fallback branch in the decision tree.

This reduces unknown logistics hubs to roughly 3% for the release.

Why

Harry and Jailson's evaluation found reasonable-to-good overlap between MDIC, bills of lading, and the SEI-PCS model, supporting use of MDIC for state-level logistics-hub patterns. Two municipality-level upsampling approaches were considered: extrapolating the solved supply-chain pattern, or distributing volume across the facility map. Both have trade-offs, but only the first was readily compatible with the supply-shed model in the current release cycle.

The team considered deeper alternatives — integrating MDIC into the supply-shed model itself, fixing the incompatibility of the facility-map method, or using a simpler production-weighted / LP approach for the affected volume — but these required more time than was available. The remaining release choices were therefore either to leave roughly 30% unknown or use the first fallback method.

The first method was judged broadly equivalent to the old branch 3.1 and therefore no worse than an approach used in previously released data. Leaving the fallback out would also make unknowns jump from roughly a quarter in v2.6 to 30–40% in v2.7 despite improvements elsewhere in the model.

Impact / implications

  • Release v2.7 with roughly 3% unknown logistics-hub volume rather than roughly 30%.
  • Treat the MDIC/known-pattern method as a pragmatic release-cycle fallback, not necessarily the final long-term solution.
  • Preserve the distinction between the higher-confidence ~70% solved directly by the improved logistics mapping and the fallback-assigned volume.
  • Quality flags are crucial for communicating these different confidence levels and the methodological differences between newer and older years.
  • Update internal documentation and FAQs to explain the fallback and quality flags.
  • Revisit the more ambitious alternatives when the team next returns to the Brazil Soy model and/or through continued supply-shed work.

References

2026 08 — Run 2022 with v2.7 for QA, but do not release the revised 2022 data yet

Decision

Run the 2022 supply chain through the v2.7 methodology for QA/comparison purposes, but do not publish a revised 2022 dataset as part of this release. The team would like to revisit historical reprocessing in a future update when there is more capacity.

Why

2022 provides a useful like-for-like year for comparing v2.6 with v2.7 and understanding the effect of the new methodology. However, the team does not currently have the bandwidth to ensure that the 2022 data is sufficiently correct in order to release it. The team also noted a known bug in the old years concerning the treatment of the EU in the MDIC stitching. Fixing that bug is not considered a high priority in this release cycle, since we expect users will to focus primarily on the newly released years.

Impact / implications

  • Keep the 2022 v2.7 run as internal QA/comparison evidence rather than a public release.
  • Do not spend current release-cycle capacity fixing that historical-only EU MDIC stitching issue.
  • Revisit whether to back-process and republish 2022 (and potentially earlier years) in a future Brazil Soy update.

Source: Team decision, 2026-08; Slack discussion on 2026-08-13.

2026 08 — ZDC ownership treatment for Glencore / Viterra / Bunge

Decision

For the Brazil Soy ZDC assessment, treat the Glencore entities appearing in the trade data as equivalent to Viterra for 2023 and 2024, and as part of Bunge for 2025 following the Viterra–Bunge merger.

Why

Viterra itself did not appear by name in the trade data, while Glencore-named entities did. The assessment therefore needed a practical ownership interpretation that followed the corporate history and the way Forest500 profiles were represented.

Impact / implications

  • ZDC assessment results for those years should use this ownership interpretation consistently.
  • Future maintainers revisiting historical results should avoid treating the Glencore-labelled records as unrelated companies simply because the trade-data names differ from the assessed company name.

Source: Slack, 2026-08-13.

2026 08 — Add NOVAAGRI → Toyota Tsusho as an exporter-group mapping

Decision

Link NOVAAGRI INFRA-ESTRUTURA DE ARMAZENAGEM E ESCOAMENTO AGRICOL to Toyota Tsusho as an exporter-group mapping.

Why

NOVAAGRI is wholly owned by Toyota Tsusho. Although the discussion noted that Forest500 assessed Toyota only for Rubber and Leather at the time, the team still considered the ownership link useful to encode.

Impact / implications

  • The exporter grouping/ownership mapping should preserve the NOVAAGRI → Toyota Tsusho relationship.
  • Forest500 commodity coverage should not be mistaken for evidence that the ownership link is absent; the mapping is based on ownership, not on whether that commodity was assessed by Forest500.

Source: Slack, 2026-08-13.

2026 08 — Release Brazil Soy without a new Data Release Package

Decision

Assume there will be no new Data Release Package for Brazil Soy for this update, and release using the existing method instead.

Why

Given the project timeline, introducing the new Data Release Package approach was judged unnecessary for this release; the existing release method was considered sufficient.

Impact / implications

  • Do not make completion of a new Data Release Package a blocker for this Brazil Soy release.
  • Release engineering/documentation should follow the existing release route for this update.
  • This is a decision for the current update, not necessarily a permanent rejection of Data Release Packages for future Brazil Soy releases.

Source: Slack, 2026-08-26.

2026 08 — FAQ can move to stakeholder review before Data Flags content is finished

Decision

Proceed with stakeholder review of the FAQ (including Paula, Andre, and Gisele) even though the Data Flags material is not yet complete. The working direction is for Data Flags to replace the existing “Branch harmonization” description, with cross-project alignment sought with the Indonesia Palm update.

Why

The FAQ was considered ready enough for useful review, and waiting for the Data Flags wording would unnecessarily delay feedback on the rest of the material.

Impact / implications

  • Feedback can be collected before the Data Flags section is final.
  • The eventual Data Flags explanation should be checked for consistency with related commodity-update work rather than developed in isolation.

Source: Slack, 2026-08-26.

2026 06 — Use Asana as the operational project record

Decision

Treat the Asana board as the primary record of work and scope. Outcomes from the kick-off were integrated directly into tasks, sub-tasks, dates, and the “Additions to Scope” area rather than written up as a separate set of kick-off notes. Status updates are to be written after context meetings; meeting notes can also be added under Asana Messages when useful.

Why

The team had already translated the kick-off discussion directly into actionable project artefacts. Keeping those items in the board avoids maintaining a separate source of truth for the same decisions and actions.

Impact / implications

  • When reconstructing why work was done, check Asana tasks, sub-tasks, dates, scope additions, status updates, and Messages rather than expecting standalone meeting minutes.
  • Future maintainers should update the board when scope or implementation decisions change, so the operational record stays aligned with the model work.

Source: Slack, 2026-06-17.

2026 06 — Named subject-matter support for GHG and broader model questions

Decision

Carina was added as the expert contact for GHG-related questions, and Erasmus was added directly to the project roles as someone the team could reach out to during the update.

Why

The team wanted the project roles to make specialist support explicit rather than leave potentially useful contributors as tentative or implicit contacts.

Impact / implications

Future work on GHG-related questions should recognise Carina as the designated expert in the project setup, while Erasmus was deliberately included as a named resource for the update.

Source: Slack, 2026-06-15 to 2026-06-16.