Skip to content

View or edit on GitHub

This page is synchronized from trase/data/brazil/logistics/silos/silo_map_v3/qa/qa_consolidated_facilities.md. Last modified on 2026-08-05 15:56 CEST by Harry Biddle. Please view or edit the original file there; changes should be reflected here after a midnight build (CET time), or manually triggering it with a GitHub action (link).

Final Agricultural Facility — Consolidation Pipeline

Overview

This notebook details the final soy facility dataset after the process of consolidating SICARM, EOFM (AI/remote-sensing detected), ABIOVE crushing facilities, and previously validated facilities. It checks whether source datasets are represented in the final output, summarizes totals (counts and storage/processing capacity) by source and category, and flags rows that may contain personal (rather than corporate) information.

1. Final dataset at a glance

| | metric | value | |-----|----------------------------------|-------| | 0 | Total facilities | 12953 | | 1 | Missing / invalid geometry | 4 | | 2 | Duplicate idx values | 0 | | 3 | Distinct states (trase_state_id) | 21 | | 4 | Distinct CNPJs | 6797 |

2. Facilities by source

Counts and capacity contributed by each source dataset in the final output.

| | source | n_facilities | total_storage_capacity | total_processing_capacity | |----|----|----|----|----| | 2 | SICARM | 10958 | 187490883\.0 | 0.0 | | 1 | EOFM | 1855 | 11018005.0 | 0.0 | | 3 | SICARM/ABIOVE | 78 | 11386927.0 | 114050.0 | | 0 | ABIOVE | 62 | 0.0 | 97920.0 |

3. Facilities by category (STORAGE vs PROCESSING)

| category | PROCESSING | STORAGE | |---------------|------------|---------| | source | | | | ABIOVE | 62 | 0 | | EOFM | 1 | 1854 | | SICARM | 0 | 10958 | | SICARM/ABIOVE | 78 | 0 |

4. Capacity comparison

Total storage and processing capacity across the final dataset.

| | capacity_type | total | |-----|---------------|----------| | 0 | Raw ABIOVE | 220070.0 | | 1 | Output | 211970.0 |

5. Source representation check

Confirms whether facilities present in the raw SICARM reference dataset are represented (by CNPJ) in the final consolidated output — a sanity check that the pipeline’s spatial/CNAE filtering and trader carve-out did not silently drop legitimate reference facilities.

| | metric | value | |-----|---------------------------------------------|-------| | 0 | Distinct CNPJs in raw SICARM | 7753 | | 1 | Distinct SICARM CNPJs found in final output | 6391 | | 2 | Representation rate | 82.4% |

6. State distribution

| | state | n_facilities | |-----|-------|--------------| | 0 | BR-43 | 3679 | | 1 | BR-51 | 2333 | | 2 | BR-41 | 1728 | | 3 | BR-52 | 922 | | 4 | BR-31 | 842 | | 5 | BR-35 | 772 | | 6 | BR-50 | 753 | | 7 | BR-42 | 609 | | 8 | BR-29 | 439 | | 9 | BR-17 | 240 | | 10 | BR-21 | 183 | | 11 | BR-22 | 152 | | 12 | BR-15 | 123 | | 13 | BR-11 | 110 | | 14 | BR-53 | 54 |

7. Spatial overview