View or edit on GitHub

This page is synchronized from doc/Data-Release-Package.md. Last modified on 2026-08-05 15:56 CEST by Harry Biddle. Please view or edit the original file there; changes should be reflected here after a midnight build (CET time), or manually triggering it with a GitHub action (link).

Data Release Package Specification

A data release package is a fixed set of release-ready files in S3 that the trase.earth website ingests directly (no "main" schema). One package = one context (country + commodity + version).

Location: s3://trase-storage/<country>/<commodity>/data_packages/<country>_<commodity>_v<X_Y_Z>/

Files

File	Format	Required	Purpose
`spatial_entities.parquet`	GeoParquet	Yes	Reference table of spatially-located entities + geometry
`spatial_metrics.parquet`	Parquet	Optional	Multi-level spatial metrics (Impact map)
`supply_chain.parquet`	Parquet	Optional	Trade flows (Sankey / exposures)
`spatial_metrics_metadata.json`	JSON	With metrics	Display / source / citation metadata per metric
`supply_chains_contexts_metadata.json`	JSON	With supply chain	Context: country, commodity, DOI, citation
`supply_chains_column_metadata.json`	JSON	With supply chain	Per-column metadata for the supply chain
`supply_chains_map_columns.json`	JSON	With supply chain	Which supply-chain columns are spatial (map tab)
`downloads/*`	any (e.g. GeoParquet)	Optional	Standalone files for users to download — no foreign-key requirements (see below)

Conventions

trase_id is the global entity key (e.g. ID-WOOD-CONCESSION-0464, ID-5103, ID-14).
year is always present and non-null. Rows are duplicated per year even when unchanged, so attributes and geometry may vary over time. Data files cover a fixed year range (e.g. 2015–2024).
Column names are snake_case; value columns carry a unit suffix (_ha, _tons, _co2, …).
Geometry is GeoParquet, EPSG:4326. spatial_entities carries simplified (preview-resolution) geometry — name the column to make that clear (e.g. simplified_geometry); the full-resolution geometry is published under downloads/.
Metadata files are JSON arrays of objects.
Joins between data files are on trase_id (and year).

`spatial_entities` — one row per `(trase_id, year)`

Column	Type	Notes
`trase_id`	string	Entity key (PK with `year`)
`parent_trase_id`	string	FK → `spatial_entities.trase_id` (self; null at top level)
`node_type_slug`	string	e.g. `wood-pulp-concession`, `kabupaten`, `province`
`node_type_name`	string	Display name of the node type
`name`	string	Entity name
`level`	string	Display level
`year`	int	Non-null
`simplified_geometry`	geometry	Simplified preview geometry, EPSG:4326 (full-resolution lives in `downloads/`)

`spatial_metrics` — one row per `(trase_id, year)`, levels mixed in

Column	Required	Notes
`year`	Yes
`trase_id`	Yes	FK → `spatial_entities`
`level`	No	e.g. concession / kabupaten / province
`<metric>_<unit>`	No	Value columns, e.g. `annual_deforestation_ha`

`supply_chain` — one row per flow

Column	Required	Notes
`year`	Yes
`production_trase_id`	Yes	FK → `spatial_entities` (producing entity)
`…_trase_id`	No	Other spatial dimensions, FK → `spatial_entities` (e.g. `production_province_trase_id`)
`<dimension>`	No	Non-spatial dimensions, e.g. `exporter_group`, `country_of_destination`
`<value>_<unit>`	No	Value columns, e.g. `volume_raw_tons`, `deforestation_exposure_ha`

Metadata files

spatial_metrics_metadata — one row per metric: backend_name, short_name/long_name, unit, tooltip, description, data_source, references, citation, doi, node_type_slugs.
supply_chains_contexts_metadata — one row per context: context_slug, country_of_production, commodity, doi, citation.
supply_chains_column_metadata — one row per supply-chain column: column_name, node_type_slug, short_name/long_name, unit, type.
supply_chains_map_columns — one row per spatially-explicit supply-chain column: column_name, node_type_slug, column_position.

Additional downloads (`downloads/`)

An optional downloads/ subfolder may hold extra files for users to download — typically the full-resolution geometry that is the precise counterpart to the simplified preview geometry in spatial_entities.

These files are standalone: they have no trase_id, year, or foreign-key requirements, are not joined to the other files, and do not appear in the relationship diagram. Each is otherwise free-form (any columns, plus a geometry column for geometry downloads).

Example (Indonesia wood pulp): downloads/province.parquet, downloads/kabupaten.parquet, and downloads/concessions_{2019,2020_2022,2023}.parquet.

Relationship diagram

This diagram gives an overview of the entire schema. Solid lines are foreign keys (joined on trase_id/year); dotted lines are "describes". (The optional downloads/ files are standalone — no foreign keys — and are not shown.)

erDiagram
    spatial_entities {
        string   trase_id PK
        int      year PK
        string   parent_trase_id FK
        string   node_type_slug
        string   node_type_name
        string   name
        string   level
        geometry geometry
    }
    spatial_metrics {
        int    year
        string trase_id FK
        string level
        float  value_columns "e.g. annual_deforestation_ha"
    }
    supply_chain {
        int    year
        string production_trase_id FK
        string other_trase_id FK "e.g. production_province_trase_id"
        string dimensions "exporter_group, country_of_destination, ..."
        float  value_columns "volume_raw_tons, deforestation_exposure_ha, ..."
    }
    spatial_metrics_metadata {
        string backend_name
        string short_name
        string unit
        string node_type_slugs
    }
    supply_chains_column_metadata {
        string column_name
        string node_type_slug
        string unit
    }
    supply_chains_map_columns {
        string column_name
        string node_type_slug
        int    column_position
    }
    supply_chains_contexts_metadata {
        string context_slug
        string country_of_production
        string commodity
        string doi
    }

    spatial_entities ||--o{ spatial_entities : "parent_trase_id"
    spatial_entities ||--o{ spatial_metrics : "trase_id + year"
    spatial_entities ||--o{ supply_chain : "production_trase_id + year"
    spatial_entities ||--o{ supply_chain : "other_trase_id + year"
    spatial_metrics ||..o{ spatial_metrics_metadata : "described by"
    supply_chain ||..o{ supply_chains_column_metadata : "columns described by"
    supply_chain ||..o{ supply_chains_map_columns : "map columns from"
    supply_chain }o..|| supply_chains_contexts_metadata : "belongs to context"

Data Release Package Specification

Files

Conventions

spatial_entities — one row per (trase_id, year)

spatial_metrics — one row per (trase_id, year), levels mixed in

supply_chain — one row per flow