Main
View or edit on GitHub
This page is synchronized from trase/models/brazil/soy_supply_sheds/main.ipynb. Last modified on 2026-05-07 15:52 CEST by Nicolas Martin.
Please view or edit the original file there; changes should be reflected here after a midnight build (CET time),
or manually triggering it with a GitHub action (link).
from trase.tools.sps import SupplyChain
sc = SupplyChain("brazil/soy_supply_sheds", year=2023)
sc.preparation()
sc.load()
df_solution = sc.run()
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/soy/sei_pcs/v2.7.0/supply_zone/SEIPCS_BRAZIL_SOY_PARTIAL_BRANCHES_V27.csv[0m
[97m took 0.2 seconds[0m
[1mRunning pre-processing for Branches[0m
Report: Before processing[0m
| Row Count: 850[0m
| Sum of vol: 118,593,571[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
Report: After processing[0m
| Row Count: 850[0m
| Sum of vol: 119,282,969[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/branches.csv[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/spatial/boundaries/gadm/gadm41_BRA_1.json[0m
[97m took 0.2 seconds[0m
[1mRunning pre-processing for BrazilBoundaries[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/brazil.geojson[0m
[1mRunning pre-processing for Brazil[0m
Report: Before processing[0m
| Row Count: 0[0m
Report: After processing[0m
| Row Count: 27[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/brazil.csv[0m
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/spatial/boundaries/ibge/2023/br_municipalities_wgs84_2023.geojson[0m
[97m took 8.2 seconds[0m
[1mRunning pre-processing for MunicipalityBoundaries[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/municipality.geojson[0m
[97mWriting data to disk took 7.8 seconds[0m
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/dictionaries/municipality.csv[0m
[97m took 0.2 seconds[0m
[1mRunning pre-processing for MunicipalityNames[0m
Report: Before processing[0m
| Row Count: 5,624[0m
Report: After processing[0m
| Row Count: 5,624[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/municipality_names.csv[0m
[1mRunning pre-processing for Municipality[0m
Report: Before processing[0m
| Row Count: 0[0m
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:141
[97mProcessing data took 67.2 seconds[0m
Report: After processing[0m
| Row Count: 5,570[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/municipality.csv[0m
[97mWriting data to disk took 22.0 seconds[0m
[97mLoading data from disk took 5.5 seconds[0m
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/soy/indicators/out/q4_2023/soy_production_IBGE_2003_2023_multilevel.csv[0m
[97m took 0.2 seconds[0m
[1mRunning pre-processing for Production[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
Report: Before processing[0m
| Row Count: 117,390[0m
| Sum of vol: 3,657,866,006[0m
| Sum of YEAR: 236,306,070[0m
Report: After processing[0m
| Row Count: 2,603[0m
| Sum of vol: 152,144,238[0m
| Sum of YEAR: 5,265,869[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/production.csv[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/logistics/abiove/out/CRUSHING_FACILITIES_2003_2024.csv[0m
[97m took 0.2 seconds[0m
[1mRunning pre-processing for CrushingDemand[0m
Report: Before processing[0m
| Row Count: 2,052[0m
Report: Filtered to 2022[0m
| Row Count: 107[0m
Report: Consolidate over trase_id/name[0m
| Row Count: 93[0m
Report: After processing[0m
| Row Count: 93[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/crushing_demand.csv[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:234: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_soybean_cake_branches["vol_beans_to_make_cake"] = (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:247: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_soybean_oil_branches["vol_beans_to_make_oil"] = (
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:256
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:317: FutureWarning: The provided callable <built-in function sum> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
.agg({"vol_based_on_capacity": sum, "capacity": sum})
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:317: FutureWarning: The provided callable <built-in function sum> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
.agg({"vol_based_on_capacity": sum, "capacity": sum})
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:334: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_excess["excess_vol"] = (
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:322
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:357
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/logistics/silos/silo_map_v3/silos_consolidated_capacity_brazil_2024_1.geojson[0m
[97m took 0.4 seconds[0m
[1mRunning pre-processing for SilosCapacities[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/silos.geojson[0m
[1mRunning pre-processing for Silos[0m
Report: Before processing[0m
| Row Count: 0[0m
Report: After processing[0m
| Row Count: 2,594[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/silos.csv[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/logistics/antt/out/antt_train_flows_station_to_station_2023.csv[0m
[97m took 0.1 seconds[0m
[1mRunning pre-processing for TrainFlows[0m
Report: Before processing[0m
| Row Count: 525[0m
| Sum of vol: 43,223,696[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:483: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
df = df.fillna("")
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/preparation.py:501: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
df = df.fillna("")
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Empty DataFrame
Columns: [station_origin, state_origin, count]
Index: []
Empty DataFrame
Columns: [station_destination, state_destination, count]
Index: []
Report: After processing[0m
| Row Count: 60[0m
| Sum of vol: 43,489,866[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/train_flows.csv[0m
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/logistics/cost_distance_matrices/v2/br_cost_matrix_osrm_2023_v2.csv[0m
[97m took 28.2 seconds[0m
[1mRunning pre-processing for Cost[0m
[97mReading /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/downloaded/cost.csv into memory took 62.0 seconds[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
Report: Before processing[0m
| Row Count: 31,024,900[0m
| Sum of cost: 57,512,903,076[0m
[97mPre-processing data took 234.5 seconds[0m
Report: Filtered out municipalities not part of the supply chains (production, silos, crushing)[0m
| Row Count: 4,382,280[0m
| Sum of cost: 5,786,004,038[0m
[97mProcessing data took 19.8 seconds[0m
Report: After processing[0m
| Row Count: 4,382,280[0m
| Sum of cost: 5,786,004,038[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/cost.csv[0m
[97mWriting data to disk took 10.7 seconds[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Extracting data from source ...[0mDownloading s3://trase-storage/brazil/logistics/freight/distance_cutoff.csv[0m
[97m took 0.1 seconds[0m
[1mRunning pre-processing for DistanceCutoff[0m
Report: Before processing[0m
| Row Count: 27[0m
| Sum of cutoff: 17,600[0m
Report: After processing[0m
| Row Count: 27[0m
| Sum of cutoff: 17,600[0m
[32mWritten /mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/2023/prepared/distance_cutoff.csv[0m
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:397: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df[column_name] = _cast_series_to_type(df[column_name], python_type)
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/tools/etl/pandas_wrapper.py:162: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
df[column].replace("", np.nan, inplace=True)
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:38
total production 152144238.0
excluded production not within range of silos 98453.0
included production 152045785.0
excluded CNPJ demand not within range of production 0.0
total cnpj silos demand 65587201.52003677
excluded capacity for silos not within range of production 0.0
total aggregator silos demand 86458583.47996324
LP1 supply 152045785.0
LP1 demand 152045785.0
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/dc88bfd0633f46e4a944ccc4dc325d82-pulp.lp -o /tmp/dc88bfd0633f46e4a944ccc4dc325d82-pulp.sol
Reading problem data from '/tmp/dc88bfd0633f46e4a944ccc4dc325d82-pulp.lp'...
3863 rows, 931492 columns, 1862984 non-zeros
497601 lines were read
GLPK Simplex Optimizer 5.0
3863 rows, 931492 columns, 1862984 non-zeros
Preprocessing...
3847 rows, 931484 columns, 1862960 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 3847
0: obj = 8.924440417e+09 inf = 7.536e+07 (546)
524: obj = 1.769386277e+10 inf = 5.948e+07 (432) 2
1005: obj = 2.366476720e+10 inf = 4.879e+07 (432) 1
1479: obj = 2.635295176e+10 inf = 4.327e+07 (363) 2
1979: obj = 3.214328109e+10 inf = 3.300e+07 (237) 2
2603: obj = 5.002851328e+10 inf = 1.539e+07 (110) 4
3113: obj = 5.649109063e+10 inf = 1.032e+07 (50) 3
3575: obj = 6.454969184e+10 inf = 7.025e+06 (23) 4
3996: obj = 6.865881841e+10 inf = 4.774e+06 (7) 3
4263: obj = 7.010137007e+10 inf = 3.547e+06 (3) 2
LP HAS NO PRIMAL FEASIBLE SOLUTION
glp_simplex: unable to recover undefined or non-optimal solution
If you need actual output for non-optimal solution, use --nopresol
Time used: 44.2 secs
Memory used: 643.1 Mb (674342636 bytes)
Writing basic solution to '/tmp/dc88bfd0633f46e4a944ccc4dc325d82-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/7f1a247ab55d4c679bd731a41fd22fd8-pulp.lp -o /tmp/7f1a247ab55d4c679bd731a41fd22fd8-pulp.sol
Reading problem data from '/tmp/7f1a247ab55d4c679bd731a41fd22fd8-pulp.lp'...
5150 rows, 932779 columns, 2797050 non-zeros
629775 lines were read
GLPK Simplex Optimizer 5.0
5150 rows, 932779 columns, 2797050 non-zeros
Preprocessing...
5134 rows, 932779 columns, 2797034 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 5134
0: obj = 0.000000000e+00 inf = 1.503e+08 (1287)
400: obj = 5.650554482e+16 inf = 9.377e+07 (887)
825: obj = 8.670464065e+16 inf = 6.357e+07 (462)
1271: obj = 1.456906287e+17 inf = 4.584e+06 (16)
1287: obj = 1.502749634e+17 inf = 0.000e+00 (0)
Perturbing LP to avoid stalling [1486]...
* 2171: obj = 1.502749634e+17 inf = 0.000e+00 (733896) 8
* 2928: obj = 1.342344864e+17 inf = 0.000e+00 (717683) 4
* 3572: obj = 8.393861816e+16 inf = 0.000e+00 (461470) 1
* 4178: obj = 7.333743070e+16 inf = 0.000e+00 (285154)
* 4765: obj = 6.736655906e+16 inf = 0.000e+00 (230274) 1
* 5359: obj = 5.703883020e+16 inf = 0.000e+00 (220203) 1
* 5964: obj = 3.827016078e+16 inf = 0.000e+00 (151225) 2
* 6517: obj = 2.187127643e+16 inf = 0.000e+00 (195997) 1
* 7025: obj = 1.718795499e+16 inf = 0.000e+00 (121246) 3
* 7511: obj = 1.013943890e+16 inf = 0.000e+00 (157789) 3
* 7899: obj = 8.048118120e+15 inf = 0.000e+00 (169195) 2
* 8292: obj = 6.499160805e+15 inf = 0.000e+00 (217372) 3
* 8660: obj = 4.334295417e+15 inf = 0.000e+00 (186985) 3
* 9054: obj = 3.546892020e+15 inf = 0.000e+00 (203198) 3
* 9349: obj = 3.546891560e+15 inf = 0.000e+00 (348167) 2
* 9671: obj = 3.546891024e+15 inf = 0.000e+00 (125965) 3
* 10024: obj = 3.546890523e+15 inf = 0.000e+00 (122913) 3
* 10323: obj = 3.546890303e+15 inf = 0.000e+00 (146188) 3
* 10661: obj = 3.546890000e+15 inf = 0.000e+00 (87068) 3
* 11093: obj = 3.546889638e+15 inf = 0.000e+00 (73564) 3
* 11441: obj = 3.546889533e+15 inf = 0.000e+00 (75174) 4
* 11918: obj = 3.546889393e+15 inf = 0.000e+00 (9894) 3
* 12391: obj = 3.546889347e+15 inf = 0.000e+00 (8565) 4
* 12992: obj = 3.546889323e+15 inf = 0.000e+00 (1508) 4
Removing LP perturbation [13186]...
* 13186: obj = 3.546889321e+15 inf = 0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
Time used: 138.5 secs
Memory used: 794.2 Mb (832768100 bytes)
Writing basic solution to '/tmp/7f1a247ab55d4c679bd731a41fd22fd8-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit 5317682.876004036
Empty DataFrame
Columns: [silo_trase_id, vol, vol_deficit, cnpj_demand_vol, non_cnpj_vol]
Index: []
vol vol_deficit cnpj_demand_vol non_cnpj_vol \
silo_trase_id
BR-1301902 1770821 1770821 1770821 0
BR-1506807 1662178 1462239 1662178 0
BR-3203908 604140 587068 604140 0
BR-3205309 1619051 1497526 1619051 0
reduction_decrease
silo_trase_id
BR-1301902 1770821
BR-1506807 1462239
BR-3203908 587068
BR-3205309 1497526
total deficit 5317682.876004036
total reduction decrease 5317659.40081007
total effective deficit 23.475193965248764
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:658
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:659
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:678: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:688: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:692: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["cnpj_demand_vol"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:708: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:709: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/deda8cc3cb9547c1b9f0827e48088a4c-pulp.lp -o /tmp/deda8cc3cb9547c1b9f0827e48088a4c-pulp.sol
Reading problem data from '/tmp/deda8cc3cb9547c1b9f0827e48088a4c-pulp.lp'...
3863 rows, 931492 columns, 1862984 non-zeros
497594 lines were read
GLPK Simplex Optimizer 5.0
3863 rows, 931492 columns, 1862984 non-zeros
Preprocessing...
3847 rows, 931484 columns, 1862960 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 3847
0: obj = 8.924441560e+09 inf = 7.536e+07 (546)
532: obj = 1.770250982e+10 inf = 5.944e+07 (432) 2
1021: obj = 2.373440997e+10 inf = 4.870e+07 (432) 1
1505: obj = 2.780301033e+10 inf = 4.065e+07 (362) 2
2022: obj = 3.259541412e+10 inf = 3.258e+07 (225) 3
2646: obj = 5.042848746e+10 inf = 1.510e+07 (101) 3
3165: obj = 5.758043151e+10 inf = 9.927e+06 (45) 3
3632: obj = 6.485089608e+10 inf = 6.852e+06 (22) 4
4069: obj = 6.935742494e+10 inf = 4.341e+06 (5) 3
4263: obj = 7.010137075e+10 inf = 3.547e+06 (3) 2
LP HAS NO PRIMAL FEASIBLE SOLUTION
glp_simplex: unable to recover undefined or non-optimal solution
If you need actual output for non-optimal solution, use --nopresol
Time used: 43.3 secs
Memory used: 643.1 Mb (674342636 bytes)
Writing basic solution to '/tmp/deda8cc3cb9547c1b9f0827e48088a4c-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/e7dfe824073e4589ae2c23334515e22a-pulp.lp -o /tmp/e7dfe824073e4589ae2c23334515e22a-pulp.sol
Reading problem data from '/tmp/e7dfe824073e4589ae2c23334515e22a-pulp.lp'...
5150 rows, 932779 columns, 2797050 non-zeros
629754 lines were read
GLPK Simplex Optimizer 5.0
5150 rows, 932779 columns, 2797050 non-zeros
Preprocessing...
5134 rows, 932779 columns, 2797034 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 5134
0: obj = 0.000000000e+00 inf = 1.503e+08 (1287)
403: obj = 5.652990809e+16 inf = 9.375e+07 (884)
831: obj = 8.686030996e+16 inf = 6.341e+07 (456)
1282: obj = 1.478399484e+17 inf = 2.435e+06 (5)
1287: obj = 1.502749634e+17 inf = 0.000e+00 (0)
Perturbing LP to avoid stalling [1486]...
* 2196: obj = 1.502749634e+17 inf = 0.000e+00 (751687) 9
* 2989: obj = 1.318470873e+17 inf = 0.000e+00 (685301) 3
* 3652: obj = 8.220299498e+16 inf = 0.000e+00 (425346) 1
* 4269: obj = 7.247013892e+16 inf = 0.000e+00 (276704)
* 4872: obj = 6.588046598e+16 inf = 0.000e+00 (229050) 1
* 5481: obj = 5.347897147e+16 inf = 0.000e+00 (193583) 1
* 6091: obj = 3.230972127e+16 inf = 0.000e+00 (176396) 2
* 6657: obj = 2.032645868e+16 inf = 0.000e+00 (195754) 2
* 7159: obj = 1.592799303e+16 inf = 0.000e+00 (221399) 3
* 7650: obj = 9.344756157e+15 inf = 0.000e+00 (125144) 3
* 8034: obj = 7.694778834e+15 inf = 0.000e+00 (312569) 2
* 8441: obj = 5.113577653e+15 inf = 0.000e+00 (289458) 3
* 8805: obj = 3.665366361e+15 inf = 0.000e+00 (233063) 3
* 9182: obj = 3.546891809e+15 inf = 0.000e+00 (317135) 3
* 9480: obj = 3.546891354e+15 inf = 0.000e+00 (257234) 3
* 9844: obj = 3.546890707e+15 inf = 0.000e+00 (139612) 3
* 10169: obj = 3.546890410e+15 inf = 0.000e+00 (273170) 3
* 10496: obj = 3.546890089e+15 inf = 0.000e+00 (160005) 3
* 10878: obj = 3.546889707e+15 inf = 0.000e+00 (43745) 3
* 11273: obj = 3.546889552e+15 inf = 0.000e+00 (50883) 3
* 11666: obj = 3.546889454e+15 inf = 0.000e+00 (19212) 4
* 12185: obj = 3.546889344e+15 inf = 0.000e+00 (12912) 3
* 12735: obj = 3.546889307e+15 inf = 0.000e+00 (4879) 4
Removing LP perturbation [13209]...
* 13209: obj = 3.546889299e+15 inf = 0.000e+00 (0) 4
OPTIMAL LP SOLUTION FOUND
Time used: 135.7 secs
Memory used: 794.2 Mb (832768100 bytes)
Writing basic solution to '/tmp/e7dfe824073e4589ae2c23334515e22a-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:190
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:196
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:208
LP1 solution 146728117.18207252
Production/LP1 matching
_matched
both 146728110
left_only 5317674
right_only 6
Name: vol, dtype: int64
LP1/exports matching
_matched
both 60269542
left_only 91776249
right_only 5317659
Name: vol, dtype: int64
demand from crushing facilities 53105366.59999999
LP2 supply 91776249.46946177
LP2 demand 91776249.46946177
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/7f9b0e6e477f4464aac3a7924b7ac3b2-pulp.lp -o /tmp/7f9b0e6e477f4464aac3a7924b7ac3b2-pulp.sol
Reading problem data from '/tmp/7f9b0e6e477f4464aac3a7924b7ac3b2-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2502: obj = 5.317692702e+15 inf = 0.000e+00 (3645) 8
* 2965: obj = 5.317692505e+15 inf = 0.000e+00 (0) 3
OPTIMAL LP SOLUTION FOUND
Time used: 6.8 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/7f9b0e6e477f4464aac3a7924b7ac3b2-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
300 BR-5103403 3.623824e+05
302 BR-5103502 1.691393e+05
314 BR-5106224 2.052981e+06
330 BR-5107602 2.251629e+06
hub_trase_id vol vol_deficit vol_hub vol_crushing \
151 BR-5103502 74018.71 1.691393e+05 2.431580e+05 0.000000e+00
155 BR-5105259 2259268.00 2.364556e+00 1.620862e+06 6.384082e+05
157 BR-5106224 2678947.60 2.052981e+06 3.420920e+06 1.311008e+06
vol_non_crushing
151 2.431580e+05
155 1.620862e+06
157 3.420920e+06
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 362382 0 1206127 0
BR-5107602 4221447 2251629 585794 4221447 585794
reduction_decrease
hub_trase_id
BR-1301902 481544
BR-5103403 362382
BR-5107602 1665834
total deficit 5317694.599753675
total reduction decrease 2509764.363246593
total effective deficit 2807930.236507082
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/938d58fd3ccc4570bee88b3f66b76ac6-pulp.lp -o /tmp/938d58fd3ccc4570bee88b3f66b76ac6-pulp.sol
Reading problem data from '/tmp/938d58fd3ccc4570bee88b3f66b76ac6-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2609: obj = 5.317690771e+15 inf = 0.000e+00 (1104) 8
* 2910: obj = 5.317690711e+15 inf = 0.000e+00 (0) 2
OPTIMAL LP SOLUTION FOUND
Time used: 6.5 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/938d58fd3ccc4570bee88b3f66b76ac6-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
300 BR-5103403 5.004594e+05
314 BR-5106224 5.793737e+05
330 BR-5107602 2.986671e+06
370 BR-5218805 7.696257e+05
hub_trase_id vol vol_deficit vol_hub vol_crushing \
47 BR-5105259 2.259266e+06 2.000000 1.620862e+06 6.384082e+05
48 BR-5106224 2.099574e+06 579373.700000 3.420920e+06 1.311008e+06
57 BR-5218805 3.685269e+06 769625.720000 8.539930e+05 3.600902e+06
178 BR-5108501 1.108883e+06 1.045162 0.000000e+00 1.048807e+06
vol_non_crushing
47 1.620860e+06
48 1.367939e+06
57 8.539926e+05
178 6.007749e+04
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 500459 0 1206127 0
BR-5107602 4221447 2986671 585794 4221447 0
reduction_decrease
hub_trase_id
BR-1301902 481544
BR-5103403 500459
BR-5107602 2986671
total deficit 5317689.194502469
total reduction decrease 3968677.663000183
total effective deficit 1349011.531502286
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/1f62125752b64c1b9e2dac6d2f89c5d8-pulp.lp -o /tmp/1f62125752b64c1b9e2dac6d2f89c5d8-pulp.sol
Reading problem data from '/tmp/1f62125752b64c1b9e2dac6d2f89c5d8-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2622: obj = 5.317690069e+15 inf = 0.000e+00 (880) 8
* 2825: obj = 5.317690050e+15 inf = 0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
Time used: 6.3 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/1f62125752b64c1b9e2dac6d2f89c5d8-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
44 BR-3170206 6.513478e+04
300 BR-5103403 5.087864e+05
302 BR-5103502 1.774684e+03
314 BR-5106224 1.742741e+05
330 BR-5107602 3.144632e+06
370 BR-5218805 7.163387e+05
374 BR-5220405 2.251892e+05
hub_trase_id vol vol_deficit vol_hub vol_crushing \
13 BR-5105259 2259264.000 2.000000 1.620862e+06 6.384082e+05
14 BR-5106224 1925299.800 174274.100000 3.420920e+06 1.311008e+06
52 BR-5108501 1108882.200 1.000000 0.000000e+00 1.048807e+06
102 BR-5102637 1165812.000 4.107178 9.469968e+05 1.915225e+05
104 BR-5103502 74018.710 1774.684321 2.431580e+05 0.000000e+00
124 BR-3170206 1868795.968 65134.783934 8.560725e+05 9.302519e+05
vol_non_crushing
13 1.620858e+06
14 7.885655e+05
52 6.007644e+04
102 9.742937e+05
104 7.579339e+04
124 1.003679e+06
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-5218805 3600902 716338 853993 3600902 84366
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 508786 0 1206127 0
BR-5107602 4221447 3144632 585794 4221447 0
BR-5220405 471963 225189 48294 471963 91284
reduction_decrease
hub_trase_id
BR-5218805 631971
BR-1301902 481544
BR-5103403 508786
BR-5107602 3144632
BR-5220405 133905
total deficit 5317694.353780562
total reduction decrease 4900842.282873297
total effective deficit 416852.07090726495
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/dc91c6a9c1c54df889225c79d256aa3d-pulp.lp -o /tmp/dc91c6a9c1c54df889225c79d256aa3d-pulp.sol
Reading problem data from '/tmp/dc91c6a9c1c54df889225c79d256aa3d-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2654: obj = 5.317689915e+15 inf = 0.000e+00 (393) 8
* 2806: obj = 5.317689902e+15 inf = 0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
Time used: 6.2 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/dc91c6a9c1c54df889225c79d256aa3d-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
44 BR-3170206 1.851331e+05
300 BR-5103403 5.368474e+05
314 BR-5106224 3.134690e+04
330 BR-5107602 3.181954e+06
370 BR-5218805 6.669043e+05
374 BR-5220405 2.339451e+05
hub_trase_id vol vol_deficit vol_hub vol_crushing \
8 BR-5105259 2259262.000 2.00 1.620862e+06 6.384082e+05
9 BR-5106224 1893952.900 31346.90 3.420920e+06 1.311008e+06
19 BR-5108501 1108881.200 1.00 0.000000e+00 1.048807e+06
46 BR-3170206 1683662.848 185133.12 8.560725e+05 9.302519e+05
vol_non_crushing
8 1.620856e+06
9 6.142914e+05
19 6.007544e+04
46 9.385441e+05
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-5218805 3600902 666904 853993 3600902 0
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 536847 0 1206127 0
BR-5107602 4221447 3181954 585794 4221447 0
BR-5220405 471963 233945 48294 471963 0
reduction_decrease
hub_trase_id
BR-5218805 666904
BR-1301902 481544
BR-5103403 536847
BR-5107602 3181954
BR-5220405 233945
total deficit 5317688.860742953
total reduction decrease 5101197.496873297
total effective deficit 216491.36386965588
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/67354b59243e4acfbcd4295285d69938-pulp.lp -o /tmp/67354b59243e4acfbcd4295285d69938-pulp.sol
Reading problem data from '/tmp/67354b59243e4acfbcd4295285d69938-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2621: obj = 5.317689839e+15 inf = 0.000e+00 (373) 8
* 2760: obj = 5.317689827e+15 inf = 0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
Time used: 6.1 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/67354b59243e4acfbcd4295285d69938-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
44 BR-3170206 9.583130e+04
300 BR-5103403 5.679664e+05
302 BR-5103502 2.868926e+02
314 BR-5106224 7.723400e+03
330 BR-5107602 3.238632e+06
370 BR-5218805 6.855521e+05
374 BR-5220405 2.401374e+05
hub_trase_id vol vol_deficit vol_hub vol_crushing \
7 BR-5105259 2259260.000 2.000000 1.620862e+06 6.384082e+05
8 BR-5106224 1886229.500 7723.400000 3.420920e+06 1.311008e+06
20 BR-3170206 1587831.548 95831.300000 8.560725e+05 9.302519e+05
91 BR-5103502 74018.710 286.892561 2.431580e+05 0.000000e+00
vol_non_crushing
7 1.620854e+06
8 5.829445e+05
20 7.534109e+05
91 7.430560e+04
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-5218805 3600902 685552 853993 3600902 0
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 567966 0 1206127 0
BR-5107602 4221447 3238632 585794 4221447 0
BR-5220405 471963 240137 48294 471963 0
reduction_decrease
hub_trase_id
BR-5218805 685552
BR-1301902 481544
BR-5103403 567966
BR-5107602 3238632
BR-5220405 240137
total deficit 5317691.744296378
total reduction decrease 5213835.012316099
total effective deficit 103856.73198027909
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/624733e20df04759ac7df21dbe67bab4-pulp.lp -o /tmp/624733e20df04759ac7df21dbe67bab4-pulp.sol
Reading problem data from '/tmp/624733e20df04759ac7df21dbe67bab4-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2632: obj = 5.317689801e+15 inf = 0.000e+00 (522) 8
* 2762: obj = 5.317689792e+15 inf = 0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
Time used: 6.1 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/624733e20df04759ac7df21dbe67bab4-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
deficit hubs/crushing hub_trase_id vol_deficit
0 BR-1301902 4.815448e+05
44 BR-3170206 4.045931e+04
282 BR-5008305 3.851782e+04
300 BR-5103403 5.803134e+05
314 BR-5106224 3.121600e+03
330 BR-5107602 3.238632e+06
370 BR-5218805 6.949489e+05
374 BR-5220405 2.401374e+05
hub_trase_id vol vol_deficit vol_hub vol_crushing \
6 BR-5105259 2.259258e+06 2.000000 1.620862e+06 6.384082e+05
7 BR-5106224 1.883108e+06 3121.600000 3.420920e+06 1.311008e+06
16 BR-3170206 1.547372e+06 40459.310000 8.560725e+05 9.302519e+05
121 BR-5102637 1.172745e+06 2.736510 9.469968e+05 1.915225e+05
180 BR-5008305 5.622738e+05 38517.815517 0.000000e+00 5.472070e+05
vol_non_crushing
6 1.620852e+06
7 5.752211e+05
16 6.575796e+05
121 9.812252e+05
180 5.358459e+04
vol vol_deficit vol_hub vol_crushing vol_non_crushing \
hub_trase_id
BR-5218805 3600902 694948 853993 3600902 0
BR-1301902 524403 481544 0 524403 0
BR-5103403 1206127 580313 0 1206127 0
BR-5107602 4221447 3238632 585794 4221447 0
BR-5220405 471963 240137 48294 471963 0
reduction_decrease
hub_trase_id
BR-5218805 694948
BR-1301902 481544
BR-5103403 580313
BR-5107602 3238632
BR-5220405 240137
total deficit 5317694.341603915
total reduction decrease 5235578.852316099
total effective deficit 82115.48928781599
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:736
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:737
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:744
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:763: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_enough["vol"] -= df_reduce_enough["vol_deficit"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["reduction_decrease"] = df_reduce_not_enough["vol_deficit"] - (
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:777: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_reduce_not_enough["vol"] = df_reduce_not_enough["vol_crushing"]
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:793: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["demand_share"] = df_increase["vol"] / increase_total_demand
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:794: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_increase["vol"] += df_increase["demand_share"] * total_effective_deficit
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:265: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
.apply(sum)
GLPSOL--GLPK LP/MIP Solver 5.0
Parameter(s) specified in the command line:
--cpxlp /tmp/f491b6db591c451aafccabd37828e966-pulp.lp -o /tmp/f491b6db591c451aafccabd37828e966-pulp.sol
Reading problem data from '/tmp/f491b6db591c451aafccabd37828e966-pulp.lp'...
1598 rows, 231610 columns, 694640 non-zeros
156144 lines were read
GLPK Simplex Optimizer 5.0
1598 rows, 231610 columns, 694640 non-zeros
Preprocessing...
1598 rows, 231610 columns, 694640 non-zeros
Scaling...
A: min|aij| = 1.000e+00 max|aij| = 1.000e+00 ratio = 1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 1598
0: obj = 0.000000000e+00 inf = 9.178e+07 (190)
190: obj = 9.177624947e+16 inf = 0.000e+00 (0)
* 2669: obj = 5.317689769e+15 inf = 0.000e+00 (475) 9
* 2781: obj = 5.317689761e+15 inf = 0.000e+00 (0)
OPTIMAL LP SOLUTION FOUND
Time used: 6.1 secs
Memory used: 197.4 Mb (206945984 bytes)
Writing basic solution to '/tmp/f491b6db591c451aafccabd37828e966-pulp.sol'...
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/lp.py:522: FutureWarning: The provided callable <built-in function sum> is currently using np.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string np.sum instead.
df_allocation.groupby(["source"])["quantity"].apply(sum).to_frame()
LP2 solution 86458560.93156005
LP1/LP2 matching
matched_lp_2
False 5317732
True 86458539
Name: vol, dtype: int64
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:398
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:405
Use the validate option with pd.merge or pass validate=None to suppress this warning
/mnt/custom-file-systems/efs/fs-049d752ef37739434/shared/shared_repos/TRASE/trase/models/brazil/soy_supply_sheds/model.py:415
before 47814376.32923957
LP2/crushed exports matching
_matched
both 17169991
left_only 30644384
right_only 2349230
Name: vol, dtype: int64
import pandas as pd
from trase.tools.aws.aws_helpers import upload_pandas_df_to_s3
df = pd.read_csv(f"df_compare_silos_demand.csv")
upload_pandas_df_to_s3(df, "brazil/soy/sei_pcs/supply_sheds/silo_capacities.csv")
df = pd.read_csv(f"df_lp_solution.csv")
upload_pandas_df_to_s3(df, "brazil/soy/sei_pcs/supply_sheds/silo_supply_sheds.csv")
df = pd.read_csv(f"df_compare_hubs_crushing_demand.csv")
upload_pandas_df_to_s3(
df, "brazil/soy/sei_pcs/supply_sheds/hubs_crushing_capacities.csv"
)
df = pd.read_csv(f"df_solution_crushed.csv")
upload_pandas_df_to_s3(df, "brazil/soy/sei_pcs/supply_sheds/crushing_supply_sheds.csv")
df = pd.read_csv(f"df_solution_exports.csv")
upload_pandas_df_to_s3(df, "brazil/soy/sei_pcs/supply_sheds/solution_exports.csv")
df = pd.read_csv(f"df_solution_domestic_unknowns.csv")
upload_pandas_df_to_s3(
df, "brazil/soy/sei_pcs/supply_sheds/solution_domestic_unknowns.csv"
)
- complete
- complete
- complete
- complete
- complete
- complete
import pandas as pd
df_prod = pd.read_csv("2023/prepared/production.csv", sep=";")
print(df_prod["vol"].sum())
for dist in [250, 500, 750, 1000]:
df_domestic_unknowns = pd.read_csv(f"df_solution_domestic_unknowns_{dist}.csv")
df_exports = pd.read_csv(f"df_solution_exports_{dist}.csv")
total_exports = df_exports["vol"].sum()
unknown_exports = df_exports[df_exports["production_trase_id"] == "UNKNOWN"][
"vol"
].sum()
known_exports = total_exports - unknown_exports
total_domestic_unknowns = df_domestic_unknowns["vol"].sum()
print(dist)
print("total exports", int(total_exports))
print("unknown exports", int(unknown_exports))
print("known exports", int(known_exports))
print("domestic/unknown exports", int(total_domestic_unknowns))
print(
"total known exports + unknown exports + domestic",
int(known_exports + total_domestic_unknowns),
)
print()
152144238.0
250
total exports 48728811
unknown exports 26262246
known exports 22466565
domestic/unknown exports 129562502
total known exports + unknown exports + domestic 152029067
500
total exports 52987393
unknown exports 17212947
known exports 35774446
domestic/unknown exports 116323392
total known exports + unknown exports + domestic 152097838
750
total exports 57113473
unknown exports 8487404
known exports 48626068
domestic/unknown exports 103517943
total known exports + unknown exports + domestic 152144011
1000
total exports 60236924
unknown exports 2797061
known exports 57439863
domestic/unknown exports 94704340
total known exports + unknown exports + domestic 152144204
import pandas as pd
df = pd.read_csv("2023/prepared/branches.csv", sep=";")
print(df["vol"].sum())
df.groupby("branch")["vol"].sum().astype(int)
117047810.62339668
branch
1.1 LINK TO FARM MUNICIPALITY WITH PRODUCTION 5969093
1.2 LINK TO FARM MUNICIPALITY NO PRODUCTION 8921830
2.1. LINK TO SILO LIST AND CADASTRO 16316415
2.2. LINK TO SILO CADASTRO ONLY 9786425
3.1.1 LINK TO CRUSH WITH SILO 3509917
3.1.2.1 LINK TO CRUSH WITH 1 SILO IN SAME MUNICIPALITY 2408669
3.1.2.2 LINK TO CRUSH WITH >1 SILO IN SAME MUNICIPALITY 11415226
3.2.1 LINK TO CRUSH NO SILOS IN MUNICIPALITY, USE SICASQ 191264
3.2.2 LINK TO CRUSH NO SILOS IN MUNICIPALITY, USE SUPPLY SHED 1994145
4.1.1 OFFICE IS NOT LOCATED AT A PORT AND TRADES >20,000 T 29759568
4.1.2 OFFICE IS LOCATED AT A PORT 6056089
5. NO SOY ECONOMIC ACTIVITY 390
UNKNOWN - MISSING INFORMATION 20718773
Name: vol, dtype: int64