Merge pull request #84 from EnergySystemsModellingLab/75_notebooks

75 notebooks

Author: dalonsoa

.github/workflows/ci.yml

@@ -9,12 +9,12 @@ on:
   workflow_call:
 
 jobs:
-  test_release:
-    name: Latest MUSE2 release
-    uses: ./.github/workflows/test-with-muse2.yml
-    with:
-      muse2_source: release
-    secrets: inherit
+  # test_release:
+  #   name: Latest MUSE2 release
+  #   uses: ./.github/workflows/test-with-muse2.yml
+  #   with:
+  #     muse2_source: release
+  #   secrets: inherit
 
   test_main:
     name: Current MUSE2 main branch

.pre-commit-config.yaml

@@ -30,6 +30,7 @@ repos:
     rev: 0.9.1
     hooks:
       - id: nbstripout
+        args: [--drop-empty-cells]
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v2.1.0
     hooks:

notebooks/capacity.ipynb

@@ -10,9 +10,7 @@
     "This notebook contains example data processing using the output of an example\n",
     "model.\n",
     "\n",
-    "Output files are mostly in CSV format. The format of output files is documented [here][output-format].\n",
-    "\n",
-    "We begin by loading the `model.toml` input file to get the list of milestone years.\n",
+    "Output files are mostly in CSV format. The format of output files is documented [in MUSE2 documentation][output-format].\n",
     "\n",
     "[output-format]: https://energysystemsmodellinglab.github.io/MUSE2/file_formats/output_files.html"
    ]
@@ -24,20 +22,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import tomllib\n",
-    "\n",
     "import pandas as pd\n",
     "\n",
-    "from muse2_data_analysis.helpers import get_example_input_dir, get_example_output_dir\n",
-    "\n",
-    "INPUT_DIR = get_example_input_dir()\n",
-    "OUTPUT_DIR = get_example_output_dir()\n",
+    "from muse2_data_analysis.helpers import get_example_output_dir\n",
     "\n",
-    "with (INPUT_DIR / \"model.toml\").open(\"rb\") as f:\n",
-    "    model = tomllib.load(f)\n",
-    "\n",
-    "# We need to know the milestone years for processing the assets file\n",
-    "years = model[\"milestone_years\"]"
+    "OUTPUT_DIR = get_example_output_dir()"
    ]
   },
   {
@@ -48,14 +37,13 @@
     "## Load and process output data\n",
     "\n",
     "We next load the output data. In this case, we want to calculate how much capacity was invested in\n",
-    "different processes for different agents. This information can be found in the `assets.csv` output\n",
-    "file.\n",
+    "different processes for different agents. This information can be found in the `asset_capacities.csv`\n",
+    "output file.\n",
     "\n",
-    "The `assets.csv` file contains information about different assets, including when they were\n",
-    "commissioned and decommissioned as well as their capacity. To calculate the overall capacity for a\n",
-    "given agent and process type, we have to process this data. Note that different assets owned by the\n",
-    "same agent may have the same process ID if the agent has reinvested in the same process type in a\n",
-    "different year."
+    "We also need some metadata, like the agent that manages the asset and the process the asset corresponds\n",
+    "to. That time-independent information is contained in the `assets.csv` file. To calculate the overall\n",
+    "capacity for a given agent and process type as a function of milestone year, we have to combine these\n",
+    "two files and process the data."
    ]
   },
   {
@@ -65,28 +53,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# The assets.csv file contains info about which assets were invested in and when\n",
     "assets = pd.read_csv(OUTPUT_DIR / \"assets.csv\")\n",
-    "\n",
-    "# Assets with no decommission_year are effectively decommissioned after time horizon\n",
-    "assets[\"decommission_year\"] = assets[\"decommission_year\"].fillna(years[-1] + 1)\n",
-    "\n",
-    "# Calculate capacity for each type of process for each agent\n",
-    "capacity = pd.DataFrame()\n",
-    "for year in years:\n",
-    "    active = assets[\n",
-    "        (year >= assets[\"commission_year\"]) & (year < assets[\"decommission_year\"])\n",
-    "    ]\n",
-    "\n",
-    "    # This only works because each agent is responsible for one and only one commodity\n",
-    "    cap_sum = active.groupby([\"agent_id\", \"process_id\"])[\"capacity\"].sum().reset_index()\n",
-    "\n",
-    "    df = pd.DataFrame(cap_sum)\n",
-    "    df[\"year\"] = year\n",
-    "\n",
-    "    capacity = pd.concat([capacity, df])\n",
-    "\n",
-    "capacity"
+    "asset_capacities = pd.read_csv(OUTPUT_DIR / \"asset_capacities.csv\")\n",
+    "\n",
+    "merged = asset_capacities.merge(assets, on=[\"asset_id\", \"group_id\"])\n",
+    "agg_capacities = merged.groupby([\"milestone_year\", \"agent_id\", \"process_id\"])[\n",
+    "    \"capacity\"\n",
+    "].sum()\n",
+    "agg_capacities_wide = agg_capacities.unstack([\"agent_id\", \"process_id\"], fill_value=0)\n",
+    "agg_capacities_wide"
    ]
   },
   {
@@ -96,10 +71,7 @@
    "source": [
     "## Plot results\n",
     "\n",
-    "Finally, we plot the results.\n",
-    "\n",
-    "Note that each of the agents has invested in only one process type; otherwise there would be\n",
-    "multiple bars per plot here."
+    "Finally, we plot the results."
    ]
   },
   {
@@ -111,22 +83,20 @@
    "source": [
     "import matplotlib.pyplot as plt\n",
     "\n",
-    "agents = capacity[\"agent_id\"].unique()\n",
-    "_, axes = plt.subplots(1, len(agents))\n",
+    "agents = agg_capacities_wide.columns.get_level_values(\"agent_id\").unique()\n",
+    "fig, axes = plt.subplots(1, len(agents), figsize=(4 * len(agents), 4))\n",
     "for ax, agent in zip(axes, agents):\n",
-    "    capacity[capacity[\"agent_id\"] == agent].pivot(\n",
-    "        index=\"year\", columns=\"process_id\", values=\"capacity\"\n",
-    "    ).plot(kind=\"bar\", ax=ax)\n",
-    "    ax.set_title(agent)\n",
-    "    ax.set_xlabel(\"Year\")\n",
-    "    ax.set_ylabel(\"Capacity\")\n",
-    "    ax.legend(title=\"Process\")"
+    "    agg_capacities_wide[agent].plot(\n",
+    "        kind=\"bar\", stacked=True, ax=ax, title=agent, xlabel=\"Year\", ylabel=\"Capacity\"\n",
+    "    )\n",
+    "\n",
+    "plt.tight_layout()"
    ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "muse2-data-analysis",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -140,7 +110,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.14.2"
+   "version": "3.14.6"
   }
  },
  "nbformat": 4,

notebooks/prices.ipynb

@@ -50,13 +50,15 @@
     "\n",
     "ax.set_xlabel(\"Milestone year\")\n",
     "ax.set_ylabel(\"Price\")\n",
-    "ax.legend(title=\"Time slice\");"
+    "ax.legend(\n",
+    "    title=\"Time slice\", bbox_to_anchor=(1.05, 1), loc=\"upper left\", borderaxespad=0.0\n",
+    ");"
    ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "muse2-data-analysis",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -70,7 +72,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.14.2"
+   "version": "3.14.6"
   }
  },
  "nbformat": 4,