{ "cells": [ { "cell_type": "markdown", "id": "ad17d3f7-f702-4444-acd2-a1a79fea00f3", "metadata": {}, "source": [ "# Getting Seasonal Means for the season December-January-February\n", "\n", "We've been getting occasionally a few questions about seasonal reduction operations (for example means).\n", "\n", "Say you have a dataset that goes on for several years, and you want to compare the seaonal means of each year against each other.\n", "\n", "For three of the four seasons this isn't a big deal, you cut the dataset into yearly chunks, then mean over the season.\n", "\n", "But for the fourth season (DJF) this doesn't work: It bunches together the January, February, and December of that same year, when in fact you want the December from the year before.\n", "\n", "Let's have a look at an overly simplified example.\n", "\n", "## Our example dataset\n", "\n", "We only use a single dimension of data, and we only use monthly values. \n", "What's more, the values of our \"dataset\" are strictly ascending. \n", "\n", "This keeps the dataset small and we can easily see what's going on.\n", "\n", "The code works just the same way with far more complex datasets" ] }, { "cell_type": "code", "execution_count": 1, "id": "75b07cb3-ec82-41b4-b480-070c04599c04", "metadata": {}, "outputs": [], "source": [ "import xarray as xr\n", "import numpy as np\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "id": "fcc0e15e-c4ee-49b1-9137-c0bcdd8c2b4a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (time: 60)>\n",
       "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,\n",
       "       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,\n",
       "       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,\n",
       "       51, 52, 53, 54, 55, 56, 57, 58, 59])\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 2000-01-01 2000-02-01 ... 2004-12-01
" ], "text/plain": [ "\n", "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,\n", " 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,\n", " 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,\n", " 51, 52, 53, 54, 55, 56, 57, 58, 59])\n", "Coordinates:\n", " * time (time) datetime64[ns] 2000-01-01 2000-02-01 ... 2004-12-01" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "time = xr.DataArray(\n", " pd.date_range(start='2000-01-01', end='2004-12-31', freq='MS'), \n", " dims=('time',)\n", ")\n", "\n", "data = xr.DataArray(\n", " np.arange(len(time)), \n", " coords={'time': time},\n", ")\n", "data" ] }, { "cell_type": "markdown", "id": "415663c7-9ea7-4448-b4ff-3df2e915073e", "metadata": {}, "source": [ "## Initial idea: groupby\n", "The `.groupby` method can combine values according to a specific option, and the season is one of these. Let's try" ] }, { "cell_type": "code", "execution_count": 3, "id": "ec221488-bfba-4ca0-a21c-faa5a8880094", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (season: 4)>\n",
       "array([28., 30., 27., 33.])\n",
       "Coordinates:\n",
       "  * season   (season) object 'DJF' 'JJA' 'MAM' 'SON'
" ], "text/plain": [ "\n", "array([28., 30., 27., 33.])\n", "Coordinates:\n", " * season (season) object 'DJF' 'JJA' 'MAM' 'SON'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.groupby(data.time.dt.season).mean(dim='time')" ] }, { "cell_type": "markdown", "id": "650f2ad2-d185-41a5-8816-a613640ba51a", "metadata": {}, "source": [ "Now this isn't what we wanted.\n", "It has basically averaged over all Decembers, Januarys, and Februaries and stored this as a single value for \"DJF\", and the same for \"MAM\", \"JJA\", and \"SON\".\n", "\n", "Note that since this is a one-dimensional array, we could have omitted the `dim='time'` option from the call to `mean()` but you will probably also have other dimensions, and this way we can guarantee that it will only collapse the time dimension.\n", "\n", "## Next idea: Resample\n", "The `.resample` method changes the frequency of the dataset, and allows for reduction methods to be applied." ] }, { "cell_type": "code", "execution_count": 4, "id": "20bfef7c-e522-48d2-950f-60dcfc6f378b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (time: 20)>\n",
       "array([ 1.,  4.,  7., 10., 13., 16., 19., 22., 25., 28., 31., 34., 37.,\n",
       "       40., 43., 46., 49., 52., 55., 58.])\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 2000-01-01 2000-04-01 ... 2004-10-01
" ], "text/plain": [ "\n", "array([ 1., 4., 7., 10., 13., 16., 19., 22., 25., 28., 31., 34., 37.,\n", " 40., 43., 46., 49., 52., 55., 58.])\n", "Coordinates:\n", " * time (time) datetime64[ns] 2000-01-01 2000-04-01 ... 2004-10-01" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.resample(time='QS').mean(dim='time')" ] }, { "cell_type": "markdown", "id": "7a617ba7-2fe8-44a7-b9ed-1bb741dfd061", "metadata": {}, "source": [ "We are now much closer. \n", "We have quartly means, except that the values don't correspond to seasons.\n", "The solution are Panda's **Anchored Offsets**\n", "\n", "Basically we can move the start and end of a period around a bit.\n", "\n", "You can read more about it [here](https://pandas.pydata.org/docs/user_guide/timeseries.html).\n", "Quarterly periods use the period letter `Q`, by default, the time coordinate will be placed at the end of the period.\n", "So a single `Q` will combine January through March, and give it the time coordinate of March 31st.\n", "\n", "The first thing we want is to have the time coordinate as the beginning of the period, we do this by appending an `S`:\n", "`QS` has the same time periods, but places the coordinate at the beginning of the period.\n", "\n", "Finally, we want to start the periods in December, so the term we use is `QS-DEC`. \n", "(Note, `QS-MAR`, `QS-JUN`, and `QS-SEP` would have had the exact same effect.)" ] }, { "cell_type": "code", "execution_count": 5, "id": "c70e9566-3d2e-45d6-af20-20227c1b54bf", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (time: 21)>\n",
       "array([ 0.5,  3. ,  6. ,  9. , 12. , 15. , 18. , 21. , 24. , 27. , 30. ,\n",
       "       33. , 36. , 39. , 42. , 45. , 48. , 51. , 54. , 57. , 59. ])\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 1999-12-01 2000-03-01 ... 2004-12-01
" ], "text/plain": [ "\n", "array([ 0.5, 3. , 6. , 9. , 12. , 15. , 18. , 21. , 24. , 27. , 30. ,\n", " 33. , 36. , 39. , 42. , 45. , 48. , 51. , 54. , 57. , 59. ])\n", "Coordinates:\n", " * time (time) datetime64[ns] 1999-12-01 2000-03-01 ... 2004-12-01" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "season_means = data.resample(time='QS-DEC').mean(dim='time')\n", "season_means" ] }, { "cell_type": "markdown", "id": "2c489a9c-0109-4147-89a9-3a4702071f7b", "metadata": {}, "source": [ "### Extracting the season we're interested in.\n", "\n", "You might notice that I've already assigned the seasonal means to a new variable. \n", "I did this because in order to extract just the **DJF** season we're interested in, I need the condensed time dimension:" ] }, { "cell_type": "code", "execution_count": 6, "id": "f1145455-3100-4650-92ce-b7dab6536644", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (time: 6)>\n",
       "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 1999-12-01 2000-12-01 ... 2004-12-01
" ], "text/plain": [ "\n", "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n", "Coordinates:\n", " * time (time) datetime64[ns] 1999-12-01 2000-12-01 ... 2004-12-01" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "season_means.sel(time=season_means.time.dt.season=='DJF')" ] }, { "cell_type": "markdown", "id": "38cf06cf-0044-4528-a339-13ab7d8096d8", "metadata": {}, "source": [ "This is the solution. 6 Values, starting with the mean of Jan/Feb 2000, then the mean of Dec 2000 and Jan and Feb 2001, and so on, until the final value is just the mean of Dec 2004." ] }, { "cell_type": "markdown", "id": "dfe8c7aa-e0ec-4bea-b139-49dc11cfd192", "metadata": {}, "source": [ "## More flexibility using time masks\n", "We were lucky above that the seasons are 3 months per season, and the quarterly periods worked so well.\n", "\n", "Starting by creating a mask of the values we want. \n", "For month-based masks, it makes sense to use the `data.time.dt.month`.\n", "In this case we could simply use the `data.time.dt.month.isin([1, 2, 12])` but I want to show how to combine masks using the *or* operator `|`.\n", "\n", "Then we drop all the values that are not in this region, and finally we use the resampling method again, this time with 'Annual Ending February': `A-FEB`\n", "\n", "(Note that we would have gotten the same values for any 'end' month outside of the season, but the time coordinate would then point to a different month, and it would be confusing.)" ] }, { "cell_type": "code", "execution_count": 7, "id": "e5b8d77a-3b88-4766-b452-5e5887ffcbae", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (time: 6)>\n",
       "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n",
       "Coordinates:\n",
       "  * time     (time) datetime64[ns] 2000-02-29 2001-02-28 ... 2005-02-28
" ], "text/plain": [ "\n", "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n", "Coordinates:\n", " * time (time) datetime64[ns] 2000-02-29 2001-02-28 ... 2005-02-28" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "DJF_mask = (data.time.dt.month == 12) | (data.time.dt.month <= 2)\n", "data_DJF = data.where(DJF_mask, drop=True)\n", "data_DJF = data_DJF.resample(time='A-FEB').mean(dim='time')\n", "data_DJF" ] }, { "cell_type": "markdown", "id": "d5cda22d-880a-4972-8e3b-3fa857fc16a4", "metadata": {}, "source": [ "I have deliberatly chosen to have the period defined by their ending date, as I want to replace the time coordinate with a year coordinate, and I want the season to be defined by the year that January is in, which is the same year that the February is in.\n", "\n", "I can now use the `assign_coords` and `swap_dims` methods to change the dimension:" ] }, { "cell_type": "code", "execution_count": 8, "id": "34f7eb8f-4057-47dc-bd37-f130e79a530f", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (year: 6)>\n",
       "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n",
       "Coordinates:\n",
       "    time     (year) datetime64[ns] 2000-02-29 2001-02-28 ... 2005-02-28\n",
       "  * year     (year) int64 2000 2001 2002 2003 2004 2005
" ], "text/plain": [ "\n", "array([ 0.5, 12. , 24. , 36. , 48. , 59. ])\n", "Coordinates:\n", " time (year) datetime64[ns] 2000-02-29 2001-02-28 ... 2005-02-28\n", " * year (year) int64 2000 2001 2002 2003 2004 2005" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_DJF = data_DJF.assign_coords(year=data_DJF.time.dt.year)\n", "data_DJF = data_DJF.swap_dims({'time': 'year'})\n", "data_DJF" ] }, { "cell_type": "markdown", "id": "6ee80493-d906-4369-b0f1-adbf16177e87", "metadata": {}, "source": [ "If you had wanted the `year` coordinate to reflect the year that December is in, it's easiest to replace the `resample` period from above from `A-FEB` (Annual Ending February) to `AS-DEC` (Annual Starting December):\n", "\n", "```python\n", "data_DJF = data_DJF.resample(time='AS-DEC').mean(dim='time')\n", "```\n", "Then the time coordinate of the resulting array would refer to December 1st, and therefore the year would be that of the December." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }