Xarray and gridded data

# initialization
import pandas as pd

Oceanographic data are often gridded: for example, we may record ocean temperature on a range of latitude and longitude, as well as a range of depth. Such data are inherently 3-dimensional.

Naively, we may store such data in a 3-dimensional numpy array. However, in doing so we’ll lose information about the coordinates of the grid, e.g., the value of depth that corresponds an index of the third axis. What we need is a higher-dimensional equivalent of pandas, where information about coordinates are stored alongside the data.

The third-party xarray module provides such an extension. As an added benefit, xarray also provides interface to load and save NETCDF files, a common external file format for gridded data.

To import xarray, run the line below (again, the as xr part is optional but standard).

import xarray as xr

Loading and inspecting netCDF file

As an example of external netCDF data, we will work with a subset of the B-SOSE (Biogeochemical Southern Ocean State Estimate) model output. The data is stored in the netCDF file bsose_monthly_velocities.nc accessible here.

To load the data, we call the xr.open_dataset() function, and assign the result to a variable:

bsose = xr.open_dataset("data/bsose_monthly_velocities.nc")

We can inspect the data using the display() function. The output is an interactive html snippet with parts that you can expand or hide:

display(bsose)

<xarray.Dataset> Size: 29MB
Dimensions:  (time: 12, depth: 10, lat: 147, lon: 135)
Coordinates:
  * time     (time) datetime64[ns] 96B 2012-01-30T20:00:00 ... 2012-12-30T12:...
  * lat      (lat) float32 588B -77.93 -77.79 -77.65 ... -31.09 -30.52 -29.94
  * lon      (lon) float32 540B 90.17 90.83 91.5 92.17 ... 178.2 178.8 179.5
  * depth    (depth) float32 40B 2.1 26.25 65.0 105.0 ... 450.0 700.0 1.1e+03
Data variables:
    U        (time, depth, lat, lon) float32 10MB ...
    V        (time, depth, lat, lon) float64 19MB ...
Attributes:
    name:     B-SOSE (Southern Ocean State Estimate) model output

xarray.Dataset

• Dimensions:
  • time: 12
  • depth: 10
  • lat: 147
  • lon: 135
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2012-01-30T20:00:00 ... 2012-12-...

    long_name :

    Time

    standard_name :

    time

    axis :

    T

    array(['2012-01-30T20:00:00.000000000', '2012-03-01T06:00:00.000000000',
           '2012-03-31T16:00:00.000000000', '2012-05-01T02:00:00.000000000',
           '2012-05-31T13:00:00.000000000', '2012-06-30T23:00:00.000000000',
           '2012-07-31T09:00:00.000000000', '2012-08-30T19:00:00.000000000',
           '2012-09-30T05:00:00.000000000', '2012-10-30T16:00:00.000000000',
           '2012-11-30T02:00:00.000000000', '2012-12-30T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • lat
    (lat)
    float32
    -77.93 -77.79 ... -30.52 -29.94

    coordinate :

    YC XC

    units :

    degrees_north

    standard_name :

    latitude

    long_name :

    latitude

    axis :

    Y

    array([-77.930405, -77.79021 , -77.64841 , -77.50499 , -77.359924, -77.21321 ,
           -77.06482 , -76.914734, -76.76295 , -76.60942 , -76.45416 , -76.29712 ,
           -76.138306, -75.97768 , -75.81523 , -75.65094 , -75.48479 , -75.31675 ,
           -75.14681 , -74.97495 , -74.80115 , -74.62538 , -74.44764 , -74.26788 ,
           -74.086105, -73.90229 , -73.7164  , -73.52843 , -73.33835 , -73.146126,
           -72.95175 , -72.75522 , -72.55647 , -72.355515, -72.15231 , -71.94685 ,
           -71.73911 , -71.52905 , -71.316666, -71.10194 , -70.88482 , -70.6653  ,
           -70.44336 , -70.21898 , -69.99213 , -69.76277 , -69.5309  , -69.29649 ,
           -69.059525, -68.819954, -68.577774, -68.33295 , -68.08548 , -67.83532 ,
           -67.58244 , -67.326836, -67.06847 , -66.80732 , -66.543365, -66.276566,
           -66.00693 , -65.7344  , -65.45897 , -65.1806  , -64.89928 , -64.614975,
           -64.327675, -64.03734 , -63.743942, -63.44747 , -63.14791 , -62.845222,
           -62.539368, -62.230347, -61.91813 , -61.602695, -61.284016, -60.96207 ,
           -60.63683 , -60.308277, -59.976387, -59.641136, -59.302505, -58.960464,
           -58.614998, -58.26608 , -57.91369 , -57.557808, -57.198425, -56.835503,
           -56.469032, -56.09899 , -55.725353, -55.348103, -54.967228, -54.58271 ,
           -54.194527, -53.802666, -53.407104, -53.007835, -52.604836, -52.19809 ,
           -51.78759 , -51.37332 , -50.95527 , -50.53342 , -50.107765, -49.6783  ,
           -49.244995, -48.807858, -48.366875, -47.922047, -47.47336 , -47.020805,
           -46.564384, -46.104095, -45.639923, -45.171875, -44.699944, -44.224136,
           -43.744453, -43.260902, -42.773476, -42.282185, -41.787025, -41.288017,
           -40.785156, -40.278465, -39.767952, -39.253616, -38.73549 , -38.21357 ,
           -37.687893, -37.158455, -36.625282, -36.088398, -35.547817, -35.00357 ,
           -34.45568 , -33.904167, -33.349068, -32.790398, -32.2282  , -31.66251 ,
           -31.093353, -30.520763, -29.939266], dtype=float32)

  • lon
    (lon)
    float32
    90.17 90.83 91.5 ... 178.8 179.5

    coordinate :

    YG XG

    units :

    degrees_east

    standard_name :

    longitude_at_f_location

    long_name :

    longitude

    axis :

    X

    c_grid_axis_shift :

    -0.5

    array([ 90.16667,  90.83333,  91.5    ,  92.16667,  92.83333,  93.5    ,
            94.16667,  94.83333,  95.5    ,  96.16667,  96.83333,  97.5    ,
            98.16667,  98.83333,  99.5    , 100.16667, 100.83333, 101.5    ,
           102.16667, 102.83333, 103.5    , 104.16667, 104.83333, 105.5    ,
           106.16667, 106.83333, 107.5    , 108.16667, 108.83333, 109.5    ,
           110.16667, 110.83333, 111.5    , 112.16667, 112.83333, 113.5    ,
           114.16667, 114.83333, 115.5    , 116.16667, 116.83333, 117.5    ,
           118.16667, 118.83333, 119.5    , 120.16667, 120.83333, 121.5    ,
           122.16667, 122.83333, 123.5    , 124.16667, 124.83333, 125.5    ,
           126.16667, 126.83333, 127.5    , 128.16666, 128.83334, 129.5    ,
           130.16666, 130.83334, 131.5    , 132.16666, 132.83334, 133.5    ,
           134.16666, 134.83334, 135.5    , 136.16666, 136.83334, 137.5    ,
           138.16666, 138.83334, 139.5    , 140.16666, 140.83334, 141.5    ,
           142.16666, 142.83334, 143.5    , 144.16666, 144.83334, 145.5    ,
           146.16666, 146.83334, 147.5    , 148.16666, 148.83334, 149.5    ,
           150.16666, 150.83334, 151.5    , 152.16666, 152.83334, 153.5    ,
           154.16666, 154.83334, 155.5    , 156.16666, 156.83334, 157.5    ,
           158.16666, 158.83334, 159.5    , 160.16666, 160.83334, 161.5    ,
           162.16666, 162.83334, 163.5    , 164.16666, 164.83334, 165.5    ,
           166.16666, 166.83334, 167.5    , 168.16666, 168.83334, 169.5    ,
           170.16666, 170.83334, 171.5    , 172.16666, 172.83334, 173.5    ,
           174.16666, 174.83334, 175.5    , 176.16666, 176.83334, 177.5    ,
           178.16666, 178.83334, 179.5    ], dtype=float32)

  • depth
    (depth)
    float32
    2.1 26.25 65.0 ... 700.0 1.1e+03

    units :

    m

    standard_name :

    depth

    long_name :

    vertical coordinate of cell center

    axis :

    -Z

    array([   2.1 ,   26.25,   65.  ,  105.  ,  146.5 ,  220.  ,  301.  ,  450.  ,
            700.  , 1100.  ], dtype=float32)

• Data variables: (2)
  • U
    (time, depth, lat, lon)
    float32
    ...

    units :

    m/s

    long_name :

    Zonal Component of Velocity (m/s)

    standard_name :

    UVEL

    mate :

    VVEL

    [2381400 values with dtype=float32]

  • V
    (time, depth, lat, lon)
    float64
    ...

    units :

    m/s

    long_name :

    Meridional Component of Velocity (m/s)

    standard_name :

    VVEL

    mate :

    UVEL

    [2381400 values with dtype=float64]

• Indexes: (4)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2012-01-30 20:00:00', '2012-03-01 06:00:00',
                   '2012-03-31 16:00:00', '2012-05-01 02:00:00',
                   '2012-05-31 13:00:00', '2012-06-30 23:00:00',
                   '2012-07-31 09:00:00', '2012-08-30 19:00:00',
                   '2012-09-30 05:00:00', '2012-10-30 16:00:00',
                   '2012-11-30 02:00:00', '2012-12-30 12:00:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • lat
    PandasIndex

    PandasIndex(Index([ -77.93040466308594,  -77.79020690917969,  -77.64840698242188,
            -77.50498962402344,  -77.35992431640625,  -77.21321105957031,
             -77.0648193359375,  -76.91473388671875,  -76.76294708251953,
            -76.60942077636719,
           ...
           -35.003570556640625,  -34.45568084716797,  -33.90416717529297,
            -33.34906768798828,  -32.79039764404297,  -32.22819900512695,
            -31.66250991821289, -31.093353271484375, -30.520763397216797,
           -29.939266204833984],
          dtype='float32', name='lat', length=147))

  • lon
    PandasIndex

    PandasIndex(Index([ 90.16667175292969,  90.83332824707031,               91.5,
            92.16667175292969,  92.83332824707031,               93.5,
            94.16667175292969,  94.83332824707031,               95.5,
            96.16667175292969,
           ...
                        173.5, 174.16665649414062, 174.83334350585938,
                        175.5, 176.16665649414062, 176.83334350585938,
                        177.5, 178.16665649414062, 178.83334350585938,
                        179.5],
          dtype='float32', name='lon', length=135))

  • depth
    PandasIndex

    PandasIndex(Index([2.0999999046325684,              26.25,               65.0,
                        105.0,              146.5,              220.0,
                        301.0,              450.0,              700.0,
                       1100.0],
          dtype='float32', name='depth'))

• Attributes: (1)

  name :

  B-SOSE (Southern Ocean State Estimate) model output

Notice that bsose is an xarray Dataset object. Note also that multiple sets of data (“Data variables”) are stored alongside the labels of the coordinates (“Coordinates”) in a single object. In addition, the dimension (a.k.a. shape in numpy lingo) of the data is presented by the “Dimensions” section at the top. Furthermore, there are metadata (“Attributes”) associated with the whole dataset, such as its name.

By clicking on the document-looking icon, you can see that each coordinate and data variable has metadata associated with them. For example, we see that depth is measured in m while U and V are measured in m/s. Finally, you can preview the actual data by clicking on the cylinder icon.

From pandas DataFrame to xarray Dataset

Occasionally, you may need to convert a pandas DataFrame into an xarray Dataset and vice versa. To give a concrete example, consider the CalCOFI data we examined in week 6. One may argue that the data is grid-like when you consider time and depth as coordinates, and there may be advantage of turning it into an xarray Dataset.

To see how this may work, we load a version of the CalCOFI subset in which the depth is binned (you can download a copy of the file here):

CalCOFI2 = pd.read_csv("data/CalCOFI_binned.csv", parse_dates = ["Datetime"])
display(CalCOFI2)

	Cast_Count	Station_ID	Datetime	Depth_bin_m	T_degC	Salinity	SigmaTheta
0	992	090.0 070.0	1950-02-06 19:54:00	5	14.040	33.1700	24.76600
1	992	090.0 070.0	1950-02-06 19:54:00	15	13.950	33.2100	24.81500
2	992	090.0 070.0	1950-02-06 19:54:00	25	13.900	33.2100	24.82600
3	992	090.0 070.0	1950-02-06 19:54:00	35	13.810	33.2180	24.85100
4	992	090.0 070.0	1950-02-06 19:54:00	55	13.250	33.1500	24.91200
...	...	...	...	...	...	...	...
6402	35578	090.0 070.0	2021-01-21 13:36:00	205	8.518	34.0402	26.44858
6403	35578	090.0 070.0	2021-01-21 13:36:00	255	8.104	34.1405	26.59119
6404	35578	090.0 070.0	2021-01-21 13:36:00	275	8.012	34.1498	26.61270
6405	35578	090.0 070.0	2021-01-21 13:36:00	305	7.692	34.1712	26.67697
6406	35578	090.0 070.0	2021-01-21 13:36:00	385	7.144	34.2443	26.81386

6407 rows × 7 columns

To convert the pandas DataFrame to an xarray Dataset, we need to tell xarray which column(s) (“variable(s)” from the xarray perspective) are coordinates. We do so by converting such columns into row (multi-)index. In our case, the relevant columns are Datetime and Depth_bin_m, and we use the .set_index() method to turn these into row index.

Next, let’s say we want the resulting xarray Dataset to contain (and contain only) T_degC, Salinity, and SigmaTheta as data variables. Then we should make sure these are the only remaining columns after the index is set. To do so we can use the .iloc[] method.

Combining, we transform our pandas DataFrame like so:

CalCOFI3 = CalCOFI2.set_index(["Datetime", "Depth_bin_m"])
CalCOFI3 = CalCOFI3.loc[:, ["T_degC", "Salinity", "SigmaTheta"]]
display(CalCOFI3)

		T_degC	Salinity	SigmaTheta
Datetime	Depth_bin_m
1950-02-06 19:54:00	5	14.040	33.1700	24.76600
	15	13.950	33.2100	24.81500
	25	13.900	33.2100	24.82600
	35	13.810	33.2180	24.85100
	55	13.250	33.1500	24.91200
...	...	...	...	...
2021-01-21 13:36:00	205	8.518	34.0402	26.44858
	255	8.104	34.1405	26.59119
	275	8.012	34.1498	26.61270
	305	7.692	34.1712	26.67697
	385	7.144	34.2443	26.81386

6407 rows × 3 columns

We are now ready to convert this DataFrame into an xarray Dataset, and all it takes is a xr.Dataset.from_dataframe() call:

CalCOFI_xr = xr.Dataset.from_dataframe(CalCOFI3)
display(CalCOFI_xr)

<xarray.Dataset> Size: 333kB
Dimensions:      (Datetime: 344, Depth_bin_m: 40)
Coordinates:
  * Datetime     (Datetime) datetime64[ns] 3kB 1950-02-06T19:54:00 ... 2021-0...
  * Depth_bin_m  (Depth_bin_m) int64 320B 5 15 25 35 45 ... 355 365 375 385 395
Data variables:
    T_degC       (Datetime, Depth_bin_m) float64 110kB 14.04 13.95 ... 7.144 nan
    Salinity     (Datetime, Depth_bin_m) float64 110kB 33.17 33.21 ... 34.24 nan
    SigmaTheta   (Datetime, Depth_bin_m) float64 110kB 24.77 24.82 ... 26.81 nan

xarray.Dataset

• Dimensions:
  • Datetime: 344
  • Depth_bin_m: 40
• Coordinates: (2)
  • Datetime
    (Datetime)
    datetime64[ns]
    1950-02-06T19:54:00 ... 2021-01-...

    array(['1950-02-06T19:54:00.000000000', '1950-04-08T17:30:00.000000000',
           '1950-05-07T17:36:00.000000000', ..., '2020-07-16T23:40:00.000000000',
           '2020-10-14T23:52:00.000000000', '2021-01-21T13:36:00.000000000'],
          dtype='datetime64[ns]')

  • Depth_bin_m
    (Depth_bin_m)
    int64
    5 15 25 35 45 ... 365 375 385 395

    array([  5,  15,  25,  35,  45,  55,  65,  75,  85,  95, 105, 115, 125, 135,
           145, 155, 165, 175, 185, 195, 205, 215, 225, 235, 245, 255, 265, 275,
           285, 295, 305, 315, 325, 335, 345, 355, 365, 375, 385, 395], dtype=int64)

• Data variables: (3)
  • T_degC
    (Datetime, Depth_bin_m)
    float64
    14.04 13.95 13.9 ... nan 7.144 nan

    array([[14.04 , 13.95 , 13.9  , ...,    nan,    nan,  6.26 ],
           [15.02 , 14.98 , 13.87 , ...,    nan,    nan,    nan],
           [14.15 , 14.17 , 14.16 , ...,    nan,  6.7  ,    nan],
           ...,
           [17.565, 17.114, 16.681, ...,    nan,  6.507,    nan],
           [18.578, 17.373, 14.791, ...,    nan,  6.716,    nan],
           [15.093, 15.092, 15.09 , ...,    nan,  7.144,    nan]])

  • Salinity
    (Datetime, Depth_bin_m)
    float64
    33.17 33.21 33.21 ... nan 34.24 nan

    array([[33.17  , 33.21  , 33.21  , ...,     nan,     nan, 34.16  ],
           [33.38  , 33.4   , 33.273 , ...,     nan,     nan,     nan],
           [33.3   , 33.308 , 33.203 , ...,     nan, 34.18  ,     nan],
           ...,
           [33.6155, 33.6188, 33.6266, ...,     nan, 34.1518,     nan],
           [33.347 , 33.3346, 33.3381, ...,     nan, 34.2113,     nan],
           [33.4595, 33.4561, 33.4562, ...,     nan, 34.2443,     nan]])

  • SigmaTheta
    (Datetime, Depth_bin_m)
    float64
    24.77 24.82 24.83 ... nan 26.81 nan

    array([[24.766  , 24.815  , 24.826  , ...,      nan,      nan, 26.86   ],
           [24.72   , 24.744  , 24.881  , ...,      nan,      nan,      nan],
           [24.843  , 24.845  , 24.767  , ...,      nan, 26.818  ,      nan],
           ...,
           [24.31839, 24.42921, 24.53744, ...,      nan, 26.82622,      nan],
           [23.86513, 24.14988, 24.73804, ...,      nan, 26.84568,      nan],
           [24.76502, 24.76328, 24.76444, ...,      nan, 26.81386,      nan]])

• Indexes: (2)
  • Datetime
    PandasIndex

    PandasIndex(DatetimeIndex(['1950-02-06 19:54:00', '1950-04-08 17:30:00',
                   '1950-05-07 17:36:00', '1950-06-13 00:42:00',
                   '1950-07-13 00:54:00', '1950-08-18 07:30:00',
                   '1950-09-18 04:12:00', '1951-02-20 03:00:00',
                   '1951-03-19 09:36:00', '1951-04-16 20:36:00',
                   ...
                   '2018-06-14 05:44:00', '2018-10-19 08:00:00',
                   '2019-02-08 08:39:00', '2019-04-07 01:30:00',
                   '2019-07-16 18:15:00', '2019-11-09 01:24:00',
                   '2020-01-09 18:13:00', '2020-07-16 23:40:00',
                   '2020-10-14 23:52:00', '2021-01-21 13:36:00'],
                  dtype='datetime64[ns]', name='Datetime', length=344, freq=None))

  • Depth_bin_m
    PandasIndex

    PandasIndex(Index([  5,  15,  25,  35,  45,  55,  65,  75,  85,  95, 105, 115, 125, 135,
           145, 155, 165, 175, 185, 195, 205, 215, 225, 235, 245, 255, 265, 275,
           285, 295, 305, 315, 325, 335, 345, 355, 365, 375, 385, 395],
          dtype='int64', name='Depth_bin_m'))

• Attributes: (0)

Note that xarray automatically “complete” the grid and fill in some missing values for us (e.g., there is no measurement near 385 m on 1950-02-06. This column is implicitly missing in the pandas DataFrame, but become explicitly missing in the xarray Dataset, since measurement near 385 m did happen on other dates):

CalCOFI2.loc[
    (CalCOFI2["Datetime"] == pd.to_datetime("1950-02-06 19:54")) & 
    (CalCOFI2["Depth_bin_m"] == 385)
]

	Cast_Count	Station_ID	Datetime	Depth_bin_m	T_degC	Salinity	SigmaTheta

CalCOFI_xr.sel(Datetime=pd.to_datetime("1950-02-06 19:54"), Depth_bin_m = 385)

<xarray.Dataset> Size: 40B
Dimensions:      ()
Coordinates:
    Datetime     datetime64[ns] 8B 1950-02-06T19:54:00
    Depth_bin_m  int64 8B 385
Data variables:
    T_degC       float64 8B nan
    Salinity     float64 8B nan
    SigmaTheta   float64 8B nan

xarray.Dataset

• Dimensions:
• Coordinates: (2)
  • Datetime
    ()
    datetime64[ns]
    1950-02-06T19:54:00

    array('1950-02-06T19:54:00.000000000', dtype='datetime64[ns]')

  • Depth_bin_m
    ()
    int64
    385

    array(385, dtype=int64)

• Data variables: (3)
  • T_degC
    ()
    float64
    nan

    array(nan)

  • Salinity
    ()
    float64
    nan

    array(nan)

  • SigmaTheta
    ()
    float64
    nan

    array(nan)

• Indexes: (0)
• Attributes: (0)

From xarray Dataset to pandas DataFrame

For the converse (from xarray to pandas), consider the tidal gauge measurement near Key West, FL, courtesy University of Hawaii Sea Level Center (you can download a copy of the netCDF file here)

gauge_xr = xr.open_dataset("data/tide_gauges.nc")
display(gauge_xr)

<xarray.Dataset> Size: 24MB
Dimensions:          (record_id: 4, time: 979146)
Coordinates:
  * time             (time) datetime64[ns] 8MB 1913-01-19T06:00:00 ... 2024-0...
  * record_id        (record_id) int16 8B 2570 7550 7620 2420
Data variables:
    sea_level        (record_id, time) float32 16MB ...
    lat              (record_id) float32 16B ...
    lon              (record_id) float32 16B ...
    station_name     (record_id) <U16 256B ...
    station_country  (record_id) <U30 480B ...
Attributes:
    title:                  UHSLC Fast Delivery Tide Gauge Data (hourly)
    ncei_template_version:  NCEI_NetCDF_TimeSeries_Orthogonal_Template_v2.0
    featureType:            timeSeries
    Conventions:            CF-1.6, ACDD-1.3
    date_created:           2024-11-07T14:27:39Z
    publisher_name:         University of Hawaii Sea Level Center (UHSLC)
    publisher_email:        philiprt@hawaii.edu, markm@soest.hawaii.edu
    publisher_url:          http://uhslc.soest.hawaii.edu
    summary:                The UHSLC assembles and distributes the Fast Deli...
    processing_level:       Fast Delivery (FD) data undergo a level 1 quality...
    acknowledgment:         The UHSLC Fast Delivery database is supported by ...

xarray.Dataset

• Dimensions:
  • record_id: 4
  • time: 979146
• Coordinates: (2)
  • time
    (time)
    datetime64[ns]
    1913-01-19T06:00:00 ... 2024-09-...

    long_name :

    time

    axis :

    T

    array(['1913-01-19T06:00:00.000000000', '1913-01-19T07:00:00.028800000',
           '1913-01-19T07:59:59.971200000', ..., '2024-09-30T21:00:00.000000000',
           '2024-09-30T22:00:00.028800000', '2024-09-30T22:59:59.971200000'],
          dtype='datetime64[ns]')

  • record_id
    (record_id)
    int16
    2570 7550 7620 2420

    long_name :

    unique identifier for each record (i.e., station and version) in the database

    array([2570, 7550, 7620, 2420], dtype=int16)

• Data variables: (5)
  • sea_level
    (record_id, time)
    float32
    ...

    long_name :

    relative sea level

    units :

    millimeters

    source :

    in situ tide gauge water level observations

    platform :

    station_name, station_country, station_country_code, uhslc_id, gloss_id, ssc_id

    [3916584 values with dtype=float32]

  • lat
    (record_id)
    float32
    ...

    standard_name :

    latitude

    units :

    degrees_north

    axis :

    Y

    valid_min :

    -90

    valid_max :

    90

    [4 values with dtype=float32]

  • lon
    (record_id)
    float32
    ...

    standard_name :

    longitude

    units :

    degrees_east

    axis :

    X

    valid_min :

    0

    valid_max :

    360

    [4 values with dtype=float32]

  • station_name
    (record_id)
    <U16
    ...

    long_name :

    station name

    [4 values with dtype=<U16]

  • station_country
    (record_id)
    <U30
    ...

    long_name :

    station country (ISO 3166-1)

    [4 values with dtype=<U30]

• Indexes: (2)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex([       '1913-01-19 06:00:00', '1913-01-19 07:00:00.028800',
                   '1913-01-19 07:59:59.971200',        '1913-01-19 09:00:00',
                   '1913-01-19 10:00:00.028800', '1913-01-19 10:59:59.971200',
                          '1913-01-19 12:00:00', '1913-01-19 13:00:00.028800',
                   '1913-01-19 13:59:59.971200',        '1913-01-19 15:00:00',
                   ...
                   '2024-09-30 13:59:59.971200',        '2024-09-30 15:00:00',
                   '2024-09-30 16:00:00.028800', '2024-09-30 16:59:59.971200',
                          '2024-09-30 18:00:00', '2024-09-30 19:00:00.028800',
                   '2024-09-30 19:59:59.971200',        '2024-09-30 21:00:00',
                   '2024-09-30 22:00:00.028800', '2024-09-30 22:59:59.971200'],
                  dtype='datetime64[ns]', name='time', length=979146, freq=None))

  • record_id
    PandasIndex

    PandasIndex(Index([2570, 7550, 7620, 2420], dtype='int16', name='record_id'))

• Attributes: (11)

  title :

  UHSLC Fast Delivery Tide Gauge Data (hourly)

  ncei_template_version :

  NCEI_NetCDF_TimeSeries_Orthogonal_Template_v2.0

  featureType :

  timeSeries

  Conventions :

  CF-1.6, ACDD-1.3

  date_created :

  2024-11-07T14:27:39Z

  publisher_name :

  University of Hawaii Sea Level Center (UHSLC)

  publisher_email :

  philiprt@hawaii.edu, markm@soest.hawaii.edu

  publisher_url :

  http://uhslc.soest.hawaii.edu

  summary :

  The UHSLC assembles and distributes the Fast Delivery (FD) dataset of hourly- and daily-averaged tide gauge water-level observations. Tide gauge operators, or data creators, provide FD data to UHSLC after a level 1 quality assessment (see processing_level attribute). The UHSLC provides an independent quality assessment of the time series and makes FD data available within 4-6 weeks of collection. This is a "fast" turnaround time compared to Research Quality (RQ) data, which are available on an annual cycle after a level 2 quality assessment. RQ data replace FD data in the data stream as the former becomes available. This file contains hybrid time series composed of RQ data when available with FD data appended to the end of each RQ series.

  processing_level :

  Fast Delivery (FD) data undergo a level 1 quality assessment (e.g., unit and timing evaluation, outlier detection, combination of multiple channels into a primary channel, etc.). In this file, FD data are appended to Research Quality (RQ) data that have received a level 2 quality assessment (e.g., tide gauge datum evaluation, assessment of level ties to tide gauge benchmarks, comparison with nearby stations, etc.).

  acknowledgment :

  The UHSLC Fast Delivery database is supported by the National Oceanic and Atmospheric Administration (NOAA) Office of Climate Observations (OCO).

Observe that the record_id dimension only has 4 coordinate values, which essentially identify the station. Thus, the data is essentially one-dimensional, and it make sense to present it in tabular form.

To convert an xarray Dataset to a pandas DataFrame, all you need is to call the .to_dataframe() method of the Dataset:

gauge_pd = gauge_xr.to_dataframe()
display(gauge_pd)

		sea_level	lat	lon	station_name	station_country
record_id	time
2570	1913-01-19 06:00:00.000000	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
	1913-01-19 07:00:00.028800	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
	1913-01-19 07:59:59.971200	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
	1913-01-19 09:00:00.000000	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
	1913-01-19 10:00:00.028800	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
...	...	...	...	...	...	...
2420	2024-09-30 19:00:00.028800	1744.0	24.552999	278.191986	Key West, FL	United States of America (the)
	2024-09-30 19:59:59.971200	1740.0	24.552999	278.191986	Key West, FL	United States of America (the)
	2024-09-30 21:00:00.000000	1801.0	24.552999	278.191986	Key West, FL	United States of America (the)
	2024-09-30 22:00:00.028800	1893.0	24.552999	278.191986	Key West, FL	United States of America (the)
	2024-09-30 22:59:59.971200	1978.0	24.552999	278.191986	Key West, FL	United States of America (the)

3916584 rows × 5 columns

Notice that the coordinates (time and record_id) of the Dataset have become (multi-)index (row labels) of the DataFrame. To convert the index back to regular columns, we apply the .reset_index() method:

gauge_pd = gauge_pd.reset_index()
display(gauge_pd)

	record_id	time	sea_level	lat	lon	station_name	station_country
0	2570	1913-01-19 06:00:00.000000	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
1	2570	1913-01-19 07:00:00.028800	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
2	2570	1913-01-19 07:59:59.971200	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
3	2570	1913-01-19 09:00:00.000000	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
4	2570	1913-01-19 10:00:00.028800	NaN	26.690001	281.016998	Settlement Point	Bahamas (the)
...	...	...	...	...	...	...	...
3916579	2420	2024-09-30 19:00:00.028800	1744.0	24.552999	278.191986	Key West, FL	United States of America (the)
3916580	2420	2024-09-30 19:59:59.971200	1740.0	24.552999	278.191986	Key West, FL	United States of America (the)
3916581	2420	2024-09-30 21:00:00.000000	1801.0	24.552999	278.191986	Key West, FL	United States of America (the)
3916582	2420	2024-09-30 22:00:00.028800	1893.0	24.552999	278.191986	Key West, FL	United States of America (the)
3916583	2420	2024-09-30 22:59:59.971200	1978.0	24.552999	278.191986	Key West, FL	United States of America (the)

3916584 rows × 7 columns

We can now manipulate this DataFrame using the usual DataFrame methods, export the results as a csv, and so on.

Combine multiple xarray Datasets

As in the case of csv files, sometimes you can only download netCDF file for a subset of the data you want, and before your analysis you’ll need to combine multiple xarray Datasets into one. As an example, the 3 .nc files below are ocean surface temperature from OISST in 2025 on Jan 15, Feb 15, and Mar 15.

oisst_20250115 = xr.open_dataset("Data/oisst-20250115.nc")
oisst_20250215 = xr.open_dataset("Data/oisst-20250215.nc")
oisst_20250315 = xr.open_dataset("Data/oisst-20250315.nc")

display(oisst_20250115)

<xarray.Dataset> Size: 17MB
Dimensions:  (time: 1, zlev: 1, lat: 720, lon: 1440)
Coordinates:
  * time     (time) datetime64[ns] 8B 2025-01-15T12:00:00
  * zlev     (zlev) float32 4B 0.0
  * lat      (lat) float32 3kB -89.88 -89.62 -89.38 -89.12 ... 89.38 89.62 89.88
  * lon      (lon) float32 6kB 0.125 0.375 0.625 0.875 ... 359.4 359.6 359.9
Data variables:
    sst      (time, zlev, lat, lon) float32 4MB ...
    anom     (time, zlev, lat, lon) float32 4MB ...
    err      (time, zlev, lat, lon) float32 4MB ...
    ice      (time, zlev, lat, lon) float32 4MB ...
Attributes: (12/37)
    Conventions:                CF-1.6, ACDD-1.3
    title:                      NOAA/NCEI 1/4 Degree Daily Optimum Interpolat...
    references:                 Reynolds, et al.(2007) Daily High-Resolution-...
    source:                     ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfin...
    id:                         oisst-avhrr-v02r01.20250115.nc
    naming_authority:           gov.noaa.ncei
    ...                         ...
    time_coverage_start:        2025-01-15T00:00:00Z
    time_coverage_end:          2025-01-15T23:59:59Z
    metadata_link:              https://doi.org/10.25921/RE9P-PT57
    ncei_template_version:      NCEI_NetCDF_Grid_Template_v2.0
    comment:                    Data was converted from NetCDF-3 to NetCDF-4 ...
    sensor:                     Thermometer, AVHRR

xarray.Dataset

• Dimensions:
  • time: 1
  • zlev: 1
  • lat: 720
  • lon: 1440
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2025-01-15T12:00:00

    long_name :

    Center time of the day

    array(['2025-01-15T12:00:00.000000000'], dtype='datetime64[ns]')

  • zlev
    (zlev)
    float32
    0.0

    long_name :

    Sea surface height

    units :

    meters

    positive :

    down

    actual_range :

    0, 0

    array([0.], dtype=float32)

  • lat
    (lat)
    float32
    -89.88 -89.62 ... 89.62 89.88

    long_name :

    Latitude

    units :

    degrees_north

    grids :

    Uniform grid from -89.875 to 89.875 by 0.25

    array([-89.875, -89.625, -89.375, ...,  89.375,  89.625,  89.875],
          dtype=float32)

  • lon
    (lon)
    float32
    0.125 0.375 0.625 ... 359.6 359.9

    long_name :

    Longitude

    units :

    degrees_east

    grids :

    Uniform grid from 0.125 to 359.875 by 0.25

    array([1.25000e-01, 3.75000e-01, 6.25000e-01, ..., 3.59375e+02, 3.59625e+02,
           3.59875e+02], dtype=float32)

• Data variables: (4)
  • sst
    (time, zlev, lat, lon)
    float32
    ...

    long_name :

    Daily sea surface temperature

    units :

    Celsius

    valid_min :

    -300

    valid_max :

    4500

    [1036800 values with dtype=float32]

  • anom
    (time, zlev, lat, lon)
    float32
    ...

    long_name :

    Daily sea surface temperature anomalies

    units :

    Celsius

    valid_min :

    -1200

    valid_max :

    1200

    [1036800 values with dtype=float32]

  • err
    (time, zlev, lat, lon)
    float32
    ...

    long_name :

    Estimated error standard deviation of analysed_sst

    units :

    Celsius

    valid_min :

    0

    valid_max :

    1000

    [1036800 values with dtype=float32]

  • ice
    (time, zlev, lat, lon)
    float32
    ...

    long_name :

    Sea ice concentration

    units :

    %

    valid_min :

    0

    valid_max :

    100

    [1036800 values with dtype=float32]

• Indexes: (4)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2025-01-15 12:00:00'], dtype='datetime64[ns]', name='time', freq=None))

  • zlev
    PandasIndex

    PandasIndex(Index([0.0], dtype='float32', name='zlev'))

  • lat
    PandasIndex

    PandasIndex(Index([-89.875, -89.625, -89.375, -89.125, -88.875, -88.625, -88.375, -88.125,
           -87.875, -87.625,
           ...
            87.625,  87.875,  88.125,  88.375,  88.625,  88.875,  89.125,  89.375,
            89.625,  89.875],
          dtype='float32', name='lat', length=720))

  • lon
    PandasIndex

    PandasIndex(Index([  0.125,   0.375,   0.625,   0.875,   1.125,   1.375,   1.625,   1.875,
             2.125,   2.375,
           ...
           357.625, 357.875, 358.125, 358.375, 358.625, 358.875, 359.125, 359.375,
           359.625, 359.875],
          dtype='float32', name='lon', length=1440))

• Attributes: (37)

  Conventions :

  CF-1.6, ACDD-1.3

  title :

  NOAA/NCEI 1/4 Degree Daily Optimum Interpolation Sea Surface Temperature (OISST) Analysis, Version 2.1 - Final

  references :

  Reynolds, et al.(2007) Daily High-Resolution-Blended Analyses for Sea Surface Temperature (available at https://doi.org/10.1175/2007JCLI1824.1). Banzon, et al.(2016) A long-term record of blended satellite and in situ sea-surface temperature for climate monitoring, modeling and environmental studies (available at https://doi.org/10.5194/essd-8-165-2016). Huang et al. (2020) Improvements of the Daily Optimum Interpolation Sea Surface Temperature (DOISST) Version v02r01, submitted.Climatology is based on 1971-2000 OI.v2 SST. Satellite data: Pathfinder AVHRR SST, Navy AVHRR SST, and NOAA ACSPO SST. Ice data: NCEP Ice and GSFC Ice.

  source :

  ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfinder_AVHRR, Navy_AVHRR, NOAA_ACSP

  id :

  oisst-avhrr-v02r01.20250115.nc

  naming_authority :

  gov.noaa.ncei

  summary :

  NOAAs 1/4-degree Daily Optimum Interpolation Sea Surface Temperature (OISST) (sometimes referred to as Reynolds SST, which however also refers to earlier products at different resolution), currently available as version v02r01, is created by interpolating and extrapolating SST observations from different sources, resulting in a smoothed complete field. The sources of data are satellite (AVHRR) and in situ platforms (i.e., ships and buoys), and the specific datasets employed may change over time. At the marginal ice zone, sea ice concentrations are used to generate proxy SSTs. A preliminary version of this file is produced in near-real time (1-day latency), and then replaced with a final version after 2 weeks. Note that this is the AVHRR-ONLY DOISST, available from Oct 1981, but there is a companion DOISST product that includes microwave satellite data, available from June 2002

  cdm_data_type :

  Grid

  history :

  Final file created using preliminary as first guess, and 3 days of AVHRR data. Preliminary uses only 1 day of AVHRR data.

  date_modified :

  2025-01-30T14:43:00Z

  date_created :

  2025-01-30T14:43:00Z

  product_version :

  Version v02r01

  processing_level :

  NOAA Level 4

  institution :

  NOAA/National Centers for Environmental Information

  creator_url :

  https://www.ncei.noaa.gov/

  creator_email :

  oisst-help@noaa.gov

  keywords :

  Earth Science > Oceans > Ocean Temperature > Sea Surface Temperature

  keywords_vocabulary :

  Global Change Master Directory (GCMD) Earth Science Keywords

  platform :

  Ships, buoys, Argo floats, MetOp-A, MetOp-B

  platform_vocabulary :

  Global Change Master Directory (GCMD) Platform Keywords

  instrument :

  Earth Remote Sensing Instruments > Passive Remote Sensing > Spectrometers/Radiometers > Imaging Spectrometers/Radiometers > AVHRR > Advanced Very High Resolution Radiometer

  instrument_vocabulary :

  Global Change Master Directory (GCMD) Instrument Keywords

  standard_name_vocabulary :

  CF Standard Name Table (v40, 25 January 2017)

  geospatial_lat_min :

  -90.0

  geospatial_lat_max :

  90.0

  geospatial_lon_min :

  0.0

  geospatial_lon_max :

  360.0

  geospatial_lat_units :

  degrees_north

  geospatial_lat_resolution :

  0.25

  geospatial_lon_units :

  degrees_east

  geospatial_lon_resolution :

  0.25

  time_coverage_start :

  2025-01-15T00:00:00Z

  time_coverage_end :

  2025-01-15T23:59:59Z

  metadata_link :

  https://doi.org/10.25921/RE9P-PT57

  ncei_template_version :

  NCEI_NetCDF_Grid_Template_v2.0

  comment :

  Data was converted from NetCDF-3 to NetCDF-4 format with metadata updates in November 2017.

  sensor :

  Thermometer, AVHRR

We can merge these xarray Dataset into one using xr.concat(), using dim="time" to tell xarray that we want to merge along the time dimension:

oisst_all = xr.concat([oisst_20250115, oisst_20250215, oisst_20250315], dim="time")
display(oisst_all)

<xarray.Dataset> Size: 50MB
Dimensions:  (time: 3, zlev: 1, lat: 720, lon: 1440)
Coordinates:
  * time     (time) datetime64[ns] 24B 2025-01-15T12:00:00 ... 2025-03-15T12:...
  * zlev     (zlev) float32 4B 0.0
  * lat      (lat) float32 3kB -89.88 -89.62 -89.38 -89.12 ... 89.38 89.62 89.88
  * lon      (lon) float32 6kB 0.125 0.375 0.625 0.875 ... 359.4 359.6 359.9
Data variables:
    sst      (time, zlev, lat, lon) float32 12MB nan nan nan ... -1.8 -1.8 -1.8
    anom     (time, zlev, lat, lon) float32 12MB nan nan nan nan ... 0.0 0.0 0.0
    err      (time, zlev, lat, lon) float32 12MB nan nan nan nan ... 0.3 0.3 0.3
    ice      (time, zlev, lat, lon) float32 12MB nan nan nan ... 0.98 0.98 0.98
Attributes: (12/37)
    Conventions:                CF-1.6, ACDD-1.3
    title:                      NOAA/NCEI 1/4 Degree Daily Optimum Interpolat...
    references:                 Reynolds, et al.(2007) Daily High-Resolution-...
    source:                     ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfin...
    id:                         oisst-avhrr-v02r01.20250115.nc
    naming_authority:           gov.noaa.ncei
    ...                         ...
    time_coverage_start:        2025-01-15T00:00:00Z
    time_coverage_end:          2025-01-15T23:59:59Z
    metadata_link:              https://doi.org/10.25921/RE9P-PT57
    ncei_template_version:      NCEI_NetCDF_Grid_Template_v2.0
    comment:                    Data was converted from NetCDF-3 to NetCDF-4 ...
    sensor:                     Thermometer, AVHRR

xarray.Dataset

• Dimensions:
  • time: 3
  • zlev: 1
  • lat: 720
  • lon: 1440
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2025-01-15T12:00:00 ... 2025-03-...

    long_name :

    Center time of the day

    array(['2025-01-15T12:00:00.000000000', '2025-02-15T12:00:00.000000000',
           '2025-03-15T12:00:00.000000000'], dtype='datetime64[ns]')

  • zlev
    (zlev)
    float32
    0.0

    long_name :

    Sea surface height

    units :

    meters

    positive :

    down

    actual_range :

    0, 0

    array([0.], dtype=float32)

  • lat
    (lat)
    float32
    -89.88 -89.62 ... 89.62 89.88

    long_name :

    Latitude

    units :

    degrees_north

    grids :

    Uniform grid from -89.875 to 89.875 by 0.25

    array([-89.875, -89.625, -89.375, ...,  89.375,  89.625,  89.875],
          dtype=float32)

  • lon
    (lon)
    float32
    0.125 0.375 0.625 ... 359.6 359.9

    long_name :

    Longitude

    units :

    degrees_east

    grids :

    Uniform grid from 0.125 to 359.875 by 0.25

    array([1.25000e-01, 3.75000e-01, 6.25000e-01, ..., 3.59375e+02, 3.59625e+02,
           3.59875e+02], dtype=float32)

• Data variables: (4)
  • sst
    (time, zlev, lat, lon)
    float32
    nan nan nan nan ... -1.8 -1.8 -1.8

    long_name :

    Daily sea surface temperature

    units :

    Celsius

    valid_min :

    -300

    valid_max :

    4500

    array([[[[  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             ...,
             [-1.73, -1.75, -1.76, ..., -1.76, -1.75, -1.73],
             [-1.74, -1.77, -1.79, ..., -1.79, -1.78, -1.74],
             [-1.8 , -1.8 , -1.8 , ..., -1.8 , -1.8 , -1.8 ]]],
    
    
           [[[  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             ...,
             [-1.73, -1.74, -1.75, ..., -1.76, -1.75, -1.73],
             [-1.74, -1.77, -1.78, ..., -1.78, -1.77, -1.75],
             [-1.8 , -1.8 , -1.8 , ..., -1.8 , -1.8 , -1.8 ]]],
    
    
           [[[  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             [  nan,   nan,   nan, ...,   nan,   nan,   nan],
             ...,
             [-1.73, -1.74, -1.75, ..., -1.75, -1.75, -1.73],
             [-1.74, -1.77, -1.78, ..., -1.78, -1.77, -1.74],
             [-1.8 , -1.8 , -1.8 , ..., -1.8 , -1.8 , -1.8 ]]]], dtype=float32)

  • anom
    (time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.0 0.0 0.0

    long_name :

    Daily sea surface temperature anomalies

    units :

    Celsius

    valid_min :

    -1200

    valid_max :

    1200

    array([[[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             ...,
             [0.07, 0.05, 0.04, ..., 0.04, 0.05, 0.07],
             [0.06, 0.03, 0.01, ..., 0.01, 0.02, 0.06],
             [0.  , 0.  , 0.  , ..., 0.  , 0.  , 0.  ]]],
    
    
           [[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             ...,
             [0.07, 0.06, 0.05, ..., 0.04, 0.05, 0.07],
             [0.06, 0.03, 0.02, ..., 0.02, 0.03, 0.05],
             [0.  , 0.  , 0.  , ..., 0.  , 0.  , 0.  ]]],
    
    
           [[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             [ nan,  nan,  nan, ...,  nan,  nan,  nan],
             ...,
             [0.07, 0.06, 0.05, ..., 0.05, 0.05, 0.07],
             [0.06, 0.03, 0.02, ..., 0.02, 0.03, 0.06],
             [0.  , 0.  , 0.  , ..., 0.  , 0.  , 0.  ]]]], dtype=float32)

  • err
    (time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.3 0.3 0.3 0.3

    long_name :

    Estimated error standard deviation of analysed_sst

    units :

    Celsius

    valid_min :

    0

    valid_max :

    1000

    array([[[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             ...,
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998]]],
    
    
           [[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
    ...
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998]]],
    
    
           [[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             ...,
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998],
             [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
              0.29999998, 0.29999998]]]], dtype=float32)

  • ice
    (time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.98 0.98 0.98

    long_name :

    Sea ice concentration

    units :

    %

    valid_min :

    0

    valid_max :

    100

    array([[[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             ...,
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996]]],
    
    
           [[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
    ...
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996]]],
    
    
           [[[       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             [       nan,        nan,        nan, ...,        nan,
                     nan,        nan],
             ...,
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996],
             [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
              0.97999996, 0.97999996]]]], dtype=float32)

• Indexes: (4)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2025-01-15 12:00:00', '2025-02-15 12:00:00',
                   '2025-03-15 12:00:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • zlev
    PandasIndex

    PandasIndex(Index([0.0], dtype='float32', name='zlev'))

  • lat
    PandasIndex

    PandasIndex(Index([-89.875, -89.625, -89.375, -89.125, -88.875, -88.625, -88.375, -88.125,
           -87.875, -87.625,
           ...
            87.625,  87.875,  88.125,  88.375,  88.625,  88.875,  89.125,  89.375,
            89.625,  89.875],
          dtype='float32', name='lat', length=720))

  • lon
    PandasIndex

    PandasIndex(Index([  0.125,   0.375,   0.625,   0.875,   1.125,   1.375,   1.625,   1.875,
             2.125,   2.375,
           ...
           357.625, 357.875, 358.125, 358.375, 358.625, 358.875, 359.125, 359.375,
           359.625, 359.875],
          dtype='float32', name='lon', length=1440))

• Attributes: (37)

  Conventions :

  CF-1.6, ACDD-1.3

  title :

  NOAA/NCEI 1/4 Degree Daily Optimum Interpolation Sea Surface Temperature (OISST) Analysis, Version 2.1 - Final

  references :

  Reynolds, et al.(2007) Daily High-Resolution-Blended Analyses for Sea Surface Temperature (available at https://doi.org/10.1175/2007JCLI1824.1). Banzon, et al.(2016) A long-term record of blended satellite and in situ sea-surface temperature for climate monitoring, modeling and environmental studies (available at https://doi.org/10.5194/essd-8-165-2016). Huang et al. (2020) Improvements of the Daily Optimum Interpolation Sea Surface Temperature (DOISST) Version v02r01, submitted.Climatology is based on 1971-2000 OI.v2 SST. Satellite data: Pathfinder AVHRR SST, Navy AVHRR SST, and NOAA ACSPO SST. Ice data: NCEP Ice and GSFC Ice.

  source :

  ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfinder_AVHRR, Navy_AVHRR, NOAA_ACSP

  id :

  oisst-avhrr-v02r01.20250115.nc

  naming_authority :

  gov.noaa.ncei

  summary :

  NOAAs 1/4-degree Daily Optimum Interpolation Sea Surface Temperature (OISST) (sometimes referred to as Reynolds SST, which however also refers to earlier products at different resolution), currently available as version v02r01, is created by interpolating and extrapolating SST observations from different sources, resulting in a smoothed complete field. The sources of data are satellite (AVHRR) and in situ platforms (i.e., ships and buoys), and the specific datasets employed may change over time. At the marginal ice zone, sea ice concentrations are used to generate proxy SSTs. A preliminary version of this file is produced in near-real time (1-day latency), and then replaced with a final version after 2 weeks. Note that this is the AVHRR-ONLY DOISST, available from Oct 1981, but there is a companion DOISST product that includes microwave satellite data, available from June 2002

  cdm_data_type :

  Grid

  history :

  Final file created using preliminary as first guess, and 3 days of AVHRR data. Preliminary uses only 1 day of AVHRR data.

  date_modified :

  2025-01-30T14:43:00Z

  date_created :

  2025-01-30T14:43:00Z

  product_version :

  Version v02r01

  processing_level :

  NOAA Level 4

  institution :

  NOAA/National Centers for Environmental Information

  creator_url :

  https://www.ncei.noaa.gov/

  creator_email :

  oisst-help@noaa.gov

  keywords :

  Earth Science > Oceans > Ocean Temperature > Sea Surface Temperature

  keywords_vocabulary :

  Global Change Master Directory (GCMD) Earth Science Keywords

  platform :

  Ships, buoys, Argo floats, MetOp-A, MetOp-B

  platform_vocabulary :

  Global Change Master Directory (GCMD) Platform Keywords

  instrument :

  Earth Remote Sensing Instruments > Passive Remote Sensing > Spectrometers/Radiometers > Imaging Spectrometers/Radiometers > AVHRR > Advanced Very High Resolution Radiometer

  instrument_vocabulary :

  Global Change Master Directory (GCMD) Instrument Keywords

  standard_name_vocabulary :

  CF Standard Name Table (v40, 25 January 2017)

  geospatial_lat_min :

  -90.0

  geospatial_lat_max :

  90.0

  geospatial_lon_min :

  0.0

  geospatial_lon_max :

  360.0

  geospatial_lat_units :

  degrees_north

  geospatial_lat_resolution :

  0.25

  geospatial_lon_units :

  degrees_east

  geospatial_lon_resolution :

  0.25

  time_coverage_start :

  2025-01-15T00:00:00Z

  time_coverage_end :

  2025-01-15T23:59:59Z

  metadata_link :

  https://doi.org/10.25921/RE9P-PT57

  ncei_template_version :

  NCEI_NetCDF_Grid_Template_v2.0

  comment :

  Data was converted from NetCDF-3 to NetCDF-4 format with metadata updates in November 2017.

  sensor :

  Thermometer, AVHRR

Occasionally you’ll need to create an extra dimension so that your individual Dataset can be merged along this new dimension. To do so we can use the .expand_dims() method of both Dataset and DataArray (which create the dimension), followed by the .assign_coords() method (which assign coordinates to the new dimension).

Take oisst_20250115 as an example, suppose we want to create a new dimension call month, we may do:

oisst_20250115.expand_dims(dim={"month": 1}).assign_coords({"month": [1]})

<xarray.Dataset> Size: 17MB
Dimensions:  (month: 1, time: 1, zlev: 1, lat: 720, lon: 1440)
Coordinates:
  * time     (time) datetime64[ns] 8B 2025-01-15T12:00:00
  * zlev     (zlev) float32 4B 0.0
  * lat      (lat) float32 3kB -89.88 -89.62 -89.38 -89.12 ... 89.38 89.62 89.88
  * lon      (lon) float32 6kB 0.125 0.375 0.625 0.875 ... 359.4 359.6 359.9
  * month    (month) int32 4B 1
Data variables:
    sst      (month, time, zlev, lat, lon) float32 4MB nan nan nan ... -1.8 -1.8
    anom     (month, time, zlev, lat, lon) float32 4MB nan nan nan ... 0.0 0.0
    err      (month, time, zlev, lat, lon) float32 4MB nan nan nan ... 0.3 0.3
    ice      (month, time, zlev, lat, lon) float32 4MB nan nan nan ... 0.98 0.98
Attributes: (12/37)
    Conventions:                CF-1.6, ACDD-1.3
    title:                      NOAA/NCEI 1/4 Degree Daily Optimum Interpolat...
    references:                 Reynolds, et al.(2007) Daily High-Resolution-...
    source:                     ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfin...
    id:                         oisst-avhrr-v02r01.20250115.nc
    naming_authority:           gov.noaa.ncei
    ...                         ...
    time_coverage_start:        2025-01-15T00:00:00Z
    time_coverage_end:          2025-01-15T23:59:59Z
    metadata_link:              https://doi.org/10.25921/RE9P-PT57
    ncei_template_version:      NCEI_NetCDF_Grid_Template_v2.0
    comment:                    Data was converted from NetCDF-3 to NetCDF-4 ...
    sensor:                     Thermometer, AVHRR

xarray.Dataset

• Dimensions:
  • month: 1
  • time: 1
  • zlev: 1
  • lat: 720
  • lon: 1440
• Coordinates: (5)
  • time
    (time)
    datetime64[ns]
    2025-01-15T12:00:00

    long_name :

    Center time of the day

    array(['2025-01-15T12:00:00.000000000'], dtype='datetime64[ns]')

  • zlev
    (zlev)
    float32
    0.0

    long_name :

    Sea surface height

    units :

    meters

    positive :

    down

    actual_range :

    0, 0

    array([0.], dtype=float32)

  • lat
    (lat)
    float32
    -89.88 -89.62 ... 89.62 89.88

    long_name :

    Latitude

    units :

    degrees_north

    grids :

    Uniform grid from -89.875 to 89.875 by 0.25

    array([-89.875, -89.625, -89.375, ...,  89.375,  89.625,  89.875],
          dtype=float32)

  • lon
    (lon)
    float32
    0.125 0.375 0.625 ... 359.6 359.9

    long_name :

    Longitude

    units :

    degrees_east

    grids :

    Uniform grid from 0.125 to 359.875 by 0.25

    array([1.25000e-01, 3.75000e-01, 6.25000e-01, ..., 3.59375e+02, 3.59625e+02,
           3.59875e+02], dtype=float32)

  • month
    (month)
    int32
    1

    array([1])

• Data variables: (4)
  • sst
    (month, time, zlev, lat, lon)
    float32
    nan nan nan nan ... -1.8 -1.8 -1.8

    long_name :

    Daily sea surface temperature

    units :

    Celsius

    valid_min :

    -300

    valid_max :

    4500

    array([[[[[  nan,   nan,   nan, ...,   nan,   nan,   nan],
              [  nan,   nan,   nan, ...,   nan,   nan,   nan],
              [  nan,   nan,   nan, ...,   nan,   nan,   nan],
              ...,
              [-1.73, -1.75, -1.76, ..., -1.76, -1.75, -1.73],
              [-1.74, -1.77, -1.79, ..., -1.79, -1.78, -1.74],
              [-1.8 , -1.8 , -1.8 , ..., -1.8 , -1.8 , -1.8 ]]]]],
          dtype=float32)

  • anom
    (month, time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.0 0.0 0.0

    long_name :

    Daily sea surface temperature anomalies

    units :

    Celsius

    valid_min :

    -1200

    valid_max :

    1200

    array([[[[[ nan,  nan,  nan, ...,  nan,  nan,  nan],
              [ nan,  nan,  nan, ...,  nan,  nan,  nan],
              [ nan,  nan,  nan, ...,  nan,  nan,  nan],
              ...,
              [0.07, 0.05, 0.04, ..., 0.04, 0.05, 0.07],
              [0.06, 0.03, 0.01, ..., 0.01, 0.02, 0.06],
              [0.  , 0.  , 0.  , ..., 0.  , 0.  , 0.  ]]]]], dtype=float32)

  • err
    (month, time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.3 0.3 0.3 0.3

    long_name :

    Estimated error standard deviation of analysed_sst

    units :

    Celsius

    valid_min :

    0

    valid_max :

    1000

    array([[[[[       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              [       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              [       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              ...,
              [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
               0.29999998, 0.29999998],
              [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
               0.29999998, 0.29999998],
              [0.29999998, 0.29999998, 0.29999998, ..., 0.29999998,
               0.29999998, 0.29999998]]]]], dtype=float32)

  • ice
    (month, time, zlev, lat, lon)
    float32
    nan nan nan nan ... 0.98 0.98 0.98

    long_name :

    Sea ice concentration

    units :

    %

    valid_min :

    0

    valid_max :

    100

    array([[[[[       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              [       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              [       nan,        nan,        nan, ...,        nan,
                      nan,        nan],
              ...,
              [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
               0.97999996, 0.97999996],
              [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
               0.97999996, 0.97999996],
              [0.97999996, 0.97999996, 0.97999996, ..., 0.97999996,
               0.97999996, 0.97999996]]]]], dtype=float32)

• Indexes: (5)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2025-01-15 12:00:00'], dtype='datetime64[ns]', name='time', freq=None))

  • zlev
    PandasIndex

    PandasIndex(Index([0.0], dtype='float32', name='zlev'))

  • lat
    PandasIndex

    PandasIndex(Index([-89.875, -89.625, -89.375, -89.125, -88.875, -88.625, -88.375, -88.125,
           -87.875, -87.625,
           ...
            87.625,  87.875,  88.125,  88.375,  88.625,  88.875,  89.125,  89.375,
            89.625,  89.875],
          dtype='float32', name='lat', length=720))

  • lon
    PandasIndex

    PandasIndex(Index([  0.125,   0.375,   0.625,   0.875,   1.125,   1.375,   1.625,   1.875,
             2.125,   2.375,
           ...
           357.625, 357.875, 358.125, 358.375, 358.625, 358.875, 359.125, 359.375,
           359.625, 359.875],
          dtype='float32', name='lon', length=1440))

  • month
    PandasIndex

    PandasIndex(Index([1], dtype='int32', name='month'))

• Attributes: (37)

  Conventions :

  CF-1.6, ACDD-1.3

  title :

  NOAA/NCEI 1/4 Degree Daily Optimum Interpolation Sea Surface Temperature (OISST) Analysis, Version 2.1 - Final

  references :

  Reynolds, et al.(2007) Daily High-Resolution-Blended Analyses for Sea Surface Temperature (available at https://doi.org/10.1175/2007JCLI1824.1). Banzon, et al.(2016) A long-term record of blended satellite and in situ sea-surface temperature for climate monitoring, modeling and environmental studies (available at https://doi.org/10.5194/essd-8-165-2016). Huang et al. (2020) Improvements of the Daily Optimum Interpolation Sea Surface Temperature (DOISST) Version v02r01, submitted.Climatology is based on 1971-2000 OI.v2 SST. Satellite data: Pathfinder AVHRR SST, Navy AVHRR SST, and NOAA ACSPO SST. Ice data: NCEP Ice and GSFC Ice.

  source :

  ICOADS, NCEP_GTS, GSFC_ICE, NCEP_ICE, Pathfinder_AVHRR, Navy_AVHRR, NOAA_ACSP

  id :

  oisst-avhrr-v02r01.20250115.nc

  naming_authority :

  gov.noaa.ncei

  summary :

  NOAAs 1/4-degree Daily Optimum Interpolation Sea Surface Temperature (OISST) (sometimes referred to as Reynolds SST, which however also refers to earlier products at different resolution), currently available as version v02r01, is created by interpolating and extrapolating SST observations from different sources, resulting in a smoothed complete field. The sources of data are satellite (AVHRR) and in situ platforms (i.e., ships and buoys), and the specific datasets employed may change over time. At the marginal ice zone, sea ice concentrations are used to generate proxy SSTs. A preliminary version of this file is produced in near-real time (1-day latency), and then replaced with a final version after 2 weeks. Note that this is the AVHRR-ONLY DOISST, available from Oct 1981, but there is a companion DOISST product that includes microwave satellite data, available from June 2002

  cdm_data_type :

  Grid

  history :

  Final file created using preliminary as first guess, and 3 days of AVHRR data. Preliminary uses only 1 day of AVHRR data.

  date_modified :

  2025-01-30T14:43:00Z

  date_created :

  2025-01-30T14:43:00Z

  product_version :

  Version v02r01

  processing_level :

  NOAA Level 4

  institution :

  NOAA/National Centers for Environmental Information

  creator_url :

  https://www.ncei.noaa.gov/

  creator_email :

  oisst-help@noaa.gov

  keywords :

  Earth Science > Oceans > Ocean Temperature > Sea Surface Temperature

  keywords_vocabulary :

  Global Change Master Directory (GCMD) Earth Science Keywords

  platform :

  Ships, buoys, Argo floats, MetOp-A, MetOp-B

  platform_vocabulary :

  Global Change Master Directory (GCMD) Platform Keywords

  instrument :

  Earth Remote Sensing Instruments > Passive Remote Sensing > Spectrometers/Radiometers > Imaging Spectrometers/Radiometers > AVHRR > Advanced Very High Resolution Radiometer

  instrument_vocabulary :

  Global Change Master Directory (GCMD) Instrument Keywords

  standard_name_vocabulary :

  CF Standard Name Table (v40, 25 January 2017)

  geospatial_lat_min :

  -90.0

  geospatial_lat_max :

  90.0

  geospatial_lon_min :

  0.0

  geospatial_lon_max :

  360.0

  geospatial_lat_units :

  degrees_north

  geospatial_lat_resolution :

  0.25

  geospatial_lon_units :

  degrees_east

  geospatial_lon_resolution :

  0.25

  time_coverage_start :

  2025-01-15T00:00:00Z

  time_coverage_end :

  2025-01-15T23:59:59Z

  metadata_link :

  https://doi.org/10.25921/RE9P-PT57

  ncei_template_version :

  NCEI_NetCDF_Grid_Template_v2.0

  comment :

  Data was converted from NetCDF-3 to NetCDF-4 format with metadata updates in November 2017.

  sensor :

  Thermometer, AVHRR

Export xarray Datasets as netCDF file

As in the case of pandas DataFrame, sometimes we want to save the Dataset obtained after some manipulations into a new netCDF file. We can do so using the .to_netcdf() method of Dataset and DataArray. Importantly, you may want to make sure that each data variable is compressed so that your file will not be exceedingly large. The way to specify compression is to supply a nested dictionary to the encoding argument, where each data variable is a key, with the value itself a dictionary specifying the compression options. As an example, to save the oisst_all Dataset we created

# NOTE: the output folder has to already exist
oisst_all.to_netcdf("output/oisst_all.nc", encoding = {
    "sst": {"zlib": True, "complevel": 9},
    "anom": {"zlib": True, "complevel": 9},
    "err": {"zlib": True, "complevel": 9},
    "ice": {"zlib": True, "complevel": 9}
})