python-xarray - 带有 GroupBy 的 xarray.apply

问题描述

我正在使用xarray.apply_ufunc()将函数应用于 xarray.DataArray 。它适用于某些 NetCDF，而与其他在尺寸、坐标等方面看起来可比的 NetCDF 则无效。但是，代码适用的 NetCDF 与代码失败的 NetCDF 之间肯定存在一些不同，希望有人可以在看到下面列出的文件的代码和一些元数据后评论问题所在。

我正在运行以执行计算的代码是这样的：

# open the precipitation NetCDF as an xarray DataSet object
dataset = xr.open_dataset(kwrgs['netcdf_precip'])

# get the precipitation array, over which we'll compute the SPI
da_precip = dataset[kwrgs['var_name_precip']]

# stack the lat and lon dimensions into a new dimension named point, so at each lat/lon
# we'll have a time series for the geospatial point, and group by these points
da_precip_groupby = da_precip.stack(point=('lat', 'lon')).groupby('point')

# apply the SPI function to the data array
da_spi = xr.apply_ufunc(indices.spi,
                        da_precip_groupby)

# unstack the array back into original dimensions
da_spi = da_spi.unstack('point')

有效的 NetCDF 如下所示：

>>> import xarray as xr
>>> ds_good = xr.open_dataset("good.nc")
>>> ds_good
<xarray.Dataset>
Dimensions:  (lat: 38, lon: 87, time: 1466)
Coordinates:
  * lat      (lat) float32 24.5625 25.229166 25.895834 ... 48.5625 49.229168
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2017-02-01
Data variables:
    prcp     (lat, lon, time) float32 ...
Attributes:
    Conventions:               CF-1.6, ACDD-1.3
    ncei_template_version:     NCEI_NetCDF_Grid_Template_v2.0
    title:                     nClimGrid
    naming_authority:          gov.noaa.ncei
    standard_name_vocabulary:  Standard Name Table v35
    institution:               National Centers for Environmental Information...
    geospatial_lat_min:        24.5625
    geospatial_lat_max:        49.354168
    geospatial_lon_min:        -124.6875
    geospatial_lon_max:        -67.020836
    geospatial_lat_units:      degrees_north
    geospatial_lon_units:      degrees_east
    NCO:                       4.7.1
    nco_openmp_thread_number:  1
>>> ds_good.prcp
<xarray.DataArray 'prcp' (lat: 38, lon: 87, time: 1466)>
[4846596 values with dtype=float32]
Coordinates:
  * lat      (lat) float32 24.5625 25.229166 25.895834 ... 48.5625 49.229168
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2017-02-01
Attributes:
    valid_min:      0.0
    units:          millimeter
    valid_max:      2000.0
    standard_name:  precipitation_amount
    long_name:      Precipitation, monthly total

失败的 NetCDF 如下所示：

>>> ds_bad = xr.open_dataset("bad.nc")   >>> ds_bad
<xarray.Dataset>
Dimensions:  (lat: 38, lon: 87, time: 1483)
Coordinates:
  * lat      (lat) float32 49.3542 48.687534 48.020866 ... 25.3542 24.687532
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2018-07-01
Data variables:
    prcp     (lat, lon, time) float32 ...
Attributes:
    date_created:              2018-02-15 10:29:25.485927
    date_modified:             2018-02-15 10:29:25.486042
    Conventions:               CF-1.6, ACDD-1.3
    ncei_template_version:     NCEI_NetCDF_Grid_Template_v2.0
    title:                     nClimGrid
    naming_authority:          gov.noaa.ncei
    standard_name_vocabulary:  Standard Name Table v35
    institution:               National Centers for Environmental Information...
    geospatial_lat_min:        24.562532
    geospatial_lat_max:        49.3542
    geospatial_lon_min:        -124.6875
    geospatial_lon_max:        -67.020836
    geospatial_lat_units:      degrees_north
    geospatial_lon_units:      degrees_east
>>> ds_bad.prcp
<xarray.DataArray 'prcp' (lat: 38, lon: 87, time: 1483)>
[4902798 values with dtype=float32]
Coordinates:
  * lat      (lat) float32 49.3542 48.687534 48.020866 ... 25.3542 24.687532
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2018-07-01
Attributes:
    valid_min:      0.0
    long_name:      Precipitation, monthly total
    standard_name:  precipitation_amount
    units:          millimeter
    valid_max:      2000.0

当我对上面的第一个文件运行代码时，它可以正常工作。使用第二个文件时，出现如下错误：

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/paperspace/git/climate_indices/scripts/process_grid_ufunc.py", line 278, in compute_write_spi
    kwargs=args_dict)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 974, in apply_ufunc
    return apply_groupby_ufunc(this_apply, *args)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 432, in apply_groupby_ufunc
    applied_example, applied = peek_at(applied)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/utils.py", line 133, in peek_at
    peek = next(gen)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 431, in <genexpr>
    applied = (func(*zipped_args) for zipped_args in zip(*iterators))
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 987, in apply_ufunc
    exclude_dims=exclude_dims)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 211, in apply_dataarray_ufunc
    result_var = func(*data_vars)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 579, in apply_variable_ufunc
    .format(data.ndim, len(dims), dims))
ValueError: applied function returned data with unexpected number of dimensions: 1 vs 2, for dimensions ('time', 'point')

任何人都可以评论可能是什么问题吗？

标签： python-xarray

python-xarray - 带有 GroupBy 的 xarray.apply_ufunc()：意外的维数

问题描述

解决方案

推荐阅读