时间戳

首页 > 解决方案 > Numba：“此错误通常是由于传递了指定函数不支持的类型的参数引起的”

python - Numba：“此错误通常是由于传递了指定函数不支持的类型的参数引起的”

问题描述

当使用 Numba 的 @jitclass 装饰器在对象上调用类方法时，似乎不支持“self”参数。不知道如何处理这里的错误。一切都编译得很好，但是尽管调用了其他 numpy 函数，但各个方法并没有运行。

spec = [('raster',numba.float32[:,:]),('height', numba.int32),('width', numba.int32),('azis', numba.int64[:]),('grid',numba.int64),('rough',numba.float64[:,:]),('maxrange',numba.float64[:,:]),('aziratio',numba.float64[:,:]),('labels',numba.float64[:,:])]

@jitclass(spec)
class raster_class(object):
    def __init__(self,raster):
        self.raster = raster
        self.height =self.raster.shape[0]
        self.width = self.raster.shape[1]
        self.azis =  np.arange(0,170,10)
        self.grid = 500
        x = np.int(self.height/self.grid)
        y = np.int(self.width/self.grid)
        self.rough = np.zeros((x,y))
        self.maxrange = np.zeros((x,y)) 
        self.aziratio = np.zeros((x,y))
        self.labels = np.zeros((x,y))


    def detrend(self):
        raster -= ndimage.gaussian_filter(self.raster,sigma=40)
        return raster


    def SR(self,image):
        image = image[~np.isnan(image)] # remove nan's
        image = np.ndarray.flatten(image)
        mean = np.mean(image)

        return np.sqrt((1/(len(image)-1))*np.sum((image-mean)**2))
    def getRange(self,mat):
    # fits an anisotropic variogram model and returns the effective range for a given azimuth
        m,n = mat.shape
        vals = np.reshape(mat,(m*n,1))
        coords = []
        for i in range(m):
            for j in range(n):
                coords.append((i,j))
                coords = np.array(coords)
                response = np.hstack((coords,vals))
                response = response[~np.isnan(response[:,-1])]
                response = response[response[:,-1] != 0]
                response = response[~np.isnan(response[:,-1])]
                coords = response[:,:2]
                response = response[:,2]
                response += np.random.normal(0,scale=0.25,size=response.shape[0]) #add noise to prevent same values

         azi_r = []
         for azi in self.azis:
            DV =  DirectionalVariogram(coords,response,azimuth=azi,tolerance=15,maxlag=250,n_lags=20) 
            azi_r.append(DV.cof[0])

        major = np.argmax(azi_r)
        large_range = azi_r[major]
        major = azis[major]

        if major  >= 90:
            perp = major - 90
        else:
            perp = major + 90
        minor = azis.index(perp)
        minor_range = azi_r(minor)
        ratio = large_range/minor_range
        return ratio,large_range

    def iterate(self):
        for i in range(0,self.height-self.grid,self.grid):
            for j in range(0,self.width-self.grid,self.grid):
                image = self.raster[i:i+self.grid,j:j+self.grid]
                indi = int(i/self.grid)
                indj = int(j/self.grid)
                roughness = self.SR(image)
                ratio,range_ = self.getRange(image)
                self.azi_ratio[indi,indj] = ratio
                self.largest_range[indi,indj] = range_
                self.response_rough[indi,indj] = roughness

if __name__ == "__main__":
    brooks = np.load("brooks_dem.npy")
    brooks_class = raster_class(brooks)
    time = time.time()
    brooks_class.iterate()
    end_time = time.time() - time
    hours = end_time/3600
    print("Computation Took {} Hours".format(hours))

错误信息

This error is usually caused by passing an argument of a type that is unsupported by the 
named function.
[1] During: typing of intrinsic-call at /home/dunbar/DEM/processraster.py (35)

File "processraster.py", line 35:
    def SR(self,image):
        image = image[~np.isnan(image)] # remove nan's
        ^

[1] During: resolving callee type: BoundFunction((<class 
'numba.types.misc.ClassInstanceType'>, 'SR') for 
instance.jitclass.raster_class#55ac81be91b8<raster:array(float32, 2d, 
A),height:int32,width:int32,azis:array(int64, 1d, A),grid:int64,rough:array(float64, 2d, 
A),maxrange:array(float64, 2d, A),aziratio:array(float64, 2d, A),labels:array(float64, 2d, 
A)>)
[2] During: typing of call at /home/dunbar/DEM/processraster.py (81)


File "processrabster.py", line 81:
    def iterate(self):
        <source elided>
                indj = int(j/self.grid)
                roughness = self.SR(image)
                ^

[1] During: resolving callee type: BoundFunction((<class 
'numba.types.misc.ClassInstanceType'>, 'iterate') for 
instance.jitclass.raster_class#55ac81be91b8<raster:array(float32, 2d, 
A),height:int32,width:int32,azis:array(int64, 1d, A),grid:int64,rough:array(float64, 2d, 
A),maxrange:array(float64, 2d, A),aziratio:array(float64, 2d, A),labels:array(float64, 2d, 
A)>)
[2] During: typing of call at <string> (3)


File "<string>", line 3:
<source missing, REPL/exec in use?>

标签： pythonnumpyparallel-processingnumba

解决方案

问题似乎出在SR方法上，但不幸的是，jitclass 错误消息除此之外并没有提供太多信息。但是，由于这是一个静态方法，调试它的一种非常简单的方法是将其作为独立函数进行测试，即SR从类中取出，删除self参数，添加@njit装饰器，然后SR在任意二维数组上运行.

当我这样做时，我发现以下两个问题：

  1. image[~np.isnan(image)]是“花式”或“高级”索引的一种形式，因为它使用布尔数组作为输入。Numba仅支持对单个维度的高级索引，但image它是二维的。

  2. 您正在调用flatten它的类中的函数ndarray，即 ie np.ndarray.flatten(image)，但 Numba 只识别更标准的方法调用image.flatten()。

您可以通过切换两行的顺序来修复第 1 点，image = image.flatten()先写（或者image = image.ravel()，因为不需要复制），然后再写image = image[~np.isnan(image)]。

幸运的是，对于您的特定应用程序，无需执行任何此操作，因为该方法似乎SR可以替换为np.nanmeanNumba 支持的对 的调用。

更一般地说，我赞同这样的评论，即用 Numba 编译这样的大型类并不是真正的预期用途（至少目前如此）；更好地通过分析识别一些瓶颈并专门编译这些瓶颈。

推荐阅读