python-3.x - Python3. How to optimize decorator for handling exceptions?

Question

I have decorator that I use as exceptions handler. I want to optimize it, because comparing with simple try...catch it is about 6 times slower.

The code of my decorator:

class ProcessException(object):

    __slots__ = ('func', 'custom_handlers', 'exclude')

    def __init__(self, custom_handlers=None):
        self.func = None
        self.custom_handlers: dict = custom_handlers
        self.exclude = [QueueEmpty, QueueFull, TimeoutError]

    def __call__(self, func):
        self.func = func
        return self.wrapper

    def wrapper(self, *args, **kwargs):
        if self.custom_handlers:
            if isinstance(self.custom_handlers, property):
                self.custom_handlers = self.custom_handlers.__get__(self, self.__class__)

        if asyncio.iscoroutinefunction(self.func):
            return self._coroutine_exception_handler(*args, **kwargs)
        else:
            return self._sync_exception_handler(*args, **kwargs)

    async def _coroutine_exception_handler(self, *args, **kwargs):
        try:
            return await self.func(*args, **kwargs)
        except Exception as e:
            if self.custom_handlers and e.__class__ in self.custom_handlers:
                return self.custom_handlers[e.__class__]()

            if e.__class__ not in self.exclude:
                raise e

    def _sync_exception_handler(self, *args, **kwargs):
        try:
            return self.func(*args, **kwargs)
        except Exception as e:
            if self.custom_handlers and e.__class__ in self.custom_handlers:
                return self.custom_handlers[e.__class__]()

            if e.__class__ not in self.exclude:
                raise e

As benchmark I used simple function with try...catch and function with my decorator:

# simple function
def divide(a, b):
    try:
        return a // b
    except ZeroDivisionError:
        return 'error'

# function with decorator
@ProcessException({ZeroDivisionError: lambda: 'err'})
def divide2(a, b):
    return a // b

Result for 10000 iterations of simple function:

timeit.timeit('divide(1, 0)', number=10000, setup='from __main__ import divide')
0.010692662000110431

And function with decorator:

timeit.timeit('divide2(1, 0)', number=10000, setup='from __main__ import divide2')
0.053688491000002614

Help me please to optimize it and please explain where is bottleneck ?

标签： python-3.xperformanceoptimizationpython-decorators

Answer 1

好吧，最后我优化了我的装饰器。所以，代码（下面的解释）：

from inspect import iscoroutinefunction

from asyncio import QueueEmpty, QueueFull
from concurrent.futures import TimeoutError


class ProcessException(object):

    __slots__ = ('func', 'handlers')

    def __init__(self, custom_handlers=None):
        self.func = None

        if isinstance(custom_handlers, property):
            custom_handlers = custom_handlers.__get__(self, self.__class__)

        def raise_exception(e: Exception):
            raise e

        exclude = {
            QueueEmpty: lambda e: None,
            QueueFull: lambda e: None,
            TimeoutError: lambda e: None
        }

        self.handlers = {
            **exclude,
            **(custom_handlers or {}),
            Exception: raise_exception
        }

    def __call__(self, func):
        self.func = func

        if iscoroutinefunction(self.func):
            def wrapper(*args, **kwargs):
                return self._coroutine_exception_handler(*args, **kwargs)
        else:
            def wrapper(*args, **kwargs):
                return self._sync_exception_handler(*args, **kwargs)

        return wrapper

    async def _coroutine_exception_handler(self, *args, **kwargs):
        try:
            return await self.func(*args, **kwargs)
        except Exception as e:
            return self.handlers.get(e.__class__, Exception)(e)

    def _sync_exception_handler(self, *args, **kwargs):
        try:
            return self.func(*args, **kwargs)
        except Exception as e:
            return self.handlers.get(e.__class__, Exception)(e)

首先，我使用 cProfile 查看装饰器进行了哪些调用。我注意到 asyncio.iscoroutinefunction 进行了额外的调用。我已经删除了它。我还删除了所有额外的if内容并为所有处理程序创建了通用字典。我的解决方案仍然不如 try...catch 快，但现在它的工作速度更快。