python - Cython 函数中高度可变的执行时间

Question

在从 Python 引擎调用的 C 编译函数（通过 scipy.weave）执行迁移到 Cython 时，我遇到了性能测量问题。

新的 cython 函数端到端分析cProfile（如果没有必要，我不会深入研究 cython 分析）记录累积测量时间高度可变。

例如。每 5 次重复执行 9 次 cython 函数的累积时间（在 5 次执行的预热之后 - 分析函数未考虑）正在占用：

• 在第一轮 215,627339 秒
• 在第二轮 235,336131 秒

每次执行都会使用不同但固定的参数多次调用函数。也许这种可变性可能取决于测试机器（云托管的专用机器）的 CPU 负载，但我想知道这种可变性（几乎 10%）是否可能取决于 cython 或缺乏优化（我已经使用了关于除法的提示，边界检查，环绕，...）。

关于如何采用可靠指标的任何想法？

标签： pythonperformancecython

Answer 1

First of all, you need to ensure that your measurement device is capable of measuring what you need: specifically, only the system resources you consume. UNIX's utime is one such command, although even that one still includes swap time. Check the documentation of your profiler: it should have capabilities to measure only the CPU time consumed by the function. If so, then your figures are due to something else.

Once you've controlled the external variations, you need to examine the internal. You've said nothing about the complexion of your function. Some (many?) functions have available short-cuts for data-driven trivialities, such as multiplication by 0 or 1. Some are dependent on an overt or covert iteration that varies with the data. You need to analyze the input data with respect to the algorithm.

One tool you can use is a line-oriented profiler to detail where the variations originate; seeing which lines take the extra time should help determine where the "noise" comes from.