时间戳

首页 > 解决方案 > 如何在 Julia 中进行正确的微基准测试？

julia - 如何在 Julia 中进行正确的微基准测试？

问题描述

Julia 1.0.0文档提供了一般提示。

它还建议不要使用 @time 宏：

对于更严格的基准测试，请考虑 BenchmarkTools.jl 包，其中包括多次评估函数以减少噪音。

它们在使用中如何比较？使用“基础”Julia 之外的东西是否值得麻烦？

标签： julia

解决方案

从统计的角度来看，@benchmark 比@time 好很多

TL;DR BenchmarkTools@benchmark宏是一个很棒的微型基准测试工具。谨慎使用@time宏，不要认真对待第一次运行。

这个简单的例子说明了用途和区别：

julia> # Fresh Julia 1.0.0 REPL

julia> # Add BenchmarkTools package using ] key package manager

(v1.0) pkg> add BenchmarkTools  
julia> # Press backspace key to get back to Julia REPL

# Load BenchmarkTools package into current REPL
julia> using BenchmarkTools

julia> # Definine a function with a known elapsed time
julia> f(n) = sleep(n)  # n is in seconds
f (generic function with 1 method)

# Expect just over 500 ms for elapsed time
julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     501.825 ms (0.00% GC)
  median time:      507.386 ms (0.00% GC)
  mean time:        508.069 ms (0.00% GC)
  maximum time:     514.496 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1

julia> # Try second run to compare consistency
julia> # Note the very close consistency in ms for both median and mean times

julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     502.603 ms (0.00% GC)
  median time:      508.716 ms (0.00% GC)
  mean time:        508.619 ms (0.00% GC)
  maximum time:     515.602 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1


julia> # Define the same function with new name for @time macro tests
julia> g(n) = sleep(n)
g (generic function with 1 method)

# First run suffers from compilation time, so 518 ms
julia> @time sleep(0.5)
  0.517897 seconds (83 allocations: 5.813 KiB)

# Second run drops to 502 ms, 16 ms drop
julia> @time sleep(0.5)
  0.502038 seconds (9 allocations: 352 bytes)

# Third run similar to second
julia> @time sleep(0.5)
  0.503606 seconds (9 allocations: 352 bytes)

# Fourth run increases over second by about 13 ms
julia> @time sleep(0.5)
  0.514629 seconds (9 allocations: 352 bytes)

这个简单的例子说明了使用宏是多么容易，@benchmark并且应该谨慎对待@time宏结果。

@benchmark是的，使用宏是值得的。

推荐阅读