时间戳

首页 > 解决方案 > 如何使用 QuickCheck 调试发散测试

haskell - 如何使用 QuickCheck 调试发散测试

问题描述

我有一些使用 Megaparsec 的解析代码，我编写了一个简单的属性来测试（它生成一个随机表达式树，漂亮地打印它，然后检查结果是否解析回原始树）。

不幸的是，似乎有一个错误，如果我没有任何限制地运行测试，我会看到 GHC 进程分配越来越多的内存，直到我杀死它或 OOM 杀手首先到达那里。

没问题，我想……但我终其一生都无法弄清楚导致分歧的原因。属性本身看起来像这样：（我已经删除了正确的测试和缩小的代码，以尽量减少实际运行的代码）

prop_parse_expr :: Property
prop_parse_expr =
  forAll arbitrary $
  (\ pe ->
     let str = prettyParExpr 0 pe in
       counterexample ("Rendered to: " ++ show str) $
       trace ("STARTING TEST: " ++ show str) $
       case parse (expr <* eof) "" str of
         Left _ -> trace "NOPE" $ False
         Right _ -> trace "GOOD" $ True)

如果我使用分析（使用stack test --profile）进行编译，我可以使用 RTS 选项运行生成的二进制文件。啊，我想，然后跑了-xc，以为当我向卡住的工作发送 SIGINT 时，我会得到一个有用的堆栈跟踪。似乎没有。与运行

./long/path/to/foo-test -j1 --test-seed 1 +RTS -xc

我看到这个输出：

STARTING TEST: "0"
GOOD
STARTING TEST: "(x [( !0 )]) "
STARTING TEST: "({ 2 {( !0 )}} ) "
STARTING TEST: "{ 2{ ( x[0? {( 0) ,( x ) } :((0 )? (x ):0) -: ( -(^( x  )) ) ]), 0**( x )} } "
STARTING TEST: "| (0? (x[({ 1{ (0)? x : ( 0 ) }} ) ]) :(~&( 0) ?( x):( (x ) ^( x ) )))"
STARTING TEST: "(0 )"
STARTING TEST: "0"
^C*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main
<snip: 2 more identical backtraces>
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Runners.Console.Utilities.hideCursorDuring,
  called from Test.Framework.Runners.Console.Run.showRunTestsTop,
  called from Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main

谁能告诉我：

  1. 尽管有-j1，为什么我看到多STARTING TEST行没有GOOD或在它们之间？NOPE

  2. 我如何获得显示测试分配所有内存的实际堆栈跟踪？

感谢您的任何想法！

标签： haskellghcquickcheck

解决方案

对于发现这个问题的任何人，我的代码的问题是我arbitrary的表达式实例没有正确限制大小，因此有时会尝试制作巨大的树。请参阅QuickCheck 手册的“生成递归数据类型”部分，了解我应该做的事情！

我发现运行命令如下：

./long/path/to/foo-test -o3 +RTS -xc

帮助我弄清楚发生了什么。奇怪的是，回溯仍然显示了几个执行线程。我真的不明白为什么，但至少我可以看到我花时间在我的“ makeAnExpr”功能上。诀窍是调整超时（以上 3 秒），这样它就不会终止测试，直到它真正卡住，但在它开始吃掉你的所有 RAM 之前停止它！

推荐阅读