linux - 多个线程的 TensorFlow 基准测试被杀死
问题描述
昨天,我正在使用 TensorFlow 的 Conda 版本运行一些性能测试,即 TF v1.12:
conda create -n tf_conda python=3.6.8 tensorflow
git clone -b cnn_tf_v1.12_compatible --single-branch https://github.com/tensorflow/benchmarks.git
cd benchmarks/scripts/tf_cnn_benchmarks/
conda activate tf_conda
python tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC --num_batches=10 --num_warmup_batches=1
今天,当我用 Conda 安装 TensorFlow 时,我最终得到了 v1.13:
conda create -n tf_conda python=3.6.8 tensorflow
git clone -b cnn_tf_v1.13_compatible --single-branch https://github.com/tensorflow/benchmarks.git
cd benchmarks/scripts/tf_cnn_benchmarks/
conda activate tf_conda
python tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC --num_batches=10 --num_warmup_batches=1
但是,基准测试失败了:
2019-04-05 08:41:04.146441: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
2019-04-05 08:41:04.164798: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2599995000 Hz
2019-04-05 08:41:04.165499: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x563d6af00290 executing computations on platform Host. Devices:
2019-04-05 08:41:04.165542: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined>
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #213: KMP_AFFINITY: cpuid leaf 11 not supported - decoding legacy APIC ids.
OMP: Info #149: KMP_AFFINITY: Affinity capable, using global cpuid info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-3
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #159: KMP_AFFINITY: 4 packages x 1 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 1
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 2
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 3
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1539 thread 0 bound to OS proc set 0
2019-04-05 08:41:04.166391: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
W0405 08:41:04.172703 140193590863680 deprecation.py:323] From /home/nsapay/anaconda3/envs/tf_conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
...
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1557 thread 1 bound to OS proc set 1
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1585 thread 3 bound to OS proc set 3
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1586 thread 4 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1584 thread 2 bound to OS proc set 2
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1556 thread 5 bound to OS proc set 1
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1588 thread 7 bound to OS proc set 3
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1587 thread 6 bound to OS proc set 2
OMP: Info #250: KMP_AFFINITY: pid 1539 tid 1589 thread 8 bound to OS proc set 0
Killed.
我完全感到困惑......我昨天和今天之间没有更新系统。我正在使用 4 个(虚拟)内核(Intel Xeon E3)和一个常规的 CentOS 7 发行版。
解决方案
确实,我已经失忆了。内存监控来自另一台虚拟机(同时与多台机器一起工作的不便)。
推荐阅读
- python - 迭代循环以获取每个不同日期变量的数据帧变量的 cumsum(非聚合)
- bash - bash 命令可以多长时间?
- c# - 如何在 C# 中以编程方式确定 Windows 7 的子版本?
- python - manage.py runserver后如何自动执行代码
- android - 尝试将 Content Uri 从活动传递到服务时出现权限错误
- macos - 子目录项目的 Composer 卸载 (Mac)
- excel - Excel VBA 中有没有办法检查一个工作簿中的数字序列,找到最后一个,然后在另一个工作簿中继续?
- node.js - 一段时间后,Socket io 连接断开
- c# - 如何访问从asp.net核心中的jquery文件上传发送的附加表单数据?
- r - 如何在 R 中重现以下涉及 for 循环的 matlab 代码?