tensorflow - Tensorflow 模型未加载 gunicorn
问题描述
我正在尝试将机器学习工作负载转移到生产中,现在工作负载正在 Flask 中运行,但它不是为生产而设计的,而且我也面临一些性能问题。我尝试将 gunicorn 与 Flask 一起使用,但在加载 tensorflow 模型和 gunicorn 服务器在加载模型之前启动时遇到了一些问题。以下是代码。
wsgi.py
from ml_service import service_api as application
import time
if __name__ == "__main__":
# initialize the session
ml_service.init()
# load model
ml_service.load_models()
application.run()
ml_service.py
def init(num_gpu=1, num_cpu=1, num_cores=4):
global session
init_op = tf.global_variables_initializer()
configuration = tf.ConfigProto(
intra_op_parallelism_threads=num_cores,
inter_op_parallelism_threads=num_cores, allow_soft_placement=True,
# gpu_options=gpu_options,
device_count={'CPU': num_cpu, 'GPU': num_gpu}
)
session = tf.Session(config=configuration)
session.run(init_op)
def load_models():
try:
#Do Stuff
except:
#Do Stuff
@service_api.route("/serviceendpoint", methods=["POST"])
def get_prediction():
#Do Stuff
return
if __name__ == "__main__":
service_api.run(host='0.0.0.0')
运行 gunicorn 的命令:
gunicorn --bind 0.0.0.0:5000 wsgi
输出:
[2019-12-17 14:16:18 +0000] [7] [INFO] Starting gunicorn 20.0.4
[2019-12-17 14:16:18 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7)
[2019-12-17 14:16:18 +0000] [7] [INFO] Using worker: sync
[2019-12-17 14:16:18 +0000] [10] [INFO] Booting worker with pid: 10
Using TensorFlow backend.
预期输出:
Using TensorFlow backend.
2019-12-17 13:17:00.497866: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-17 13:17:02.556270: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-17 13:17:02.556854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2019-12-17 13:17:02.556902: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-12-17 13:17:02.970751: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-17 13:17:02.970851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-12-17 13:17:02.970879: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-12-17 13:17:02.971248: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10759 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
2019-12-17 13:17:03.197540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-12-17 13:17:03.197611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-17 13:17:03.197626: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-12-17 13:17:03.197636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-12-17 13:17:03.197822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10759 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
../posenet/converter/config.py:9: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
cfg = yaml.load(cfg_f)
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
解决方案
推荐阅读
- angular - 如何删除当前点击的html div标签角4
- ios - 在 TableView 上添加搜索栏但没有响应
- replace - 同时替换多个列值
- android - Retrofit2 未找到任何改造注释。(参数#1)
- python - 如何使用 MinMaxScaler sklearn 标准化训练和测试数据
- asp.net - IIS - 可以使用 localhost 访问端口 8080 的网站,但不能使用外部 IP 地址
- c# - 在 mvc 视图模型中获取当前操作
- android - 单击我正在调用异步任务的按钮时出错?
- ionic-framework - 如何使用 Ionic 3 永久保存数据?
- ios - 无法将“__NSCFNumber”(0x111189058)类型的值转换为“NSString”错误