时间戳

首页 > 解决方案 > 本地风暴集群设置 - 未执行

apache - 本地风暴集群设置 - 未执行

问题描述

我正在尝试在同一台机器上运行 zookeeper、nimbus 和 supervisor 来模拟 Storm 集群（我理解 Storm 被设计为分布式系统，但为了学习想要模拟单台机器上的工作方式）。

Zookeeper、Nimbus 和 Supervisor 启动良好，没有错误，我提交了一个 Storm-starter 拓扑（RollingTopWords）。Storm.yaml 的 zookeeper 和 nimbus 都指向 127.0.0.1。Nimbus 得到了拓扑并且似乎开始下载到主管，但由于某种原因在主管上看不到任何执行。除了启动消息之外，主管日志也是空的。

Nimbus 日志在下面（最后几个），之后我看不到任何东西。

2019-12-13 02:43:01.818 o.a.s.d.n.Nimbus timer [INFO] Assigning top1-1-1576204977 to 1 slots 2019-12-13 02:43:01.818 o.a.s.d.n.Nimbus timer [INFO] Assign executors: [[14, 14], [12, 12], [10, 10], [8, 8], [6, 6], [4, 4], [2, 2], [7, 7], [5, 5], [3, 3], [1, 1], [15, 15], [13, 13], [11, 11], [9, 9]] 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Fragmentation after scheduling is: 0.0 MB, 0 PCore CPUs 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Node Id: 1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3 Total Mem: 4096.0, Used Mem: 1920.0, Available Mem: 2176.0, Total CPU: 400.0, Used CPU: 150.0, Available CPU: 250.0, fragmented: false 2019-12-13 02:43:01.819 o.a.s.d.n.Nimbus timer [INFO] Setting new assignment for topology id top1-1-1576204977: Assignment(master_code_dir:storm-local, node_host:{1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3=centos-7-1.asia-southeast1-b.c.mandava-forever.internal}, executor_node_port:{[14, 14]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [12, 12]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [10, 10]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [8, 8]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [6, 6]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [4, 4]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [2, 2]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [7, 7]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [5, 5]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [3, 3]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [1, 1]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [15, 15]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [13, 13]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [11, 11]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700]), [9, 9]=NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700])}, executor_start_time_secs:{[2, 2]=1576204981, [4, 4]=1576204981, [6, 6]=1576204981, [8, 8]=1576204981, [10, 10]=1576204981, [12, 12]=1576204981, [14, 14]=1576204981, [1, 1]=1576204981, [3, 3]=1576204981, [5, 5]=1576204981, [7, 7]=1576204981, [9, 9]=1576204981, [11, 11]=1576204981, [13, 13]=1576204981, [15, 15]=1576204981}, worker_resources:{NodeInfo(node:1a8a2795-2b6f-4529-862c-61c45d7bf536-10.148.0.3, port:[6700])=WorkerResources(mem_on_heap:1920.0, mem_off_heap:0.0, cpu:150.0, shared_mem_on_heap:0.0, shared_mem_off_heap:0.0, resources:{onheap.memory.mb=1920.0, cpu.pcore.percent=150.0, offheap.memory.mb=0.0}, shared_resources:{})}, total_shared_off_heap:{}, owner:root) 2019-12-13 02:43:02.849 o.a.s.d.n.Nimbus pool-29-thread-41 [INFO] Created download session 2e19912c-f36f-4eb6-9b1e-df51b94adb24 for top1-1-1576204977-stormjar.jar 2019-12-13 02:43:02.851 o.a.s.d.n.Nimbus pool-29-thread-39 [INFO] Created download session 1798dab4-32d0-4112-b20e-e7b96eee4eaf for top1-1-1576204977-stormcode.ser 2019-12-13 02:43:02.859 o.a.s.d.n.Nimbus pool-29-thread-34 [INFO] Created download session b0523592-8374-43f1-b536-22c724bb8836 for top1-1-1576204977-stormconf.ser

标签： apacheapache-zookeeperapache-storm

解决方案

我想通了 - 它实际上正在运行，并且执行日志存储在风暴的每个工作日志中。在我的设置中，它正在登录 /logs/workers-artifacts/-x-xxxx/670x/worker.log

670X 是指主管上的工人端口号。例如 worker1 将有 6701 等。

推荐阅读