ruby-on-rails - Kafka/Ruby on Rails - 我不理解运行“bundle exec karafka server”后生成的日志
问题描述
我是微服务领域的新手。我正在将一个整体 Web 应用程序迁移到一个微服务应用程序。我在将负责身份验证的 Ruby on Rails Web 应用程序与 KAFKA 连接时遇到问题。
Ruby on Rails 本身没有任何问题。POC 可在此处获得https://github.com/maelfosso/authentication-kafka。HTTP 请求工作正常。
现在我想将它与 Apache Kafka 连接起来。我正在使用Karafka。感谢他的作者，我终于成功设置了它。但是在启动它时，我得到了一个我不明白的日志。你可以在这里阅读https://gist.github.com/maelfosso/3be57de6bede85f67b36b0143fd6ae58
这是我对 Apache Kafka 的第一次体验，我尽可能简单地开始它，如快速入门部分所述
从一个 NodeJs 微服务，我可以产生一条消息。我可以使用bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic business.
这是一些多次出现的部分
Fetching cluster metadata from kafka://192.168.8.101:9092
[topic_metadata] Opening connection to 192.168.8.101:9092 with client id nkapsi_accounts_development...
[topic_metadata] Sending topic_metadata API request 1 to 192.168.8.101:9092
[topic_metadata] Waiting for response 1 from 192.168.8.101:9092
[topic_metadata] Received response 1 from 192.168.8.101:9092
Discovered cluster metadata; nodes: 192.168.8.101:9092 (node_id=0)
Closing socket to 192.168.8.101:9092
我不明白它为什么要关闭。
[[nkapsi_accounts_development_business] {business: 0}:] Fetching cluster metadata from kafka://192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [topic_metadata] Opening connection to 192.168.8.101:9092 with client id nkapsi_accounts_development...
[[nkapsi_accounts_development_business] {business: 0}:] [topic_metadata] Sending topic_metadata API request 1 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [topic_metadata] Waiting for response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [topic_metadata] Received response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Discovered cluster metadata; nodes: 192.168.8.101:9092 (node_id=0)
[[nkapsi_accounts_development_business] {business: 0}:] Closing socket to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Opening connection to 192.168.8.101:9092 with client id nkapsi_accounts_development...
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Sending fetch API request 1 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Waiting for response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Received response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Fetching batches
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Sending fetch API request 2 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Waiting for response 2 from 192.168.8.101:9092
等待响应 2（如果我不停止响应，最多 9 个）？它在等什么？有没有我错过的配置？
New [Business] event:
Params deserialization error for business topic: 767: unexpected token at '{
activities: [],
_id: 5f0dd8df919fec6c2294fd28,
delivrables: [],
__v: 0
}'
[[nkapsi_accounts_development_business] {}:] Exception raised when processing business/0 in offset range 0..0 -- Karafka::Errors::DeserializationError: 767: unexpected token at '{
activities: [],
_id: 5f0dd8df919fec6c2294fd28,
delivrables: [],
__v: 0
}'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/serialization/json/deserializer.rb:22:in `rescue in call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/serialization/json/deserializer.rb:19:in `call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/params/params.rb:60:in `block in deserialize'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `block in instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:11:in `measure'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/params/params.rb:59:in `deserialize'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/params/params.rb:51:in `deserialize!'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/params/params_batch.rb:42:in `first'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/consumers/single_params.rb:11:in `params'
/home/maelfosso/Documents/Projects/Nkapsi/api/accounts/app/consumers/business_consumer.rb:5:in `consume'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/backends/inline.rb:12:in `block in process'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `block in instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:11:in `measure'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/backends/inline.rb:12:in `process'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/base_consumer.rb:38:in `call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/batch_delegator.rb:43:in `block (2 levels) in call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/batch_delegator.rb:38:in `each'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/batch_delegator.rb:38:in `block in call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `block in instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:11:in `measure'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/dry-monitor-0.3.2/lib/dry/monitor/notifications.rb:43:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/batch_delegator.rb:18:in `call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/listener.rb:46:in `block in fetch_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/client.rb:36:in `block in fetch_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:331:in `block (3 levels) in each_batch'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/activesupport-6.0.3.2/lib/active_support/notifications.rb:182:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/instrumenter.rb:21:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/instrumenter.rb:35:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:329:in `block (2 levels) in each_batch'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:310:in `each'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:310:in `block in each_batch'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/patches/ruby_kafka.rb:28:in `block in consumer_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:414:in `block in consumer_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/activesupport-6.0.3.2/lib/active_support/notifications.rb:182:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/instrumenter.rb:21:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/instrumenter.rb:35:in `instrument'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:412:in `consumer_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/patches/ruby_kafka.rb:16:in `consumer_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:307:in `each_batch'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/client.rb:36:in `fetch_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/listener.rb:39:in `fetch_loop'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/connection/listener.rb:24:in `call'
/home/maelfosso/.rvm/gems/ruby-2.6.5/gems/karafka-1.3.6/lib/karafka/fetcher.rb:18:in `block (2 levels) in call'
[[nkapsi_accounts_development_business] {business: 0}:] [fetch] Received response 2 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Handling fetcher command: stop
[nkapsi_accounts_development_business] {business: 0}: Fetcher thread exited.
[[nkapsi_accounts_development_business] {}:] Leaving group `nkapsi_accounts_development_business`
[[nkapsi_accounts_development_business] {}:] [leave_group] Sending leave_group API request 6 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {}:] [leave_group] Waiting for response 6 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {}:] [leave_group] Received response 6 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {}:] Disconnecting broker 0
[[nkapsi_accounts_development_business] {}:] Closing socket to 192.168.8.101:9092
Client fetch loop error: 767: unexpected token at '{
activities: [],
_id: 5f0dd8df919fec6c2294fd28,
delivrables: [],
__v: 0
}'
我不明白这个问题。它只是从NodeJS应用程序发送的 json 消息。就是这里
{
"email": "contact@nkapsi.cm",
"name": "nkapsi",
"website: "nkapsi.cm",
"phone": "666555444"
}
还有那个消息显示了很多次
[[nkapsi_accounts_development_business] {business: 0}:] Joining group `nkapsi_accounts_development_business`
[[nkapsi_accounts_development_business] {business: 0}:] Handling fetcher command: configure
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Opening connection to 192.168.8.101:9092 with client id nkapsi_accounts_development...
[[nkapsi_accounts_development_business] {business: 0}:] Fetching batches
[[nkapsi_accounts_development_business] {business: 0}:] There are no partitions to fetch from, sleeping for 1s
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Sending join_group API request 1 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Waiting for response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Received response 1 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Failed to join group; resetting member id and retrying in 1s...
[[nkapsi_accounts_development_business] {business: 0}:] Fetching batches
[[nkapsi_accounts_development_business] {business: 0}:] There are no partitions to fetch from, sleeping for 1s
[[nkapsi_accounts_development_business] {business: 0}:] Joining group `nkapsi_accounts_development_business`
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Sending join_group API request 2 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Waiting for response 2 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [join_group] Received response 2 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Joined group `nkapsi_accounts_development_business` with member id `nkapsi_accounts_development-81a33adb-ff6d-4066-8a7f-b54fd83aeee7`
[[nkapsi_accounts_development_business] {business: 0}:] Chosen as leader of group `nkapsi_accounts_development_business`
[[nkapsi_accounts_development_business] {business: 0}:] [sync_group] Sending sync_group API request 3 to 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [sync_group] Waiting for response 3 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] [sync_group] Received response 3 from 192.168.8.101:9092
[[nkapsi_accounts_development_business] {business: 0}:] Partitions assigned for `business`: 0
我不明白。
我检查了安装的 Kafka192.168.8.101:9092并且它正在工作。我可以从 NodeJS 应用程序或控制台生产者生成消息，并使用控制台消费者使用它们。
请问，有什么问题吗？完整的日志文件在这里https://gist.github.com/maelfosso/3be57de6bede85f67b36b0143fd6ae58和 Ruby on Rails POC 在这里https://github.com/maelfosso/authentication-kafka
谢谢
解决方案
推荐阅读
- ios - 如何映射 API？
- ansible - 在复杂列表变量中使用模板
- visual-studio-code - 如何在VSCode集成终端Ctrl+Shift+←/→选择上一个或下一个单词？
- python - 如何在 Python 中获取 dBA 噪声级别？
- angular - 如何设置Angular 8轮事件的灵敏度？
- python - Python 根据其他列表求和列表元素
- javascript - 通过计时器重置文本框文本并再次关注它，在javascript中
- java - 用右移编码整数
- c++ - 在 C++ 中按下左键和释放左键时如何获取鼠标光标坐标？
- .htaccess - CakePHP 3 htaccess 忽略目录