java - 将数据加载到 Druid 时出错
问题描述
我是 druid 新手,试图通过本地文件加载数据。我已经设置了节点和 zookeeper 实例。我在 Ubuntu 18.04 中尝试过这个并且工作正常,但我尝试在 lUbuntu 中使用它,我看到以下错误:
2018-07-30T12:25:03,390 ERROR [main] io.druid.cli.CliPeon - Error when starting up. Failing.
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error in custom provider, java.lang.OutOfMemoryError
at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:110) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.DruidProcessingModule)
at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:110) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.DruidProcessingModule)
while locating io.druid.collections.NonBlockingPool<java.nio.ByteBuffer> annotated with @io.druid.guice.annotations.Global()
for the 2nd parameter of io.druid.query.groupby.GroupByQueryEngine.<init>(GroupByQueryEngine.java:81)
at io.druid.guice.QueryRunnerFactoryModule.configure(QueryRunnerFactoryModule.java:88) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.QueryRunnerFactoryModule)
while locating io.druid.query.groupby.GroupByQueryEngine
for the 2nd parameter of io.druid.query.groupby.strategy.GroupByStrategyV1.<init>(GroupByStrategyV1.java:77)
while locating io.druid.query.groupby.strategy.GroupByStrategyV1
for the 2nd parameter of io.druid.query.groupby.strategy.GroupByStrategySelector.<init>(GroupByStrategySelector.java:43)
while locating io.druid.query.groupby.strategy.GroupByStrategySelector
for the 1st parameter of io.druid.query.groupby.GroupByQueryQueryToolChest.<init>(GroupByQueryQueryToolChest.java:104)
at io.druid.guice.QueryToolChestModule.configure(QueryToolChestModule.java:101) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.QueryRunnerFactoryModule)
while locating io.druid.query.groupby.GroupByQueryQueryToolChest
while locating io.druid.query.QueryToolChest annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=80, type=MAPBINDER, keyType=java.lang.Class<? extends io.druid.query.Query>)
at io.druid.guice.DruidBinders.queryToolChestBinder(DruidBinders.java:45) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.QueryRunnerFactoryModule -> com.google.inject.multibindings.MapBinder$RealMapBinder)
while locating java.util.Map<java.lang.Class<? extends io.druid.query.Query>, io.druid.query.QueryToolChest>
for the 1st parameter of io.druid.query.MapQueryToolChestWarehouse.<init>(MapQueryToolChestWarehouse.java:36)
while locating io.druid.query.MapQueryToolChestWarehouse
while locating io.druid.query.QueryToolChestWarehouse
for the 1st parameter of io.druid.server.QueryLifecycleFactory.<init>(QueryLifecycleFactory.java:52)
at io.druid.server.QueryLifecycleFactory.class(QueryLifecycleFactory.java:52)
while locating io.druid.server.QueryLifecycleFactory
for the 1st parameter of io.druid.server.QueryResource.<init>(QueryResource.java:113)
at io.druid.server.QueryResource.class(QueryResource.java:78)
while locating io.druid.server.QueryResource
Caused by: java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at io.druid.offheap.OffheapBufferGenerator.get(OffheapBufferGenerator.java:53)
at io.druid.offheap.OffheapBufferGenerator.get(OffheapBufferGenerator.java:29)
at io.druid.collections.StupidPool.makeObjectWithHandler(StupidPool.java:112)
at io.druid.collections.StupidPool.<init>(StupidPool.java:83)
at io.druid.guice.DruidProcessingModule.getIntermediateResultsPool(DruidProcessingModule.java:115)
at io.druid.guice.DruidProcessingModule$$FastClassByGuice$$8e266e5c.invoke(<generated>)
at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264)
at com.google.inject.internal.ProviderMethod$Factory.provision(ProviderMethod.java:401)
at com.google.inject.internal.ProviderMethod$Factory.get(ProviderMethod.java:376)
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
我确实尝试增加中层管理人员的内存,假设这是导致问题的原因,但没有奏效。任何帮助,将不胜感激。谢谢 !
解决方案
我认为您需要增加直接缓冲内存。参考:https ://groups.google.com/forum/#!topic/druid-user/clGruLMHAgU
在中间管理器配置中,您会发现此选项 druid.indexer.runner.javaOpts 将其设置为更高的值 -XX:MaxDirectMemorySize=4g
参考:druid.io/docs/latest/configuration/index.html
推荐阅读
- r - 在 y 轴上的下一个值之间留出空间
- python - 如何通过网络抓取获取 Airbnb 房源的纬度和经度值
- apache-spark - Spark UI -> SQL 选项卡不显示所有(旧)阶段
- mysql - MySQL 更新查询滞后
- amazon-web-services - 无法将 AWS SES 配置为从 Cognito 发送电子邮件
- html - 如何将我的网站主页设置为“/”?
- python - 检查产品的可用性
- c++ - 类析构函数改变程序结果
- java - Java 使用 JNA 读取其他进程的内存
- powershell - 对powershell中的空字符串应用拆分后,相等运算符不产生真/假输出的原因是什么?