java - 构建本机映像时,如何调试“映像堆中不允许...的实例”?
问题描述
我有一个使用 Micronaut 2.0.0 实现 RESTful API 的小型 Java 应用程序。在底层,它使用 Redisson 3.13.1 去 Redis。反过来,Redisson 使用 Netty (4.1.49)。
该应用程序在“经典”java 中运行良好(在 HotSpot 上,Java 8 和 11)。
我正在尝试使用 GraalVM 从该应用程序中构建本机映像。
命令大概是这样的:
native-image --no-server --no-fallback -H:+TraceClassInitialization -H:+PrintClassInitialization --report-unsupported-elements-at-runtime --initialize-at-build-time=reactor.core.publisher.Flux,reactor.core.publisher.Mono -H:ConfigurationFileDirectories=target/config -cp target/app-1.0.0-SNAPSHOT.jar com.app.AppApplication target/app
这是我得到的:
Error: Unsupported features in 4 methods
Detailed message:
Error: No instances of java.net.Inet4Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace: Object was reached by
reading field io.netty.channel.socket.InternetProtocolFamily.localHost of
constant io.netty.channel.socket.InternetProtocolFamily@593f1f62 reached by
scanning method io.netty.resolver.dns.DnsNameResolver.preferredAddressType(DnsNameResolver.java:481)
Call path from entry point to io.netty.resolver.dns.DnsNameResolver.preferredAddressType(ResolvedAddressTypes):
at io.netty.resolver.dns.DnsNameResolver.preferredAddressType(DnsNameResolver.java:478)
at io.netty.resolver.dns.DnsNameResolver.<init>(DnsNameResolver.java:436)
at io.netty.resolver.dns.DnsNameResolverBuilder.build(DnsNameResolverBuilder.java:473)
at io.netty.resolver.dns.DnsAddressResolverGroup.newNameResolver(DnsAddressResolverGroup.java:111)
at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:91)
at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:76)
at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:70)
at org.redisson.cluster.ClusterConnectionManager$1.run(ClusterConnectionManager.java:251)
at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:125)
at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:75)
at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:141)
at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:184)
at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
这只是输出的一部分,它还会生成关于其他 3 个错误的类似报告。
我仍在努力理解这个问题,但我想,正如其中java.net.InetAddress
的本机方法一样,它和它的子类java.net.Inet4Address
都不能在构建时初始化。这意味着Inet4Address
对于在构建时(在初始化阶段,用 Java 术语)初始化的代码,一个实例是不可见的。原生图像生成器找到了一种方法,可以达到这样一个对象可见的程度。它甚至显示了跟踪,但问题ClusterConnectionManager$1
是Runnable
它只在运行时提交Executor
(在静态初始化之后)。
你如何调试这种情况?即:
- 你如何找到罪魁祸首?
- 找到罪魁祸首后如何解决?
PS。如果我添加--initialize-at-run-time=java.net.InetAddress
,它会以不同的方式失败:
Error: The class java.net.InetAddress has already been initialized; it is too late
to register java.net.InetAddress for build-time initialization (from the command
line). java.net.InetAddress has been initialized without the native-image
initialization instrumentation and the stack trace can't be tracked. Try avoiding
this conflict by avoiding to initialize the class that caused initialization of
java.net.InetAddress or by not marking java.net.InetAddress for build-time
initialization.
Java 将自身报告为build 25.252-b09-jvmci-20.1-b02, mixed mode
.
聚苯乙烯。我发现这个No instance of ... are allowed in the image heap 因为这个类应该在图像运行时初始化,并且似乎 Quarkus 问题已修复。但我仍然不明白如何解决手头的问题。任何帮助,将不胜感激。
解决方案
TLDR;答案末尾有一小部分带有摘要。
一点理论
在 Java 中,每个类都必须在使用前进行初始化。初始化意味着执行静态字段初始化器和静态初始化块。当然,在标准 JVM(如 HotSpot)中,这发生在运行时。
但是对于本机图像,您有两种选择。一个类可能仍然在运行时被初始化,或者它的初始化可能在构建时被执行。后者有一个明显的好处,就是在原生镜像启动时避免了这项工作,这使得镜像启动更快。但是对于某些类,在构建时初始化它们是没有意义的。这样的示例可能是一个类,它在其初始化时根据环境(环境变量、配置文件等)做出一些决定(例如,创建这个或那个类的实例)。
在构建/运行时初始化选项之间进行选择有一些限制:
- 如果一个类在构建时初始化,它的所有超类必须在构建时初始化
- 如果一个类在运行时初始化,它的所有子类必须在运行时初始化
- 某些类必须始终在运行时初始化(见下文)
- 在运行时初始化的类的实例不能存在于映像堆中(这意味着没有构建时初始化的类或其实例可以(直接或间接)引用这样的运行时初始化的类实例)
Inet地址问题
从版本 19.3.0 开始,native-image
工具要求java.net.InetAddress
始终在运行时初始化该类。这(通过限制 2)意味着它的子类,java.net.Inet4Address
并且java.net.Inet6Address
还必须在运行时初始化,这反过来(通过限制 4)意味着您不能有任何InetAddress
被构建时初始化的类引用。
我们在这里遇到的所有构建失败都是由同一个问题引起的:要么在图像堆中,Inet4Address
要么Inet6Address
在图像堆中。
但是为什么 Netty 相关的类会在构建时尝试初始化呢?
原来netty-codec-http
包含以下内容
Args = --initialize-at-build-time=io.netty \
在其native-image.properties
下方META-INF
,并且 micronaut 具有netty-codec-http
依赖项,因此
io.netty
默认情况下所有类都在构建时初始化(因为native-image
工具尊重此类
native-image.properties
文件)。
样板工程
这里https://github.com/rpuch/netty-InetAddress-native-image-diagnosing是一个模拟问题的项目,我进一步使用它来展示如何解决问题。其main()
方法如下:
public static void main(String[] args) throws Exception {
NioEventLoopGroup group = new NioEventLoopGroup(1, new DefaultThreadFactory("netty"));
DnsAddressResolverGroup resolverGroup = new DnsAddressResolverGroup(NioDatagramChannel.class,
DnsServerAddressStreamProviders.platformDefault());
AddressResolver<InetSocketAddress> resolver = resolverGroup.getResolver(group.next());
System.out.println(resolver);
resolver.close();
group.shutdownGracefully().get();
}
它会产生与以下代码相同的效果(关于 Netty):
Config config = new Config();
config.useSingleServer().setAddress(redisUri);
config.useSingleServer().setPassword(redisPassword);
return Redisson.createReactive(config);
该项目还在--initialize-at-build-time=io.netty
其构建脚本中模拟基于 micronaut 的项目行为。
因此,它是解决这个问题的原始项目的有用替代品。
GraalVM 版本
我在这里使用的是 20.2.0 版本(截至撰写本文时最新发布的版本)。
诊断和修复
1
构建失败并出现以下错误:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.net.Inet4Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace:
at parsing io.netty.resolver.dns.DnsNameResolver.resolveHostsFileEntry(DnsNameResolver.java:659)
Call path from entry point to io.netty.resolver.dns.DnsNameResolver.resolveHostsFileEntry(String):
at io.netty.resolver.dns.DnsNameResolver.resolveHostsFileEntry(DnsNameResolver.java:651)
at io.netty.resolver.dns.DnsNameResolver.doResolve(DnsNameResolver.java:884)
at io.netty.resolver.dns.DnsNameResolver.doResolve(DnsNameResolver.java:733)
at io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:61)
at io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:53)
at io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:55)
at io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:31)
at io.netty.resolver.AbstractAddressResolver.resolve(AbstractAddressResolver.java:106)
at io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:206)
at io.netty.bootstrap.Bootstrap.access$000(Bootstrap.java:46)
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:180)
DnsNameResolver:659
是
return LOCALHOST_ADDRESS;
它引用了名为LOCALHOST_ADDRESS
type的静态字段InetAddress
。让我们通过在命令中添加以下内容来避免它在构建时初始化native-image
:
--initialize-at-run-time=io.netty.resolver.dns.DnsNameResolver
错误消失。
2
现在还有一个:
Error: No instances of java.net.Inet6Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace: Object was reached by
reading field java.util.HashMap$Node.value of
constant java.util.HashMap$Node@26eb0f30 reached by
indexing into array
constant java.util.HashMap$Node[]@63e95621 reached by
reading field java.util.HashMap.table of
constant java.util.HashMap@563992d1 reached by
reading field java.util.Collections$UnmodifiableMap.m of
constant java.util.Collections$UnmodifiableMap@38a9945c reached by
reading field io.netty.resolver.DefaultHostsFileEntriesResolver.inet6Entries of
constant io.netty.resolver.DefaultHostsFileEntriesResolver@7ef4ba7e reached by
scanning method io.netty.resolver.dns.DnsNameResolverBuilder.<init>(DnsNameResolverBuilder.java:56)
Call path from entry point to io.netty.resolver.dns.DnsNameResolverBuilder.<init>():
at io.netty.resolver.dns.DnsNameResolverBuilder.<init>(DnsNameResolverBuilder.java:68)
at io.netty.resolver.dns.DnsAddressResolverGroup.<init>(DnsAddressResolverGroup.java:54)
at Main.main(Main.java:18)
DnsNameResolverBuilder:56
是
private HostsFileEntriesResolver hostsFileEntriesResolver = HostsFileEntriesResolver.DEFAULT;
让我们推迟HostsFileEntriesResolver
初始化:
--initialize-at-run-time=io.netty.resolver.HostsFileEntriesResolver
3
现在,还有另一个错误:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.net.Inet6Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace:
at parsing io.netty.resolver.dns.DnsQueryContextManager.getOrCreateContextMap(DnsQueryContextManager.java:111)
Call path from entry point to io.netty.resolver.dns.DnsQueryContextManager.getOrCreateContextMap(InetSocketAddress):
at io.netty.resolver.dns.DnsQueryContextManager.getOrCreateContextMap(DnsQueryContextManager.java:96)
DnsQueryContextManager:111
参考资料NetUtil.LOCALHOST6
。NetUtil
在构建时初始化,但其字段
LOCALHOST4
和分别LOCALHOST6
包含Inet4Address
和的实例Inet6Address
。这可以通过替换来解决。我们只需将以下类添加到我们的项目中:
@TargetClass(NetUtil.class)
final class NetUtilSubstitutions {
@Alias
@InjectAccessors(NetUtilLocalhost4Accessor.class)
public static Inet4Address LOCALHOST4;
@Alias
@InjectAccessors(NetUtilLocalhost6Accessor.class)
public static Inet6Address LOCALHOST6;
private static class NetUtilLocalhost4Accessor {
static Inet4Address get() {
// using https://en.wikipedia.org/wiki/Initialization-on-demand_holder_idiom
return NetUtilLocalhost4LazyHolder.LOCALHOST4;
}
static void set(Inet4Address ignored) {
// a no-op setter to avoid exceptions when NetUtil is initialized at run-time
}
}
private static class NetUtilLocalhost4LazyHolder {
private static final Inet4Address LOCALHOST4;
static {
byte[] LOCALHOST4_BYTES = {127, 0, 0, 1};
// Create IPv4 loopback address.
try {
LOCALHOST4 = (Inet4Address) InetAddress.getByAddress("localhost", LOCALHOST4_BYTES);
} catch (Exception e) {
// We should not get here as long as the length of the address is correct.
PlatformDependent.throwException(e);
throw new IllegalStateException("Should not reach here");
}
}
}
private static class NetUtilLocalhost6Accessor {
static Inet6Address get() {
// using https://en.wikipedia.org/wiki/Initialization-on-demand_holder_idiom
return NetUtilLocalhost6LazyHolder.LOCALHOST6;
}
static void set(Inet6Address ignored) {
// a no-op setter to avoid exceptions when NetUtil is initialized at run-time
}
}
private static class NetUtilLocalhost6LazyHolder {
private static final Inet6Address LOCALHOST6;
static {
byte[] LOCALHOST6_BYTES = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1};
// Create IPv6 loopback address.
try {
LOCALHOST6 = (Inet6Address) InetAddress.getByAddress("localhost", LOCALHOST6_BYTES);
} catch (Exception e) {
// We should not get here as long as the length of the address is correct.
PlatformDependent.throwException(e);
throw new IllegalStateException("Should not reach here");
}
}
}
}
这个想法是用我们控制的方法调用来替换大量有问题的字段。这是通过替换(注@TargetClass
和)来实现@Alias
的@InjectAccessors
。结果,这些InetAddress
值不再存储在图像堆中。错误消失。
4
我们现在有另一个:
Error: No instances of java.net.Inet6Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace: Object was reached by
reading field io.netty.channel.socket.InternetProtocolFamily.localHost of
constant io.netty.channel.socket.InternetProtocolFamily@5dc39065 reached by
scanning method io.netty.resolver.dns.DnsNameResolver.preferredAddressType(DnsNameResolver.java:487)
Call path from entry point to io.netty.resolver.dns.DnsNameResolver.preferredAddressType(ResolvedAddressTypes):
at io.netty.resolver.dns.DnsNameResolver.preferredAddressType(DnsNameResolver.java:481)
从 的代码中可以看出InternetProtocolFamily
,每个枚举常量都存储了一个 的实例InetAddress
,因此如果在构建时初始化的任何类初始化,图像堆就会被实例InternetProtocolFamily
污染
。InetAddress
这也可以通过替换来解决:
@TargetClass(InternetProtocolFamily.class)
final class InternetProtocolFamilySubstitutions {
@Alias
@InjectAccessors(InternetProtocolFamilyLocalhostAccessor.class)
private InetAddress localHost;
private static class InternetProtocolFamilyLocalhostAccessor {
static InetAddress get(InternetProtocolFamily family) {
switch (family) {
case IPv4:
return NetUtil.LOCALHOST4;
case IPv6:
return NetUtil.LOCALHOST6;
default:
throw new IllegalStateException("Unsupported internet protocol family: " + family);
}
}
static void set(InternetProtocolFamily family, InetAddress address) {
// storing nothing as the getter derives all it needs from its argument
}
}
}
错误消失。
5
这次还有一个:
Error: No instances of java.net.Inet4Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Detailed message:
Trace: Object was reached by
reading field java.net.InetSocketAddress$InetSocketAddressHolder.addr of
constant java.net.InetSocketAddress$InetSocketAddressHolder@34913c36 reached by
reading field java.net.InetSocketAddress.holder of
constant java.net.InetSocketAddress@ad1fe10 reached by
reading field io.netty.resolver.dns.SingletonDnsServerAddresses.address of
constant io.netty.resolver.dns.SingletonDnsServerAddresses@79fd599 reached by
scanning method io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.nameServerAddressStream(DefaultDnsServerAddressStreamProvider.java:115)
Call path from entry point to io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.nameServerAddressStream(String):
at io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.nameServerAddressStream(DefaultDnsServerAddressStreamProvider.java:115)
at io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder$1.nameServerAddressStream(DnsServerAddressStreamProviders.java:131)
at io.netty.resolver.dns.DnsNameResolver.doResolveAllUncached0(DnsNameResolver.java:1070)
首先,让我们将初始化移动io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider
到运行时:
--initialize-at-run-time=io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider
现在,错误是相似的,但仍然略有不同:
Error: No instances of java.net.Inet4Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Trace: Object was reached by
reading field java.net.InetSocketAddress$InetSocketAddressHolder.addr of
constant java.net.InetSocketAddress$InetSocketAddressHolder@5537c5de reached by
reading field java.net.InetSocketAddress.holder of
constant java.net.InetSocketAddress@fb954f8 reached by
reading field io.netty.resolver.dns.SingletonDnsServerAddresses.address of
constant io.netty.resolver.dns.SingletonDnsServerAddresses@3ec9baab reached by
reading field io.netty.resolver.dns.UnixResolverDnsServerAddressStreamProvider.defaultNameServerAddresses of
constant io.netty.resolver.dns.UnixResolverDnsServerAddressStreamProvider@1b7f0339 reached by
reading field io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder$1.currentProvider of
constant io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder$1@2d249be7 reached by
scanning method io.netty.resolver.dns.DnsServerAddressStreamProviders.unixDefault(DnsServerAddressStreamProviders.java:104)
Call path from entry point to io.netty.resolver.dns.DnsServerAddressStreamProviders.unixDefault():
at io.netty.resolver.dns.DnsServerAddressStreamProviders.unixDefault(DnsServerAddressStreamProviders.java:104)
at io.netty.resolver.dns.DnsServerAddressStreamProviders.platformDefault(DnsServerAddressStreamProviders.java:100)
at Main.main(Main.java:18)
好的,让我们io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder
也将初始化移动到运行时:
'--initialize-at-run-time=io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder'
(注意单引号:没有它们$
和后面的字符将被解释sh
为空字符串并替换为空字符串)。
错误消失。
请注意,订单在这里很重要。当我第一次将
io.netty.resolver.dns.DnsServerAddressStreamProviders$DefaultProviderHolder
初始化移到运行时但没有触及io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider
初始化时,错误报告并没有改变。所以需要一点耐心和试验(或者一些我没有的知识,唉)。
现在我们有了这个:
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.net.Inet6Address are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.
Detailed message:
Trace:
at parsing io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.<clinit>(DefaultDnsServerAddressStreamProvider.java:87)
Call path from entry point to io.netty.resolver.dns.DefaultDnsServerAddressStreamProvider.<clinit>():
no path found from entry point to target method
好的,它NetUtil.LOCALHOST
被引用了,所以让我们也为它添加一个替换(to NetUtilSubstitutions
):
@Alias
@InjectAccessors(NetUtilLocalhostAccessor.class)
public static InetAddress LOCALHOST;
// NOTE: this is the simpliest implementation I could invent to just demonstrate the idea; it is probably not
// too efficient. An efficient implementation would only have getter and it would compute the InetAddress
// there; but the post is already very long, and NetUtil.LOCALHOST computation logic in Netty is rather cumbersome.
private static class NetUtilLocalhostAccessor {
private static volatile InetAddress ADDR;
static InetAddress get() {
return ADDR;
}
static void set(InetAddress addr) {
ADDR = addr;
}
}
这使得最终错误消失。
感谢@NicolasFilotto 对第 5 项的建议,我比原来更喜欢他的解决方案,实际上第 5 项是他想法的实现。
技术总结
- 首先,你可以找到一个类,它被移到运行时初始化阶段,导致失败消失。为此,您可以在提供的堆栈跟踪中跟踪引用。要考虑的最佳候选者是静态字段。
- 如果第 1 项没有帮助,您可以尝试创建替换
- 理论上也有可能使用更轻的变体:
@RecomputeFieldValue
,但它受到更多限制,我无法让它在这个与 Netty 相关的任务中工作。
PS。替换相关代码的灵感来自https://github.com/quarkusio/quarkus/pull/5353/files
聚苯乙烯。第 5 项解决方案的灵感来自 @NicolasFilotto
推荐阅读
- ansible - 为什么 ansible 选项“--private-key”在一台主机上有效,而在另一台主机上无效?
- jekyll - 如何使用 netlify cms 配置 jekyll 来编辑和创建页面
- html - HTML 服务器发送事件
- python - 使用来自另一个 DataFrame 的 Series 映射多个列
- javascript - 我的代码中的画布仅在我的 for 循环之后显示。我在追求动画效果
- java - 我无法为在无头模式下运行的 ChromeDriver 设置 cookie
- netlogo - 如何让一只乌龟一步一步走向另一只乌龟?
- excel - 如何计算数据透视表中的文本 - EXCEL?
- django - PyDev 可以自动补全继承的方法名,但不能自动补全字段名
- azure - Azure 应用程序规模错误:无法更改为目标 SKU“免费”,因为公共证书计数将超过“0”的新限制