1. 首页
  2. 问答
  3. [alibaba/arthas]启动attach成功，connect server报错Connection refused，NoClassDefFoundError: Could ...

[alibaba/arthas]启动attach成功，connect server报错Connection refused，NoClassDefFoundError: Could not initialize class com.alibaba.arthas.deps.ch.qos.logback.core.spi.FilterReply

2024-07-17 764 views
2
环境信息
  • arthas-boot.jar 或者 as.sh 的版本： 3.5.3
  • 操作系统版本: centos7
  • 目标进程的JVM版本: jdk1.8.0_291
  • 执行arthas-boot的版本: 3.6.7
  • JAVA_OPTS="-Xmx4500M -Xms4500M -Xmn2800M -XX:SurvivorRatio=7 -Xss256k -XX:MaxMetaspaceSize=200M -XX:MetaspaceSize=128M" -XX:MaxTenuringThreshold=15 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=80 -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses -XX:+CMSClassUnloadingEnabled -XX:+ParallelRefProcEnabled -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=200M -XX:+CMSScavengeBeforeRemark -XX:ErrorFile=./hs_err_pid%p.log -Xloggc:./gc-%t.log -XX:HeapDumpPath=. -XX:+PrintGCDetails -XX:+PrintGCDateStamps -verbose:gc -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintTenuringDistribution -XX:+PrintHeapAtGC"
重现问题的步骤
  1. 该进程，之前有两次是attach成功，并且每次都retransform class成功 ，stop正常退出
  2. 第三次启动时，就显示attach process success, 但 connect too telnet server error: Connection refused. 并且查看artash.log日志文件，有提示 Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.alibaba.arthas.deps.ch.qos.logback.core.spi.FilterReply at com.alibaba.arthas.deps.ch.qos.logback.classic.LoggerContext.getTurboFilterChainDecision_0_3OrMore(LoggerContext.java:267) 4，切换端口 --telnet-port其他端口 启动也是报上面一样的错。 5，第三次启动arthas时，进程触发生成了 java_pid27493.hprof 文件。里面分析了下，暂未发现有内存泄漏的点。 但进程任然是正常运行的，正常提供服务！
期望的结果

能正常attach, 然后可以retransform class类 image

实际运行的结果

实际运行结果，最好有详细的日志，异常栈。尽量贴文本。

image

把异常信息贴到这里
这是arthas.log日志文件里输出的内容：

2023-02-22 18:28:23 [arthas-command-execute] INFO  c.t.arthas.core.advisor.Enhancer -Success to reset classes: []
2023-02-22 18:28:23 [arthas-NettyHttpTelnetBootstrap-3-1] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0xe691f2c2, L:/127.0.0.1:3658] INACTIVE
2023-02-22 18:28:23 [arthas-NettyHttpTelnetBootstrap-3-1] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0xe691f2c2, L:/127.0.0.1:3658] UNREGISTERED
2023-02-22 18:28:23 [arthas-NettyWebsocketTtyBootstrap-4-1] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0x66683e02, L:/127.0.0.1:8563] CLOSE
2023-02-22 18:28:23 [arthas-NettyWebsocketTtyBootstrap-4-1] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0x66683e02, L:/127.0.0.1:8563] INACTIVE
2023-02-22 18:28:23 [arthas-NettyWebsocketTtyBootstrap-4-1] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0x66683e02, L:/127.0.0.1:8563] UNREGISTERED
2023-02-22 18:28:23 [arthas-NettyWebsocketTtyBootstrap-4-2] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0x2587e324, L:local:arthas-netty-localaddress] INACTIVE
2023-02-22 18:28:23 [arthas-NettyWebsocketTtyBootstrap-4-2] INFO  c.a.a.d.i.n.h.logging.LoggingHandler -[id: 0x2587e324, L:local:arthas-netty-localaddress] UNREGISTERED
2023-02-22 18:28:23 [arthas-command-execute] INFO  c.t.a.core.server.ArthasBootstrap -as-server destroy completed.
Arthas server agent start...
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.taobao.arthas.agent334.AgentBootstrap.bind(AgentBootstrap.java:182)
    at com.taobao.arthas.agent334.AgentBootstrap.access$000(AgentBootstrap.java:20)
    at com.taobao.arthas.agent334.AgentBootstrap$1.run(AgentBootstrap.java:152)
Caused by: java.lang.OutOfMemoryError: Metaspace
Arthas server agent start...
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.taobao.arthas.agent334.AgentBootstrap.bind(AgentBootstrap.java:182)
    at com.taobao.arthas.agent334.AgentBootstrap.access$000(AgentBootstrap.java:20)
    at com.taobao.arthas.agent334.AgentBootstrap$1.run(AgentBootstrap.java:152)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.alibaba.arthas.deps.ch.qos.logback.core.spi.FilterReply
    at com.alibaba.arthas.deps.ch.qos.logback.classic.LoggerContext.getTurboFilterChainDecision_0_3OrMore(LoggerContext.java:267)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:373)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.error(Logger.java:538)
    at com.taobao.arthas.core.server.ArthasBootstrap.bind(ArthasBootstrap.java:474)
    at com.taobao.arthas.core.server.ArthasBootstrap.<init>(ArthasBootstrap.java:156)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:581)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:569)
    ... 7 more
Arthas server agent start...
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.taobao.arthas.agent334.AgentBootstrap.bind(AgentBootstrap.java:182)
    at com.taobao.arthas.agent334.AgentBootstrap.access$000(AgentBootstrap.java:20)
    at com.taobao.arthas.agent334.AgentBootstrap$1.run(AgentBootstrap.java:152)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.alibaba.arthas.deps.ch.qos.logback.core.spi.FilterReply
    at com.alibaba.arthas.deps.ch.qos.logback.classic.LoggerContext.getTurboFilterChainDecision_0_3OrMore(LoggerContext.java:267)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:373)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.error(Logger.java:538)
    at com.taobao.arthas.core.server.ArthasBootstrap.bind(ArthasBootstrap.java:474)
    at com.taobao.arthas.core.server.ArthasBootstrap.<init>(ArthasBootstrap.java:156)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:581)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:569)
    ... 7 more
Arthas server agent start...
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.taobao.arthas.agent334.AgentBootstrap.bind(AgentBootstrap.java:182)
    at com.taobao.arthas.agent334.AgentBootstrap.access$000(AgentBootstrap.java:20)
    at com.taobao.arthas.agent334.AgentBootstrap$1.run(AgentBootstrap.java:152)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.alibaba.arthas.deps.ch.qos.logback.core.spi.FilterReply
    at com.alibaba.arthas.deps.ch.qos.logback.classic.LoggerContext.getTurboFilterChainDecision_0_3OrMore(LoggerContext.java:267)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:373)
    at com.alibaba.arthas.deps.ch.qos.logback.classic.Logger.error(Logger.java:538)
    at com.taobao.arthas.core.server.ArthasBootstrap.bind(ArthasBootstrap.java:474)
    at com.taobao.arthas.core.server.ArthasBootstrap.<init>(ArthasBootstrap.java:156)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:581)
    at com.taobao.arthas.core.server.ArthasBootstrap.getInstance(ArthasBootstrap.java:569)
    ... 7 more

wickywe

回答

5

Caused by: java.lang.OutOfMemoryError: Metaspace

OOM了，所以没办法加载新的类了。每次加载Arthas都是一个新的ClassLoader。

hengyunabc

9

大佬你好，这个“每次加载Arthas都是一个新的ClassLoader”的意思 是每次启动arthas 后，之前的类，都会在 元空间里面重新加载一份吗？ 因为这个原因，所以造成我的元空间溢出了，是这个意思吗？ 这是我的jvm参数 "-Xmx4500M -Xms4500M -Xmn2800M -XX:SurvivorRatio=7 -Xss256k -XX:MaxMetaspaceSize=200M -XX:MetaspaceSize=128M"

请问后续我该怎么样操作，能尽量避免这样的情况呢

wickywe

8

我是自己调小了堆内存导致的，调大点就好了

KaiHavertz