[seata]XA全局事务超时后回滚失败,一直报XAER_RMFAIL

2024-07-15 162 views
3

XA全局事务超时后回滚失败,一直报XAER_RMFAIL

流程: 全局事务超时时间配置为10秒。在A服务上配置了全局事务注解,A服务更新数据库然后调用B服务。B服务更新数据库后休眠15秒,15秒后提交B服务的事务。A、B服务使用同一个数据库实例的多张表。

2022-06-16 21:21:55.612 INFO  [rpcDispatch_RMROLE_1_12_16] i.s.c.r.p.c.RmBranchRollbackProcessor - rm handle branch rollback process:xid=10.90.13.27:8091:2549311505400693112,branchId=234461296923190606,branchType=XA,resourceId=jdbc:mysql://192.168.201.120:3306/spring_clond_demo,applicationData=null
2022-06-16 21:21:55.612 INFO  [rpcDispatch_RMROLE_1_12_16] io.seata.rm.AbstractRMHandler - Branch Rollbacking: 10.90.13.27:8091:2549311505400693112 234461296923190606 jdbc:mysql://192.168.201.120:3306/spring_clond_demo
2022-06-16 21:21:55.621 INFO  [rpcDispatch_RMROLE_1_12_16] i.s.r.d.xa.ResourceManagerXA - 10.90.13.27:8091:2549311505400693112-234461296923190606 rollback failed since XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
com.mysql.cj.jdbc.MysqlXAException: XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
    at com.mysql.cj.jdbc.MysqlXAConnection.mapXAExceptionFromSQLException(MysqlXAConnection.java:344)
    at com.mysql.cj.jdbc.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:329)
    at com.mysql.cj.jdbc.MysqlXAConnection.rollback(MysqlXAConnection.java:243)
    at io.seata.rm.datasource.xa.ConnectionProxyXA.xaRollback(ConnectionProxyXA.java:141)
    at io.seata.rm.datasource.xa.ConnectionProxyXA.xaRollback(ConnectionProxyXA.java:132)
    at io.seata.rm.datasource.xa.ResourceManagerXA.finishBranch(ResourceManagerXA.java:115)
    at io.seata.rm.datasource.xa.ResourceManagerXA.branchRollback(ResourceManagerXA.java:100)
    at io.seata.rm.AbstractRMHandler.doBranchRollback(AbstractRMHandler.java:125)
    at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:67)
    at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:63)
    at io.seata.core.exception.AbstractExceptionHandler.exceptionHandleTemplate(AbstractExceptionHandler.java:131)
    at io.seata.rm.AbstractRMHandler.handle(AbstractRMHandler.java:63)
    at io.seata.rm.DefaultRMHandler.handle(DefaultRMHandler.java:68)
    at io.seata.core.protocol.transaction.BranchRollbackRequest.handle(BranchRollbackRequest.java:35)
    at io.seata.rm.AbstractRMHandler.onRequest(AbstractRMHandler.java:150)
    at io.seata.core.rpc.processor.client.RmBranchRollbackProcessor.handleBranchRollback(RmBranchRollbackProcessor.java:63)
    at io.seata.core.rpc.processor.client.RmBranchRollbackProcessor.process(RmBranchRollbackProcessor.java:58)
    at io.seata.core.rpc.netty.AbstractNettyRemoting.lambda$processMessage$2(AbstractNettyRemoting.java:281)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
    at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
    at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:762)
    at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:646)
    at com.mysql.cj.jdbc.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:323)
    ... 20 common frames omitted
2022-06-16 21:21:55.621 INFO  [rpcDispatch_RMROLE_1_12_16] io.seata.rm.AbstractRMHandler - Branch Rollbacked result: PhaseTwo_RollbackFailed_Retryable
2022-06-16 21:21:56.058 INFO  [rpcDispatch_RMROLE_1_13_16] i.s.c.r.p.c.RmBranchRollbackProcessor - rm handle branch rollback process:xid=10.90.13.27:8091:2549311505400693112,branchId=234461296923190606,branchType=XA,resourceId=jdbc:mysql://192.168.201.120:3306/spring_clond_demo,applicationData=null
2022-06-16 21:21:56.058 INFO  [rpcDispatch_RMROLE_1_13_16] io.seata.rm.AbstractRMHandler - Branch Rollbacking: 10.90.13.27:8091:2549311505400693112 234461296923190606 jdbc:mysql://192.168.201.120:3306/spring_clond_demo
2022-06-16 21:21:56.065 INFO  [rpcDispatch_RMROLE_1_13_16] i.s.r.d.xa.ResourceManagerXA - 10.90.13.27:8091:2549311505400693112-234461296923190606 rollback failed since XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
com.mysql.cj.jdbc.MysqlXAException: XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
    at com.mysql.cj.jdbc.MysqlXAConnection.mapXAExceptionFromSQLException(MysqlXAConnection.java:344)
    at com.mysql.cj.jdbc.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:329)
    at com.mysql.cj.jdbc.MysqlXAConnection.rollback(MysqlXAConnection.java:243)
    at io.seata.rm.datasource.xa.ConnectionProxyXA.xaRollback(ConnectionProxyXA.java:141)
    at io.seata.rm.datasource.xa.ConnectionProxyXA.xaRollback(ConnectionProxyXA.java:132)
    at io.seata.rm.datasource.xa.ResourceManagerXA.finishBranch(ResourceManagerXA.java:115)
    at io.seata.rm.datasource.xa.ResourceManagerXA.branchRollback(ResourceManagerXA.java:100)
    at io.seata.rm.AbstractRMHandler.doBranchRollback(AbstractRMHandler.java:125)
    at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:67)
    at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:63)
    at io.seata.core.exception.AbstractExceptionHandler.exceptionHandleTemplate(AbstractExceptionHandler.java:131)
    at io.seata.rm.AbstractRMHandler.handle(AbstractRMHandler.java:63)
    at io.seata.rm.DefaultRMHandler.handle(DefaultRMHandler.java:68)
    at io.seata.core.protocol.transaction.BranchRollbackRequest.handle(BranchRollbackRequest.java:35)
    at io.seata.rm.AbstractRMHandler.onRequest(AbstractRMHandler.java:150)
    at io.seata.core.rpc.processor.client.RmBranchRollbackProcessor.handleBranchRollback(RmBranchRollbackProcessor.java:63)
    at io.seata.core.rpc.processor.client.RmBranchRollbackProcessor.process(RmBranchRollbackProcessor.java:58)
    at io.seata.core.rpc.netty.AbstractNettyRemoting.lambda$processMessage$2(AbstractNettyRemoting.java:281)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: XAER_RMFAIL: The command cannot be executed when global transaction is in the  ACTIVE state
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
    at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
    at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:762)
    at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:646)
    at com.mysql.cj.jdbc.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:323)
    ... 20 common frames omitted
2022-06-16 21:21:56.065 INFO  [rpcDispatch_RMROLE_1_13_16] io.seata.rm.AbstractRMHandler - Branch Rollbacked result: PhaseTwo_RollbackFailed_Retryable

有时会在70秒后出现Branch Rollbacked result: PhaseTwo_RollbackFailed_XAER_NOTA_Retryable,然后回滚成功,多数情况下会一直报错下去。

  • JDK version : 1.8
  • Seata version: 1.5.1
  • OS : Windows
  • MySQL: 5.7.30

回答

2

多tc下由于mysql必须使用registry的rm节点进行二阶段处理,所以默认的client.rm.connectionTwoPhaseHoldTimeoutXA配置为10s,当超过10s,会断开registry的rm对应的分支事务使用的connection,此时其他的rm便可以帮助回滚,所以在多tc+多rm节点下会增加该情况出现的频率,你可以暂时将client.rm.connectionTwoPhaseHoldTimeoutXA 配置10s改为合理的时间,假设你一次调用完整连接timeout为1s,那么client.rm.connectionTwoPhaseHoldTimeoutXA应该配置在1s内,将这个配置放到你所使用的配置中心中即可

9

@a364176773 您好,配置成1s后现象还是一样的,与之前相比分支事务并没有提前成功回滚

0

你的业务rt是多少?就是这个接口一个完整调用如果是100ms,那1s肯定不会提前回滚成功,你业务都结束了,连接没释放,你可以改成1个tc或那个回滚要很久的rm就1个节点先试下,肯定是没这个问题的,这个问题存在与rm注册的时候到每个tc都是不同的remoteaddress,没有唯一标识,导致begin的时候在tc1,registry在tc3,commit或rollback在tc2,tc2就找不到rm注册的那个channel,就会出现这种情况,只能靠无限重试和rm缩短释放连接的时间来优化,后续我们会做根据xid的负载均衡,这种适合xa这种模式,begin在tc1,registry也会在tc1,commit也会在tc1

8

好的,了解了,感谢回复。只能暂时考虑其他模式了。

3

下周二我们会发布1.5.2默认为xid的负载均衡算法来优化这个现象,欢迎届时使用新版本进行测试