启动Apache Atlas时报错

夹胡碰

发布于 2020-10-14 15:41:48

2K0

发布于 2020-10-14 15:41:48

1. 问题描述

启动Apache Atlas时，报错信息如下

Caused by: org.janusgraph.diskstorage.PermanentBackendException: Read 1 locks with our rid  97- 99- 49- 50- 49- 48- 50- 54- 49- 51- 56- 49- 54- 45-106-102-112- 49 but mismatched timestamps; no lock column contained our timestamp (2020-10-12T02:09:41.393999800Z)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSeniority(ConsistentKeyLocker.java:528)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:454)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118)
    at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351)
    ... 107 more

2. 解决办法

JDK版本问题，JDK11的时间精度不兼容此版本JanusGraph默认的时间精度，JAVA_HOME替换成JDK8启动正常(其他JDK版本也可，JDK8亲测可以)

3. 详细原因

从Java 11开始Instant.now()的默认时间精度从milis(毫秒)提高到nano(纳秒)，这使得JanusGraphTimestampProviders无论实际配置了什么枚举值，始终使用NANO精度。而cassandra仍然使用MICRO精度，使得JanusGraph时间戳不兼容。

不同JDK版本下运行测试运行

public static void main(String[] args) {
    Instant now = Instant.now();
    System.out.println("default: " + now);

    // JDK8 => out 默认为MILLTS精度
    //default: 2020-10-12T08:09:45.609Z

    // JDK11 => out 默认为NANOS精度
    // default: 2020-10-12T08:05:21.304999800Z
}

以下参考 JanusGraph-issues https://github.com/JanusGraph/janusgraph/issues/1979

Starting from Java 11 the default precision of Instant.now() has been raised for some OSes from milis to nano. See similar issue: h2database/h2database#1178 This makes JanusGraph TimestampProviders always uses NANO precision no matter what enum value is actually configured. Still cassandra uses MICRO precision which makes Janus timestamps incompatible - see stacktrace below. Fixed version of TimestampProviders:

MICRO {
        @Override
        public Instant getTime() {
            // original => return Instant.now();
            return Instant.now().truncatedTo(ChronoUnit.MICROS);
        }

        @Override
        public Instant getTime(long sinceEpoch) {
            return Instant.ofEpochSecond(0, (sinceEpoch * 1000L));
        }

        @Override
        public ChronoUnit getUnit() {
            return ChronoUnit.MICROS;
        }

        @Override
        public long getTime(Instant timestamp) {
            return timestamp.getEpochSecond() * 1000000L + timestamp.getNano()/1000;

        }
    },

    MILLI {
        @Override
        public Instant getTime() {
            // original => return Instant.now();
            return Instant.now().truncatedTo(ChronoUnit.MILLIS);
        }

        @Override
        public Instant getTime(long sinceEpoch) {
            return Instant.ofEpochMilli(sinceEpoch);
        }

        @Override
        public ChronoUnit getUnit() {
            return ChronoUnit.MILLIS;
        }

        @Override
        public long getTime(Instant timestamp) {
            return timestamp.getEpochSecond() * 1000 + timestamp.getNano() / 1000000;
        }
    };

Note lines: return Instant.now().truncatedTo(ChronoUnit.MICROS); return Instant.now().truncatedTo(ChronoUnit.MILLIS); The above two lines fix the issue.

4. 详细错误堆栈信息如下

2020-10-12 10:09:51,410 ERROR - [main:] ~ Could not commit transaction [1] due to storage exception in system-commit (StandardJanusGraph:724)
org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
    at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196)
    at org.janusgraph.diskstorage.BackendTransaction.commit(BackendTransaction.java:150)
    at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:721)
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1425)
    at org.janusgraph.graphdb.database.management.ManagementSystem.commit(ManagementSystem.java:195)
    at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphManagement.commit(AtlasJanusGraphManagement.java:129)
    at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.commit(GraphBackedSearchIndexer.java:943)
    at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.initialize(GraphBackedSearchIndexer.java:363)
    at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.<init>(GraphBackedSearchIndexer.java:129)
    at org.apache.atlas.repository.graph.GraphBackedSearchIndexer.<init>(GraphBackedSearchIndexer.java:116)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:122)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:271)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1314)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1280)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1178)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1094)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1136)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1314)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1280)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1178)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1094)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
    at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
    at org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
    at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
    at org.eclipse.jetty.server.Server.start(Server.java:422)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.server.Server.doStart(Server.java:389)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98)
    at org.apache.atlas.Atlas.main(Atlas.java:133)
Caused by: org.janusgraph.diskstorage.locking.PermanentLockingException: Permanent locking failure
    at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:359)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91)
    at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68)
    at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54)
    ... 100 more
Caused by: org.janusgraph.diskstorage.PermanentBackendException: Read 1 locks with our rid  97- 99- 49- 50- 49- 48- 50- 54- 49- 51- 56- 49- 54- 45-106-102-112- 49 but mismatched timestamps; no lock column contained our timestamp (2020-10-12T02:09:41.393999800Z)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSeniority(ConsistentKeyLocker.java:528)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:454)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118)
    at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351)
    ... 107 more