Uploaded image for project: 'bgpcep'
  1. bgpcep
  2. BGPCEP-315

Lithium BGP project fails to deploy in 3 nodes cluster environment

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Cannot Reproduce
    • Bugzilla Migration
    • Bugzilla Migration
    • BGP
    • None

    • Operating System: Linux
      Platform: All

    • 4676

    Description

      When we deploy BGP (and all compatible features) in a 3 nodes cluster environment we see this exception:

      2015-11-24 07:07:08,755 | INFO | config-pusher | ConfigPusherImpl | 157 - org.opendaylight.controller.config-persister-impl - 0.3.3.Lithium-SR3 | Pushing configuration snapshot 30-programming.xml(odl-bgpcep-pcep-all,odl-bgpcep-pcep-all)
      2015-11-24 07:07:08,771 | INFO | 9]-nio2-thread-8 | ChannelSession | 28 - org.apache.sshd.core - 0.12.0 | Executing command: bundle:list

      2015-11-24 07:07:08,872 | ERROR | config-pusher | ConfigTransactionControllerImpl | 139 - org.opendaylight.controller.config-manager - 0.3.3.Lithium-SR3 | Commit failed on ModuleIdentifier

      {factoryName='instruction-scheduler-impl', instanceName='global-instruction-scheduler'} in transaction TransactionIdentifier{name='ConfigTransaction-201-203'}
      java.lang.IllegalStateException: Conflicting instruction queue found
      at com.google.common.base.Preconditions.checkState(Preconditions.java:173)
      at org.opendaylight.bgpcep.programming.impl.ProgrammingServiceImpl.<init>(ProgrammingServiceImpl.java:123)
      at org.opendaylight.controller.config.yang.programming.impl.InstructionSchedulerImplModule.createInstance(InstructionSchedulerImplModule.java:57)
      at org.opendaylight.controller.config.spi.AbstractModule.getInstance(AbstractModule.java:73)
      at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at org.opendaylight.controller.config.manager.impl.dependencyresolver.DependencyResolverManager$ModuleInvocationHandler.handleInvocation(DependencyResolverManager.java:150)
      at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
      at com.sun.proxy.$Proxy35.getInstance(Unknown Source)
      at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.secondPhaseCommit(ConfigTransactionControllerImpl.java:399)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.secondPhaseCommit(ConfigRegistryImpl.java:280)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.commitConfig(ConfigRegistryImpl.java:227)
      at sun.reflect.GeneratedMethodAccessor223.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)[:1.7.0_85]
      at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)[:1.7.0_85]
      at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy$InvokeHandler.invoke(MXBeanProxy.java:150)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:167)[:1.7.0_85]
      at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:258)[:1.7.0_85]
      at com.sun.proxy.$Proxy16.commitConfig(Unknown Source)[120:org.opendaylight.controller.config-api:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.config.util.ConfigRegistryJMXClient.commitConfig(ConfigRegistryJMXClient.java:102)[134:org.opendaylight.controller.config-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.transactions.TransactionProvider.commitTransaction(TransactionProvider.java:138)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.operations.Commit.handleWithNoSubsequentOperations(Commit.java:54)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractLastNetconfOperation.handle(AbstractLastNetconfOperation.java:33)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractNetconfOperation.handle(AbstractNetconfOperation.java:100)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.sendRequestGetResponseCheckIsOK(ConfigPusherImpl.java:354)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfig(ConfigPusherImpl.java:308)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfigWithConflictingVersionRetries(ConfigPusherImpl.java:147)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.internalPushConfigs(ConfigPusherImpl.java:106)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.process(ConfigPusherImpl.java:77)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.osgi.ConfigPersisterActivator$InnerCustomizer$1.run(ConfigPersisterActivator.java:181)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at java.lang.Thread.run(Thread.java:745)[:1.7.0_85]
      2015-11-24 07:07:08,882 | ERROR | config-pusher | ConfigRegistryImpl | 139 - org.opendaylight.controller.config-manager - 0.3.3.Lithium-SR3 | Configuration Transaction failed on 2PC, server is unhealthy
      java.lang.IllegalStateException: Error - getInstance() failed for ModuleIdentifier{factoryName='instruction-scheduler-impl', instanceName='global-instruction-scheduler'}

      in transaction TransactionIdentifier

      {name='ConfigTransaction-201-203'}
      at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.secondPhaseCommit(ConfigTransactionControllerImpl.java:405)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.secondPhaseCommit(ConfigRegistryImpl.java:280)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.commitConfig(ConfigRegistryImpl.java:227)
      at sun.reflect.GeneratedMethodAccessor223.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)[:1.7.0_85]
      at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)[:1.7.0_85]
      at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy$InvokeHandler.invoke(MXBeanProxy.java:150)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:167)[:1.7.0_85]
      at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:258)[:1.7.0_85]
      at com.sun.proxy.$Proxy16.commitConfig(Unknown Source)[120:org.opendaylight.controller.config-api:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.config.util.ConfigRegistryJMXClient.commitConfig(ConfigRegistryJMXClient.java:102)[134:org.opendaylight.controller.config-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.transactions.TransactionProvider.commitTransaction(TransactionProvider.java:138)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.operations.Commit.handleWithNoSubsequentOperations(Commit.java:54)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractLastNetconfOperation.handle(AbstractLastNetconfOperation.java:33)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractNetconfOperation.handle(AbstractNetconfOperation.java:100)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.sendRequestGetResponseCheckIsOK(ConfigPusherImpl.java:354)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfig(ConfigPusherImpl.java:308)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfigWithConflictingVersionRetries(ConfigPusherImpl.java:147)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.internalPushConfigs(ConfigPusherImpl.java:106)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.process(ConfigPusherImpl.java:77)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.osgi.ConfigPersisterActivator$InnerCustomizer$1.run(ConfigPersisterActivator.java:181)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at java.lang.Thread.run(Thread.java:745)[:1.7.0_85]
      Caused by: java.lang.IllegalStateException: Conflicting instruction queue found
      at com.google.common.base.Preconditions.checkState(Preconditions.java:173)
      at org.opendaylight.bgpcep.programming.impl.ProgrammingServiceImpl.<init>(ProgrammingServiceImpl.java:123)
      at org.opendaylight.controller.config.yang.programming.impl.InstructionSchedulerImplModule.createInstance(InstructionSchedulerImplModule.java:57)
      at org.opendaylight.controller.config.spi.AbstractModule.getInstance(AbstractModule.java:73)
      at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at org.opendaylight.controller.config.manager.impl.dependencyresolver.DependencyResolverManager$ModuleInvocationHandler.handleInvocation(DependencyResolverManager.java:150)
      at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
      at com.sun.proxy.$Proxy35.getInstance(Unknown Source)
      at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.secondPhaseCommit(ConfigTransactionControllerImpl.java:399)
      ... 35 more
      2015-11-24 07:07:08,889 | ERROR | config-pusher | ConfigPusherImpl | 157 - org.opendaylight.controller.config-persister-impl - 0.3.3.Lithium-SR3 | Failed to apply configuration snapshot: 30-programming.xml(odl-bgpcep-pcep-all,odl-bgpcep-pcep-all)
      java.lang.IllegalStateException: Error - getInstance() failed for ModuleIdentifier{factoryName='instruction-scheduler-impl', instanceName='global-instruction-scheduler'} in transaction TransactionIdentifier{name='ConfigTransaction-201-203'}

      at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.secondPhaseCommit(ConfigTransactionControllerImpl.java:405)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.secondPhaseCommit(ConfigRegistryImpl.java:280)
      at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.commitConfig(ConfigRegistryImpl.java:227)
      at sun.reflect.GeneratedMethodAccessor223.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)[:1.7.0_85]
      at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)[:1.7.0_85]
      at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy$InvokeHandler.invoke(MXBeanProxy.java:150)[:1.7.0_85]
      at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:167)[:1.7.0_85]
      at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:258)[:1.7.0_85]
      at com.sun.proxy.$Proxy16.commitConfig(Unknown Source)[120:org.opendaylight.controller.config-api:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.config.util.ConfigRegistryJMXClient.commitConfig(ConfigRegistryJMXClient.java:102)[134:org.opendaylight.controller.config-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.transactions.TransactionProvider.commitTransaction(TransactionProvider.java:138)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.confignetconfconnector.operations.Commit.handleWithNoSubsequentOperations(Commit.java:54)[154:org.opendaylight.controller.config-netconf-connector:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractLastNetconfOperation.handle(AbstractLastNetconfOperation.java:33)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.util.mapping.AbstractNetconfOperation.handle(AbstractNetconfOperation.java:100)[152:org.opendaylight.controller.netconf-util:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.sendRequestGetResponseCheckIsOK(ConfigPusherImpl.java:354)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfig(ConfigPusherImpl.java:308)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.pushConfigWithConflictingVersionRetries(ConfigPusherImpl.java:147)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.internalPushConfigs(ConfigPusherImpl.java:106)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.process(ConfigPusherImpl.java:77)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at org.opendaylight.controller.netconf.persist.impl.osgi.ConfigPersisterActivator$InnerCustomizer$1.run(ConfigPersisterActivator.java:181)[157:org.opendaylight.controller.config-persister-impl:0.3.3.Lithium-SR3]
      at java.lang.Thread.run(Thread.java:745)[:1.7.0_85]
      Caused by: java.lang.IllegalStateException: Conflicting instruction queue found
      at com.google.common.base.Preconditions.checkState(Preconditions.java:173)
      at org.opendaylight.bgpcep.programming.impl.ProgrammingServiceImpl.<init>(ProgrammingServiceImpl.java:123)
      at org.opendaylight.controller.config.yang.programming.impl.InstructionSchedulerImplModule.createInstance(InstructionSchedulerImplModule.java:57)
      at org.opendaylight.controller.config.spi.AbstractModule.getInstance(AbstractModule.java:73)
      at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_85]
      at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_85]
      at org.opendaylight.controller.config.manager.impl.dependencyresolver.DependencyResolverManager$ModuleInvocationHandler.handleInvocation(DependencyResolverManager.java:150)
      at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
      at com.sun.proxy.$Proxy35.getInstance(Unknown Source)
      at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.secondPhaseCommit(ConfigTransactionControllerImpl.java:399)
      ... 35 more

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Unassigned Unassigned
            ecelgp Luis Gomez
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: