[#OVSDB-434] br-int not created after failing over one controller in 3 node cluster

steps seem easy enough. I don't have a clustered setup at this exact moment or I'd try them:

If I get access to a clustered setup locally, I'll try to remember to check this.

Having said that, I noticed that this is probably an OVSDB bug and maybe not all the way up the food chain
to netvirt, even though I know those are the features listed as installed. I'm just looking at the Exception.

We have OVSDB Cluster Jobs and they pass most ofthe time, but they do fail periodically. I scanned some
karaf logs in the 5 most recent failed jobs and one of them looks to have a similar exception:

2018-05-09T17:25:31,776 | WARN  | transaction-invoker-impl-0 | SouthboundUtil                   | 312 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT | Read Operational/DS for Node failed! KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/8fba589d-fb24-44ab-8efa-82ba6fd92e3a}}]]}
org.opendaylight.controller.md.sal.common.api.data.ReadFailedException: Error executeRead ReadData for path /(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=ovsdb:1}]/node/node[{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node-id=ovsdb://uuid/8fba589d-fb24-44ab-8efa-82ba6fd92e3a}]
	at org.opendaylight.controller.cluster.datastore.compat.LegacyDOMStoreAdapter$1.newWithCause(LegacyDOMStoreAdapter.java:43) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.compat.LegacyDOMStoreAdapter$1.newWithCause(LegacyDOMStoreAdapter.java:39) ~[?:?]
	at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:91) ~[?:?]
	at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:40) ~[?:?]
	at org.opendaylight.mdsal.common.api.MappingCheckedFuture.mapException(MappingCheckedFuture.java:62) ~[?:?]
	at org.opendaylight.mdsal.common.api.MappingCheckedFuture.wrapInExecutionException(MappingCheckedFuture.java:66) ~[?:?]
	at org.opendaylight.mdsal.common.api.MappingCheckedFuture.get(MappingCheckedFuture.java:79) ~[?:?]
	at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:168) ~[?:?]
	at com.google.common.util.concurrent.Futures.getDone(Futures.java:1436) ~[?:?]
	at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:85) ~[?:?]
	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398) ~[?:?]
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1015) ~[?:?]
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:868) ~[?:?]
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:713) ~[?:?]
	at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:54) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.NoOpTransactionContext.executeRead(NoOpTransactionContext.java:67) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.TransactionProxy$1.invoke(TransactionProxy.java:96) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.TransactionContextWrapper.executePriorTransactionOperations(TransactionContextWrapper.java:192) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.onFindPrimaryShardFailure(AbstractTransactionContextFactory.java:109) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.access$100(AbstractTransactionContextFactory.java:37) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory$1.onComplete(AbstractTransactionContextFactory.java:136) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory$1.onComplete(AbstractTransactionContextFactory.java:130) ~[?:?]
	at akka.dispatch.OnComplete.internal(Future.scala:260) ~[?:?]
	at akka.dispatch.OnComplete.internal(Future.scala:258) ~[?:?]
	at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188) ~[?:?]
	at akka.dispatch.japi$CallbackBridge.apply(Future.scala:185) ~[?:?]
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) ~[356:org.scala-lang.scala-library:2.12.5.v20180316-130912-VFINAL-30a1428]
	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) ~[?:?]
	at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91) ~[?:?]
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) [356:org.scala-lang.scala-library:2.12.5.v20180316-130912-VFINAL-30a1428]
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) [356:org.scala-lang.scala-library:2.12.5.v20180316-130912-VFINAL-30a1428]
	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:43) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [40:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [40:com.typesafe.akka.actor:2.5.11]
Caused by: org.opendaylight.mdsal.common.api.DataStoreUnavailableException: Shard member-1-shard-topology-operational currently has no leader. Try again later.
	at org.opendaylight.controller.cluster.datastore.NoOpTransactionContext.executeRead(NoOpTransactionContext.java:63) ~[?:?]
	... 22 more
Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-topology-operational currently has no leader. Try again later.
	at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.createNoShardLeaderException(ShardManager.java:955) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:786) ~[?:?]
	at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:253) ~[?:?]
	at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44) ~[?:?]
	at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:275) ~[?:?]
	at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104) ~[?:?]
	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:608) ~[?:?]
	at akka.actor.Actor.aroundReceive(Actor.scala:517) ~[?:?]
	at akka.actor.Actor.aroundReceive$(Actor.scala:515) ~[?:?]
	at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:273) ~[?:?]
	at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:691) ~[?:?]
	at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:192) ~[?:?]
	at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:191) ~[?:?]
	at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:273) ~[?:?]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) ~[?:?]
	at akka.actor.ActorCell.invoke(ActorCell.scala:559) ~[?:?]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) ~[?:?]
	at akka.dispatch.Mailbox.run(Mailbox.scala:224) ~[?:?]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234) ~[?:?]
	... 4 more

I didn't see this exception in any of the other karaf logs in the other 4 failing jobs I looked at.

[OVSDB-434] br-int not created after failing over one controller in 3 node cluster Created: 13/Apr/16 Updated: 02/May/19 Resolved: 02/May/19
Status:	Resolved
Project:	ovsdb
Component/s:	Southbound.Open_vSwitch
Affects Version/s:	Carbon-SR3
Fix Version/s:	None

[OVSDB-434] br-int not created after failing over one controller in 3 node cluster Created: 13/Apr/16 Updated: 02/May/19 Resolved: 02/May/19