https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-1790 controller <p><a href="https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-3node-openstack-ocata-upstream-stateful-carbon/179/odl_3/odl3_exception.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-3node-openstack-ocata-upstream-stateful-carbon/179/odl_3/odl3_exception.log.gz</a></p> <p>Messages below happen as the leader node is taken down and the followers have the messages below.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>ROBOT MESSAGE: Starting test Take Down ODL1 ROBOT MESSAGE: Killing ODL1 10.29.14.5 </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2017-11-15 02:53:11,625 | WARN | ult-dispatcher-5 | NettyTransport | 178 - com.typesafe.akka.slf4j - 2.4.18 | Remote connection to null failed with java.net.ConnectException: Connection refused: /10.29.14.5:2550 2017-11-15 02:53:11,626 | WARN | ult-dispatcher-5 | ReliableDeliverySupervisor | 178 - com.typesafe.akka.slf4j - 2.4.18 | Association with remote system [akka.tcp://opendaylight-cluster-data@10.29.14.5:2550] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://opendaylight-cluster-data@10.29.14.5:2550]] Caused by: [Connection refused: /10.29.14.5:2550] 2017-11-15 02:53:16,646 | WARN | lt-dispatcher-38 | NettyTransport | 178 - com.typesafe.akka.slf4j - 2.4.18 | Remote connection to null failed with java.net.ConnectException: Connection refused: /10.29.14.5:2550 2017-11-15 02:53:20,936 | INFO | ult-dispatcher-3 | kka://opendaylight-cluster-data) | 178 - com.typesafe.akka.slf4j - 2.4.18 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.29.15.176:2550] - Leader can currently not perform its duties, reachability status: [akka.tcp://opendaylight-cluster-data@10.29.15.176:2550 -&gt; akka.tcp://opendaylight-cluster-data@10.29.14.5:2550: Unreachable [Unreachable] (1), akka.tcp://opendaylight-cluster-data@10.29.15.237:2550 -&gt; akka.tcp://opendaylight-cluster-data@10.29.14.5:2550: Unreachable [Unreachable] (1)], member status: [akka.tcp://opendaylight-cluster-data@10.29.14.5:2550 Up seen=false, akka.tcp://opendaylight-cluster-data@10.29.15.176:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@10.29.15.237:2550 Up seen=true] </pre> </div></div> CONTROLLER-1790

Leader can currently not perform its duties

Bug Medium Resolved Won't Do Unassigned Sam Hague csit:3node Fri, 17 Nov 2017 21:27:02 +0000 Thu, 23 Aug 2018 06:15:31 +0000 Thu, 23 Aug 2018 06:15:31 +0000 clustering 0 3 <p>That's reported from akka when connection is lost to a node. Looks like all normal stuff. </p> <p>Could the way we kill nodes cause any issues? The current method to kill the node is below.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>${NODE_KILL_COMMAND} ps axf | grep org.apache.karaf | grep -v grep | awk '{print \"kill -9 \" $1}' | sh </pre> </div></div> <p>That shouldn't matter - that message is normal akka behavior when a node becomes unavailable to the leader for any reason. By "duties", it means it can't let any <b>new</b> node into the cluster until all existing ones are either reachable or specifically declared as "down". That's the way it works - all existing nodes have to be able to reach the new node and agree to allow it in.</p> <p>Do you see anything actionable here or OK to close?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=shague" class="user-hover" rel="shague">shague</a> ok to close here?</p> <p>Wondering if this came from a blown up test because of <a href="https://jira.opendaylight.org/browse/CONTROLLER-1849" title="controller not coming up healthy after being killed and restarted (401 after 5m)" class="issue-link" data-issue-key="CONTROLLER-1849"><del>CONTROLLER-1849</del></a>? Let's find the next job that has this problem and double check.</p> <p>We have an example of a 100% passing <a href="https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-fluorine/25/" class="external-link" target="_blank" rel="nofollow noopener">netvirt CSIT job</a> where will still see this message "Leader can currently not perform its duties" will show up, which as <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=tpantelis" class="user-hover" rel="tpantelis">tpantelis</a> notes is an expected message when<br/> we are downing nodes in the cluster. I think it's safe to close this one for now. If we have a more specific failure or if we find this message is coming at an inappropriate<br/> time, we can re-open this or a new one.</p> Relates CONTROLLER-1849 Development Epic Link NETVIRT-996 Rank 0|i037z7: