OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[BGPCEP-491] bgp-ls update not being shown in the bgp topology in ODL https://jira.opendaylight.org/browse/BGPCEP-491 bgpcep <p>bgp-ls update not being shown in the bgp topology in ODL</p> <p>Operating System: All<br/> Platform: All</p> BGPCEP-491

bgp-ls update not being shown in the bgp topology in ODL

Bug Resolved Duplicate Claudio David Gasparini Claudio David Gasparini Wed, 6 Jul 2016 08:50:13 +0000 Sun, 3 Mar 2019 11:49:48 +0000 Wed, 10 Aug 2016 23:29:49 +0000 Bugzilla Migration Bugzilla Migration BGP 0 4 <p>Attachment karaf.log has been added with description: karaf.log</p> <p>Attachment karaf.log.1.gz has been added with description: karaf.log1</p> <p>Attachment karaf.log.2.gz has been added with description: karaf.log2</p> <p>Attachment karaf.log.3.gz has been added with description: karaf.log3</p> <p>Attachment karaf.log.4.gz has been added with description: karaf.log.4</p> <p>Attachment karaf.log.5.gz has been added with description: karaf.log5</p> <p>Attachment karaf.log.6.gz has been added with description: karaf.log6</p> <p>This issue occurs when a link is shut and goes down. It is removed from show mpls traffic-eng topology output, but not from the ODL topology. After link was shut(no shut) successive GETS on bgpls toplolgy (curl -u admin:admin <a href="http://localhost:8181/restconf/operational/network-topology:network-topology/topology/example-linkstate-topology" class="external-link" target="_blank" rel="nofollow noopener">http://localhost:8181/restconf/operational/network-topology:network-topology/topology/example-linkstate-topology</a>) do not reflect change link state, i.e., the link-id object is always present. sh bgp li li on bgp speaker has 256,232 entries. </p> <p>Summary of steps to reproduce this issue:</p> <p>1. GET bgp-ls topology from ODL<br/> 2. bring link down<br/> 3. GET bgp-ls topology from ODL</p> <p>no difference in ODL topology</p> <p>4. bring link up<br/> 5. GET bgp-ls topology from ODL</p> <p>no differences in ODL topology</p> <p>Throttle timer or distribution is set:</p> <p>distribute bgp-ls instance 2 throttle 10</p> <p>Hi Al, Could you please attach package capture from both UPDATES. (Advertisement and Withdrawal) so I can reproduce it locally.<br/> I tested in Be SR2 using application Peer and it worked well, from the logs attached I don't see any problem.<br/> Once I have the packages I ll able to trace it better what can be causing this behavior.</p> <p>Thanks</p> <p>Not following. What are you referring to with 'package capture'?</p> <p>Adding additional info. bgp link-state on router shows no route, yet ODL still holds the link-id.</p> <p>RP/0/RP0/CPU0:HUCRS1#sh bgp li li <span class="error">[E]</span><span class="error">[O]</span><span class="error">[I0x2]</span>[N<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span>[a0.0.0.0$<br/> Wed Jul 6 12:41:41.469 EDT<br/> BGP routing table entry for <span class="error">[E]</span><span class="error">[O]</span><span class="error">[I0x2]</span>[N<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span><span class="error">[a0.0.0.0]</span><span class="error">[r10.0.0.208]</span>][R<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span><span class="error">[a0.0.0.0]</span><span class="error">[r10.0.0.209]</span>][L<span class="error">[i10.0.7.98]</span><span class="error">[n10.0.7.97]</span>]/792<br/> Versions:<br/> Process bRIB/RIB SendTblVer<br/> Speaker 869532 869532<br/> Last Modified: Jul 6 12:28:22.012 for 00:13:19<br/> Paths: (0 available, no best path)<br/> Not advertised to any peer<br/> RP/0/RP0/CPU0:HUCRS1#sh bgp li li <span class="error">[E]</span><span class="error">[O]</span><span class="error">[I0x2]</span>[N<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span>[a0.0.0.0$<br/> Wed Jul 6 12:42:03.220 EDT<br/> BGP routing table entry for <span class="error">[E]</span><span class="error">[O]</span><span class="error">[I0x2]</span>[N<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span><span class="error">[a0.0.0.0]</span><span class="error">[r10.0.0.209]</span>][R<span class="error">[c65060]</span><span class="error">[b10.0.0.208]</span><span class="error">[a0.0.0.0]</span><span class="error">[r10.0.0.208]</span>][L<span class="error">[i10.0.7.97]</span><span class="error">[n10.0.7.98]</span>]/792<br/> Versions:<br/> Process bRIB/RIB SendTblVer<br/> Speaker 869529 869529<br/> Last Modified: Jul 6 12:28:22.013 for 00:13:41<br/> Paths: (0 available, no best path)<br/> Not advertised to any peer<br/> RP/0/RP0/CPU0:HUCRS1#</p> <p>ODL topology:</p> <p><span class="error">[acg000@automation2 ~]</span>$ grep link-id.*10.0.7.9<span class="error">[78]</span> bgpls.core4 | grep area=0<br/> "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",</p> <p>Continued to perform GET on bgp-ls tolology, and link-id is now removed from topology. The time difference is ~50 minutes. I expect topology change in ODL to be instantaneous.</p> <p><del>rw-r</del><del>r</del>-. 1 acg000 staff 43995514 Jul 6 13:20 bgpls.core5<br/> <del>rw-r</del><del>r</del>-. 1 acg000 staff 44009411 Jul 6 12:33 bgpls.core4</p> <p>link-id only present in bgpls.core4 file.</p> <p><span class="error">[acg000@automation2 ~]</span>$ grep link-id.*10.0.7.9<span class="error">[78]</span> bgpls.core<span class="error">[45]</span> | grep area=0<br/> bgpls.core4: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> bgpls.core4: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",</p> <p>"package capture" = using wireshark capture the package that are sent and attach the .cap.</p> <p>After no shut link advertisement not shown in ODL. I ran tshark for a few minutes (em1.pcap enclosed), for your review. I will GET topology every 5 minutes to see when the topology reflects the no shut link.</p> <p>Enclosed is the pcap file from our ODL server which was run for several minutes during link advertisement.</p> <p>Attachment em1.pcap.gz has been added with description: pcap from ODL server for link advertisement</p> <p>Hi Al, what I'm expecting to find in .cap is the message announcing the link and the one announcing the withdrawn, so based on what you said you ran wireshark after fail. Therefore I can do really much about it.</p> <p>What I suspect can be issue, is BUG-6083. When a new bet Path is announced, is not updated. Once the fix for this bug is merged I will let you know and ask you to test against the latest version of Be which I ll provide to you.</p> <p>Meanwhile you can attach the new .cap and add more information about the message that contains the update so we can try to replicate or identify from the capture.</p> <p>The pcap I sent was when the link was advertised, not after. I ran for a few minutes thinking that the advertisement would be captured.</p> <p>Hi Al, the withdrawal is not advertised, we expect to see an MP_UNREACH_NLRI<br/> You can check it by using filter (bgp.mp_unreach_nlri_ipv4_prefix) || (bgp.mp_unreach_nlri_ipv6_prefix)</p> <p>So it might be that router is not sending the notification.<br/> There can be some missing configuration, like distribute bgp-ls</p> <p><a href="http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r5-1/routing/command/reference/b_routing_cr51xasr9k/b_routing_cr51xasr9k_chapter_00.html#wp1456509002" class="external-link" target="_blank" rel="nofollow noopener">http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r5-1/routing/command/reference/b_routing_cr51xasr9k/b_routing_cr51xasr9k_chapter_00.html#wp1456509002</a></p> <p>Cisco already confirmed that updates were sent from the router. I thought the purpose of sending you the pcap file was for you to confirm that the updates were being handled properly by ODL? The distribute configuration has been configured since last year.</p> <p>(In reply to Al Goddard from comment #18)<br/> > Cisco already confirmed that updates were sent from the router. I thought<br/> > the purpose of sending you the pcap file was for you to confirm that the<br/> > updates were being handled properly by ODL? The distribute configuration<br/> > has been configured since last year.</p> <p>Hi Al,<br/> We would need a packet capture and ODL logs from moment when the link is going down, so we can investigate what is happening in ODL.</p> <p>I see that the filter:</p> <p>bgp.mp_reach_nlri_ipv4_prefix</p> <p>does not return any matches, but this filter:</p> <p>bgp.update.path_attribute.mp_reach_nlri.afi</p> <p>does return matches.</p> <p>Are you specifically looking not for an UPDATE message, but a bgp.mp_reach* message?</p> <p>(In reply to Al Goddard from comment #20)<br/> > I see that the filter:<br/> > <br/> > bgp.mp_reach_nlri_ipv4_prefix<br/> > <br/> > does not return any matches, but this filter:<br/> > <br/> > bgp.update.path_attribute.mp_reach_nlri.afi<br/> > <br/> > does return matches.<br/> > <br/> > Are you specifically looking not for an UPDATE message, but a bgp.mp_reach*<br/> > message?</p> <p>We are looking for specific MP_UNREACH NLRI Path Attribute (included in Update Message) which express the link is no longer reachable, so can be removed from ODL BGP-LS topology.<br/> Filter is: bgp.update.path_attribute.mp_unreach_nlri.afi</p> <p>Please see Comment 16. There are hundreds of type 'bgp.update.path_attribute.mp_reach_nlri.afi', as this test was done when when the link came up, and hence an mp_reach would be generated.</p> <p>Hi Al, seems that you didn't receive the email I sent last week. I ll attach it here:</p> <p>Hi,<br/> For keep a proper record of the bug, our main communication is via the BUG opened <a href="https://bugs.opendaylight.org/show_bug.cgi?id=6160" class="external-link" target="_blank" rel="nofollow noopener">https://bugs.opendaylight.org/show_bug.cgi?id=6160</a>.<br/> As requested in the bug we need proper information so we can trace and replicate the bug.<br/> Logs and and packet capture needs to be done when link goes down. The ones provided doesn't contain that information, <br/> This can mean that mp_unreach is not sent or the record attached is not correct. </p> <p>Some things that needs to be checked : </p> <ul class="alternate" type="square"> <li>check via wireshark at the moment that links goes down, using filter bgp.update.path_attribute.mp_unreach_nlri.afi.</li> <li>Ensure that message is sent at the moment link goes down and not later, which could be sent by neighbour expiration based on lsp timeout for example.(based on comments that after 50 min it dissapeared)</li> </ul> <p>This will ensure that issue is in ODL side. Then provide the .cap to us for further analysis, attaching it to the bug.<br/> Same for logs, since it seems that you are experiencing some issues when activating TRACE logs, ensure at least that message is sent without activate the logs, this will ensure that we go in the correct path.</p> <p>Regards,</p> <p>Cisco BGP DE are investigating router debugs + wireshark of a recent repro of this issue. I am attachment of the relevant bgp.update.path_attribute.mp_unreach_nlri.afi packets for this test when the link was shut down, and a bgp withdrawal update was sent.</p> <p>The pcap clearly shows the packet was sent by the router and received on the controller. Why wasn't this link removed from the topology?</p> <p>These are the packets that contain MP_UNREACH updates for the link that was withdrawn. Summary extract from capture:</p> <p> Path Attribute - MP_UNREACH_NLRI<br/> Type Code: MP_UNREACH_NLRI (15)<br/> Link State SAFI 72 NLRI<br/> NLRI Type: Link NLRI (2)<br/> NLRI Length: 97<br/> Link-State NLRI Link NLRI<br/> Area ID TLV<br/> Area ID: 167898919 (0x0a01ef27)<br/> Area ID TLV<br/> Area ID: 167898919 (0x0a01ef27)<br/> IPv4 Interface Address: 10.0.7.98<br/> IPv4 Neighbor Address: 10.0.7.97<br/> Link State SAFI 72 NLRI<br/> NLRI Type: Link NLRI (2)<br/> NLRI Length: 97<br/> Link-State NLRI Link NLRI<br/> Area ID TLV<br/> Area ID: 0 (0x00000000)<br/> Area ID TLV<br/> Area ID: 0 (0x00000000)<br/> IPv4 Interface Address: 10.0.7.98<br/> IPv4 Neighbor Address: 10.0.7.97<br/> Link State SAFI 72 NLRI<br/> NLRI Type: Link NLRI (2)<br/> NLRI Length: 97<br/> Link-State NLRI Link NLRI<br/> Area ID TLV<br/> Area ID: 0 (0x00000000)<br/> Area ID TLV<br/> Area ID: 0 (0x00000000)<br/> IPv4 Interface Address: 10.0.7.97<br/> IPv4 Neighbor Address: 10.0.7.98<br/> Link State SAFI 72 NLRI<br/> NLRI Type: Link NLRI (2)<br/> NLRI Length: 97<br/> Link-State NLRI Link NLRI<br/> Area ID TLV<br/> Area ID: 167898919 (0x0a01ef27)<br/> Area ID TLV<br/> Area ID: 167898919 (0x0a01ef27)<br/> IPv4 Interface Address: 10.0.7.97<br/> IPv4 Neighbor Address: 10.0.7.98</p> <p>Attachment unreach.txt has been added with description: MP_UNREACH capture from wireshark</p> <p>Cisco BGP DE has confirmed that withdraw update was sent by using a debug SMU for bgp and correlating with wireshark capture. This is an urgent issue pointing at ODL. Please continue to pursue to resolution.</p> <p>-------</p> <p>We were able to turn on the actual date/time from the View menu. The withdrawal packet was sent out by CRS router within 2sec of the link being shut down. Wireshark also indicates the same.</p> <p>LC/0/13/CPU0:Jul 20 14:04:27.574 : ifmgr<span class="error">[197]</span>: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/13/0/0, changed state to Administratively Down</p> <p>And the withdraws were packed into the packets at 14:04:29</p> <p>Jul 20 14:04:29.387 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[GEN]</span>:7664: CSCva25043 bgp_send_update(7664) withdraw to be formatted for pelem 10db8084<br/> Jul 20 14:04:29.387 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[unknown 0x10007d2/6]</span> 0x0000123d 0x10e42910 0x0000123d 0x10db8084 0x10e42938 0x00000001<br/> Jul 20 14:04:29.387 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[GEN]</span>:4736: CSCva25043 bgp4_find_or_format_withdraw(4736): adv bit reset for pelem 10db8084, fgrp 1<br/> Jul 20 14:04:29.388 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[GEN]</span>:7664: CSCva25043 bgp_send_update(7664) withdraw to be formatted for pelem 10db8474<br/> Jul 20 14:04:29.388 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[unknown 0x10007d2/6]</span> 0x0000123d 0x10e4298c 0x0000123d 0x10db8474 0x10e429b4 0x00000001<br/> Jul 20 14:04:29.388 default-bgp/spkr-tr2-gen 0/RP0/CPU0 t20 <span class="error">[GEN]</span>:4736: CSCva25043 bgp4_find_or_format_withdraw(4736): adv bit reset for pelem 10db8474, fgrp 1</p> <p>So investigation now shifts to ODL team. Please continue to communicate with them via 6160.</p> <p>Hi Al, thanks for the update. Please attach the .cap.<br/> Could attach also the logs from controller?</p> <p>Regards,</p> <p>Attached is the .pcap file of the MP_UNREACH NRLI for the route withdrawal that is not shown in the bgpl-ls datastore topology.</p> <p>Attachment bgpls.pcap.gz has been added with description: .pcap file showing route withdrawal</p> <p>Although reproduced on Be SR2, we are using Brocade bvc 3.0 in production (problem also observed on this release). I don't have any karaf logs for this issue. Shal we continue to work this via this bug, or defer to Brocade for handling escalation?</p> <p>Hi Al,</p> <p>Once the link down event happens, can you check the LOC-RIB to see if link information is still present there?<br/> http://<controller-ip-addr>:8181/restconf/operational/bgp-rib:bgp-rib/rib/example-bgp-rib/loc-rib/</p> <p>Also can you please enable bgpcep debugs and collect controller logs a minute before and a couple of minutes after the link is shut? Getting pcaps during those few minutes will also help as we can then correlate update messages to how the controller handles them</p> <p>log:set DEBUG org.opendaylight.bgpcep<br/> log:set DEBUG org.opendaylight.protocol</p> <p>Thanks<br/> Ajay</p> <p>Hi Al,</p> <p>Once the link down event happens, can you check the LOC-RIB to see if link information is still present there?<br/> http://<controller-ip-addr>:8181/restconf/operational/bgp-rib:bgp-rib/rib/example-bgp-rib/loc-rib/</p> <p>Also can you please enable bgpcep debugs and collect controller logs a minute before and a couple of minutes after the link is shut? Getting pcaps during those few minutes will also help as we can then correlate update messages to how the controller handles them</p> <p>log:set DEBUG org.opendaylight.bgpcep<br/> log:set DEBUG org.opendaylight.protocol</p> <p>Thanks<br/> Ajay(In reply to Ajay L from comment #30)<br/> > Hi Al,<br/> > <br/> > Once the link down event happens, can you check the LOC-RIB to see if link<br/> > information is still present there?<br/> > http://<controller-ip-addr>:8181/restconf/operational/bgp-rib:bgp-rib/rib/<br/> > example-bgp-rib/loc-rib/<br/> > <br/> > Also can you please enable bgpcep debugs and collect controller logs a<br/> > minute before and a couple of minutes after the link is shut? Getting pcaps<br/> > during those few minutes will also help as we can then correlate update<br/> > messages to how the controller handles them<br/> > <br/> > log:set DEBUG org.opendaylight.bgpcep<br/> > log:set DEBUG org.opendaylight.protocol<br/> > <br/> > Thanks<br/> > Ajay</p> <p>Getting "show mpls traffic-eng topology" and "show bgp link-state link-state" (with appropriate filters if number of entries in excessive) from the router before & after link down while collecting above output will help</p> <p>Hi Al, I was able to announce and withdrawal the link using the same packets you provided.<br/> reach from the first .cap </p> <p>bgp.update.path_attribute.mp_reach_nlri.afi && bgp.ls.nlri_ipv4_interface_address == 10.0.7.98 && bgp.ls.nlri_ipv4_neighbor_address == 10.0.7.97</p> <p>unreach from the second</p> <p>bgp.update.path_attribute.mp_unreach_nlri.afi && bgp.ls.nlri_ipv4_interface_address == 10.0.7.98 && bgp.ls.nlri_ipv4_neighbor_address == 10.0.7.97</p> <p>The behavior was the expected one and everything worked fine. Link was announced and removed correctly when it was withdrawn.<br/> In resume, we need the logs to see if it can add some light to the problem you are experiencing.</p> <p>Regards,</p> <p>Just repro the issues today and captured ODL DEBUG/wireshark. Will update bug with detailed information soon.</p> <p>The topology is quite large. Local rib after json formatting is 500M.</p> <p>This clearly looks like topology not being updated.</p> <p>Here’s a timeline:</p> <p>RP/0/RP0/CPU0:Jul 26 09:24:34.451 : ifmgr<span class="error">[250]</span>: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether218, changed state to Down<br/> RP/0/RP0/CPU0:Jul 26 09:24:34.451 : ifmgr<span class="error">[250]</span>: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether218, changed state to Down</p> <p>Pre capture:</p> <p><del>rw-r</del><del>r</del>-. 1 acg000 staff 47074820 Jul 26 09:15 bgpls.odl1-12<br/> <del>rw-r</del><del>r</del>-. 1 acg000 staff 433888305 Jul 26 09:20 bgpls.odl1-12-rib</p> <p>Post capture:</p> <p><del>rw-r</del><del>r</del>-. 1 acg000 staff 47074820 Jul 26 09:25 bgpls.odl1-13<br/> <del>rw-r</del><del>r</del>-. 1 acg000 staff 433851492 Jul 26 09:29 bgpls.odl1-13-rib</p> <p>Link in pre/post topologies:</p> <p><span class="error">[acg000@automation2 ~]</span>$ grep link-id bgpls.odl1-1<span class="error">[23]</span> | grep 10.0.7.9<span class="error">[78]</span><br/> bgpls.odl1-12: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=167898919&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=167898919&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> bgpls.odl1-12: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",<br/> bgpls.odl1-12: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> bgpls.odl1-12: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=167898919&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=167898919&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",</p> <p>bgpls.odl1-13: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=167898919&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=167898919&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> bgpls.odl1-13: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",<br/> bgpls.odl1-13: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=0&local-router=167772369&remote-as=65060&remote-domain=167772368&remote-area=0&remote-router=167772368&ipv4-iface=10.0.7.97&ipv4-neigh=10.0.7.98",<br/> bgpls.odl1-13: "link-id": "bgpls://Ospf:2/type=link&local-as=65060&local-domain=167772368&local-area=167898919&local-router=167772368&remote-as=65060&remote-domain=167772368&remote-area=167898919&remote-router=167772369&ipv4-iface=10.0.7.98&ipv4-neigh=10.0.7.97",<br/> <span class="error">[acg000@automation2 ~]</span>$</p> <p>Link only in pre local rib:</p> <p><span class="error">[acg000@automation2 ~]</span>$ grep ipv4-inter.*address bgpls.odl1-1<span class="error">[23]</span>-rib | grep 10.0.7.9<span class="error">[78]</span><br/> bgpls.odl1-12-rib: "ipv4-interface-address": "10.0.7.97",<br/> bgpls.odl1-12-rib: "ipv4-interface-address": "10.0.7.97",<br/> bgpls.odl1-12-rib: "ipv4-interface-address": "10.0.7.98",<br/> bgpls.odl1-12-rib: "ipv4-interface-address": "10.0.7.98",</p> <p>Ajay,</p> <p>Here are the karaf logs. As Al mentioned, the loc rib is updated but the topology is not. As I previously mentioned, there are many ping pong messages. The pcap did not work since I used wrong interface during the test but I don't think you need that since we are seeing loc rib updated so we know we are getting the update. Let me know if you need us to rerun this test.</p> <p>Lynn</p> <p>Attachment bug6160.zip has been added with description: karaf log during link down</p> <p>Hi Lynn, based on the logs and test I did, this is not an issue with the packets or how their are handled. This is an error that occurs before and therefore affects topology update.</p> <p>Logs shows IllegalStateException: New transaction PingPongTransaction<br/> which indicates that its a replica of an already open bug (BUG-6111).</p> <p>Could you confirm that this logs ERRORs are already seen before links goes down, and some trace if possible when they start to be present? </p> <p>Regards,</p> <p>(In reply to Lynn Rivera from comment #34)<br/> > Created attachment 1111 <span class="error">[details]</span><br/> > karaf log during link down<br/> > <br/> > Ajay,<br/> > <br/> > Here are the karaf logs. As Al mentioned, the loc rib is updated but the<br/> > topology is not. As I previously mentioned, there are many ping pong<br/> > messages. The pcap did not work since I used wrong interface during the<br/> > test but I don't think you need that since we are seeing loc rib updated so<br/> > we know we are getting the update. Let me know if you need us to rerun this<br/> > test.<br/> > <br/> > Lynn</p> <p>Yep there are a large number of PingPongTransaction error messages. Also given the observation that on link down loc-rib is updated but not topology, this is most likely same issue as in <a href="https://bugs.opendaylight.org/show_bug.cgi?id=6111" class="external-link" target="_blank" rel="nofollow noopener">https://bugs.opendaylight.org/show_bug.cgi?id=6111</a></p> <p>Yes, I am confirming we have the PING PONG before and after the link up/down. </p> <p>Lynn</p> <p>Fix for BUG-6111 and BUG-6342 has fixed this issue</p> Blocks BGPCEP-486 Duplicate BGPCEP-506 Development External issue ID 6160 External issue URL Issue Type Rank 0|i02cm7: