ODL doesn't establish the thrift connection with qbgp which causes ODL can't be configured as BGP server and not able proceed with BGP peer with Quagga VM and datapath testing.
Version: ppa:odl-team/nitrogen
Here are the error logs:
karaf log:
2018-01-29 11:16:31,959 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | number of retries left 93 delay factor 87
2018-01-29 11:16:32,959 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Trying to connect BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:32,960 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | number of retries left 92 delay factor 88
2018-01-29 11:16:33,960 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Trying to connect BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:33,961 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | number of retries left 91 delay factor 89
2018-01-29 11:16:34,600 | INFO | Thread-52 | BgpThriftService | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Bgp thrift server create context event
2018-01-29 11:16:34,600 | INFO | Thread-52 | BgpThriftService | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Bgp thrift server old context is null nothing to close
2018-01-29 11:16:34,608 | INFO | pool-57-thread-1 | BgpThriftService | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | BGP (re)started
2018-01-29 11:16:34,610 | ERROR | pool-57-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | cancelling already running bgp replay task
2018-01-29 11:16:34,610 | ERROR | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Exception while waiting for next try
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)[:1.8.0_151]
at org.opendaylight.netvirt.bgpmanager.RetryOnException.waitUntilNextTry(RetryOnException.java:74)[295:org.opendaylight.netvirt.bgpmanager-impl:0.5.0]
at org.opendaylight.netvirt.bgpmanager.RetryOnException.errorOccured(RetryOnException.java:47)[295:org.opendaylight.netvirt.bgpmanager-impl:0.5.0]
at org.opendaylight.netvirt.bgpmanager.thrift.client.BgpRouter.connect(BgpRouter.java:162)[295:org.opendaylight.netvirt.bgpmanager-impl:0.5.0]
at org.opendaylight.netvirt.bgpmanager.BgpConfigurationManager.lambda$bgpRestarted$2(BgpConfigurationManager.java:1585)[295:org.opendaylight.netvirt.bgpmanager-impl:0.5.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_151]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)[:1.8.0_151]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_151]
at java.lang.Thread.run(Thread.java:748)[:1.8.0_151]
2018-01-29 11:16:34,624 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Trying to connect BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:34,624 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Connected to BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:34,628 | INFO | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Connected to BGP config server at 127.0.0.1:7644
2018-01-29 11:16:34,628 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | no config to push in bgp replay task
2018-01-29 11:16:36,610 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | running bgp replay task
2018-01-29 11:16:36,611 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | connecting to bgp host 127.0.0.1
2018-01-29 11:16:36,611 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Trying to connect BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:36,612 | ERROR | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Connected to BGP config server at 127.0.0.1 : 7644
2018-01-29 11:16:36,612 | INFO | pool-39-thread-1 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Connected to BGP config server at 127.0.0.1:7644
2018-01-29 11:16:36,612 | INFO | pool-39-thread-1 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | no config to push in bgp replay task
2018-01-29 11:16:46,507 | INFO | 6]-nio2-thread-1 | ServerSession | 129 - org.apache.sshd.core - 0.14.0 | Server session created from /127.0.0.1:49600
2018-01-29 11:16:46,524 | INFO | 6]-nio2-thread-2 | SimpleGeneratorHostKeyProvider | 129 - org.apache.sshd.core - 0.14.0 | Generating host key...
2018-01-29 11:16:51,465 | INFO | 6]-nio2-thread-4 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication attempt - karaf
2018-01-29 11:16:51,466 | INFO | 6]-nio2-thread-4 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication failed - karaf
2018-01-29 11:16:51,485 | INFO | 6]-nio2-thread-6 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication attempt - karaf
2018-01-29 11:16:51,486 | INFO | 6]-nio2-thread-6 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication succeeded - karaf
2018-01-29 11:16:51,492 | INFO | 6]-nio2-thread-6 | ServerUserAuthService | 129 - org.apache.sshd.core - 0.14.0 | Session karaf@/127.0.0.1:49600 authenticated
2018-01-29 11:16:51,535 | INFO | 6]-nio2-thread-8 | ChannelSession | 129 - org.apache.sshd.core - 0.14.0 | Executing command: bgp-connect --host 127.0.0.1 --port 7644 add
2018-01-29 11:16:51,692 | INFO | atchingManager-0 | ResourceBatchingManager | 238 - org.opendaylight.genius.mdsalutil-api - 0.3.0 | Total taken ##time = 13ms for resourceList of size 1 for resourceType BGP-RESOURCES
2018-01-29 11:16:51,695 | ERROR | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | received add Bgp config
2018-01-29 11:16:51,699 | ERROR | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | IO Exception got while activating mip: Cannot run program "cluster": error=2, No such file or directory
2018-01-29 11:16:51,700 | ERROR | eChangeHandler-0 | BgpRouter | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | Config server updated while connecting to server 127.0.0.1 7644
2018-01-29 11:16:51,700 | ERROR | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | config-server Add failed; Config store updated; undo with Delete if needed.
2018-01-29 11:16:52,548 | INFO | 6]-nio2-thread-6 | ServerSession | 129 - org.apache.sshd.core - 0.14.0 | Server session created from /127.0.0.1:49644
2018-01-29 11:16:52,729 | INFO | 6]-nio2-thread-3 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication attempt - karaf
2018-01-29 11:16:52,729 | INFO | 6]-nio2-thread-3 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication failed - karaf
2018-01-29 11:16:52,736 | INFO | 6]-nio2-thread-4 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication attempt - karaf
2018-01-29 11:16:52,736 | INFO | 6]-nio2-thread-4 | LogAuditLoginModule | 109 - org.apache.karaf.jaas.modules - 4.0.9 | Authentication succeeded - karaf
2018-01-29 11:16:52,737 | INFO | 6]-nio2-thread-4 | ServerUserAuthService | 129 - org.apache.sshd.core - 0.14.0 | Session karaf@/127.0.0.1:49644 authenticated
2018-01-29 11:16:52,769 | INFO | 6]-nio2-thread-6 | ChannelSession | 129 - org.apache.sshd.core - 0.14.0 | Executing command: odl:configure-bgp -op start-bgp-server --as-num 100 --router-id 172.29.239.65
2018-01-29 11:16:52,834 | INFO | atchingManager-0 | ResourceBatchingManager | 238 - org.opendaylight.genius.mdsalutil-api - 0.3.0 | Total taken ##time = 8ms for resourceList of size 1 for resourceType BGP-RESOURCES
2018-01-29 11:16:52,835 | ERROR | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | received bgp add asid AsId{getLocalAs=100, getRouterId=IpAddress [_ipv4Address=Ipv4Address [_value=172.29.239.65]], getStalepathTime=0, isAnnounceFbit=false, augmentations={}}
2018-01-29 11:16:52,835 | WARN | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | as-id : configuration received when BGP is inactive
2018-01-29 11:16:52,835 | ERROR | eChangeHandler-0 | BgpConfigurationManager | 295 - org.opendaylight.netvirt.bgpmanager-impl - 0.5.0 | as-id Unable to process add for asNum 100; 102
2018-01-29 11:17:58,015 | INFO | eChangeHandler-0 | IdManager | 229 - org.opendaylight.genius.idmanager-impl - 0.3.0 | Got pool IdLocalPool [poolName=elan.ids.pool.2130706689, availableIds=AvailableIdHolder [low=5000, high=5499, cur=5000], releasedIds=ReleasedIdHolder [availableIdCount=0, timeDelaySec=30, delayedEntries=[]]]
2018-01-29 11:17:58,015 | INFO | eChangeHandler-0 | IdManager | 229 - org.opendaylight.genius.idmanager-impl - 0.3.0 | The newIdValues [5001] for the idKey bd8db3a8-2b30-4083-a8b3-b3fd46401142
2018-01-29 11:17:58,019 | INFO | nPool-1-worker-6 | UpdateIdEntryJob | 229 - org.opendaylight.genius.idmanager-impl - 0.3.0 | Updated id entry with idValues [5001], idKey bd8db3a8-2b30-4083-a8b3-b3fd46401142, pool elan.ids.pool.2130706689
2018-01-29 11:20:39,051 | INFO | er@6739462fTimer | teInvalidatingHashSessionManager | 384 - org.ops4j.pax.web.pax-web-jetty - 4.3.0 | Timing out for 1 session(s) with id 1h2yggdry4x2hfaknda0pjysi
2018-01-29 11:20:39,052 | INFO | er@6739462fTimer | teInvalidatingHashSessionManager | 384 - org.ops4j.pax.web.pax-web-jetty - 4.3.0 | Timing out for 1 session(s) with id jgsxciwf7cgw6z2q1svve3n
2018-01-29 11:23:06,750 | INFO | er@1f8174c2Timer | teInvalidatingHashSessionManager | 384 - org.ops4j.pax.web.pax-web-jetty - 4.3.0 | Timing out for 1 session(s) with id 1g7gc4namynfthx4j6cg8fk3i
2018-01-29 11:23:06,750 | INFO | er@1f8174c2Timer | teInvalidatingHashSessionManager | 384 - org.ops4j.pax.web.pax-web-jetty - 4.3.0 | Timing out for 1 session(s) with id 1i31htqv2ryah1v2jjb544xae
further info:
root@controller00-neutron-server-container-e75c1444:~# netstat -a | grep 7644
tcp 0 0 :7644 *: LISTEN
root@controller00-neutron-server-container-e75c1444:~# cat /opt/quagga/etc/bgpd.conf
hostname bgpd
password sdncbgpc
service advanced-vty
log stdout
line vty
exec-timeout 0 0
debug bgp
debug bgp updates
debug bgp events
root@controller00-neutron-server-container-e75c1444:/opt/quagga/var/log/quagga# cat zrpcd.init.log
2018/01/29 11:16:34 ZRPC: Can't fopen pid lock file /opt/quagga/var/run/quagga/bgpd.pid (2), continuing
2018/01/29 11:16:34 ZRPC: Can't fopen pid lock file /opt/quagga/var/run/quagga/bgpd.pid (2), continuing
2018/01/29 11:16:34 ZRPC: bgpUpdater check connection with 0.0.0.0:6644 OK
2018/01/29 11:16:34 ZRPC: bgpUpdater check connection with 0.0.0.0:6644 OK
2018/01/29 11:16:34 ZRPC: zrpcd starting: zrpc@<all>:7644 pid 7920
2018/01/29 11:16:34 ZRPC: zrpcd starting: zrpc@<all>:7644 pid 7920
2018/01/29 11:16:34 ZRPC: zrpc_accept : new connection (fd 11) from 127.0.0.1:39310
2018/01/29 11:16:34 ZRPC: zrpc_accept : new connection (fd 11) from 127.0.0.1:39310
-
- (process:7920): WARNING **: error reading start of message: failed to read 4 bytes - Resource temporarily unavailable
2018/01/29 11:16:36 ZRPC: zrpcd_read_packet: close connection (fd 11)
2018/01/29 11:16:36 ZRPC: zrpcd_read_packet: close connection (fd 11)
2018/01/29 11:16:36 ZRPC: zrpc_accept : new connection (fd 11) from 127.0.0.1:39318
2018/01/29 11:16:36 ZRPC: zrpc_accept : new connection (fd 11) from 127.0.0.1:39318
2018/01/29 11:16:45 ZRPC: zrpc_accept : new connection (fd 12) from 127.0.0.1:39366
2018/01/29 11:16:45 ZRPC: zrpc_accept : new connection (fd 12) from 127.0.0.1:39366
2018/01/29 11:16:45 ZRPC: zrpc_accept : a new connection from same src IP. Ignoring it (fd 11)
2018/01/29 11:16:45 ZRPC: zrpc_accept : a new connection from same src IP. Ignoring it (fd 11)
-
- (process:7920): WARNING **: error reading start of message: failed to read 4 bytes - Resource temporarily unavailable
2018/01/29 11:16:45 ZRPC: zrpcd_read_packet: close connection (fd 12)
2018/01/29 11:16:45 ZRPC: zrpcd_read_packet: close connection (fd 12)
-
- (process:7920): WARNING **: error reading start of message: failed to read 4 bytes - Resource temporarily unavailable
2018/01/29 11:16:51 ZRPC: zrpcd_read_packet: close connection (fd 11)
2018/01/29 11:16:51 ZRPC: zrpcd_read_packet: close connection (fd 11)
root@controller00-neutron-server-container-e75c1444:/opt/quagga/var/log/quagga# curl -s -u admin:admin -X GET http://172.29.236.11:8180/restconf/config/ebgp:bgp
{"bgp":{"config-server":
{"host":"127.0.0.1","port":7644},"as-id":{"announce-fbit":false,"router-id":"172.29.239.65","local-as":100,"stalepath-time":0}}}root@controller00-neutron-server-container-e75c1444:/opt/quagga/var/log/quagga#
root@controller00-neutron-server-container-e75c1444:/opt/quagga/var/log/quagga# ssh karaf@localhost -p8101
Password authentication
Password:
_______ ________ . . ._ __
____ \ ______ ____ ____ _____ \ ____ ___..| | || ____ | |_/ |
/ |
____ _/ __ \ / \ | |
__ \< | || | | |/ ___| | \ __\
/ | \ |> > ___/| | | ` \/ __
___ || || / /_/ > Y \ |
______ / __/ _ >_| /_____ (__ / ____||__/_ /|_| /_|
\/|_| \/ \/ \/ \/\/ /____/ \/
Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight.
opendaylight-user@root>display-bgp-config --debug
is ODL Connected to Q-BGP: FALSE
ODL BGP Router transport is open: FALSE
Last ODL connection attempt TS: Mon Jan 29 11:16:51 UTC 2018
Last Successful connection TS: Mon Jan 29 11:16:36 UTC 2018
Last ODL started BGP at: Thu Jan 01 00:00:00 UTC 1970
Last Quagga BGP, sent reSync at: Mon Jan 29 11:16:34 UTC 2018
Time taken to create stale fib : 0 ms
Time taken to create replay configuration : 0 ms
Time taken for Stale FIB cleanup : 0 ms
Total stale entries created 0
Total stale entries cleared 0
Mon Jan 29 11:15:05 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=bgp}] amIOwner:false hasOwner:false
Mon Jan 29 11:15:05 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=bgp}] amIOwner:true hasOwner:true
Mon Jan 29 11:15:08 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=interface_config}] amIOwner:false hasOwner:false
Mon Jan 29 11:15:08 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=interface_config}] amIOwner:true hasOwner:true
Mon Jan 29 11:15:08 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=interface_service_binding}] amIOwner:true hasOwner:true
Mon Jan 29 11:15:09 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=netvirt-acl}] amIOwner:false hasOwner:false
Mon Jan 29 11:15:09 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=netvirt-acl}] amIOwner:true hasOwner:true
Mon Jan 29 11:15:09 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=elan}] amIOwner:false hasOwner:false
Mon Jan 29 11:15:09 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=elan}] amIOwner:true hasOwner:true
Mon Jan 29 11:15:11 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=arpmonitoring}] amIOwner:false hasOwner:false
Mon Jan 29 11:15:11 UTC 2018 entity : /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)entity/entity[\{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:core:general-entity?revision=2015-08-20)name=arpmonitoring}] amIOwner:true hasOwner:true
Configuration Server
Host 127.0.0.1
Port 7644
BGP Router
AS-Number 100
Router-ID 172.29.239.65
Stale-Path-Time default
F-bit ON
opendaylight-user@root>