1. Aim
   To connect three node cluster to the netconf testtool and look behind mount point.

1. Test Artifact
   Last successful Nitrogen auto-release build
   https://nexus.opendaylight.org/content/repositories/autorelease-1919

1. 1. Test Setup
      Member-1: 10.10.199.105
      Member-2: 10.10.199.88
      Member-3: 10.10.199.183
      Netconf device: 10.10.199.185

java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

1. 1. Steps
      Configure a 3 node cluster
      Start odl on members

Set the log levels for clustering and netconf

log:set DEBUG org.opendaylight.controller.cluster
log:set INFO org.opendaylight.controller.cluster.datastore.node.utils
log:set DEBUG org.opendaylight.controller.netconf
log:set DEBUG org.opendaylight.controller.sal.connect.netconf

1. Load features
   feature:install odl-jolokia odl-netconf-clustered-topology odl-restconf

@ 11.04 UTC
Run postman collection to

1) find leaders of 'shard-topology-config' and 'shard-inventory-config'. It was member-1
2) Confirm netconf topology is available.
3) To confirm no device is connected.

1. 1. Example of REST APIs used.

Leader of shard-topology-config on DistributedConfigDatastore on each member

curl --request GET \
--url 'http://10.10.199.XXX:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-topology-config,type=DistributedConfigDatastore' \
--header 'accept: application/json' \
--header 'authorization: Basic YWRtaW46YWRtaW4=' \
--header 'cache-control: no-cache' \
--header 'content-type: application/json'

Leader of shard-inventory-config on DistributedConfigDatastore on each member

curl --request GET \
--url 'http://10.10.199.XXX:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-inventory-config,type=DistributedConfigDatastore' \
--header 'accept: application/json' \
--header 'authorization: Basic YWRtaW46YWRtaW4=' \
--header 'cache-control: no-cache' \
--header 'content-type: application/json'

Confirm no device added to network topology

curl --request GET \
--url http://10.10.199.105:8181/restconf/operational/network-topology:network-topology \
--header 'accept: application/xml' \
--header 'authorization: Basic YWRtaW46YWRtaW4=' \
--header 'cache-control: no-cache'

@11.05 UTC
Start device on testtools VM

java -Xmx1G -XX:MaxPermSize=256M -Dorg.apache.sshd.registerBouncyCastle=false -jar netconf-testtool-1.3-Nitrogen-executable.jar --device-count 1 --debug true --schemas-dir ./schemas --md-sal true

@11.05 UTC
Connect to device from member 1

curl --request PUT \
--url http://10.10.199.105:8181/restconf/config/network-topology:network-topology/topology/topology-netconf/node/netconf-test-device \
--header 'authorization: Basic YWRtaW46YWRtaW4=' \
--header 'cache-control: no-cache' \
--header 'content-type: application/xml' \
--data ' <node xmlns="urn:TBD:params:xml:ns:yang:network-topology">\r\n <node-id>netconf-test-device</node-id>\r\n <host xmlns="urn:opendaylight:netconf-node-topology">10.10.199.185</host>\r\n <port xmlns="urn:opendaylight:netconf-node-topology">17830</port>\r\n <username xmlns="urn:opendaylight:netconf-node-topology">root</username>\r\n <password xmlns="urn:opendaylight:netconf-node-topology">root</password>\r\n <tcp-only xmlns="urn:opendaylight:netconf-node-topology">false</tcp-only>\r\n <keepalive-delay xmlns="urn:opendaylight:netconf-node-topology">0</keepalive-delay>\r\n </node>\r\n \r\n '

@11.06 UTC
Run postman collection to see connecting status Try-1

@11.10 UTC
Run postman collection to see connecting status Try-2

1. 1. Results
      Device showed as connected but HTTP 500 errors were seen on every node when trying to look beyond the mount point.

The postman results are attached as a zip
The full logs from each member are attached as a zip

Extracts from the member-1 ,used to make the PUT request

2017-08-17 11:05:42,364 | ERROR | lt-dispatcher-37 | AAAEncryptionServiceImpl | 134 - org.opendaylight.aaa.encrypt-service - 0.6.0.Nitrogen | Failed to decrypt encoded data
javax.crypto.IllegalBlockSizeException: Input length must be multiple of 16 when decrypting with padded cipher
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:934)[sunjce_provider.jar:1.8.0_112]
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:845)[sunjce_provider.jar:1.8.0_112]
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)[sunjce_provider.jar:1.8.0_112]

2017-08-17 11:05:42,522 | ERROR | on-dispatcher-71 | DataTreeChangeListenerActor | 275 - org.opendaylight.controller.sal-clustering-commons - 1.6.0.Nitrogen | member-1-shard-topology-config: Error notifying listener org.opendaylight.controller.md.sal.binding.impl.BindingClusteredDOMDataTreeChangeListenerAdapter@2b7930d6
java.lang.NullPointerException: RemoteDevice

{netconf-test-device}

: Device communicator was not created.
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:799)[55:com.google.guava:22.0.0]