[NETCONF-821] Mounting a device does not work when multiple TLS Certificates are present Created: 15/Sep/21  Updated: 22/Jan/24

Status: In Progress
Project: netconf
Component/s: netconf
Affects Version/s: 1.13.2
Fix Version/s: 7.0.0, 6.0.7

Type: Bug Priority: Medium
Reporter: Ravi Pendurty Assignee: Ruslan Kashapov
Resolution: Unresolved Votes: 0
Labels: pick-next, pt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Microsoft Word [NETCONF-821] Steps to reproduce.rtf     Zip Archive files.zip    
Issue Links:
Relates
relates to NETCONF-1205 Support private keys and trusted cert... Open

 Description   

Configured netconf-keystore model with 2 sets of keystores. Following scenarios were tried out -

  1. Both sets are the same i.e, There are 2 key-id values - ODL_private_key_0 and ODL_private_key_1 and both have the same values  - In such a scenario, mounting of a device using either keys was successful.
  2. One is a valid key and the other is invalid i.e., the valid set (client.key, client.crt and trustedCertificates.crt) was taken, a copy of it was made and the client.crt was edited to include some invalid data. Both sets (valid and invalid) were used to create entries in the netconf-keystore. In this scenario, mounting a device with either of the keys is unsuccessful. The following is the exception in the karaf.log -
2021-09-14T04:58:18,310 | INFO | nioEventLoopGroupCloseable-3-10 | AbstractNetconfSessionNegotiator | 352 - org.opendaylight.netconf.netty-util - 1.13.2 | - | Unexpected error during negotiation
java.lang.IllegalStateException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Empty input
 at org.opendaylight.netconf.sal.connect.util.SslHandlerFactoryImpl.createSslHandler(SslHandlerFactoryImpl.java:82) ~[bundleFile:?]
 at org.opendaylight.netconf.sal.connect.util.SslHandlerFactoryImpl.createSslHandler(SslHandlerFactoryImpl.java:45) ~[bundleFile:?]
 at org.opendaylight.netconf.client.TlsClientChannelInitializer$ChannelActiveSentry.channelActive(TlsClientChannelInitializer.java:56) ~[bundleFile:?]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230) [bundleFile:4.1.63.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216) [bundleFile:4.1.63.Final]
 at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209) [bundleFile:4.1.63.Final]
 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398) [bundleFile:4.1.63.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230) [bundleFile:4.1.63.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216) [bundleFile:4.1.63.Final]
 at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:305) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:335) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) [bundleFile:4.1.63.Final]
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [bundleFile:4.1.63.Final]
 at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [bundleFile:4.1.63.Final]
 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [bundleFile:4.1.63.Final]
 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [bundleFile:4.1.63.Final]
 at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Empty input
 at sun.security.provider.X509Factory.engineGenerateCertificate(Unknown Source) ~[?:?]
 at java.security.cert.CertificateFactory.generateCertificate(Unknown Source) ~[?:?]
 at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfKeystoreAdapter.getCertificateChain(NetconfKeystoreAdapter.java:159) ~[bundleFile:?]
 at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfKeystoreAdapter.getJavaKeyStore(NetconfKeystoreAdapter.java:113) ~[bundleFile:?]
 at org.opendaylight.netconf.sal.connect.util.SslHandlerFactoryImpl.createSslHandler(SslHandlerFactoryImpl.java:51) ~[bundleFile:?]
 ... 19 more

Expectation - When the key-id from the valid set is used, mounting of the device should be successful.

 Comments   
Comment by Ivan Martiniak [ 09/Dec/21 ]

pendurty, Can you please refer further details about issue:
1. What is the type of device in which you wanted to create mount point?
2. What is the version of ONAP where you found out issue?
    I assume you have been following this guide: 
    https://docs.onap.org/projects/onap-sdnc-oam/en/istanbul/cert_installation.html, where is
    mentioned: 
    "Client.crt represents the client certificate and the client.key is the private key that is to be
    used. Only a single client/client cert is supported as of the Dublin release and multiple clients
    are not supported."
3. If you do not use ONAP, which requests have you issued with ODL RESTCONF/NETCONF?

Comment by Ravi Pendurty [ 09/Dec/21 ]

Hi ivanm1996 ,

  1. What is the type of device in which you wanted to create mount point?  [RAVI] I was trying on an O-RAN device. It supports NETCONF
  2. What is the version of ONAP where you found out issue? - Honolulu and Istanbul. Yes I have seen the guide earlier. However releases after Dublin support multiple zip files to be provided and the InstallCerts.py script ([https://gerrit.onap.org/r/gitweb?p=sdnc/oam.git;a=blob;f=installation/sdnc/src/main/scripts/installCerts.py) installs multiple keys into the netconf-keystore.
  3. I used SDNC component to issue the requests to ODL

I believe using ONAP is not the problem. When there are multiple keys configured in the netconf-keystore model, ODL fails to mount even though a valid key is passed while creating the mountpoint.

Comment by Ivan Hrasko [ 09/Dec/21 ]

pendurty what do you mean by " was edited to include some invalid data"? Was it still valid certificate?

Comment by Ivan Hrasko [ 09/Dec/21 ]

pendurty How did you specify which key should be used when mounting a device? Can you provide requests/steps to reproduce?

Comment by Ravi Pendurty [ 09/Dec/21 ]

Instead of creating a new valid certificate, I replicated an existing certificate and edited it. The edited certificate is not valid, however the basic structure of the certificate is intact, i.e., it is enclosed between  ----BEGIN CERTIFICATE---- and ----END CERTIFICATE---- but the contents in between are modified.

Comment by Ravi Pendurty [ 09/Dec/21 ]

Following is the request URL and the payload -

http://172.18.0.3:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=TLS_Device

Payload - (The key is specified in the <key-id> element. This key is added to the netconf-keystore model using the installCerts.py script that I shared earlier)

<node xmlns="urn:TBD:params:xml:ns:yang:network-topology">
 <node-id>TLS_Device</node-id>
 <key-based xmlns="urn:opendaylight:netconf-node-topology">
 <key-id xmlns="urn:opendaylight:netconf-node-topology">ODL_private_key_0</key-id>
 <username xmlns="urn:opendaylight:netconf-node-topology">netconf</username>
 </key-based>
 <host xmlns="urn:opendaylight:netconf-node-topology">172.18.0.4</host>
 <port xmlns="urn:opendaylight:netconf-node-topology">831</port>
 <tcp-only xmlns="urn:opendaylight:netconf-node-topology">false</tcp-only>
 <protocol xmlns="urn:opendaylight:netconf-node-topology">
 <name xmlns="urn:opendaylight:netconf-node-topology">TLS</name> 
 </protocol>
 <max-connection-attempts xmlns="urn:opendaylight:netconf-node-topology">2</max-connection-attempts>
</node>
Comment by Ruslan Kashapov [ 05/Dec/23 ]

Few issues here 

While key-based/key-id is eligible option in topology node configuration the purpose of this option is key based authentication for SSH transport, not TLS. It means the key-id value is ignored if protocol value is TLS. See DefaultNetconfClientConfigurationBuilderFactory invoking DefaultSslHandlerFactoryProvider

While no specific key is expected on per device basis the SslHandler is built based on KeyStore instance containing ALL the configured keys and trusted certificates. Upon KeyStore preparation all the defined private keys and certificates are parsed from binary form into Java objects. As result any single invalid (non-parseable) entry will cause SslHandler build failure for every TLS device unless this entry is removed from datastore.

 

Generated at Wed Feb 07 20:15:59 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.