https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/MDSAL-213 mdsal <p>Working with a version close to stable/boron.<br/> Trying to deserialize different BindingAware DataObjects to JSON, and sometimes getting this exception:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2016-12-21 14:10:17,114 | ERROR | pool-63-thread-4 | FederationProducerMgr | 360 - org.opendaylight.federation.service-impl - 0.1.2.SNAPSHOT | com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbNodeAugmentation$StreamWriter: frozen class (cannot edit) com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbNodeAugmentation$StreamWriter: frozen class (cannot edit) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)[38:com.google.guava:18.0.0] at com.google.common.cache.LocalCache.get(LocalCache.java:3937)[38:com.google.guava:18.0.0] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)[38:com.google.guava:18.0.0] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)[38:com.google.guava:18.0.0] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)[38:com.google.guava:18.0.0] at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry.getSerializer(BindingNormalizedNodeCodecRegistry.java:72)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SNAPSHOT] at org.opendaylight.yangtools.binding.data.codec.util.AugmentableDispatchSerializer.emitAugmentation(AugmentableDispatchSerializer.java:77)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SNAPSHOT] at org.opendaylight.yangtools.binding.data.codec.util.AugmentableDispatchSerializer.serialize(AugmentableDispatchSerializer.java:51)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SNAPSHOT] at org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node$StreamWriter.serialize(DataObjectSerializerPrototype.java)[57:org.opendaylight.mdsal.model.ietf-topology:2013.10.21.9_2-SNAPSHOT] at org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology$StreamWriter.serialize(DataObjectSerializerPrototype.java)[57:org.opendaylight.mdsal.model.ietf-topology:2013.10.21.9_2-SNAPSHOT] at org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology$StreamWriter.serialize(DataObjectSerializerPrototype.java)[57:org.opendaylight.mdsal.model.ietf-topology:2013.10.21.9_2-SNAPSHOT] at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry$DataObjectSerializerProxy.serialize(BindingNormalizedNodeCodecRegistry.java:295)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SNAPSHOT] at org.opendaylight.federation.service.api.message.BindingAwareJsonConverter.jsonStringFromDataObject(BindingAwareJsonConverter.java:147)[359:org.opendaylight.federation.service-api:0.1.2.SNAPSHOT]</pre> </div></div> <p>Operating System: All<br/> Platform: All</p> MDSAL-213

Serializing DataObject to JSON causes frozen class exception

Bug Resolved Cannot Reproduce Michael Vorburger Guy Sela Wed, 21 Dec 2016 12:45:21 +0000 Tue, 12 Jun 2018 12:59:05 +0000 Wed, 6 Jun 2018 11:28:50 +0000 Binding runtime 0 5 <p>Attachment karafagain.zip has been added with description: karaf log</p> <p>Attachment karaf.zip has been added with description: Another karaf log experiencing this issue</p> <p>It seems like an issue in binding generator, hence moved to md-sal.</p> <p>Hi</p> <p>what do you mean by "close to stable" ?<br/> Since issue occurs only sometimes, I decrease severity to MAJOR.</p> <p>It looks like we came across this issue in the past</p> <p><a href="https://lists.opendaylight.org/pipermail/sfc-dev/2015-November/002344.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/sfc-dev/2015-November/002344.html</a><br/> <a href="https://git.opendaylight.org/gerrit/#/c/18820" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/18820</a></p> <p>investigating ...</p> <p>Synced from stable boron at 15/12/2016.<br/> I think it is critical because when it happens, the workflow completely fails.<br/> The fact that it is a race bug, which is not deterministic, is a stronger reason to fix that, because any mechanism that will try to workaround it until fixed, can't be actually verified.<br/> Is there a theoretical workaround?<br/> Maybe retrying the serialization can help ? Add a sleep before ? I really don't know what is the possible root cause, so it would really help to get some directions from you guys.</p> <p>Now I got the same exception while calling tx.merge in a write transaction. It made me think that maybe the concurrent operations i'm doing are:</p> <ul> <li>Serializing a DataObject to JSON</li> <li>Merging a write transaction with the same type of DataObject into MD-SAL.</li> </ul> <p>Can this explain the failure? <br/> If it does, will it happen only in the first time serializing the object?</p> <p>I'm attaching the log of the new failure</p> <p>Attachment karaf.log.gz has been added with description: Karaf log that fails in merge tx</p> <p>So as a workaround I added a Thread.sleep of 500 millis and retried the operation, and it actually fixed the problem.</p> <p>Okay so apparently I can't apply this workaround, because the frozen class issue can stem from places that are not in my control:<br/> com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.ela<br/> n.rev150602.elan.interfaces.ElanInterface$StreamWriter: frozen class (cannot edit)<br/> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)<span class="error">&#91;38:com.google.guava:18.0.0&#93;</span><br/> at com.google.common.cache.LocalCache.get(LocalCache.java:3937)<span class="error">&#91;38:com.google.guava:18.0.0&#93;</span><br/> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)<span class="error">&#91;38:com.google.guava:18.0.0&#93;</span><br/> at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)<span class="error">&#91;38:com.google.guava:18.0.0&#93;</span><br/> at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)<span class="error">&#91;38:com.google.guava:18.0.0&#93;</span><br/> at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry.getSerializer(BindingNormalizedNodeCodecRegistry.java:72)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SNAPSH<br/> OT]<br/> at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry.toNormalizedNode(BindingNormalizedNodeCodecRegistry.java:106)[92:org.opendaylight.mdsal.binding-dom-codec:0.9.2.SN<br/> APSHOT]<br/> at org.opendaylight.controller.md.sal.binding.impl.BindingToNormalizedNodeCodec.toNormalizedNode(BindingToNormalizedNodeCodec.java:132)[146:org.opendaylight.controller.sal-binding-broker-impl:1.4.2.SNAPS<br/> HOT]<br/> at org.opendaylight.controller.md.sal.binding.impl.AbstractWriteTransaction.put(AbstractWriteTransaction.java:39)<span class="error">&#91;146:org.opendaylight.controller.sal-binding-broker-impl:1.4.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.genius.mdsalutil.MDSALUtil.syncWrite(MDSALUtil.java:521)<span class="error">&#91;281:org.opendaylight.genius.mdsalutil-api:0.1.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanServiceProvider.addElanInterface(ElanServiceProvider.java:271)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanServiceProvider.createExternalElanNetwork(ElanServiceProvider.java:489)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanServiceProvider.lambda$createExternalElanNetworks$0(ElanServiceProvider.java:469)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanServiceProvider.handleExternalElanNetworks(ElanServiceProvider.java:702)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanServiceProvider.createExternalElanNetworks(ElanServiceProvider.java:468)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanOvsdbNodeListener.add(ElanOvsdbNodeListener.java:97)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.netvirt.elan.internal.ElanOvsdbNodeListener.add(ElanOvsdbNodeListener.java:30)<span class="error">&#91;325:org.opendaylight.netvirt.elanmanager-impl:0.3.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.genius.mdsalutil.AbstractDataChangeListener.createData(AbstractDataChangeListener.java:68)<span class="error">&#91;281:org.opendaylight.genius.mdsalutil-api:0.1.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.genius.mdsalutil.AbstractDataChangeListener.onDataChanged(AbstractDataChangeListener.java:54)<span class="error">&#91;281:org.opendaylight.genius.mdsalutil-api:0.1.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker$TranslatingDataChangeInvoker.onDataChanged(AbstractForwardedDataBroker.java:143)<span class="error">&#91;146:org.opendaylight.controller.sal-binding-broker-impl:1.4.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.controller.cluster.datastore.DataChangeListener.dataChanged(DataChangeListener.java:71)<span class="error">&#91;176:org.opendaylight.controller.sal-distributed-datastore:1.4.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.controller.cluster.datastore.DataChangeListener.handleReceive(DataChangeListener.java:43)<span class="error">&#91;176:org.opendaylight.controller.sal-distributed-datastore:1.4.2.SNAPSHOT&#93;</span><br/> at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:26)<span class="error">&#91;170:org.opendaylight.controller.sal-clustering-commons:1.4.2.SNAPSHOT&#93;</span><br/> at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.actor.Actor$class.aroundReceive(Actor.scala:484)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.actor.ActorCell.invoke(ActorCell.scala:495)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.dispatch.Mailbox.run(Mailbox.scala:224)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)<span class="error">&#91;158:com.typesafe.akka.actor:2.4.7&#93;</span><br/> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)<span class="error">&#91;154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8&#93;</span><br/> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)<span class="error">&#91;154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8&#93;</span><br/> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)<span class="error">&#91;154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8&#93;</span><br/> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)<span class="error">&#91;154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8&#93;</span><br/> Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.elan.interfaces.ElanInterface$StreamWriter: frozen class (cannot edit)</p> <p>I suspect we need to move away from javassist to get rid of this issue. I suspect this happens because the class loaders are not completely cleaned up and we end up with untracked codec classes.</p> <p>We've hit this one again, and apparently we (not myself but <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=aswins" class="user-hover" rel="aswins">aswins</a> <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=JankiChhatbar" class="user-hover" rel="JankiChhatbar">JankiChhatbar</a> <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=smalleni" class="user-hover" rel="smalleni">smalleni</a> ?) can now fairly reliably reproduce this...</p> <p>MDSAL maintainers <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=rovarga" class="user-hover" rel="rovarga">rovarga</a> <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=tpantelis" class="user-hover" rel="tpantelis">tpantelis</a> <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=martin.ciglan" class="user-hover" rel="martin.ciglan">martin.ciglan</a> can you help with this? </p> <p>Otherwise I guess I'll have to dig into this code to see if I can make any sense of this... <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>FTR: As was discussed in <a href="https://jira.opendaylight.org/browse/MDSAL-230" title="Caused by: java.lang.RuntimeException: org.opendaylight.yang.gen...$StreamWriter: frozen class (cannot edit)" class="issue-link" data-issue-key="MDSAL-230"><del>MDSAL-230</del></a>, following <a href="https://jira.opendaylight.org/browse/MDSAL-200" title="Cleanup package names to remove &#39;yangtools&#39;" class="issue-link" data-issue-key="MDSAL-200"><del>MDSAL-200</del></a> the stack trace is now .mdsal. instead of .yangtools. but this is the very same issue, here's a current stack trace, from our <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1573273" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.redhat.com/show_bug.cgi?id=1573273</a> :</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2018-04-30T15:08:33,051 | ERROR | org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode_AsyncClusteredDataTreeChangeListenerBase-DataTreeChangeHandler-0 | AsyncClusteredDataTreeChangeListenerBase | 263 - org.opendaylight.genius.mdsalutil-api - 0.4.0.redhat-8 | Thread terminated due to uncaught exception: org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode_AsyncClusteredDataTreeChangeListenerBase-DataTreeChangeHandler-0 com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.model.match.types.rev131026.match.Metadata$StreamWriter: frozen class (cannot edit) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218) [32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) [32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) [32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) [32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) [32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry.getSerializer(BindingNormalizedNodeCodecRegistry.java:72) [290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry.toNormalizedNode(BindingNormalizedNodeCodecRegistry.java:107) [290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.controller.md.sal.binding.impl.BindingToNormalizedNodeCodec.toNormalizedNode(BindingToNormalizedNodeCodec.java:134) [220:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-8] at org.opendaylight.controller.md.sal.binding.impl.AbstractWriteTransaction.put(AbstractWriteTransaction.java:39) [220:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-8] at org.opendaylight.genius.mdsalutil.internal.MDSALManager.writeFlowEntityInternal(MDSALManager.java:183) [264:org.opendaylight.genius.mdsalutil-impl:0.4.0.redhat-8] at org.opendaylight.genius.mdsalutil.internal.MDSALManager.installFlowInternal(MDSALManager.java:142) [264:org.opendaylight.genius.mdsalutil-impl:0.4.0.redhat-8] at org.opendaylight.genius.mdsalutil.internal.MDSALManager.installFlow(MDSALManager.java:609) [264:org.opendaylight.genius.mdsalutil-impl:0.4.0.redhat-8] at Proxydde4f757_0177_4930_af4f_a514886e67a6.installFlow(Unknown Source) [?:?] at Proxy12933f77_31f6_4e8b_b7ca_4582b0b424a9.installFlow(Unknown Source) [?:?] at org.opendaylight.netvirt.ipv6service.Ipv6NodeListener.createTableMissEntry(Ipv6NodeListener.java:91) [356:org.opendaylight.netvirt.ipv6service-impl:0.6.0.redhat-8] at org.opendaylight.netvirt.ipv6service.Ipv6NodeListener.add(Ipv6NodeListener.java:74) [356:org.opendaylight.netvirt.ipv6service-impl:0.6.0.redhat-8] at org.opendaylight.netvirt.ipv6service.Ipv6NodeListener.add(Ipv6NodeListener.java:36) [356:org.opendaylight.netvirt.ipv6service-impl:0.6.0.redhat-8] at org.opendaylight.genius.datastoreutils.AsyncClusteredDataTreeChangeListenerBase$DataTreeChangeHandler.run(AsyncClusteredDataTreeChangeListenerBase.java:155) [263:org.opendaylight.genius.mdsalutil-api:0.4.0.redhat-8] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?] at java.lang.Thread.run(Thread.java:748) [?:?] Caused by: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.model.match.types.rev131026.match.Metadata$StreamWriter: frozen class (cannot edit) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.getSerializer(AbstractStreamWriterGenerator.java:87) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:283) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:280) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] ... 20 more Caused by: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.model.match.types.rev131026.match.Metadata$StreamWriter: frozen class (cannot edit) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.loadSerializerFor(AbstractStreamWriterGenerator.java:97) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataObjectSerializerSource.staticInvokeEmitter(DataObjectSerializerSource.java:144) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChildInner(DataNodeContainerSerializerSource.java:184) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChild(DataNodeContainerSerializerSource.java:158) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitBody(DataNodeContainerSerializerSource.java:148) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.getSerializerBody(DataNodeContainerSerializerSource.java:83) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.lambda$generateEmitter0$0(AbstractStreamWriterGenerator.java:205) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.yangtools.util.ClassLoaderUtils.withClassLoader(ClassLoaderUtils.java:51) ~[425:org.opendaylight.yangtools.util:2.0.1.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:204) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.getSerializer(AbstractStreamWriterGenerator.java:87) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:283) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:280) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] ... 20 more Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.model.match.types.rev131026.match.Metadata$StreamWriter: frozen class (cannot edit) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.loadSerializerFor(AbstractStreamWriterGenerator.java:97) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataObjectSerializerSource.staticInvokeEmitter(DataObjectSerializerSource.java:144) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChildInner(DataNodeContainerSerializerSource.java:184) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChild(DataNodeContainerSerializerSource.java:158) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitBody(DataNodeContainerSerializerSource.java:148) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.getSerializerBody(DataNodeContainerSerializerSource.java:83) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.lambda$generateEmitter0$0(AbstractStreamWriterGenerator.java:205) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.yangtools.util.ClassLoaderUtils.withClassLoader(ClassLoaderUtils.java:51) ~[425:org.opendaylight.yangtools.util:2.0.1.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:204) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.loadSerializerFor(AbstractStreamWriterGenerator.java:97) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataObjectSerializerSource.staticInvokeEmitter(DataObjectSerializerSource.java:144) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChildInner(DataNodeContainerSerializerSource.java:184) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChild(DataNodeContainerSerializerSource.java:158) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitBody(DataNodeContainerSerializerSource.java:148) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.getSerializerBody(DataNodeContainerSerializerSource.java:83) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.lambda$generateEmitter0$0(AbstractStreamWriterGenerator.java:205) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.yangtools.util.ClassLoaderUtils.withClassLoader(ClassLoaderUtils.java:51) ~[425:org.opendaylight.yangtools.util:2.0.1.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:204) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.getSerializer(AbstractStreamWriterGenerator.java:87) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:283) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:280) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] ... 20 more Caused by: java.lang.RuntimeException: org.opendaylight.yang.gen.v1.urn.opendaylight.model.match.types.rev131026.match.Metadata$StreamWriter: frozen class (cannot edit) at javassist.ClassPool.checkNotFrozen(ClassPool.java:617) ~[66:javassist:3.21.0.GA] at javassist.CtClassType.setName(CtClassType.java:353) ~[66:javassist:3.21.0.GA] at javassist.ClassPool.getAndRename(ClassPool.java:408) ~[66:javassist:3.21.0.GA] at org.opendaylight.mdsal.binding.generator.util.JavassistUtils.instantiatePrototype(JavassistUtils.java:127) ~[293:org.opendaylight.mdsal.binding-generator-impl:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:208) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.loadSerializerFor(AbstractStreamWriterGenerator.java:97) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataObjectSerializerSource.staticInvokeEmitter(DataObjectSerializerSource.java:144) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChildInner(DataNodeContainerSerializerSource.java:184) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChild(DataNodeContainerSerializerSource.java:158) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitBody(DataNodeContainerSerializerSource.java:148) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.getSerializerBody(DataNodeContainerSerializerSource.java:83) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.lambda$generateEmitter0$0(AbstractStreamWriterGenerator.java:205) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.yangtools.util.ClassLoaderUtils.withClassLoader(ClassLoaderUtils.java:51) ~[425:org.opendaylight.yangtools.util:2.0.1.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:204) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.loadSerializerFor(AbstractStreamWriterGenerator.java:97) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataObjectSerializerSource.staticInvokeEmitter(DataObjectSerializerSource.java:144) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChildInner(DataNodeContainerSerializerSource.java:184) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitChild(DataNodeContainerSerializerSource.java:158) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.emitBody(DataNodeContainerSerializerSource.java:148) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.DataNodeContainerSerializerSource.getSerializerBody(DataNodeContainerSerializerSource.java:83) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.lambda$generateEmitter0$0(AbstractStreamWriterGenerator.java:205) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.yangtools.util.ClassLoaderUtils.withClassLoader(ClassLoaderUtils.java:51) ~[425:org.opendaylight.yangtools.util:2.0.1.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.generateEmitter0(AbstractStreamWriterGenerator.java:204) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.access$300(AbstractStreamWriterGenerator.java:45) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.generateSerializer(AbstractStreamWriterGenerator.java:138) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:124) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator$SerializerImplementationLoader.load(AbstractStreamWriterGenerator.java:100) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.get(LocalCache.java:4147) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5146) ~[32:com.google.guava:23.3.0.jre] at org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator.getSerializer(AbstractStreamWriterGenerator.java:87) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:283) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at org.opendaylight.mdsal.binding.dom.codec.impl.BindingNormalizedNodeCodecRegistry$GeneratorLoader.load(BindingNormalizedNodeCodecRegistry.java:280) ~[290:org.opendaylight.mdsal.binding-dom-codec:0.12.0.redhat-8] at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299) ~[32:com.google.guava:23.3.0.jre] at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212) ~[32:com.google.guava:23.3.0.jre] ... 20 more</pre> </div></div> <p>&gt; Otherwise I guess I'll have to dig into this code to see if I can make any sense of this...</p> <p>This having been a purely rhetorical <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/wink.png" height="16" width="16" align="absmiddle" alt="" border="0"/> Q, I now at least understand what can cause these Princess Elsa of Arendelle (frozen) situations:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> @Test <span class="code-keyword">public</span> void testMDSAL_213() <span class="code-keyword">throws</span> NotFoundException, CannotCompileException { JavassistUtils javassistUtils = JavassistUtils.forClassPool(ClassPool.getDefault()); <span class="code-object">Class</span>&lt;<span class="code-object">String</span>&gt; type = <span class="code-object">String</span>.class; CtClass poolClass = javassistUtils.instantiatePrototype(type.getName(), <span class="code-quote">"Something$StreamWriter"</span>, cls -&gt; { }); @SuppressWarnings({ <span class="code-quote">"unchecked"</span>, <span class="code-quote">"unused"</span> }) <span class="code-object">Class</span>&lt;? <span class="code-keyword">extends</span> DataObjectSerializerImplementation&gt; javaClass = poolClass.toClass(type.getClassLoader(), type.getProtectionDomain()); javassistUtils.instantiatePrototype(type.getName(), <span class="code-quote">"Something$StreamWriter"</span>, cls -&gt; { }); }</pre> </div></div> <p>Above is inspired from the real code in <tt>org.opendaylight.mdsal.binding.dom.codec.gen.impl.AbstractStreamWriterGenerator</tt> but simplified a bit (and we use <tt>String</tt> as the <tt>type</tt> instead of the real <tt>DataObjectSerializerPrototype</tt> just because that is not available in <tt>mdsal-binding-dom-codec</tt>). If we copy/paste that e.g. into <tt>org.opendaylight.mdsal.binding.generator.util.JavassistUtilsTest</tt>, then the <b>2nd</b> instantiatePrototype() indeed causes:</p> <p><tt>java.lang.RuntimeException: Something$StreamWriter: frozen class (cannot edit)</tt></p> <p>But if, and only if, the poolClass.toClass() already happened! So this smells of some concurrency problem - we may be creating the $StreamWriter in parallel? The fact that this problem is only seen in cluster load tests, but not eay gone standard CSITs supports that hypothesis. The trouble is... the way I currently understand the code in <tt>AbstractStreamWriterGenerator</tt> it SHOULD be concurrency/thread safe? But clearly it's not, as we are hitting this problem (and can reproduce it) - so what am I missing? If not found, we could perhaps also build a dumb band aid fix... the way I understand things, that 2nd new $StreamWriter should be the same as the original one? So... we could cheat, and brush the frozen yogurt RuntimeException under the carpet, IFF we meanwhile have the class in the cache? I'll propose something like that - for discussion. I'll also add some additional logging to see if perhaps that could help to make more sense of what is really going on here.</p> <p>I think this boils down to classloader vs. classpool vs. cache lifecycle. I do not believe we can just sweep it under the carpet – for that you need to be sure you are sweeping, not smearing ...</p> <p>&gt; I think this boils down to classloader vs. classpool vs. cache lifecycle. </p> <p>right... so the ClassPool (currently) ultimately used in AbstractStreamWriterGenerator is fixed, unless I'm completely mistaken, it's the  private static final JavassistUtils JAVASSIST = JavassistUtils.forClassPool(ClassPool.getDefault()); from the BindingToNormalizedNodeCodecFactory?</p> <p>the classloader vs. cache lifecycle is a little unclear to me so far, but one of the changes I've raised today under this JIRA (see above) adds more (debug) logging; if I can get that to run on where we can apparently reproduce this fairly reliably, perhaps we'll learn more.</p> <p>Just for the record, this looks funky, but I don't thinks is our problem here, because we do not evict, at least not programmatically (but GC?):</p> <p>* <a href="https://softwaremill.com/race-condition-cache-guava-caffeine/" class="external-link" target="_blank" rel="nofollow noopener">https://softwaremill.com/race-condition-cache-guava-caffeine/</a></p> <ul> <li><a href="https://github.com/google/guava/issues/1881" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/google/guava/issues/1881</a></li> </ul> <p>but we DO have weak Class keys - one of my Gerrits today proposes to remove them.</p> <p>While we are using weak keys, it is on Class objects – which means that a GC-induced invalidation of the key simply cannot happen while we are using it for lookup.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=jluhrsen" class="user-hover" rel="jluhrsen">jluhrsen</a> meanwhile has also hit this in <a href="https://jira.opendaylight.org/browse/NETVIRT-1089" title="Add trace:transactions to suite teardowns" class="issue-link" data-issue-key="NETVIRT-1089">NETVIRT-1089</a>, and will try to provide us the new debug logs I recently added.</p> <p>I've just stumbled upon <a href="https://jira.opendaylight.org/browse/CONTROLLER-1832" title="Transaction Trace tool wiring creates second BindingToNormalizedNodeCodec" class="issue-link" data-issue-key="CONTROLLER-1832"><del>CONTROLLER-1832</del></a>, which could be related / causing this... TBC if <em>THE</em> or <em>A</em> cause.</p> <p><a href="https://jira.opendaylight.org/browse/CONTROLLER-1834" title="Transaction Trace tool wiring for ping-pong DataBroker is wrong" class="issue-link" data-issue-key="CONTROLLER-1834"><del>CONTROLLER-1834</del></a> has something else which seems vaguely related - not claiming / no proof (yet) it's causing this, but at least worth noting.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=jluhrsen" class="user-hover" rel="jluhrsen">jluhrsen</a>'s  <a href="https://jira.opendaylight.org/browse/NETVIRT-1089" title="Add trace:transactions to suite teardowns" class="issue-link" data-issue-key="NETVIRT-1089">NETVIRT-1089</a> is no longer able to reproduce this after <del><a href="https://jira.opendaylight.org/browse/CONTROLLER-1832" title="Transaction Trace tool wiring creates second BindingToNormalizedNodeCodec" class="issue-link" data-issue-key="CONTROLLER-1832"><del>CONTROLLER-1832</del></a></del>.</p> <p>FTR: The additional DEBUG logigng for this (c/71890) went into Oxygen SR2 (not only SR3; which <a href="https://jira.opendaylight.org/browse/CONTROLLER-1834" title="Transaction Trace tool wiring for ping-pong DataBroker is wrong" class="issue-link" data-issue-key="CONTROLLER-1834"><del>CONTROLLER-1834</del></a> and <a href="https://jira.opendaylight.org/browse/CONTROLLER-1832" title="Transaction Trace tool wiring creates second BindingToNormalizedNodeCodec" class="issue-link" data-issue-key="CONTROLLER-1832"><del>CONTROLLER-1832</del></a> will).</p> Blocks NETVIRT-1089 CONTROLLER-1834 Duplicate MDSAL-230 Relates CONTROLLER-1832 Development External issue ID 7420 External issue URL Rank 0|i02x0f: