OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[OPNFLWPLUG-799] Disabling statistic poll breaks plugin https://jira.opendaylight.org/browse/OPNFLWPLUG-799 OpenFlowPlugin <p>Disabling statistics poll:</p> <p>PUT <a href="http://localhost:8181/restconf/config/openflow-provider-config:openflow-provider-config/" class="external-link" target="_blank" rel="nofollow noopener">http://localhost:8181/restconf/config/openflow-provider-config:openflow-provider-config/</a><br/> {<br/> "openflow-provider-config": </p> { "is-statistics-polling-off": false } <p>}</p> <p>Breaks the plugin as after doing this:</p> <ul class="alternate" type="square"> <li>Empty topology when switch connects</li> <li>Empty inventory when switch connects</li> <li>Flow push does not work</li> </ul> <p>BR/Luis</p> <p>Operating System: All<br/> Platform: All</p> OPNFLWPLUG-799

Disabling statistic poll breaks plugin

Bug Resolved Cannot Reproduce Jozef Bacigal Luis Gomez Fri, 14 Oct 2016 15:58:57 +0000 Mon, 27 Sep 2021 09:01:56 +0000 Thu, 27 Oct 2016 15:24:48 +0000 General 0 2 <p>probably restarting the openflow bundles are causing these. We are moving the config parameters to a .cfg file in carbon, will c/p the changes here too. need to check if the error exists post that,</p> <p>Can I see somehow logs ?</p> <p>(In reply to Jozef Bacigal from comment #2)<br/> > Can I see somehow logs ?</p> <p>This is the error in the karaf console:</p> <p>opendaylight-user@root>feature:install odl-restconf-all<br/> opendaylight-user@root>feature:install odl-mdsal-all<br/> opendaylight-user@root>feature:install odl-openflowplugin-southbound<br/> opendaylight-user@root>feature:install odl-openflowplugin-flow-services<br/> opendaylight-user@root>Exception in thread "Thread-132" io.netty.channel.unix.Errors$NativeIoException: bind() failed: Address already in use<br/> at io.netty.channel.unix.Errors.newIOException(Errors.java:115)<br/> at io.netty.channel.unix.Socket.bind(Socket.java:204)<br/> at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)<br/> at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:490)<br/> at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1198)<br/> at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:481)<br/> at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:466)<br/> at io.netty.channel.ChannelDuplexHandler.bind(ChannelDuplexHandler.java:38)<br/> at io.netty.handler.logging.LoggingHandler.bind(LoggingHandler.java:197)<br/> at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:481)<br/> at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:466)<br/> at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:944)<br/> at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:203)<br/> at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:350)<br/> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)<br/> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)<br/> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)<br/> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)<br/> at java.lang.Thread.run(Thread.java:745)<br/> Exception in thread "Thread-131" io.netty.channel.unix.Errors$NativeIoException: bind() failed: Address already in use<br/> at io.netty.channel.unix.Errors.newIOException(Errors.java:115)<br/> at io.netty.channel.unix.Socket.bind(Socket.java:204)<br/> at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:91)<br/> at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:490)<br/> at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1198)<br/> at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:481)<br/> at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:466)<br/> at io.netty.channel.ChannelDuplexHandler.bind(ChannelDuplexHandler.java:38)<br/> at io.netty.handler.logging.LoggingHandler.bind(LoggingHandler.java:197)<br/> at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:481)<br/> at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:466)<br/> at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:944)<br/> at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:203)<br/> at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:350)<br/> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)<br/> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)<br/> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)<br/> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)<br/> at java.lang.Thread.run(Thread.java:745)</p> <p>====================================================================================<br/> And this is the corresponding tailed logs:</p> <p>2016-10-17 08:06:12,612 | INFO | erRestartService | printContainerRestartServiceImpl | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Restarting blueprint containers for bundle org.opendaylight.openflowplugin.blueprint-config_0.3.1.SNAPSHOT <span class="error">[289]</span> and its dependent bundles [org.opendaylight.openflowplugin.impl_0.3.1.SNAPSHOT <span class="error">[286]</span>, org.opendaylight.openflowjava.openflow-protocol-impl_0.8.1.SNAPSHOT <span class="error">[276]</span>]<br/> 2016-10-17 08:06:12,625 | INFO | erRestartService | BlueprintExtender | 15 - org.apache.aries.blueprint.core - 1.6.1 | Destroying BlueprintContainer for bundle org.opendaylight.openflowjava.openflow-protocol-impl/0.8.1.SNAPSHOT<br/> 2016-10-17 08:06:12,675 | INFO | erRestartService | BlueprintExtender | 15 - org.apache.aries.blueprint.core - 1.6.1 | Destroying BlueprintContainer for bundle org.opendaylight.openflowplugin.impl/0.3.1.SNAPSHOT<br/> 2016-10-17 08:06:12,707 | INFO | erRestartService | BlueprintExtender | 15 - org.apache.aries.blueprint.core - 1.6.1 | Destroying BlueprintContainer for bundle org.opendaylight.openflowplugin.blueprint-config/0.3.1.SNAPSHOT<br/> 2016-10-17 08:06:12,754 | INFO | erRestartService | printContainerRestartServiceImpl | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Restarting blueprint container for bundle org.opendaylight.openflowplugin.blueprint-config_0.3.1.SNAPSHOT <span class="error">[289]</span> with paths <span class="error">[bundleentry://289.fwk195984832/org/opendaylight/blueprint/openflowplugin.xml]</span><br/> 2016-10-17 08:06:12,817 | INFO | erRestartService | BlueprintContainerImpl | 15 - org.apache.aries.blueprint.core - 1.6.1 | Bundle org.opendaylight.openflowplugin.blueprint-config/0.3.1.SNAPSHOT is waiting for dependencies <span class="error">[(objectClass=org.opendaylight.openflowjava.protocol.spi.connection.SwitchConnectionProviderFactory), (objectClass=org.opendaylight.openflowplugin.api.openflow.OpenFlowPluginProviderFactory)]</span><br/> 2016-10-17 08:06:12,818 | INFO | rint Extender: 3 | BlueprintContainerImpl | 15 - org.apache.aries.blueprint.core - 1.6.1 | Bundle org.opendaylight.openflowplugin.blueprint-config/0.3.1.SNAPSHOT is waiting for dependencies <span class="error">[(objectClass=org.opendaylight.openflowjava.protocol.spi.connection.SwitchConnectionProviderFactory), (objectClass=org.opendaylight.openflowplugin.api.openflow.OpenFlowPluginProviderFactory)]</span><br/> 2016-10-17 08:06:12,819 | INFO | rint Extender: 3 | BlueprintContainerImpl | 15 - org.apache.aries.blueprint.core - 1.6.1 | Bundle org.opendaylight.openflowplugin.blueprint-config/0.3.1.SNAPSHOT is waiting for dependencies <span class="error">[(objectClass=org.opendaylight.openflowjava.protocol.spi.connection.SwitchConnectionProviderFactory), (objectClass=org.opendaylight.openflowplugin.api.openflow.OpenFlowPluginProviderFactory)]</span><br/> 2016-10-17 08:06:12,820 | INFO | erRestartService | printContainerRestartServiceImpl | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Restarting blueprint container for bundle org.opendaylight.openflowplugin.impl_0.3.1.SNAPSHOT <span class="error">[286]</span> with paths <span class="error">[bundleentry://286.fwk195984832/org/opendaylight/blueprint/commands.xml, bundleentry://286.fwk195984832/org/opendaylight/blueprint/openflowplugin-impl.xml]</span><br/> 2016-10-17 08:06:12,842 | INFO | rint Extender: 1 | BlueprintContainerImpl | 15 - org.apache.aries.blueprint.core - 1.6.1 | Bundle org.opendaylight.openflowplugin.blueprint-config/0.3.1.SNAPSHOT is waiting for dependencies <span class="error">[(objectClass=org.opendaylight.openflowjava.protocol.spi.connection.SwitchConnectionProviderFactory)]</span><br/> 2016-10-17 08:06:12,847 | INFO | ntAdminThread #3 | BlueprintBundleTracker | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Blueprint container for bundle org.opendaylight.openflowplugin.impl_0.3.1.SNAPSHOT <span class="error">[286]</span> was successfully created<br/> 2016-10-17 08:06:12,847 | INFO | erRestartService | printContainerRestartServiceImpl | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Restarting blueprint container for bundle org.opendaylight.openflowjava.openflow-protocol-impl_0.8.1.SNAPSHOT <span class="error">[276]</span> with paths <span class="error">[bundleentry://276.fwk195984832/org/opendaylight/blueprint/openflow-protocol-impl.xml]</span><br/> 2016-10-17 08:06:12,885 | INFO | ntAdminThread #4 | BlueprintBundleTracker | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Blueprint container for bundle org.opendaylight.openflowjava.openflow-protocol-impl_0.8.1.SNAPSHOT <span class="error">[276]</span> was successfully created<br/> 2016-10-17 08:06:12,943 | INFO | rint Extender: 1 | penFlowPluginProviderFactoryImpl | 286 - org.opendaylight.openflowplugin.impl - 0.3.1.SNAPSHOT | Initializing new OFP southbound.<br/> 2016-10-17 08:06:12,960 | WARN | rint Extender: 1 | OpenFlowPluginProviderImpl | 286 - org.opendaylight.openflowplugin.impl - 0.3.1.SNAPSHOT | Error registering MBean {}<br/> javax.management.InstanceAlreadyExistsException: MXBean already registered with name org.opendaylight.openflowplugin.impl.statistics.ofpspecific:type=MessageIntelligenceAgencyMXBean<br/> at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:151)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:160)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.mbeanserver.MBeanSupport.preRegister2(MBeanSupport.java:173)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:930)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)<span class="error">[:1.8.0_73]</span><br/> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)<span class="error">[:1.8.0_73]</span><br/> at org.opendaylight.openflowplugin.impl.OpenFlowPluginProviderImpl.registerMXBean(OpenFlowPluginProviderImpl.java:288)<br/> at org.opendaylight.openflowplugin.impl.OpenFlowPluginProviderImpl.initialize(OpenFlowPluginProviderImpl.java:227)<br/> at org.opendaylight.openflowplugin.impl.OpenFlowPluginProviderFactoryImpl.newInstance(OpenFlowPluginProviderFactoryImpl.java:68)<br/> at Proxy3fb871b3_ca45_481d_9092_7545ffec048e.newInstance(Unknown Source)<br/> at Proxy9bf2f9d3_0c0a_447e_8dad_4c59ebd43d38.newInstance(Unknown Source)<br/> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<span class="error">[:1.8.0_73]</span><br/> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)<span class="error">[:1.8.0_73]</span><br/> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<span class="error">[:1.8.0_73]</span><br/> at java.lang.reflect.Method.invoke(Method.java:497)<span class="error">[:1.8.0_73]</span><br/> at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:299)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BeanRecipe.invoke(BeanRecipe.java:980)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BeanRecipe.getInstanceFromFactory(BeanRecipe.java:295)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:278)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:830)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:811)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<span class="error">[:1.8.0_73]</span><br/> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.di.RefRecipe.internalCreate(RefRecipe.java:62)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:106)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.ServiceRecipe.createService(ServiceRecipe.java:285)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.ServiceRecipe.internalGetService(ServiceRecipe.java:252)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.ServiceRecipe.internalCreate(ServiceRecipe.java:149)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<span class="error">[:1.8.0_73]</span><br/> at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:255)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:186)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:724)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:411)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:276)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<span class="error">[:1.8.0_73]</span><br/> at org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(ExecutorServiceWrapper.java:106)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:48)<span class="error">[15:org.apache.aries.blueprint.core:1.6.1]</span><br/> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)<span class="error">[:1.8.0_73]</span><br/> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)<span class="error">[:1.8.0_73]</span><br/> at java.lang.Thread.run(Thread.java:745)<span class="error">[:1.8.0_73]</span><br/> 2016-10-17 08:06:12,990 | INFO | rint Extender: 1 | penFlowPluginProviderFactoryImpl | 286 - org.opendaylight.openflowplugin.impl - 0.3.1.SNAPSHOT | Configured values, StatisticsPollingOff:true, SwitchFeaturesMandatory:false, BarrierCountLimit:25600, BarrierTimeoutLimit:500, EchoReplyTimeout:2000, ThreadPoolMinThreads:1, ThreadPoolMaxThreads:32000, ThreadPoolTimeout:60, NotificationFlowRemovedOff:false<br/> 2016-10-17 08:06:13,041 | INFO | ntAdminThread #6 | BlueprintBundleTracker | 174 - org.opendaylight.controller.blueprint - 0.5.1.SNAPSHOT | Blueprint container for bundle org.opendaylight.openflowplugin.blueprint-config_0.3.1.SNAPSHOT <span class="error">[289]</span> was successfully created</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/47004/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/47004/</a></p> <p>will provide the necessary relief.</p> <p>If you disable statistics you should not see anything in operational inventory. But if statistics disabled you can alway make one time statistics gathering of all or just part you want update with RPC</p> <p>opendaylight-direct-statistics</p> <p>But anyway this is why Shuva did create all setting in cfg file to be able do changes no the fly.</p> <p>(In reply to Jozef Bacigal from comment #5)<br/> > If you disable statistics you should not see anything in operational<br/> > inventory. But if statistics disabled you can alway make one time statistics<br/> > gathering of all or just part you want update with RPC<br/> > <br/> > opendaylight-direct-statistics<br/> > <br/> > But anyway this is why Shuva did create all setting in cfg file to be able<br/> > do changes no the fly.</p> <p>But with statistics turned off, will not itemLifecycleListener be non-null thereby call itemLifecycleListener.onAdded() on rpc success?</p> <p>even with changing on the fly, although no stats goes on the wire, i am not able to push any flows, i donot see flow_mods going on the wire</p> <p>(In reply to Shuva Jyoti Kar from comment #6)<br/> > (In reply to Jozef Bacigal from comment #5)<br/> > > If you disable statistics you should not see anything in operational<br/> > > inventory. But if statistics disabled you can alway make one time statistics<br/> > > gathering of all or just part you want update with RPC<br/> > > <br/> > > opendaylight-direct-statistics<br/> > > <br/> > > But anyway this is why Shuva did create all setting in cfg file to be able<br/> > > do changes no the fly.<br/> > <br/> > <br/> > But with statistics turned off, will not itemLifecycleListener be non-null<br/> > thereby call itemLifecycleListener.onAdded() on rpc success?</p> <p>i do understand this part now, as LifeCycleSource salFlowService is currently not registered in MdSalRegistrationUtils,so itemLifecycleListener is never set.</p> <p>If LifeCycleSource is registered, then special RPC "change-statistics-work-mode" need to be called yet. Thereafter flows are written to the operational DS.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/47414/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/47414/</a></p> <p>This patch, already merged, should help to use direct statistics RPC.</p> <p>I'll close this bug because if used with changing cfg file it is working fine. With the yang configuration is not properly implemented in plugin. Created a new milestone to improve this through configuration DS and change the statistics gathering behavior changes.</p> <p><a href="https://jira.opendaylight.org/browse/OPNFLWPLUG-804" title="Improvement in OpenFlow plugin Statistics collection." class="issue-link" data-issue-key="OPNFLWPLUG-804">OPNFLWPLUG-804</a></p> Development External issue ID 6941 External issue URL Rank 0|i033dr: