Operating System: All
Platform: All

Additional analysis shows that the problem is interaction between blueprint's (and by extension end-user's) expectation versus how RPC routing works.

Current BP odl:rpc-service and odl:rpc-implementation can only work as envisioned for binding FooService completely composed of global RPCs. It acts as a dependency activation gate before the component activates.

Routed RPCs ('actions' to use RFC7950 term) work in a rather different way, as they come and go as their contexts change. In this case BP wiring really wants to express a dependency on 'someone has promised to instantiate actions'.

That contract is currently 'provided' by sal-remoterpc-connector by blanked-subscribing as an implementation for all actions. This has the side-effect of rendering BP dependency wiring in this case not working, for example for OFP's lldp-speaker.xml – it depends on PacketProcessingService, which is supposed to be exported from openflowplugin's blueprint (i.e. OFP instance registers actions for each connect OF switch). But lldp-speaker activation will not be blocked waiting for openflowplugin, but rather it will activate immediately due to sal-remoterpc-connector pretending to provide that promise.

Another layer of features is to have beans dynamically start when a specific action (on a context) is registered, but making that work means we have to actually react to actions and RPCs disappearing – this is out of scope of this issue, but needs to be considered in the solution domain here.

To solve this issue and BUG-3128 without breaking the world, we will need to provide a partial fix for this issue, then fix sal-remoterpc-connector (and DOMRpcBroker) and then finalize the fix for this issue.

The fix is to provide a clear split between actions and rpcs, in a way which is forward-compatible with YANG 1.1 and Java Binding V2 (where each action is its own entity).

This will be done by defining a odl:action-service and odl:action-provider, which mirror the semantics of odl:rpc-

.

An odl:action-provider designates a bean, which holds the promise to instantiate action instances. As soon as that bean is instantiated, the promise is fulfilled and the promise is populated into DOMRpcRegistry.

An odl:action-service declares a dependency to have at least one odl:action-provider active.

The terms odl:action-instance and odl:action-implementation are reserved for future use. odl:action-implementation will provide an implementation of an action on a particular context (much like odl:routed-rpc-implementation). odl:action-instance will be used to reference to action-implementation. This will be added to XSD, but will be left unimplemented until such time as true dynamic services are needed.

The first part of the fix is to provide these concepts and add a basic compatible implementation, i.e. odl:action-service will be always considered to be fulfilled if a model for it exists. The code will also detect and warn about ambiguous use of odl:rpc-service when the specified interface also contains an action (routed RPC).

DOMRpcService contract clarification: https://git.opendaylight.org/gerrit/51034

New action-provider/action-service contracts: https://git.opendaylight.org/gerrit/50784
OFP fix (requiring the above): https://git.opendaylight.org/gerrit/51115
Final activation: https://git.opendaylight.org/gerrit/51114

boron: https://git.opendaylight.org/gerrit/51204

rpc error

Attachment rpcerror.txt has been added with description: rpc error

log

Attachment karaf_log_error.txt has been added with description: log

This does not quite work in cluster singleton service

our blueprint configuration
https://github.com/opendaylight/groupbasedpolicy/blob/master/renderers/vpp/src/main/resources/org/opendaylight/blueprint/vpp-renderer.xml

<bean id="vppRenderer" class="org.opendaylight.controller.config.yang.config.vpp_provider.impl.GbpVppProviderInstance"
init-method="initialize" destroy-method="close">
<argument ref="dataBroker"/>
<argument ref="broker"/>
<argument ref="clusterSingletonService" />
</bean>

<odl:rpc-implementation ref="vppRenderer"/>

Errors are returned, when sending RPCs to follower node (not leader).
Please see rpc_error and log in the attachment.

NPE is caught here on line 104

https://github.com/opendaylight/groupbasedpolicy/blob/master/renderers/vpp/src/main/java/org/opendaylight/controller/config/yang/config/vpp_provider/impl/GbpVppProviderInstance.java

@Override
public Future<RpcResult<Void>> createVirtualBridgeDomainOnNodes(CreateVirtualBridgeDomainOnNodesInput input)

However, RPCs are processed correctly when sent to the leader node.

Of course it does not. The reason for that is that the GbpVppProviderInstance is not fully initialized by the time it gets registered as a provider – i.e. instantiateServiceInstance() has not been called.

This effectively means you cannot use odl:rpc-implementation to inject the bean as an implementation – you have to do it yourself from instantiateServiceInstance()... unless there is a blueprint way to have GbpVppProviderInstance bean created based on singleton service.

So with the current setup you are registering an implementation on each node, even before it is fully initialized. Once singleton does its thing, one of the implementations will become operational – but there is a time window when you already have registered it, but it is not initialized.

RPCs are always routed to the closest registered instance, in this case it will end up always being the local one – whether it is operational or not.