Change event notification subscription tutorial ----------------------------------------------- Subscribing to data change notifications makes it possible to obtain notifications about data manipulation (insert, change, delete) which are done on any specified **path** of any specified **datastore** with specific **scope**. In following examples *{odlAddress}* is address of server where ODL is running and *{odlPort}* is port on which OpenDaylight is running. OpenDaylight offers two methods for receiving notifications: Server-Sent Events (SSE) and WebSocket. SSE is the default notification mechanism used in OpenDaylight. SSE notifications subscription process ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In this section we will learn what steps need to be taken in order to successfully subscribe to data change event notifications. Create stream ^^^^^^^^^^^^^ In order to use event notifications you first need to call RPC that creates notification stream that you can later listen to. You need to provide three parameters to this RPC: - **path**: data store path that you plan to listen to. You can register listener on containers, lists and leaves. - **datastore**: data store type. *OPERATIONAL* or *CONFIGURATION*. - **scope**: Represents scope of data change. Possible options are: - BASE: only changes directly to the data tree node specified in the path will be reported - ONE: changes to the node and to direct child nodes will be reported - SUBTREE: changes anywhere in the subtree starting at the node will be reported The RPC to create the stream can be invoked via RESTCONF like this: - URI: http://{odlAddress}:{odlPort}/rests/operations/sal-remote:create-data-change-event-subscription - HEADER: Content-Type=application/json Accept=application/json - OPERATION: POST - DATA: .. code:: json { "input": { "path": "/toaster:toaster/toaster:toasterStatus", "sal-remote-augment:datastore": "OPERATIONAL", "sal-remote-augment:scope": "ONE" } } The response should look something like this: .. code:: json { "sal-remote:output": { "stream-name": "data-change-event-subscription/toaster:toaster/toaster:toasterStatus/datastore=CONFIGURATION/scope=SUBTREE" } } **stream-name** is important because you will need to use it when you subscribe to the stream in the next step. .. note:: Internally, this will create a new listener for *stream-name* if it did not already exist. Subscribe to stream ^^^^^^^^^^^^^^^^^^^ In order to subscribe to stream and obtain SSE location you need to call *GET* on your stream path. The URI should generally be http://{odlAddress}:{odlPort}/rests/data/ietf-restconf-monitoring:restconf-state/streams/stream/{streamName}, where *{streamName}* is the *stream-name* parameter contained in response from *create-data-change-event-subscription* RPC from the previous step. - URI: http://{odlAddress}:{odlPort}/rests/data/ietf-restconf-monitoring:restconf-state/streams/stream/data-change-event-subscription/toaster:toaster/datastore=CONFIGURATION/scope=SUBTREE - OPERATION: GET The subscription call may be modified with the following query parameters defined in the RESTCONF RFC: - `filter `__ - `start-time `__ - `end-time `__ In addition, the following ODL extension query parameter is supported: :odl-leaf-nodes-only: If this parameter is set to "true", create and update notifications will only contain the leaf nodes modified instead of the entire subscription subtree. This can help in reducing the size of the notifications. :odl-skip-notification-data: If this parameter is set to "true", create and update notifications will only contain modified leaf nodes without data. This can help in reducing the size of the notifications. The response should look something like this: .. code:: json { "subscribe-to-notification:location": "http://localhost:8181/rests/notif/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE" } .. note:: During this phase there is an internal check for to see if a listener for the *stream-name* from the URI exists. If not, new a new listener is registered with the DOM data broker. Receive notifications ^^^^^^^^^^^^^^^^^^^^^ Once you got SSE location you can now connect to it and start receiving data change events. The request should look something like this: :: curl -v -X GET http://localhost:8181/rests/notif/data-change-event-subscription/toaster:toaster/toasterStatus/datastore=OPERATIONAL/scope=ONE -H "Content-Type: text/event-stream" -H "Authorization: Basic YWRtaW46YWRtaW4=" WebSocket notifications subscription process ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Enabling WebSocket notifications in OpenDaylight requires a manual setup before building the project. The following steps can be followed to enable WebSocket notifications in OpenDaylight: 1. Open the file `restconf8040.cfg`, which is located at `netconf/restconf/restconf-nb-rfc8040/src/main/resources/` directory. 2. Locate the `use-sse` configuration parameter and change its value from `true` to `false`. 3. Uncomment the `use-sse` parameter if it is commented out. 4. Save the changes made to the `restconf8040.cfg` file. Once these steps are completed, WebSocket notifications will be enabled in OpenDaylight, and they can be used for receiving notifications instead of SSE. WebSocket Notifications subscription process is the same as SSE until you receive a location of WebSocket. You can follow steps given above and after subscribing to a notification stream over WebSocket, you will receive a response indicating that the subscription was successful: .. code:: json { "subscribe-to-notification:location": "ws://localhost:8181/rests/notif/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE" } You can use this WebSocket to listen to data change notifications. To listen to notifications you can use a JavaScript client or if you are using chrome browser you can use the `Simple WebSocket Client `__. Also, for testing purposes, there is simple Java application named WebSocketClient. The application is placed in the *-sal-rest-connector-classes.class* project. It accepts a WebSocket URI as and input parameter. After starting the utility (WebSocketClient class directly in Eclipse/InteliJ Idea) received notifications should be displayed in console. Notifications are always in XML format and look like this: .. code:: xml 2014-09-11T09:58:23+02:00 /meae:toaster updated Example use case ~~~~~~~~~~~~~~~~ The typical use case is listening to data change events to update web page data in real-time. In this tutorial we will be using toaster as the base. When you call *make-toast* RPC, it sets *toasterStatus* to "down" to reflect that the toaster is busy making toast. When it finishes, *toasterStatus* is set to "up" again. We will listen to this toaster status changes in data store and will reflect it on our web page in real-time thanks to WebSocket data change notification. Simple javascript client implementation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We will create simple JavaScript web application that will listen updates on *toasterStatus* leaf and update some element of our web page according to new toaster status state. Create stream ^^^^^^^^^^^^^ First you need to create stream that you are planing to subscribe to. This can be achieved by invoking "create-data-change-event-subscription" RPC on RESTCONF via AJAX request. You need to provide data store **path** that you plan to listen on, **data store type** and **scope**. If the request is successful you can extract the **stream-name** from the response and use that to subscribe to the newly created stream. The *{username}* and *{password}* fields represent your credentials that you use to connect to OpenDaylight via RESTCONF: .. note:: The default user name and password are "admin". .. code:: javascript function createStream() { $.ajax( { url: 'http://{odlAddress}:{odlPort}/rests/operations/sal-remote:create-data-change-event-subscription', type: 'POST', headers: { 'Authorization': 'Basic ' + btoa('{username}:{password}'), 'Content-Type': 'application/json' }, data: JSON.stringify( { 'input': { 'path': '/toaster:toaster/toaster:toasterStatus', 'sal-remote-augment:datastore': 'OPERATIONAL', 'sal-remote-augment:scope': 'ONE' } } ) }).done(function (data) { // this function will be called when ajax call is executed successfully subscribeToStream(data.output['stream-name']); }).fail(function (data) { // this function will be called when ajax call fails console.log("Create stream call unsuccessful"); }) } Subscribe to stream ^^^^^^^^^^^^^^^^^^^ The Next step is to subscribe to the stream. To subscribe to the stream you need to call *GET* on *http://{odlAddress}:{odlPort}/rests/data/ietf-restconf-monitoring:restconf-state/streams/stream/{stream-name}*. If the call is successful, you get WebSocket address for this stream in **Location** parameter inside response header. You can get response header by calling *getResponseHeader(\ *Location*)* on HttpRequest object inside *done()* function call: .. code:: javascript function subscribeToStream(streamName) { $.ajax( { url: 'http://{odlAddress}:{odlPort}/rests/data/ietf-restconf-monitoring:restconf-state/streams/stream/' + streamName; type: 'GET', headers: { 'Authorization': 'Basic ' + btoa('{username}:{password}'), } } ).done(function (data, textStatus, httpReq) { // we need function that has http request object parameter in order to access response headers. listenToNotifications(httpReq.getResponseHeader('Location')); }).fail(function (data) { console.log("Subscribe to stream call unsuccessful"); }); } Receive notifications ^^^^^^^^^^^^^^^^^^^^^ Once you got WebSocket server location you can now connect to it and start receiving data change events. You need to define functions that will handle events on WebSocket. In order to process incoming events from OpenDaylight you need to provide a function that will handle *onmessage* events. The function must have one parameter that represents the received event object. The event data will be stored in *event.data*. The data will be in an XML format that you can then easily parse using jQuery. .. code:: javascript function listenToNotifications(socketLocation) { try { var notificatinSocket = new WebSocket(socketLocation); notificatinSocket.onmessage = function (event) { // we process our received event here console.log('Received toaster data change event.'); $($.parseXML(event.data)).find('data-change-event').each( function (index) { var operation = $(this).find('operation').text(); if (operation == 'updated') { // toaster status was updated so we call function that gets the value of toasterStatus leaf updateToasterStatus(); return false; } } ); } notificatinSocket.onerror = function (error) { console.log("Socket error: " + error); } notificatinSocket.onopen = function (event) { console.log("Socket connection opened."); } notificatinSocket.onclose = function (event) { console.log("Socket connection closed."); } // if there is a problem on socket creation we get exception (i.e. when socket address is incorrect) } catch(e) { alert("Error when creating WebSocket" + e ); } } The *updateToasterStatus()* function represents function that calls *GET* on the path that was modified and sets toaster status in some web page element according to received data. After the WebSocket connection has been established you can test events by calling make-toast RPC via RESTCONF. .. note:: for more information about WebSockets in JavaScript visit `Writing WebSocket client applications `__ .. _config_subsystem: Config Subsystem ---------------- Overview ~~~~~~~~ The Controller configuration operation has three stages: - First, a Proposed configuration is created. Its target is to replace the old configuration. - Second, the Proposed configuration is validated, and then committed. If it passes validation successfully, the Proposed configuration state will be changed to Validated. - Finally, a Validated configuration can be Committed, and the affected modules can be reconfigured. In fact, each configuration operation is wrapped in a transaction. Once a transaction is created, it can be configured, that is to say, a user can abort the transaction during this stage. After the transaction configuration is done, it is committed to the validation stage. In this stage, the validation procedures are invoked. If one or more validations fail, the transaction can be reconfigured. Upon success, the second phase commit is invoked. If this commit is successful, the transaction enters the last stage, committed. After that, the desired modules are reconfigured. If the second phase commit fails, it means that the transaction is unhealthy - basically, a new configuration instance creation failed, and the application can be in an inconsistent state. .. figure:: ./images/configuration.jpg :alt: Configuration states Configuration states .. figure:: ./images/Transaction.jpg :alt: Transaction states Transaction states Validation ~~~~~~~~~~ To secure the consistency and safety of the new configuration and to avoid conflicts, the configuration validation process is necessary. Usually, validation checks the input parameters of a new configuration, and mostly verifies module-specific relationships. The validation procedure results in a decision on whether the proposed configuration is healthy. Dependency resolver ~~~~~~~~~~~~~~~~~~~ Since there can be dependencies between modules, a change in a module configuration can affect the state of other modules. Therefore, we need to verify whether dependencies on other modules can be resolved. The Dependency Resolver acts in a manner similar to dependency injectors. Basically, a dependency tree is built. APIs and SPIs ~~~~~~~~~~~~~ This section describes configuration system APIs and SPIs. SPIs ^^^^ **Module** org.opendaylight.controller.config.spi. Module is the common interface for all modules: every module must implement it. The module is designated to hold configuration attributes, validate them, and create instances of service based on the attributes. This instance must implement the AutoCloseable interface, owing to resources clean up. If the module was created from an already running instance, it contains an old instance of the module. A module can implement multiple services. If the module depends on other modules, setters need to be annotated with @RequireInterface. **Module creation** 1. The module needs to be configured, set with all required attributes. 2. The module is then moved to the commit stage for validation. If the validation fails, the module attributes can be reconfigured. Otherwise, a new instance is either created, or an old instance is reconfigured. A module instance is identified by ModuleIdentifier, consisting of the factory name and instance name. | **ModuleFactory** org.opendaylight.controller.config.spi. The ModuleFactory interface must be implemented by each module factory. | A module factory can create a new module instance in two ways: - From an existing module instance - | An entirely new instance | ModuleFactory can also return default modules, useful for populating registry with already existing configurations. A module factory implementation must have a globally unique name. APIs ^^^^ +--------------------------------------+--------------------------------------+ | ConfigRegistry | Represents functionality provided by | | | a configuration transaction (create, | | | destroy module, validate, or abort | | | transaction). | +--------------------------------------+--------------------------------------+ | ConfigTransactionController | Represents functionality for | | | manipulating with configuration | | | transactions (begin, commit config). | +--------------------------------------+--------------------------------------+ | RuntimeBeanRegistratorAwareConfiBean | The module implementing this | | | interface will receive | | | RuntimeBeanRegistrator before | | | getInstance is invoked. | +--------------------------------------+--------------------------------------+ Runtime APIs ^^^^^^^^^^^^ +--------------------------------------+--------------------------------------+ | RuntimeBean | Common interface for all runtime | | | beans | +--------------------------------------+--------------------------------------+ | RootRuntimeBeanRegistrator | Represents functionality for root | | | runtime bean registration, which | | | subsequently allows hierarchical | | | registrations | +--------------------------------------+--------------------------------------+ | HierarchicalRuntimeBeanRegistration | Represents functionality for runtime | | | bean registration and | | | unreregistration from hierarchy | +--------------------------------------+--------------------------------------+ JMX APIs ^^^^^^^^ | JMX API is purposed as a transition between the Client API and the JMX platform. +--------------------------------------+--------------------------------------+ | ConfigTransactionControllerMXBean | Extends ConfigTransactionController, | | | executed by Jolokia clients on | | | configuration transaction. | +--------------------------------------+--------------------------------------+ | ConfigRegistryMXBean | Represents entry point of | | | configuration management for | | | MXBeans. | +--------------------------------------+--------------------------------------+ | Object names | Object Name is the pattern used in | | | JMX to locate JMX beans. It consists | | | of domain and key properties (at | | | least one key-value pair). Domain is | | | defined as | | | "org.opendaylight.controller". The | | | only mandatory property is "type". | +--------------------------------------+--------------------------------------+ Use case scenarios ^^^^^^^^^^^^^^^^^^ | A few samples of successful and unsuccessful transaction scenarios follow: **Successful commit scenario** 1. The user creates a transaction calling creteTransaction() method on ConfigRegistry. 2. ConfigRegisty creates a transaction controller, and registers the transaction as a new bean. 3. Runtime configurations are copied to the transaction. The user can create modules and set their attributes. 4. The configuration transaction is to be committed. 5. The validation process is performed. 6. After successful validation, the second phase commit begins. 7. Modules proposed to be destroyed are destroyed, and their service instances are closed. 8. Runtime beans are set to registrator. 9. The transaction controller invokes the method getInstance on each module. 10. The transaction is committed, and resources are either closed or released. | **Validation failure scenario** | The transaction is the same as the previous case until the validation process. 1. If validation fails, (that is to day, illegal input attributes values or dependency resolver failure), the validationException is thrown and exposed to the user. 2. The user can decide to reconfigure the transaction and commit again, or abort the current transaction. 3. On aborted transactions, TransactionController and JMXRegistrator are properly closed. 4. Unregistration event is sent to ConfigRegistry. Default module instances ^^^^^^^^^^^^^^^^^^^^^^^^ The configuration subsystem provides a way for modules to create default instances. A default instance is an instance of a module, that is created at the module bundle start-up (module becomes visible for configuration subsystem, for example, its bundle is activated in the OSGi environment). By default, no default instances are produced. The default instance does not differ from instances created later in the module life-cycle. The only difference is that the configuration for the default instance cannot be provided by the configuration subsystem. The module has to acquire the configuration for these instances on its own. It can be acquired from, for example, environment variables. After the creation of a default instance, it acts as a regular instance and fully participates in the configuration subsystem (It can be reconfigured or deleted in following transactions.)