OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[COE-36] Get CoE as-is manually running on K8s an OpenStack (RDO) Cloud https://jira.opendaylight.org/browse/COE-36 coe <p><a href="https://github.com/vorburger/opendaylight-coe-kubernetes-openshift" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/vorburger/opendaylight-coe-kubernetes-openshift</a> ...</p> <p>The goal of this first step is to Manually “cobble together” COE run on K8s on RDO Cloud.</p> COE-36

Get CoE as-is manually running on K8s an OpenStack (RDO) Cloud

Story Medium Resolved Done Michael Vorburger Michael Vorburger Wed, 5 Dec 2018 12:06:09 +0000 Wed, 19 Dec 2018 14:16:08 +0000 Wed, 19 Dec 2018 13:19:56 +0000 0 2 <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=fkautz" class="user-hover" rel="fkautz">fkautz</a> & <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=shague" class="user-hover" rel="shague">shague</a> I'm currently stuck at this, when can you help me get past this:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [fedora@coe-node ~]$ sudo ovs-vsctl show e29c4851-cc9d-4833-ba3a-906d87fd72b7 Manager <span class="code-quote">"tcp:192.168.0.5:6640"</span> is_connected: <span class="code-keyword">true</span> Bridge br-<span class="code-object">int</span> Controller <span class="code-quote">"tcp:192.168.0.5:6653"</span> is_connected: <span class="code-keyword">true</span> Port <span class="code-quote">"vethb206b14b"</span> Interface <span class="code-quote">"vethb206b14b"</span> error: <span class="code-quote">"could not open network device vethb206b14b (No such device)"</span> Port <span class="code-quote">"veth741cafb2"</span> Interface <span class="code-quote">"veth741cafb2"</span> Port <span class="code-quote">"veth9e3d3ff1"</span> Interface <span class="code-quote">"veth9e3d3ff1"</span> error: <span class="code-quote">"could not open network device veth9e3d3ff1 (No such device)"</span> Port <span class="code-quote">"vethae78875f"</span> Interface <span class="code-quote">"vethae78875f"</span> Port <span class="code-quote">"enp0s9"</span> Interface <span class="code-quote">"enp0s9"</span> type: system error: <span class="code-quote">"could not open network device enp0s9 (No such device)"</span> Port br-<span class="code-object">int</span> Interface br-<span class="code-object">int</span> type: internal ovs_version: <span class="code-quote">"2.8.1"</span> </pre> </div></div> <p>Above is from the node (minion), this is the master:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [fedora@coe-master ~]$ sudo ovs-vsctl show 0b2f6636-f32f-4bc0-a6f9-651b61563aae Manager <span class="code-quote">"tcp:192.168.0.5:6640"</span> is_connected: <span class="code-keyword">true</span> Bridge br-<span class="code-object">int</span> Controller <span class="code-quote">"tcp:192.168.0.5:6653"</span> is_connected: <span class="code-keyword">true</span> Port <span class="code-quote">"veth3c325425"</span> Interface <span class="code-quote">"veth3c325425"</span> Port <span class="code-quote">"veth6c86e2c9"</span> Interface <span class="code-quote">"veth6c86e2c9"</span> Port br-<span class="code-object">int</span> Interface br-<span class="code-object">int</span> type: internal ovs_version: <span class="code-quote">"2.8.1"</span></pre> </div></div> <p> </p> <p>Doing the following:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[fedora@coe-node ~]$ sudo ovs-vsctl set O . other_config:local_ip=192.168.0.8 [fedora@coe-master ~]$ sudo ovs-vsctl set O . other_config:local_ip=192.168.0.5</pre> </div></div> <p>added this <tt>tunb5dbe23b682</tt> which <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=k.faseela" class="user-hover" rel="k.faseela">k.faseela</a> says is good news:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[fedora@coe-node ~]$ sudo ovs-vsctl show e29c4851-cc9d-4833-ba3a-906d87fd72b7 Manager "tcp:192.168.0.5:6640" is_connected: true Bridge br-int Controller "tcp:192.168.0.5:6653" is_connected: true Port "vethb206b14b" Interface "vethb206b14b" error: "could not open network device vethb206b14b (No such device)" Port "veth741cafb2" Interface "veth741cafb2" error: "could not open network device veth741cafb2 (No such device)" Port "veth9e3d3ff1" Interface "veth9e3d3ff1" error: "could not open network device veth9e3d3ff1 (No such device)" Port "tunb5dbe23b682" Interface "tunb5dbe23b682" type: vxlan options: {key=flow, local_ip="192.168.0.8", remote_ip="192.168.0.5"} Port "vethae78875f" Interface "vethae78875f" error: "could not open network device vethae78875f (No such device)" Port "enp0s9" Interface "enp0s9" type: system error: "could not open network device enp0s9 (No such device)" Port br-int Interface br-int type: internal ovs_version: "2.8.1"</pre> </div></div> <p>but the <tt>veth</tt> errors are still there. I'm also hitting <a href="https://jira.opendaylight.org/browse/COE-42" title="Watcher output "400 Bad Request"" class="issue-link" data-issue-key="COE-42">COE-42</a> and <a href="https://jira.opendaylight.org/browse/NETVIRT-1532" title="NFE in CoeUtils.buildElanInstance()" class="issue-link" data-issue-key="NETVIRT-1532"><del>NETVIRT-1532</del></a> ...</p> <p>These veth ports error won't cause any issue for you, they are actually stale veth ports that remain. I have seen the same issue, but have not clearly seen when they start showing up, but ofcourse this is a bug.</p> <p>OK then I'll ignore that issue (veth "could not open network device") here, and just gocus on tgetting the ping test working, and we can look at this problem later; I've created <a href="https://jira.opendaylight.org/browse/COE-43" title="veth interface error: "could not open network device veth9e3d3ff1 (No such device)"" class="issue-link" data-issue-key="COE-43"><del>COE-43</del></a> so that we don't forget it.</p> <p>I'm currently blocked by the various isuses linked to <a href="https://jira.opendaylight.org/browse/COE-10" title="COE Deployment and Packaging for Kubernetes" class="issue-link" data-issue-key="COE-10">COE-10</a>, and need help from CoE experts to proceed.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=vorburger" class="user-hover" rel="vorburger">vorburger</a> : I just now noticed that COE CSIT is broken on master, with the same error as you have reported <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/> stable/fluorine works fine though. Could you please checkout fluorine ODL, as well as build fluorine cni plugin and watcher, and test this out?</p> <p>Sure I could try that (probably only on Wednesday). So the basic CoE use case already worked in Fluorine?</p> <p>Everything was working fine all the way from stable/oxygen.. Something is broken recently.. I will take a look</p> <p>I'll first try out the patch on <a href="https://jira.opendaylight.org/browse/NETVIRT-1533" title="IllegalArgumentException: Node elan-interface is missing mandatory descendant /elan-instance-name" class="issue-link" data-issue-key="NETVIRT-1533"><del>NETVIRT-1533</del></a> to see if that helps; if not, I'll attempt FluorineOxygen instead.</p> <p>The inter-container ping on a single node VM meanwhile works in <a href="https://github.com/vorburger/opendaylight-coe-kubernetes-openshift" class="external-link" target="_blank" rel="nofollow noopener">my (scipted) test environment on the RDO Cloud</a>.</p> <p>I'll set up a 2nd node and test pings between busybox containers on different nodes next.</p> <p><a href="https://lists.opendaylight.org/pipermail/coe-dev/2018-December/000449.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/coe-dev/2018-December/000449.html</a></p> <p>It works now, see coe-dev list reply. (<a href="https://github.com/vorburger/opendaylight-coe-kubernetes-openshift/commit/b38fcdecd97427bb4e38f7e58fdf0c49a60f6a55" class="external-link" target="_blank" rel="nofollow noopener">This did the trick</a>.)</p> <p>With this, <a href="https://github.com/vorburger/opendaylight-coe-kubernetes-openshift" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/vorburger/opendaylight-coe-kubernetes-openshift</a> lets you create a K8s cluster with CoE SDN on OpenStack. The next step is to containerizing odlovs-cni, watcher & ODL and reducing the number of manual steps and moving pieces.</p> <p>I've used 1 master and 2 worker nodes VMs (like CSIT), but my scripts would letus create 99 works nodes as well, which could make this a useful set-up for CoE scale testing in the future.</p> <p>Considering this task Done.</p> <p>The various issues I've seen here are now linked to <a href="https://jira.opendaylight.org/browse/COE-46" title="Production Quality CoE" class="issue-link" data-issue-key="COE-46">COE-46</a> in addition to <a href="https://jira.opendaylight.org/browse/COE-10" title="COE Deployment and Packaging for Kubernetes" class="issue-link" data-issue-key="COE-10">COE-10</a>.</p> Blocks COE-10 COE-37 COE-39 NETVIRT-1532 Development Rank 0|i03k3w: Sprint COE Sprint 1