Uploaded image for project: 'VOLTHA'
  1. VOLTHA
  2. VOL-1266

vcore stuck in loop due to etcd DNSLookupError

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Medium
    • Resolution: Won't Fix
    • Affects Version/s: VOLTHA v1.7
    • Fix Version/s: None
    • Component/s: etcd, ro-core, rw-core
    • Labels:
      None

      Description

      "Seba-in-a-box" builds have been failing more often than not and we have isolated the issue being that voltha comes up in a bad state. 

      The logs of voltha have this error:

      20181004T174942.819 ERROR MainThread coordinator_etcd._retry {event: DNS lookup failed: no results for hostname lookup: etcd-cluster.default.svc.cluster.local., exception: Traceback (most recent call last): File "/voltha/voltha/coordinator_etcd.py", line 571, in _retry result = yield operation(*args, **kw) File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/voltha/voltha/coordinator_etcd.py", line 235, in _create_session self.lease = yield etcd.lease(self.session_time_to_live) File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python2.7/dist-packages/txaioetcd/_client_tx.py", line 725, in lease response = yield treq.post(url, data, headers=self._REQ_HEADERS, timeout=(timeout or self._timeout)) File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/local/lib/python2.7/dist-packages/twisted/internet/endpoints.py", line 954, in startConnectionAttempts "no results for hostname lookup: {}".format(self._hostStr) DNSLookupError: DNS lookup failed: no results for hostname lookup: etcd-cluster.default.svc.cluster.local., instance_id: vcore-0}
      

      Seems to be failing to connect to etcd cluster.

      Seba-in-a-box build job - https://jenkins.opencord.org/view/SIAB/job/seba-in-a-box-build-test/

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            kailash Kailash Khalasi
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes