Uploaded image for project: 'VOLTHA'
  1. VOLTHA
  2. VOL-4771

Error deleting device via voltctl when OLT is unreachable

    XMLWordPrintable

    Details

    • Story Points:
      5

      Description

      I noticed that, when the OLT is unreachable, the voltctl device disable and delete commands can lead to an unexpected behavior.

      If you disconnect the OLT mgmt, execute one of these commands, then reconnect its mgmt interface, there will still be resources present at the OLT, while Voltha and ONOS have cleaned their own.

      For the OLT delete case, when the command is issued, an error is thrown at the olt adapter:

      {"level":"error","ts":"2022-08-18T11:48:53.003Z","caller":"olterrors/olterrors.go:138","msg":"olt-reboot-failed","instanceId":"adapter-furukawa-olt-6846f7586f-nb96l","device-id":"ccb848f2-bd5d-4ef8-a466-a113f8e44f0c","wrapped":"rpc error: code = Canceled desc = grpc: the client connection is closing","stacktrace":"github.com/opencord/voltha-openolt-adapter/internal/pkg/olterrors.(*ErrAdapter).LogAt\n\t/go/src/internal/pkg/olterrors/olterrors.go:138\ngithub.com/opencord/voltha-openolt-adapter/internal/pkg/olterrors.(*ErrAdapter).Log\n\t/go/src/internal/pkg/olterrors/olterrors.go:118\ngithub.com/opencord/voltha-openolt-adapter/internal/pkg/core.(*DeviceHandler).DeleteDevice\n\t/go/src/internal/pkg/core/device_handler.go:2149\ngithub.com/opencord/voltha-openolt-adapter/internal/pkg/core.(*OpenOLT).DeleteDevice\n\t/go/src/internal/pkg/core/openolt.go:234\ngithub.com/opencord/voltha-protos/v5/go/adapter_service._AdapterService_DeleteDevice_Handler.func1\n\t/go/src/vendor/github.com/opencord/voltha-protos/v5/go/adapter_service/adapter_service.pb.go:716\ngithub.com/opencord/voltha-lib-go/v7/pkg/grpc.mkServerInterceptor.func1\n\t/go/src/vendor/github.com/opencord/voltha-lib-go/v7/pkg/grpc/server.go:158\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/go/src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing/server_interceptors.go:38\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n\t/go/src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1\n\t/go/src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34\ngithub.com/opencord/voltha-protos/v5/go/adapter_service._AdapterService_DeleteDevice_Handler\n\t/go/src/vendor/github.com/opencord/voltha-protos/v5/go/adapter_service/adapter_service.pb.go:718\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/src/vendor/google.golang.org/grpc/server.go:1007\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/src/vendor/google.golang.org/grpc/server.go:1287\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/go/src/vendor/google.golang.org/grpc/server.go:722"}
      

      the connection was closed and the OLT could not be rebooted. But this error is not reported back to the vCore.

      If you reprovision the OLT and its mgmt interface is up, its flows and ACLs would still be installed, and, ONOS will not be able to install flows:

      # Flows at ONOS
      karaf@root > flows -s                                                                                                                                                                 11:50:51
      deviceId=of:000080a235f54292, flowRuleCount=0
      
      # Flows at Voltha
      administrator@lvqsdnk8s01:~$ voltctl device flows ab4d153c-dd49-4f0a-815b-722861cb5cbd 
      *** NO FLOWS ***

        Attachments

        1. image-2022-09-12-15-56-01-339.png
          image-2022-09-12-15-56-01-339.png
          91 kB
        2. image-2022-09-12-15-54-53-675.png
          image-2022-09-12-15-54-53-675.png
          27 kB
        3. image-2022-09-12-15-43-58-382.png
          image-2022-09-12-15-43-58-382.png
          61 kB
        4. image-2022-09-12-15-43-02-831.png
          image-2022-09-12-15-43-02-831.png
          27 kB
        5. image.png
          image.png
          27 kB
        6. delete-olt-when-unreachable.zip
          400 kB

          Issue Links

          # Subject Branch Project Status CR V

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              danielgarbi Daniel Garbi
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes