-
Type:
Bug
-
Status: Resolved (View Workflow)
-
Priority:
Medium
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: VOLTHA v2.10
-
Component/s: openonu-adapter
-
Labels:
-
Story Points:3
-
Epic Link:
-
Sprint:VOLTHA 2.10 - Sprint 6
The issue is that openonu adapter crashes in PM module due to nil reference during device delete and some stale data is left on etcd (w.r.to PM data). As part of PrepareForGarbageCollection some pointers are set to nil to free up references for garbage collection by go runtime engine. This function is invoked during device delete. These nil pointers are referenced by PM FSM that is still preparing to shutdown. Working on a possible solution.
Below is the panic back trace.
Issue reproducible with this commit -https://github.com/opencord/voltha-openonu-adapter-go/commit/12609a10eacc719b5444e35c7bfbdbab948fd045 and this job https://jenkins.opencord.org/view/VOLTHA-2.X-Tests/job/periodic-voltha-memory-leak-test-bbsim/
voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: panic: runtime error: invalid memory address or nil pointer dereference voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1115929] voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: goroutine 62639 [running]: voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/opencord/voltha-openonu-adapter-go/internal/pkg/pmmgr.(*OnuMetricsManager).l2PmFsmCreatePM(0xc0013b7b80, 0x1679ae0, 0xc000919680, 0xc000efcf50, 0x7, 0xc0010bf0e0) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/internal/pkg/pmmgr/onu_metrics_manager.go:1478 +0x1f49 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/opencord/voltha-openonu-adapter-go/internal/pkg/pmmgr.(*OnuMetricsManager).initializeL2PmFsm.func7(0xc000efcf50) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/internal/pkg/pmmgr/onu_metrics_manager.go:2590 +0x46 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/looplab/fsm.(*FSM).enterStateCallbacks(0xc00150d9e0, 0xc000efcf50) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/vendor/github.com/looplab/fsm/fsm.go:403 +0x102 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/looplab/fsm.(*FSM).Event.func1() voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/vendor/github.com/looplab/fsm/fsm.go:308 +0x9e voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/looplab/fsm.transitionerStruct.transition(...) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/vendor/github.com/looplab/fsm/fsm.go:354 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/looplab/fsm.(*FSM).doTransition(...) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/vendor/github.com/looplab/fsm/fsm.go:339 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/looplab/fsm.(*FSM).Event(0xc00150d9e0, 0x14a55a2, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/vendor/github.com/looplab/fsm/fsm.go:322 +0x608 voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: github.com/opencord/voltha-openonu-adapter-go/internal/pkg/pmmgr.(*OnuMetricsManager).AddGemPortForPerfMonitoring.func1(0xc0013b7b80, 0x1679ae0, 0xc0015fae70) voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/internal/pkg/pmmgr/onu_metrics_manager.go:2698 +0x5d voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: created by github.com/opencord/voltha-openonu-adapter-go/internal/pkg/pmmgr.(*OnuMetricsManager).AddGemPortForPerfMonitoring voltha/voltha-voltha-adapter-openonu-69fd555cf8-btt6s[adapter-open-onu]: /go/src/internal/pkg/pmmgr/onu_metrics_manager.go:2697 +0x6e5