-
Type: Bug
-
Status: Resolved (View Workflow)
-
Priority: Medium
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: VOLTHA v2.11
-
Component/s: onos, openolt-adapter, rw-core
-
Labels:None
-
Story Points:5
-
Epic Link:
I saw that the subscriber was provisioned too soon before the olt adapter could reconcile and also establish inter-adapter connections. The flows failed to add because the adapters are not ready yet. It takes about 30s after adapter restart, but flow is provisioned few seconds right after adapter restart. Flow is continuously trying to be installed and it fails, and at some point ONOS removes the meters for some reason. I see errors around this area. When adapters are completely ready, the flow add fails because the referenced meters are not present (they were removed earlier).
I have attached logs in logs.tgz. Look for "GGC:" in logs of openolt adapter, openonu adapter, rw-core and onos to follow the sequence/timestamps.
There is especially below log snippet in ONOS OLT App logs that triggers my attention and I think we should pay some attention to this.
=======
GGC: Zero counter reference for meter and it is getting removed. - 18:23:15.707
default/voltha-infra-onos-classic-0[onos-classic]: 18:23:15.707 INFO [OltMeterService] Zero Count Reference event is received for meter f on of:00000a0a0a0a0a0a, incrementing counter
default/voltha-infra-onos-classic-0[onos-classic]: 18:23:15.710 INFO [OltMeterService] Meter f on device of:00000a0a0a0a0a0a is unused, removing it
default/voltha-infra-onos-classic-0[onos-classic]: 18:23:15.712 INFO [OltMeterService] Zero Count Reference event is received for meter 10 on of:00000a0a0a0a0a0a, incrementing counter
default/voltha-infra-onos-classic-0[onos-classic]: 18:23:15.717 INFO [OltMeterService] Meter 10 on device of:00000a0a0a0a0a0a is unused, removing it
GGC: Some exception seen here - 18:23:15.720
default/voltha-infra-onos-classic-0[onos-classic]: 18:23:15.720 ERROR [OltMeterService] Uncaught exception on onos-olt-pending-removal-meters-2
default/voltha-infra-onos-classic-0[onos-classic]: java.util.ConcurrentModificationException
default/voltha-infra-onos-classic-0[onos-classic]: at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1493)
========