[VOL-1690] Voltha 2.0 etcd-cluster OOM - OpenCORD JIRA

XML

Word

Printable

Details

Type: Bug
Status: Resolved (View Workflow)
Priority: Blocker
Resolution: Fixed
Affects Version/s: None
Fix Version/s: VOLTHA v2.1
Component/s: etcd
Labels:
- stabilization

Story Points:
1
Epic Link:
Maintenance

Description

Voltha 2.0 was left running over the weekend and the etcd-cluster pods failed due to memory issues.

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

Attachments

6-10-19-voltha-2-etcd-failure-completed.txt
92 kB
2019/06/10 6:17 AM
6-10-19-voltha-2-etcd-failure-journalctl.txt
90 kB
2019/06/10 6:21 AM
6-10-19-voltha-2-etcd-failure-k8s-info.txt
26 kB
2019/06/10 6:17 AM
6-10-19-voltha-2-etcd-failure-OOMKilled.txt
58 kB
2019/06/10 6:17 AM
completed-etcd-cluster-logs3.txt
223 kB
2019/07/05 4:46 AM
completed-etcd-cluster-logs3.txt
223 kB
2019/07/05 4:46 AM
oomkilled-etcd-cluster-logs.txt
31 kB
2019/07/05 4:46 AM
oomkilled-etcd-cluster-logs.txt
31 kB
2019/07/05 4:46 AM
oomkilled-etcd-cluster-logs2.txt
256 kB
2019/07/05 4:46 AM
oomkilled-etcd-cluster-logs2.txt
256 kB
2019/07/05 4:46 AM
oomkilled-k8s-logs.txt
26 kB
2019/07/05 4:46 AM

Options
- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

Gerrit Reviews

#	Subject	Branch	Project	Status	CR	V
15413,2	VOL-1690 VOL-1330: No longer store config nor alarm MIB data in etcd	master	voltha-openonu-adapter	Status: MERGED	+2	+1
15440,2	VOL-1690 VOL-1330: No longer store config nor alarm MIB data in etcd	voltha-2.1	voltha-openonu-adapter	Status: MERGED	+2	+1
15459,6	VOL-1690: Use revision based compaction	master	voltha-helm-charts	Status: MERGED	+2	+1
15541,1	VOL-1690: Close the etcd watcher channels after we are done watching for etcd events. Also ensure that the etcd watcher context is canceled after we are done using the etcd Watcher interface.	voltha-2.1	voltha-go	Status: MERGED	+2	+1
15542,1	VOL-1690: Close the etcd watcher channels after we are done watching for etcd events. Also ensure that the etcd watcher context is canceled after we are done using the etcd Watcher interface.	master	voltha-go	Status: MERGED	+2	+1
15560,1	VOL-1690: Don't close the event channel for watch events on the receiver. Close it in the sender instead.	voltha-2.1	voltha-go	Status: MERGED	+2	+1
15561,1	VOL-1690: Don't close the event channel for watch events on the receiver. Close it in the sender instead.	master	voltha-go	Status: MERGED	+2	+1

Activity

People

Assignee:: Matt Jeanneret

Reporter:: Kevin Riordan (Inactive)

Watchers:: 7 Start watching this issue

Dates

Created:: 2019/06/10 6:22 AM

Updated:: 2019/10/03 1:17 PM

Resolved:: 2019/10/03 1:17 PM

Gerrit Reviews

There are no open Gerrit changes

Show There are 7 closed Gerrit changes

Hide There are 7 closed Gerrit changes

VOL-1690 VOL-1330: No longer store config nor alarm MIB data in etcd: Gerrit Review:

VOL-1690 VOL-1330: No longer store config nor alarm MIB data in etcd: Gerrit Review:

VOL-1690: Use revision based compaction: Gerrit Review:

VOL-1690: Close the etcd watcher channels after we are done watching for etcd events. Also ensure that the etcd watcher context is canceled after we are done using the etcd Watcher interface.: Gerrit Review:

VOL-1690: Close the etcd watcher channels after we are done watching for etcd events. Also ensure that the etcd watcher context is canceled after we are done using the etcd Watcher interface.: Gerrit Review:

VOL-1690: Don't close the event channel for watch events on the receiver. Close it in the sender instead.: Gerrit Review:

VOL-1690: Don't close the event channel for watch events on the receiver. Close it in the sender instead.: Gerrit Review: