[VOL-4077] Improve storage usage on etcd - OpenCORD JIRA

XML

Word

Printable

Details

Type: Story
Status: Resolved (View Workflow)
Priority: Medium
Resolution: Done
Affects Version/s: None
Fix Version/s: VOLTHA v2.8
Component/s: openolt-adapter
Labels:
- onf

Story Points:
8
Epic Link:
Maintenance
Sprint:
VOLTHA Sprint 3, VOLTHA Sprint 4, VOLTHA Sprint 5

Description

This work is a result of study on the types of data stored by the voltha stack on etcd as described here https://docs.google.com/document/d/1WEJqDGGnvhzg2ql7ER31KGcF0HmOEXrBnPTF5Dg4hbQ/edit?pli=1#heading=h.jguvl76negl1. This was presented in the TST call on May11, 2021.

The goal here is to remove data from etcd that does not require persistence by the voltha stack. At the moment we store over a GB of data on the etcd for 1024 subscribers and when extrapolated for 10K subscribers the size of the etcd DB is just too large for stable operation of the cluster of voltha stacks.

Attachments

Options

Sub-Tasks

1.	Changes in agent to deduce symmetric flow-id, and also to pass {onu,uni} information in PacketIndication		Resolved	Girish Gowdra
2.	Changes in openolt adapter for storage improvements		Resolved	Girish Gowdra
3.	BBSIM changes for storage improvements		Resolved	Girish Gowdra
4.	Verify agent changes with openolt-scale-tester		Resolved	Girish Gowdra
5.	Scale testing with BBSIM after all coding is complete		Resolved	Girish Gowdra
6.	Test openolt adapter custom image + rw-core custom image with storage improvements		Resolved	Girish Gowdra
7.	Create protobuf message definitions for techprofile template, techprofile instance and resource instance		Resolved	Girish Gowdra
8.	Changes to techprofile module for using protobuf definitions of techprofile template and instance		Resolved	Girish Gowdra
9.	Merge changes to voltha protos, lib-go, openolt, openonu adapter and openolt agent		Resolved	Girish Gowdra

Options
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

Gerrit Reviews

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

Activity

People

Assignee:: Girish Gowdra

Reporter:: Matteo Scandolo

Watchers:: 1 Start watching this issue

Dates

Created:: 2021/05/11 9:40 AM

Updated:: 2021/06/28 7:11 AM

Resolved:: 2021/06/28 7:11 AM

Gerrit Reviews

There are no open Gerrit changes

Show There are 6 closed Gerrit changes

Hide There are 6 closed Gerrit changes

VOL-4077: Improve storage usage - send onu, uni port number information in packet indications to VOLTHA - adjust the resource id limits to be realistic enough for the votha-2.8 requirements: Gerrit Review:

VOL-4077: Improve storage usage on etcd - send onu_id and uni_id in packet in (eap, dhcp, igmp): Gerrit Review:

VOL-4077: Improve storage usage on etcd - Do away with unnecessary data storage on etcd if it can be reconciled on adapter restart - For data that needs storage, use lesser footprint if possible - Use write-through-cache for all data stored on etcd via resource manager module - Use ResourceManager module per interface to localize lock contention per PON port: Gerrit Review:

VOL-4077: Add onu-id and uni-id in PacketIndication message - The openolt-agent application shall pass this information for PacketIndication message arriving from the PON port.: Gerrit Review:

VOL-4077: Improve storage usage - Use etcd chart 6.2.5 autoCompactionMode=revision and autocompactionRetention=1 - Allow configurability of ETCD_ELECTION_TIMEOUT and ETCD_HEARTBEAT_INTERVAL for etcd (defaults to 5000 and 1000 respectively). - Also create a cron job to defragement etcd DB every 5 mins: Gerrit Review:

VOL-4077: Improve storage usage on etcd - the onu adapter will now receive the tp instance on the inter-container kafka message and need not reach the etcd store to fetch it. - on reconcile, the onu adapter need to go to the kv store to fetch the tp instance, but request the tp instance via a new API towards openolt adapter which in-turn will fetch from the cache. - re-org the code in onu-metrics-manager to restore pm-data on reconcile to avoid panics by accessing uninitialzed data if ani-fsm were to try adding gem port for monitoring before pm-data is initialized properly.: Gerrit Review: