Uploaded image for project: 'VOLTHA'
  1. VOLTHA
  2. VOL-4077

Improve storage usage on etcd

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      VOLTHA Sprint 3, VOLTHA Sprint 4, VOLTHA Sprint 5

      Description

      This work is a result of study on the types of data stored by the voltha stack on etcd as described here https://docs.google.com/document/d/1WEJqDGGnvhzg2ql7ER31KGcF0HmOEXrBnPTF5Dg4hbQ/edit?pli=1#heading=h.jguvl76negl1. This was presented in the TST call on May11, 2021.

      The goal here is to remove data from etcd that does not require persistence by the voltha stack. At the moment we store over a GB of data on the etcd for 1024 subscribers and when extrapolated for 10K subscribers the size of the etcd DB is just too large for stable operation of the cluster of voltha stacks. 

        Attachments

        # Subject Branch Project Status CR V
        24743,5 VOL-4077: Improve storage usage - send onu, uni port number information in packet indications to VOLTHA - adjust the resource id limits to be realistic enough for the votha-2.8 requirements master openolt Status: MERGED +2 +1
        24746,4 VOL-4077: Improve storage usage on etcd - send onu_id and uni_id in packet in (eap, dhcp, igmp) master bbsim Status: MERGED +2 +1
        24766,14 VOL-4077: Improve storage usage on etcd - Do away with unnecessary data storage on etcd if it can be reconciled on adapter restart - For data that needs storage, use lesser footprint if possible - Use write-through-cache for all data stored on etcd via resource manager module - Use ResourceManager module per interface to localize lock contention per PON port master voltha-openolt-adapter Status: MERGED +2 +1
        24788,1 VOL-4077: Add onu-id and uni-id in PacketIndication message - The openolt-agent application shall pass this information for PacketIndication message arriving from the PON port. master voltha-protos Status: MERGED +2 +1
        25069,26 VOL-4077: Improve storage usage - Use etcd chart 6.2.5 autoCompactionMode=revision and autocompactionRetention=1 - Allow configurability of ETCD_ELECTION_TIMEOUT and ETCD_HEARTBEAT_INTERVAL for etcd (defaults to 5000 and 1000 respectively). - Also create a cron job to defragement etcd DB every 5 mins master voltha-helm-charts Status: MERGED +2 +1
        25132,12 VOL-4077: Improve storage usage on etcd - the onu adapter will now receive the tp instance on the inter-container kafka message and need not reach the etcd store to fetch it. - on reconcile, the onu adapter need to go to the kv store to fetch the tp instance, but request the tp instance via a new API towards openolt adapter which in-turn will fetch from the cache. - re-org the code in onu-metrics-manager to restore pm-data on reconcile to avoid panics by accessing uninitialzed data if ani-fsm were to try adding gem port for monitoring before pm-data is initialized properly. master voltha-openonu-adapter-go Status: MERGED +2 +1

          Activity

            People

            Assignee:
            ggowdra Girish Gowdra
            Reporter:
            teone Matteo Scandolo
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes