Multicast tests in TT workflow

The multicast tests in the TT workflow need to be fixed.

Running the tests with the berlin-community-pod-1-gpon-TT.yaml configuration, the culprit seemed to be the ONU, since the counters in the OLT NNI and GEM ports were both increasing when the multicast traffic was being sent, but the RG was not receiving the packets. The tests were run with the GPON ONUs SCOM00001c82 and SCOM00001C89, both of them connected to the same EdgeCore OLT box. Since both ONUs were failing the mcast tests, we needed to find other ONUs to run the tests with:

• The other SCOM ONUs connected to that same OLT were probably the same model/version, so no point in testing with them.
• The Adtran ONUs and OLTs have never been tested with the TT workflow and Mahir did not think it would work.
• The other EdgeCore XGS-PON OLT did not have any config file nor pipeline ready, so again, we could not test with it.
• The only option was to try with the Zyxel Combo OLT S210Z14009247. This OLT has 2 Zyxel ONUs connected to it: a PM7300-T0 XGSPON ONU and a PMG1005-T20C GPON ONU. The first one seemed to have all the cabling ok and it was showing in the openolt logs, so it was being discovered, but the openonu adapter logs showed LOS OMCI messages (although the LED light in the box was not showing this). The second one had the LOS led light in red and was not showing in the openolt logs (it was not being discovered in the port).

After some troubleshooting, I was able to see both ONUs in VOLTHA, by configuring the OLT with PON device 1 as ANYPON, with port 24 as GPON (default for ANYPON device) and port 30 as XGSPON. But none of the serial numbers is the one in the berlin-community-pod-2-xgspon-zyxel-TT.yaml file, so this should be changed. The data for the discovered Zyxel ONUs is this:

• Zyxel PMG1005-T20C GPON ONU with serial number ZYWNC6018139. It is connected to port 13 of the Zyxel Combo OLT (10.34.90.53) with a Zyxel GPON-OLT-Class C+ transceiver (POMG5SFP-H1). Its power port is connected to port 3 of 10.34.90.49. It is connected to the eno7 interface of the node-6, but it doesn't seem to have a RG LXC container set up.
• Zyxel PM7300-T0 XGSPON ONU with serial number ZYXE8CABE30B. It is connected to port 16 of the Zyxel Combo OLT (10.34.90.53) with a Zyxel XGSPON&GPON-OLT-N2-C+ transceiver (POM-C1-SFP-H-1). Its power port is connected to port 4 of 10.34.90.61. It is connected to the eno6 interface of the node-6, which seems to be the RG LXC container with name "ZYXE8CAC752C".

Also, since the OLT device/port configuration that is set by the CI/CD pipeline does not work, I believe the OpenOLT build used should also be changed, probably a new one would need to be built. As a reference, to configure the OLT so that both ONUs show, I used these BAL CLI commands:

1. Reset the PON devices:

/api/oper object=device sub=reset device_id=0
/api/oper object=device sub=reset device_id=1

2. Enable the PON devices as ANYPON:

/api/oper object=device sub=connect device_id=0 comm_mode=pcie system_mode=itu_any_pon__8_x inni_config={mux=two_to_one mode=all_12_p_5_g} ras_ddr_mode=two_ddrs
/api/oper object=device sub=connect device_id=1 comm_mode=pcie system_mode=itu_any_pon__8_x inni_config={mux=two_to_one mode=all_12_p_5_g} ras_ddr_mode=two_ddrs

3. Configure port 30 as XGSPON:

/a/o object=pon_interface sub=switch_pon_type pon_ni=30 new_pon_type=xgs_gpon_wdma

4. Enable port 24 and port 30 (where the two ONUs are):

/api/set object=pon_interface pon_ni=24 discovery={control=enable onu_post_discovery_mode=activate}
/api/set object=pon_interface pon_ni=30 discovery={control=enable onu_post_discovery_mode=activate}
/api/oper object=pon_interface sub=set_pon_interface_state pon_ni=24 operation=active_working
/api/oper object=pon_interface sub=set_pon_interface_state pon_ni=30 operation=active_working

After both ONUs were being discovered by the OLT, I added the OLT to VOLTHA and enabled it. Now both ONUs can be seen in VOLTHA, but only the XGSPON ONU reaches an ACTIVE state, the GPON ONU stays as DISCOVERED:

$ voltctl device list
ID                                      TYPE                 ROOT     PARENTID                                SERIALNUMBER     ADMINSTATE    OPERSTATUS    CONNECTSTATUS    REASON
1d523018-8f47-4dbe-89e7-07205c2a6651    brcm_openomci_onu    false    f3883119-4ee8-4592-8935-b4efeffa084a    ZYWNc6018139     ENABLED       DISCOVERED    REACHABLE        activating-onu
d9559ad6-6683-465c-b391-4927f031240b    brcm_openomci_onu    false    f3883119-4ee8-4592-8935-b4efeffa084a    ZYXE8cabe30b     ENABLED       ACTIVE        REACHABLE        initial-mib-downloaded
f3883119-4ee8-4592-8935-b4efeffa084a    openolt              true     70ccd1f4-1612-44f2-865d-e83750647ed9    S210Z14009247    ENABLED       ACTIVE        REACHABLE

Checking the OpenOLT logs, I see these lines for the activation process, with an error for the GPON ONU and all good for the XGSPON ONU:

[2716786: I OPENOLT ] core_api_handler.cc 1201| Activate ONU : old state = 0, current state = 0
[2716786: I OPENOLT ] core_api_handler.cc 1224| Configuring ONU 1 on PON 24 : vendor id ZYWN, vendor specific C6018139, pir 1000000
[2716786: E OPENOLT ] core_api_handler.cc 1242| Failed to configure ONU 1 on PON 24, err = itu.xgpon: Invalid configuration in PON mode=gpon
 (-11)
[2716787: I OPENOLT ] core_api_handler.cc 1201| Activate ONU : old state = 0, current state = 0
[2716787: I OPENOLT ] core_api_handler.cc 1224| Configuring ONU 1 on PON 30 : vendor id ZYXE, vendor specific 8CABE30B, pir 1000000
[2716789: I OPENOLT ] core_api_handler.cc 1280| Activated ONU, onu_id 1 on PON 30

As additional info, just in case it adds any more value, this is the OS info for the Zyxel Combo OLT:

root@localhost:~# uname -a
Linux localhost 4.19.81-OpenNetworkLinux #1 SMP Tue Dec 28 02:59:04 UTC 2021 x86_64 GNU/Linux
root@localhost:~# cat /etc/debian_version
8.11

And this is what the BAL CLI shows when run:

root@localhost:/opt/bcm68620# ./example_user_appl
[2050602: I default ] bcm_dev_log_task.c 1474| Logging started
bcmolt_board_specific_init#1824> INF: bcmolt_board_specific_init - Plz support this
bcmtr_init: init transport library
Launching with a unix domain connection
[2050708: I API_KA_0 ] bcmolt_api_conn_mgr.c 252| Connected with OLT 0. Software version: 3.10.0. Model revision: 6. Build time: Wed, 06 Oct 2021 01:46:19 -0700
[2050708: I API_KA_1 ] bcmolt_api_conn_mgr.c 215| Logical OLT 1 doesn't exist
Building itu_ds_omci CLI for OLT 0
Building itu_statistics CLI for OLT 0
Building itu_rssi CLI for OLT 0
Building itu_sn_discovery CLI for device 0
Building itu_stress CLI for OLT 0
Building itu_onu_tuning CLI for OLT 0
[2050729: I itu_coop_dba_example] bcmolt_user_appl_coop_dba_example.c 602| ITU Coop DBA example initialized: 'OK'
[2050739: I default ] example_user_appl.c 662| Starting CLI