2024 VOLTHA TST Meeting Minutes

Owned by Michelle Roth
Last updated: yesterday at 4:35 pm
33 min read

Meeting Information

        

LF Broadband community webinar scheduled for December 3 - View agenda and register here

2024-11-19

Recorded Meeting: 

Agenda:

  • BISDN status

    • ONU, OLT diagram and other cabling diagrams presented. TST members to review.

    • BISDN requested permission for certain repos for +2 and +verify. A Task is created for this. Mahir to create another for removing the permissions after the work is finished.

    • TST to check the documents that are prepared by BISDN.

    • Radisys OLT currently have problems. Praneeth is on this.

    • Everything needed from LF for Netbox setup is ready, reported by Eric.

    • Eric will grant TST members access to all the links in the documents.

      • The links are accesible only through the VPN. Eric will create temporary VPN usernames for TST members to access the lab through the VPN.

    • Lab diagram is updated according to the feedbacks from TST.

    • Jokain presented the OLT-ONU diagram, and gave status.

    • Abhilash will sahre the new FW for Radisys OLT.

    • M1.3.2 and M1.4 is in progress

    • TST to review and approve the documentation for M1.3.2

    • M1.4

      • Radisys OLT 1600x is up and running, Jan making changes to the test suite to point to ROLT

      • Adtran OLT , still work in progress.

      • Zyxel OLT , work in progress.

    • Access to the Lab needs to be discussed on who has to be given access to.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

      • periodic-voltha-multiple-olts-openonu-go-test-bbsim , we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • periodic-voltha-openonu-go-test-bbsim, we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • We have found the issue in the above test suites, now to verify the fix how do we run these test on demand ? like if we have to test out a fix in voltha components or in voltha-system-tests ?

      • voltha-system-tests fix merged.

      • We should able to test it up this week probably.

      • A branch is requested from Eric. Abhilash and Praneeth are on this.

      • Eric to give permissions to create a branch on the voltha-system-tests repo. Abhilash to rerun the suite with the private branch

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

  • Proto changes to offload PPPoe and DHCP RA apps. MR: https://gerrit.opencord.org/c/voltha-protos/+/35278

    • There are comments on the patch to be answered.

    • Merge conflict needs to be resolved

    • Amit is working on the comments, will be addressed by this week.

  • Release 12 in Jira hasn’t been released yet.

    • Eric will give permission to TST to release.

    • Abhilash to create a ticket for this.

  • VGC build pipelines to be setup.

    • Changes done , review pending.

  • The AWS is running very old Ubuntu 1.18 and Kubernetes versions,

    • Abhilash and Mahir to sync up on this.

2024-11-12

Recorded Meeting: 

Agenda:

  • BISDN status

    • ONU, OLT diagram and other cabling diagrams presented. TST members to review.

    • BISDN requested permission for certain repos for +2 and +verify. A Task is created for this. Mahir to create another for removing the permissions after the work is finished.

    • TST to check the documents that are prepared by BISDN.

    • Radisys OLT currently have problems. Praneeth is on this.

    • Everything needed from LF for Netbox setup is ready, reported by Eric.

    • Eric will grant TST members access to all the links in the documents.

      • The links are accesible only through the VPN. Eric will create temporary VPN usernames for TST members to access the lab through the VPN.

    • Lab diagram is updated according to the feedbacks from TST.

    • Jokain presented the OLT-ONU diagram, and gave status.

    • Abhilash will sahre the new FW for Radisys OLT.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

      • periodic-voltha-multiple-olts-openonu-go-test-bbsim , we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • periodic-voltha-openonu-go-test-bbsim, we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • We have found the issue in the above test suites, now to verify the fix how do we run these test on demand ? like if we have to test out a fix in voltha components or in voltha-system-tests ?

      • voltha-system-tests fix merged.

      • We should able to test it up this week probably.

      • A branch is requested from Eric. Abhilash and Praneeth are on this.

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

  • Proto changes to offload PPPoe and DHCP RA apps. MR: https://gerrit.opencord.org/c/voltha-protos/+/35278

    • There are comments on the patch to be answered.

    • Merge conflict needs to be resolved

  • Release 12 in Jira hasn’t been released yet.

    • Eric will give permission to TST to release.

  • VGC build pipelines to be setup

2024-11-5

Recorded Meeting: 

Agenda:

  • BISDN status

    • TST to check the documents that are prepared by BISDN.

    • Eric will grant TST members access to all the links in the documents.

      • The links are accesible only through the VPN. Eric will create temporary VPN usernames for TST members to access the lab through the VPN.

    • Lab diagram is updated according to the feedbacks from TST.

    • Jokain presented the OLT-ONU diagram, and gave status.

    • Abhilash will sahre the new FW for Radisys OLT.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

      • periodic-voltha-multiple-olts-openonu-go-test-bbsim , we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • periodic-voltha-openonu-go-test-bbsim, we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • We have found the issue in the above test suites, now to verify the fix how do we run these test on demand ? like if we have to test out a fix in voltha components or in voltha-system-tests ?

      • voltha-system-tests fix merged.

      • We should able to test it up this week probably.

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

  • Proto changes to offload PPPoe and DHCP RA apps. MR: https://gerrit.opencord.org/c/voltha-protos/+/35278

    • There are comments on the patch to be answered.

    • Merge conflict needs to be resolved

  • Release 12 in Jira hasn’t been released yet.

    • Eric will give permission to TST to release.

  • VGC build pipelines to be setup

 

2024-10-29

Recorded Meeting: here

Agenda:

2024-10-22

Recorded Meeting: here

Agenda:

  • BISDN status

    • Jan and Eric to sync up on the VPN

    • Demo on the work done on M1.1, M1.2 and M131 next TST call.

      • These works are presented by Jan. The document are under the “Community Lab”.

      • TST to check the documents

      • Eric will grant TST members access to all the links in the documents.

    • The Netbox to be hosted in LF (servers/VMs).

    • Which Pod and which OLTs should be used? Should we use Radisys OLT in pod-1 instead of Edgecore?

      • There is a Radisys GPON OLT ready. We can use it.

      • The OLTs will be in Berlin by End of Next week.

      • Bjoern informed that the OLT is received and will be handed over to Jan.

  • How can we check and accept BISDN’s work?

    • Acceptance document/checklist preparation

    • The document is presneted by Mahir Gunyel . link here

  • VOLTHA Release 2.13 preparations.

    • Target dates, pre-conditions, VGC readiness.

      • VGC will be ready by the end of the year.

      • Test pipeline for VGC also will be ready.

      • We will proceed with ONOS and VGC together for this release.

      • Let’s check HW test at mid-Nov. Then decide again if we will go without HW tests or not.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

      • periodic-voltha-multiple-olts-openonu-go-test-bbsim , we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • periodic-voltha-openonu-go-test-bbsim, we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • We have found the issue in the above test suites, now to verify the fix how do we run these test on demand ? like if we have to test out a fix in voltha components or in voltha-system-tests ?

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

2024-10-15

Recorded Meeting: here

Agenda:

  • BISDN status

    • Where will BISDN start? Action plan

      • Jan will check if the contract is received.

      • First will install the HW provided

      • Fix the GW and then install the infra.

      • Jan will prepare a plan for the meeting TST for the next steps.

        • Wokring on milestones M1.1 and M1.2

        • Jan and Eric to sync up on the VPN

        • Demo on the work done on M1.1, M1.2 and M131 next TST call.

        • The Netbox to be hosted in LF (servers/VMs).

    • Which Pod and which OLTs should be used? Should we use Radisys OLT in pod-1 instead of Edgecore?

      • There is a Radisys GPON OLT ready. We can use it.

      • This can be done after the existing HW checked by BISDN

      • The OLTs will be in berlin by End of Next week.

  • How can we check and accept BISDN’s work?

    • Acceptance document/checklist preparation

    • TST will put the SoW items with the acceptance criterias to a checklist document.

    • Mahir will come up with acceptance criterias.

  • VOLTHA Release 2.13 preparations.

    • Target dates, pre-conditions, VGC readiness.

      • VGC will be ready by the end of the year.

      • Test pipeline for VGC also will be ready.

      • We will proceed with ONOS and VGC together for this release.

      • Let’s check HW test at mid-Nov. Then decide again if we will go without HW tests or not.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

      • periodic-voltha-multiple-olts-openonu-go-test-bbsim , we have found out the RCA, will share the fixes ( in test suites or in the application code).

      • periodic-voltha-openonu-go-test-bbsim, we have found out the RCA, will share the fixes ( in test suites or in the application code).

  • Update voltha-docker-tools with new versions of the tools

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

2024-10-01

Recorded Meeting: here

Agenda:

  •  

  • BISDN status

    • Where will BISDN start? Action plan

      • Jan will check if the contract is received.

      • First will install the HW provided

      • Fix the GW and then install the infra.

      • Jan will prepare a plan for the meeting TST for the next steps.

    • Which Pod and which OLTs should be used? Should we use Radisys OLT in pod-1 instead of Edgecore?

      • There is a Radisys GPON OLT ready. We can use it.

      • This can be done after the existing HW checked by BISDN

  • How can we check and accept BISDN’s work?

    • Acceptance document/checklist preparation

    • TST will put the SoW items with the acceptance criterias to a checklist document.

  • VOLTHA Release 2.13 preparations.

    • Target dates, pre-conditions, VGC readiness.

      • VGC will be ready by the end of the year.

      • Test pipeline for VGC also will be ready.

      • We will proceed with ONOS and VGC together for this release.

      • Let’s check HW test at mid-Nov. Then decide again if we will go without HW tests or not.

  • Test suites which needs to be in green so that we allow merges to start

    • Abhilash will check the onu-adapter-go test pipeline to solve the problem. Then we can start the merges again.

  • Update voltha-docker-tools with new versions of the tools

  • Abhilash will come with a document next meeting for openonu-adapter memory consumption, optimizing the memory usage.

  • Abhilash will come with a document next meeting for onu-adapter to check onu config in onu-adapter restarts.

  • Because of the NetworkX, the meeting next week will be canceled and the next meeting will be the week after.

2024-09-24

Recorded Meeting: here

Agenda:

2024-09-10

Recorded Meeting: here

Agenda:

  • Announcement:

    • Abhilash Laxmeshwar and Mahir Gunyel are appointed as the new TST co-chairs by the Board.

  • Status on Amazon Machine Image(AMI) problem

    • Eric has done with this problem. The existing problems are now about the sw and the tests.

    • We are using the ubuntu18.04 image.

    • Eric will take the owner of the patches belong to Joey.

  • Testing patch for Robot Framework update.

    • Eric will create a branch with the name “python312_updates” on voltha-system-tests. And will share the link on slack.

  • Any update on editing SEBA reference design document (Michelle)

    • Michele is creating the doc version of the last version.

  • Abhilash requested review for some patches on slack.

  •  

2024-09-03

Recorded Meeting: here

Agenda:

  • Status on Amazon Machine Image(AMI) problem(Joey, Eric)

    • Review on pipelines. Passing and failing pipes.

    • A common error : “voltctl: command not found"

    • Another one with TLS error: “

      12:52:26 [91mwget: note: TLS certificate validation not implemented 12:52:26 [0m[91mwget: TLS error from peer (alert code 80): 80 12:52:26 [0m[91mwget: error getting response: Connection reset by peer 12:52:26 [0mThe command '/bin/sh -c wget -q -O /usr/bin/voltctl.tmp https://github.com/opencord/voltctl/releases/download/v1.8.45/voltctl-1.8.45-linux-amd64' returned a non-zero code: 1

      "

  • Given that the verification jobs have started to succeed, everyone should resume code reviews.

    • Review but do not merge. Wait for the problems with the test pipes to be solved.

  • Status on SEBA reference design edit options(Michelle)

    • Who edits and who reviews?

    • How is it published and released?

  •  

2024-08-27

Recorded Meeting: here

Agenda:

  • Amazon Machine Image(AMI) problem(Joey)

    • Eric also working on the problem

    • Patch upstreaming problems

    • Working on two AMI images.

    • Some pipes are running now.

  • Python and Robot Framework version upgrade

    • We will check the failing pipes and see if they succeed with the ubuntu image prepared by Eric.

  • Status on LF release engineering team’s SoW document?(Joey,  Michelle)

  • SEBA reference design edit options(Michelle)

    • The options listed by Michele:

      1. The Broadband technical charter has a clause that says the TST can publish documentation under the Creative Commons Attribution License 4.0. As I understand it, this includes copyright but no patent coverage. This can be set up and executed quite quickly. 

      2. If patent coverage is needed, the next best option would be a community specification license. We would set up a working group who would do the updates in github, and folks would have 45 days to review and ask for exclusions. This includes copyright and patent license, and would be relatively easy to spin up. 

      3. The third option would be a Joint Development Foundation Project. This is a more formal process that requires extensive legal review from everyone involved, additional membership agreements signed, and most likely additional funds for PM resources spent. 

    • First option seems more reasonable.

    • Who will be the reviewers then? According to the licence in option 1, TST can review and publish.

    • Michelle to check its mechanics.

 

2024-08-20

Recorded Meeting:  here

Agenda:

  • Wiki page access problem is solved.

  • A scope is defined by the TST and the Board for 3rd party companies to get the Berlin lab operational. Work items are defined as below. BISDN requested to clarify the acceptance criterias.

    • Lab BoM, inventory and network diagram with deployment design (IP addresses, VLANs, Ports, Rules etc…) is documented and completed

    • Installation, startup, shutdown, recovery processes are documented, stable and repeatable

    • Any missing or replacement parts are procured, installed, integrated

    • Reference OLT model and ONU models (defined by TST) upgraded and fully functional

    • Workflow testing for DT activated and running

    • Workflow testing for TT activated and running

    • Test pipelines for nightly testing activated and running

    • Test pipelines for upstreamed patches activated and running

    • Connectivity between lab and AWS Infrastructure (for CI/CD and other tools) configured(if needed) and documented.

    • DNS, DHCP and NTP services installed on a infra host, automated and documented

    • ONOS controller that is managing the aggregation switch is installed on the same infra host and documented

    • The aggregation switch is upgraded and its configuration is documented

Previously defined acceptance criterias :

  •  

    • Documentation of the infrastructure should be accepted by the TST

    • Pipelines should be green

    • When a test is triggered from pushing of a patchset the tests should be executed successfully.

    • Percolate relevant changes made in the Physical infrastructure to LF so that it is replicated in the simulated environment

    • Updated/configured list of tools, libraries and software/firmware should be documented and demonstrated to the TST

  • Amazon Machine Image(AMI) problem(Joey)

  • Status on LF release engineering team’s SoW document?(Joey,  Michelle)

  • SEBA reference design edits. How to edit, what should be the correct process(Michelle)

    • This is a placeholder to follow at the next meetings (after the board meeting)

2024-08-13

Recorded Meeting: here

  • Problem accessing wiki page. Unable to access meeting minutes.(Michelle)

    • How to separate the tickets in Jira.

    • Tickets need to be assigned to Eric(ones that are assigned to Joey).

    • There is a plan to open an LF BB Jira and merge the existing to the new one.

  • Amazon Machine Image(AMI) problem(Joey)

    • https://jira.opencord.org/browse/VOL-5352

    • Need to upgrade the ubuntu version(20.04) to resolve system problmes introduced by upgrading python

      • 20.04 includes a newer version of python v3.8.2 and golang.

    • Can anyone from LF take this over?

  • Status on LF release engineering team’s SoW document?(Joey,  Michelle)

    • Joey is trying to update release documentation and fix AMI image problem.

    • Haven’t written up a checklist yet.

  • Go controller status(Praneeth)

    • DT wf verified, TT needs to be done

    • Waiting for the new image including golang version upgrade

    • Need to prepare pipelines for vgc.

  • SEBA reference design edits. How to edit, what should be the correct process(Michelle)

    • Michelle prepares a slide for the Board to discuss.

2024-08-06

Recorded Meeting: here

  • Problem accessing wiki page. Unable to access meeting minutes.

  • Amazon Machine Image(AMI) problem

  • What is the workflow between the LF release engineering team and the community?

    • What are the tasks that will be covered by the LF release engineering team?

    • Who are the people related? Is there a dedicated resource? Will they join the TST meetings?

    • Do we assign tasks to them? Do they have VOLTHA Jira accounts?

  • Jira Tasks review

  • Things to be done with Joey until the end of the month:

    • Release documentation

    • Responsibilities/tasks categorization.

    • Jira task categorization. Which ones to LF Release team, which ones to community

    • Gerrit patches review.

    • Handover the access rights of the infra to LF(Jenkins, gerrit, AWS etc)

    • Handover the access rights of the Lab to LF.

  • Go controller

    • What is the current status

  • SEBA reference design edits - what is the process to edit RD?

    • Meeting tomorrow to find answers

    • Where is the editable version of the document served?

2024-07-30

Recorded Meeting: 2024-07-??

Announcements:

  • [michelle] - 2024 TST election results:

    • I am pleased to announce the results of the 2024 VOLTHA TST election.

    • The elected nominees are Mahir Gunyel (Netsia) and Praneeth Nalmas (Radisys).

    • Praneeth has been added as a TST member:

      • Granted +2 patch approval attribute.

      • Can now access the voltha-tst slack channel,

  • [joey] - on vacation next week, can someone emcee the TST meeting 2024-08-06 ?

    • Moving forward from August 1, TST Chairs are responsible for TST meeting agendas

  • [joey] - Atlassian cloud migration is underway (completed?):

  • Reminder: New project mailing list(s) and wiki/TST Agenda are being used:

    • See 2024-07-23 agenda for details.

    • Links and mailing lists can also be found at the top of this page.

 

Items:

 

Updates:

  • VPN / colocation:

    • A new tunnel has been setup with jenkins traffic routed through it to the Berlin lab.

    • COLO network interfaces for VPN(s) have been taken down to block traffic flow.

    • Spot checking for any lingering peripheral dependencies, the new tunnel is live.

  • Jenkins:

    • Jenkins pipelines are being updated to use the new/replacement AMI image.

    • Pipeline Job Status

    • System resources per-job will need to revisit later, resize resources.

      • 1c-1g, 1c-2g, 2c-4g, 4c-8g, 8c-15g  (c-processors, g-ram)

      • Current image (label:voltha-1804-micro) allocated as 4c-8g.

      • A new system image with each resourcepair above will be created from the template AMI.

    • Change jenkins node allocation (EC2) from static to dynamic.

  • Documentation / Release:

    • Also working on release items between AMI evaluation builds.

    • Reviewing and updating release content in docs.voltha.org and for LF (release engineering wiki).

 

The Floor is Open for discussion:

  • [todo] - Diagnose repo:voltha-protos hang/timeout once docker permission problems are resolved.

  • [serkant] - HOWTO: Seba document editing.  Michelle will check on it.

2024-07-23

Recorded Meeting: 2024-07-24

 

Announcements:

Items:

  • [Serkant] -  Activities and updates from the LF-BBF tiger team.

    • Topic: alignment points between SEBA and LF-BBF

      • Two interface points can be aligned between SEBA and the cloud (TR-411)

    • Topic: 

    • Old ONF designs went through a release process.

    • Michelle will check on the process at LF.

    • Question: Is there still a reference design team ?

      • ONF: reviewed by company & community members.

      • Who should have input on the SEBA refrence design ?

  • [Serkant] -  (email thread) In addition, I'd appreciate some information on how to incorporate future LF-

Updates:

  • [joey] - Re-create AMI image (jenkins nodes):

    • Status tracking page on github

    • A new AMI/image was created with more processors, ram and disk.

      • Disk size for prototype image was small and filled quickly by most repositories  

    • Most interpreters, tools and versions used by v2.12 have been restored:

      • helm and a few other tools (jq, etc) are being updated to the latest version.

    • voltha-protos:

      • manual builds succeed

      • jenkins builds: gerrit:35288  job: 664 – dial unix /var/run/docker.sock: connect: permission denied"

        • Failure is not due to the common problem of unix group membership.

        • If it were building locally would fail for the same reason.

    • voltha-lib-go:

      • jenkin builds: gerrit:35288  job: 664 – permission denied, I/O on mounted directories.

        • manual build & test succeeds

    • voltha-docs: gerrit:35291

      • linkcheck and linting problems to cleanup.

    • voltha-openolt-adapter: gerrit:35071, job: 1986

 

The Floor is Open for discussion:

  • [michelle] - TST election input is still needed, submit your nominees by tomorrow.

  • [manuel] - VPN status ?

  • Lab documentation requests ?

 

Misc:

 

2024-07-16

  • Announcements / Requests:

  • Updates:

    • AWS AMI/Jenkins Nodes

  •  

    •  

      • Repo:voltha-protos and repo:voltha-lib-go built and tested cleanly.

      • Building up a clean inventory of tools, interpreters, packages and versions.

      • Initial AMI was created based on the free aws.tiny model (1cpu, 4gb).

      • Two additional AMIs need to be created with additional resources to avoid high loads causing severe thrashing

      • Still need to add the re-created AMI to jenkins so pipelines can start launching jobs agin.

  • Items:

  •  

    •  

      • An effort is underway to migrate some of the jenkins pipeline into github actions.

      • No major change in functionality will be visible:

        • GA can function as a dispatcher launching commands in place of JJB.

        • Literally jenkins pipeline code can be simplified, GA would simply invoke make targets directly.

      • Benefits:

        • Future proof logic, all LF projects are migrating to Github Actions.

        • Actions can be performed multiple times with multiple triggers.  Linting / pre-commit hook can be performed on checkins more easily than creating or augmenting JJB pipelines.

        • Python upgrade: using GA will help isolate python virtual machine use,  Upgrading python for pipeline use will no longer introduce an implicit danger for jenkinsh.  JJB and repo:voltha-system-tests will use distinct virtual machines.python interpreters so there will be little danger of breaking jenkins while attempting to upgrade python for the testing.

        • Most importantly: LF Releng involvement.

          • More people available to work on jenkins & build problems.

          • Releng has established workflows used by many projects that can help reduce dependency and complexity added by JJB.

  • Atlassian Server Cloud Migration slides

  • Project documentation for LF Release Engineering

    • Actively creating cookbooks and docs.voltha.org links for people to use.

    • Reference material will also be helpful during handoff in August.

  • The Floor is Open for discussion:

    • [serkant]  - summary of items and a few questions.

    • [michelle] - Upcoming TST elections:

      • Please review governance & charters

      • Upcoming elections, a notice will need to be sent out soon.

      • Suggest nominations.

 

 

2024-07-09 - Cancelled

 

 

2024-07-02 [Amit] covering

  • Docker pipeline - Joey working on it

Announcements / Requests:

  • [joey] Question: can someone emcee the TST meeting on 07-09 ?

Updates:

[joey] - Jenkins – recreate AMI for job nodes:

  •  

    • VOL-5352 - AWS AMI recreation:

    •  

      • This ticket documents current status.

      • See subtasks attached to this story, assign & work on any of interest.

      • Build & test problems, compiler warnings, etc encountered during image testing / clean rebuilds / detailed log review.

    • A new/clean AMI created with former interpreter & tool versions created.

    • Successful manual build & tested:

    •  

      • repo:voltha-protos

      • repo:voltha-lib-go

    • 06/28 - replacing pipeline jobs with the new AMI to restore builds.

    • olt-agent build

    •  

      • Manual build induced thrashing/heavy swap, machine unresponsive.

      • Prototype AMI is aws type micro, limited resources (1cpu, 2-4gb)

      • Creating another AMI based on a larger resource model

[joey] - LF will help with VPN migration:

Items:

  • [joey] - V13 release target

wiki.lfbroadband.org :: Release Tracking

  •  

    • Code Freeze date is yet to be finalized

    • The date cannot be finalized unless the problems seen currently in the release process are sorted out.

The Floor is Open for discussion:

 

 

2024-06-25

Announcements:

  • [joey] - LF will be migrating Atlassian services (confluence, jira, etc) in bulk from data center to cloud.  Slides and material for projects will be provided later.

  •  

Updates:

[joey] - Jenkins – recreate AMI for job nodes:[joey] - LF will help with VPN migration:

Items:

  • [joey] - V13 release target

wiki.lfbroadband.org :: Release Tracking

The Floor is Open for discussion:

 

 

2024-06-18

Announcements:

[joey] - Taking vacation 1st week in July

  •  

    • Can someone cover the TST meeting on 2nd and 9th ?

    • [Amit] can cover on the 2nd.

Updates:

[joey] - Jenkins – recreate AMI for job nodes:[joey] - LF will help with VPN migration:

Items:

  • [joey] - V13 release target

wiki.lfbroadband.org :: Release Tracking

The Floor is Open for discussion:

 

2024-06-11

Announcements:

  • CFP for OCP Global Summit: https://www.opencompute.org/summit/global-summit

  •  

    • October 15–17, 2024  San Jose, California

    • OCP: Open Compute Project:

  • [joey] - git commit –signoff  (LF Required)

    • Create an ed25519 GPG signing key (gpg2 –expert –full-key-gen).

    • Add key to .gitconfig (gpgsigningkey = {xxxxxxxxx}! – note bang(!) suffix)

    • git commit –signoff –gpg-sign

  •  

    • See also: TST meeting notes from 2024-04-09

Updates:

[joey] - Jenkins – recreate AMI for job nodes:

  •  

    • No status update

[joey] - LF will help with VPN migration:

  •  

    • Admin/login access added

Items:

    [joey] - V13 release target

Create a list of pending items for the next release.A release date is needed, Dec may be too far off.AMI image problem, VGC/golang upgrade, transient problems, etc

The Floor is Open for discussion:

 

2024-06-04

Announcements:

Updates:

[joey] - Jenkins – recreate AMI for job nodes (WIP)[joey] - LF will help with VPN migration:

  •  

    • TODO: grant admin and remote access.

Items:

The Floor is Open for discussion:

 

2024-05-28

Announcements:

Updates:

Jira Tickets: Patch grid, Transient test failures

Code Reviews: gerrit?status=open

REPO:onf-scripts : gerrit-search, jenkins-search, jira-search

The Floor is Open for discussion:

[joey] Question: .vscode/ in repositories

  • Gerrit:35248 - voltha-lib/go/.vscode/launch.json checked in (circa: 2021):

    • Will this freeze editor behavior until updated ?

    • If .vscode/ is needed a new git repository can be created to support a common core set of options then cloned as a submodule into all votlha repositories.

  •  

    • Personal editor preferences ?

    • Maintained (?) configs contain hardcoded version strings.

    • Single use case or is vscode in widespread use ?

    • Thoughts ?  Is .vscode needed and should a central config be maintained ?

    • [mahir] - .vscode can be removed.

VPN update

 

2024-05-21

Announcements:

New public calendar for meetings:

New Broadband/VOLTHA wiki created

AWS AMI mishap last week: Amazon Machine Image

  • An AMI was inadvertently deleted during aws cleanup.

  • This ami is the base image for all running VOLTHA jobs.

  • In the interim jobs can report two different messages/states:

  •  

    • All nodes of label ‘ubuntu18.04-basebuild-4c-8g’ are offline

    • Patch status: no build

  • We are working to recreate the missing AMI.

Updates:

  • Colo deprecation / VPN relocation

  •  

    • TSTs presented to the board and a 30 day extension has been approved.

Jira Tickets: Patch grid, Transient test failures

Open Code Reviews: gerrit?status=open

The Floor is Open for discussion:

[mahir] - Reminder: VERSION file needs to be edited with each patch submission.


2024-05-14

Announcements:

New public calendar for meetings: https://zoom-lfx.platform.linuxfoundation.org/meetings/broadband-fund

VPN discussion

  • Currently hosted in ONF colosite (CoreSite), co-located (and paid for) with Aether and P4 equipment

  • Aether and P4 pulled or turned off equipment, and will not pay for colo moving forward

  • VPN still needs to be reverse-engineered

  • Options: 

  •  

    • Ask LF BB governing board to approve funds to continue paying for colo until VPN issue can be resolved (~$2,500 / month)

    • Determine if VPN issue can be resolved within 30 days (required notice to turn colo site off)

    • Turn off VPN and lose all hardware tests (and possibly other things that are currently unknown)

  • Next steps: Amit/Mahir will bring to board on Thursday 

[serkant] - edits needed in bbsim to support testing

Implement logic in repo:voltha-go-controller, merge patches.Periodic updates during TST meeting (?)Are new unit tests needed to test this feature (~bbsim):[Serkant] investigation into bbsim enhancements.Update: BBSIM assumes only one NNI port exists in the OLT. All of the NNI up/down notifications and uplink messages sent are based on this assumption. Changes need to be made.

[joey] - jenkins.opencord

  • Job queue is backed up and long this morning.

  • Status: All nodes of label ‘ubuntu18.04-base*’ are offline

  • Researching

Updates:

  • [joey] - Triage failing patches - commit message(s) decorated with jira tickets.

Jira Tickets: Patch grid, Transient test failures

VOL-5331

Test Perform BBSim Sanity :: Validates the BBSim Functionality for

VOL-5331

Voltha DT PODTests :: Test various end-to-end scenarios

VOL-5335

Voltha TT PODTests :: Test various functional end-to-end scenarios for TT

VOL-5346

Voltha-go-controller timeout (10 min)

VOL-5347

Repo:voltha-protos timeout (20 min)

  • Individual tickets have affected gerrit patches linked to them.

Repository: voltha-go-controller

  • Gerrit: 35217, 35267

    • Abhilash has done some builds locally

    •  

      • No code complexity issues contributing to long runtime.

      • Local builds not affected, problem appears to manifest in batch.

  •  

    • VOL-5346 - 10 minute timeout during linting

Repository: voltha-protos

  • Gerrit: 35268, 34992, 35128

  •  

    • Build timed out (after 20 minutes). Marking the build as failed.

    • Verbosity and instrumentation are being added to the makefile.

Open Code Reviews: gerrit?status=open

Patch

Repo

Notes

 

 

 

 

 

 

The Floor is Open for discussion:

 

2024-05-07

Announcements:

Submitting Patches: docs.voltha.org

  • All patches require a Jira Ticket in the Commit Message.

  • All patches should increment VERSION file

  • Update Copyright Notice whenever sources are modified:

  •  

    • Date spans last year modified (NOTE: *-present not valid).

    • Also include ‘SPDX’ tokens for linting project licenses:

# SPDX-FileCopyrightText: 2017-2024 Open Networking Foundation Contributors

# SPDX-License-Identifier: Apache-2.0

Items:

Pipelines have been affected by transient problems lately.Timeouts, end-2-end connectivity tests, etc.

[serkant] - edits needed in bbsim to support testing

Implement logic in repo:voltha-go-controller, merge patches.Periodic updates during TST meeting (?)Are new unit tests needed to test this feature (~bbsim):[Serkant] investigation into bbsim enhancements.Update: BBSIM assumes only one NNI port exists in the OLT. All of the NNI up/down notifications and uplink messages sent are based on this assumption. Changes need to be made.

Updates:

  • Jenkins / JJB SW upgrades: temporarily on hold due to peripheral failures.

  • Repo:todo - Grid created to track open patches, common/transient failures.

Deployments

Item

Repos

Installed

Patched

Notes

pre-commit

14

11

2

Linting tool; make pre-commit

install virtualenv, invoke pre-commit

REPO:onf-make

14

5

3

Library makefiles.  Added as a git submodule, lightly used.

Jira Tickets:

Common / Transient failures

  • Repo:todo - Grid created to track open patches, common/transient failures.

  • VOL-5335 - NOP patch, TT E2E ping test failure.

VOL-5305, VOL-5331 - DT PODTests, E2E ping failing

voltha-protos

Under research

Open Reviews: gerrit?status=open

Patch

Repo

Notes

G:35248

voltha-lib-go

Lint cleanups

 

 

 

The Floor is Open for discussion:

 

2024-04-30

Announcements:

Items:

  • [Fridolin] presentation on the support for multiple NNI ports in VOLTHA

  •  

Implement logic in repo:voltha-go-controller, merge patches.Periodic updates during TST meeting (?)Are new unit tests needed to test this feature (~bbsim):[Serkant] investigation into bbsim enhancements.Update: BBSIM assumes only one NNI port exists in the OLT. All of the NNI up/down notifications and uplink messages sent are based on this assumption. Changes need to be made.

Updates:

Jira Tickets:

Golang upgrade / grpc

  • VOL-5222 - Upgrade docker image (~golang) for grpc:   

  •  

    • Gerrit:34870 - golang upgrade indirectly triggered a ‘break the world’ dependency.

    • Jenkins:52 - more packages within the image will need to be updated.

    • Hoping the base alpine image will not upgrade python right now.  That event will prompt updating the robot testing harness across all repos.

voltctl

Open Reviews:

Pre-commit / lint cleanups

  • Gerrit:35248  repo:voltha-lib-go                             lint cleanups for yaml src

  • Gerrit:35254  repo:voltha-docs                              add pre-commit

  • Gerrit:35249  repo:voltha-go-controller                  add pre-commit

  • Gerrit:35255 repo:voltha-openonu-adatpter-go     add pre-commit

  • [Abhilash] % gerrit --wip --message 'olt'

  •  

    • Code reviews needed

    • Patches to be merged after voltha-protos job failure is resolved.

Resources:

Scripts: repo:onf-scripts

  • Gerrit -  gerrit.sh  –wip –before ‘2024-04-24’ –message ‘bbsim’

  •  

pre-commit

  • .pre-commit-config.yaml

  •  

    • Common config being deployed to all repos.

    • Latest hooks and versions frozen for release.

    • Contains a list of directory exclusions (.venv, makefiles, lf)

  • copyright notice - Include SPDX tokens and date range spans 2024

docs.votha.org Submitting Code

The Floor is Open for discussion:

 

2024-04-23

Announcements:

  • Fridolin(s) will be giving a presentation next week on support for Multiple NNI ports.

Updates:

Items:

  • VOL-5306 - port forwarding VS ingress controller:

  •  

    • Effort needed to complete transition to ingress controllers.

    • Ingress replaced port forwarding in VOLTHA proper to improve reliability.

    • Forwarding and kill logic still exists in repositories.  Inventory to determine if unused logic can be removed and finish transition for lingering dependencies.

  • Jenkins:1148 - repo:voltha-helm-charts

  •  

    • volthaStackDeploy timeout resurfaced

Jira Tickets:

Bbsim - test failures unrelated to patch

Test Perform BBSim Sanity :: Validates the BBSim Functionality for... | FAIL |

  •  

    • No match found for 'dhcpl2relay-remove-allocation of:00000a0a0a0a0a0a 256'

  • VOL-5311

  •  

    • Gerrit:35225 - yaml lint cleanups

    • ERROR: Keyword 'Verify Subscriber Access Flows Added For ONU DT' failed

  • VOL-5313 - Build timed out (after 20 minutes). Marking the build as failed

  •  

    • Gerrit:35182 - no logic change, simply adding a git submodule.

  • gerrit: 35225 - yamllint cleanups

  •  

    • ERROR: Keyword 'Verify Subscriber Access Flows Added For ONU' failed

  • NOTE: “Verify Subscriber Access” failed jobs in several different repos.

repo:voltha-go-controller

  • VOL-5309 - Makefile edits needed.

  • VOL-5318 - Cryptic failure in a NOP (copyright update) patch.

repo:voltctl

  • VOL-5314

  •  

    • Gerrit:35198 - simple VERSION file update.

    • ERROR: Keyword 'Verify Subscriber Access Flows Added For ONU' failed

Open Code Reviews: 

The Floor is Open for discussion:

 

2024-04-16

Announcements:

  • VOLTHA v2.12 Techinar

  •  

    • Thank you: Amit, Mahir & Abhilash !

  • Some jobs are failing due to AWS nodes being unavailable:

  •  

    • Jobs launch and nodes startup but eventually fail due to a missing key.

    • Problem seems related to longer startup time.

    • Timeout limit to be increased temporarily.

    • Debugging ongoing.

  • Pre-commit - A framework for managing pre-commit hooks

  •  

  • Very nice tool for linting tasks (source, licenses, etc)

    • [bbsim] git clone bbsim

    • cd bbsim

    • make venv # install python virtualenv w/modules

    • source .venv/bin/activate

    • Pre-commit

  •  

    • Gerrit:35218 - Deployed to a few repositories (WIP: repo:voltha-go)

Updates:

Items:

  • [Amit] We need to add in the agenda for the 2024-04-30 TST meeting a presentation from Fridolin on the support for multiple NNI ports in VOLTHA that he has worked on:

  •  

    • VOL-5255 - Use all uplink interfaces not only interface 0 at OLT

    • Repo:voltha-protos - discuss failures.

  • Pre-commit - A framework for managing pre-commit hooks

  •  

  • Very nice tool for linting tasks (source, licenses, etc)

    • [bbsim] git clone bbsim

    • cd bbsim

    • make venv # install python virtualenv w/modules

    • source .venv/bin/activate

    • Pre-commit

  •  

    • Gerrit:35218 - Deployed to a few repositories (WIP: repo:voltha-go)

PSA:

Jira Tickets:

VOL-5300 - (story) repo:voltha-go report of stale content (*.yaml) in voltha deploy.

Open Reviews [Abhilash]

On-demand diagnosis for metrics being addedWould like to close the ticket this week.https://jira.opencord.org/browse/VOL-5291https://gerrit.opencord.org/c/voltha-protos/+/35128https://gerrit.opencord.org/c/voltha-openolt-adapter/+/35173https://gerrit.opencord.org/c/openolt/+/35195https://gerrit.opencord.org/c/voltctl/+/35152https://jira.opencord.org/browse/VOL-5305https://gerrit.opencord.org/c/voltctl/+/35216

The Floor is Open for discussion:

  • Pipelines are not running for most of the repos.

    • Likely related to the aws node startup problem

  • Resume work on getting VGC release as part for next release

    • On joey(s) todo list and aware of it.

 

2024-04-09

Announcements:

Updates:

Linux Foundation

  • Gather monthly AWS billing and usage for the project.

  • VPN transition work for the Berlin lab ongoing.

  • JJB Upgrades are ongoing:

  •  

    • Blocker: lftools upgrade dropped support for older python interpreter versions.

    • Interpreter will need to be upgraded to continue JJB upgrades.

    • Python upgrades will affect robot testing and repo:voltha-docs

Items:

  • Jenkins

  •  

    • Unused job views have been deleted.

  • VOL-5296 - Publish voltctl version v1.11.2   (gerrit:35192)

  •  

    • Github signing key for releasing expired, re-generate and re-publish.

    • Jenkins: publish task ran and failed after patch merge and ‘job success’.

    • Cannot ‘recheck’ to trigger a side effect of publishing the artifact.

    • Bump VERSION string to trigger a clean publishing job.

PSA(s):

  • VOL-5296 - voltctl release failure: copy & paste jenkins log text

  •  

    • Pasting jenkins logs into jira

    • A simple paste of log text into Description strips newlines and formatting.

    • Use the [+v] menu, option preformat then paste.

  • Consider DCO and signing commits:

    • Asserts project copyright/license covers the current patch.

    • Can be signed from gpg or ssh-keygen

    • Validation that you created the patch.

    • [user]

    •  

      • email =

      • name =

      • gpgsigningkey = xxxxxxxxx!    # Note bang(!) suffix

    • [alias]

    •  

      • ci   =  checkin

      • co  = checkout

      • cs = commit --signoff —gpg-sign

      • st = status

      • unpushed = diff --name-only origin/master..

    • [commit]

    •  

      • gpgsign = true

  •  

    • DCO - Developer certificate of origin  ( % git commit –signoff ).

    • Signing Commits  (% git commit –gpg-sign)

    • ~/.gitconfig

  • Alias Usage:

    • Sign a commit, checkin message read from disk.

    • Include all locally modified files in patch.

    • Checkin edits with commit message prompting.

  •  

    • git cs –F ../my-checkin-message file[ .. file(s) ]

    • git add –all && git ci

[joey] - Refresh bbsim problem list and repos.

Code Review requests:

Voltha.docs:

The Floor is Open for discussion:

 

2024-04-02 - cancelled

 

2024-03-26

Announcements:

  • V2.12 techinar to be scheduled:

  •  

    • [amit] - proposed week of 2024-04-08

Updates:

JJB:jenkins job builder

  • Global-jjb upgrade to v0.67.0

    • Add makefile target lint-bashate

    • Update source so linting tools run successfully.

  •  

    • Commit hook added: bashate

    • Similar to shellcheck but enabled for all repos by default.

    • No fallout from upgrade, linting limited to modified source in a patch.

    • TODO:

  • Suggestion: Create 2 patches:

    • docs.voltha.org: make lint-{source} targets:

    •  

      • % make help | grep lint

      • % make lint lint-shell lint-yaml

    • NOTE: Linting tools generally support exclusions.  Cleanup whatever can be but if edits are excessive individual files can be filtered from checking to allow gradual cleanup.

    • Patch #1 - Cleanup source problems until linting tools run successfully.

    • Review and merge patch containing lint cleanups.

    • Patch #2 - Normal development, edit source to add features or fix bugs.

    • Two distinct logic patches will be easier to review.

    • Logic isolation helps identify the source when jobs fail.

  •  

    • Bulk source cleanup forced by jjb upgrades and future build system answers.

    • When submitting code create two patches:

Items:

Code Reviews:

The Floor is Open for discussion:

  • [Mahir] - checking on bbsim test failures:

    • Likely a hard coded version that needs to be updated for release.

    • Groovy script queries for tags in github not gerrit:

    •  

      • Answer should be consistent across repositories.

      • Gerrit holds the truth, github is a mirror.

      • Identify and cleanup discrepancies.

    • Failing due to a tag discrepancy between the repositories.

    • Timestamp between two counter periods failing:

    •  

      • Expected value of 4 fails on time difference of 5.

    • Problems:

    •  

      • Stale data found in etcd.

      • Port was not removed by ONOS after device delete.

      • Race conditions (?)

    • A lot more tests are running green.

    • We were seeing this behavior last year.

    • BISDN mentioned hardware tests also exhibited behavior at device level

    • Reverted patch was masking other failures in the code base.

    • We should also actively remove ‘|| true’ commands suffixed 

  •  

    • bbsim test failure

    • Gustov(s) patch submitted earlier was backed out.

    • Voltctl: older version in use:

    • Software-upgrade tests:

    • PM tests

    • VOLTHA-2.X-Tests

    • [joey]  Intermittent test failures are not new:

  • JJB (jenkins job builder) is currently unmaintained:

  •  

    • May become a failure point over time due to stagnation or interpreter upgrades force code changes and the repository is no longer being maintained.

    • Gradle may be an option if needed

    • Other packages or systems to consider ?

Voltha.docs:

 

2024-03-19

Announcements:

  • V2.12 techinar to be scheduled:

  •  

    • [amit] - proposed week of 2024-04-08

    • [michelle] - 3 week lead time for scheduling, registration, etc

  • VOLTHA Project Page on youtube:

  •  

    • TST Meeting videos are now uploaded to this channel.

Updates:

Items:

JJB - Jenkins Job Builder

  • global-jjb v0.63.1

  •  

    • Upgrades will active two commit hooks: bashate and yamllint.

    • Bulk cleanup across multiple repositories may be needed.

    • lint targets for local development:

make lint-shell

shellcheck, bashate to be added.

make lint-yaml

yamllint

 

  • Look into tox deployment

  • Lftools - python upgrade needed, affects robotframework.

Robot Test Framework

  • python --version  (testing Python 3.11.6)

  • Repo:voltha-system-tests messages:

  •  

    • [ ERROR ] option --noncritical not recognized

    • [ WARN ] voltha-system-tests/libraries/bbsim.robot' on line 101: The '[Return]' setting is deprecated. Use the 'RETURN' statement instead.

  • [Return] statement will be removed from robotframework:

  •  

    • If it has not already in the latest version.

  • gerrit:35161 Tool robotidy can apply bulk syntax cleanups:

  •  

    • Upgrade robotframework:3.1.2 to at least v4.0

The Floor is Open for discussion:

  • Any VOLTHA features or internals of interest to mention ?

  • [Amit] - Working on slide deck for techinar, sent out for review input:

    • Options: free form meeting.

    • Options: zoom pro meeting shared between projects ?

    • Writeup a short blog on the release notes.

    • Lf-marketing can help after Kubecon

  •  

    • Who would like to present.

    • [Michelle] - waiting for feedback from sales for the event:

Code Reviews:

  • [Abhilash] Gerrit:35128 - OnDemand stats repo:voltha-protos

  • [Joey] Gerrit:35137 - repo:voltha-openolt-adapter Makefile cleanups

 

 

2024-03-12

Announcements:

Updates:

  • Linux Foundation

    • Only visible internally, will need to re-home bbsim and JJB below.

    • A request to create a server will be initiated soon.

  •  

BBSIM test failureshttps://github.com/joey-onf/todo/tree/origin/master/bbsim
JJB Upgradesrepo:global-jjbUpgrade v0.57.0 -> v0.57.3

  •  

    • repo:lf-common-packer

      • Update packer-merge aws jobs to use packer-builder=aws

    •  

      • Upgrade v0.7.0 -> v0.7.6

      • V0.8.0 blocker:

    • lf-ansible

    •  

      • Upgrade to latest changeset (circa: 2018)

    • repo:lf-tools

      • Old python versions deprecated

      • Interpreter upgrade needed

    •  

The Floor is Open for discussion:

  • Any VOLTHA features or internals of interest to mention ?

Code Review Requests:

TODO Items:

  • Download and publish TST youtube video from 2024-03-05

  • Triage jira tickets:

  •  

    • decorate, assign, resolve, etc – cleanup as needed

  • Two scripts/repositories exist able to query for gerrit patches & jira records:

  •  

    • Search by string or tickets decorated with attributes & labels.

    • Assigned to ‘me’

    • Status = [pending | resolved | unresolved]

    • Sharp edges lingering, cleanup and publish needed so queries are available for TST meetings.

  • Testing:

  •  

    • Remove ‘|| true’ from commands and testing

    • Allow commands to fail, debug and resolve underlying issues

  • Vgc

  •  

    • VOL-5220 - new jenkins pipeline for VGC jobs.

    •  golang upgrade – update and validate docker images

  • Python

    • Affects repo:voltha-docs and robot testing harness.

    • Remove custom tarball used to freeze packages.

  •  

    • Upgrade interpreter

  • Makefiles:

  •  

    • Update VOLTHA repositories to use git submodule repo:onf-make

    • Provides consistent target behavior and features.

The Floor is Open for discussion:

  • [abhilash] - [wip] Changes on the way, pointers will be posted in slack

 

 

2024-03-05

Announcements:

  • V2.12 techinar:

    • Mahir will also be attending

  •  

    • Volunteer(s) – Amit will begin working on the V2.12 Techinar

    • Ongoing call for helper elves.

    • Scheduling: within the next 30 days

    • Michelle will coordinate with Amit to set things up:

  • LFID - https://identity.linuxfoundation.org

    • The login is used to access project resources: gerrit, wiki, etc

    • LF email address

  •  

Updates:

BBSIM test failures

  •  

    • Bbsim test failures page in need of visibility, mahir was having problems accessing with his LF ID.  Problem related to personal page access, strange no restrictions for viewing are imposed, maybe blocked at login/credentials level.

JJB UpgradesGlobal-jjb v0.57.0Common-packer: v0.7.0

Items:

Code review requests

The Floor is Open for discussion

  • [Mahir] volunteers needed to diagnose bbsim testing failures.

 

2024-02-27

Announcements:

  • V2.12 techinar:

    • Mahir will also be attending

  •  

    • Volunteer(s) – Amit will begin working on the V2.12 Techinar

    • Ongoing call for helper elves.

    • Scheduling: within the next 30 days

    • Michelle will coordinate with Amit to set things up:

Updates:

  • Meeting problems from last week:

    • Headset and webcam blocked by usbguard (long story)

    • DNSSEC was contributing to a few problems.

    • Not identified as host

    • Devices not selectable

    • Checking for firewalled ports.

  •  

    • New laptop + SE Linux + usbguard:

  • Question asked: was the invite only sent to TST members ?

  •  

    • No - broadcast email sent to voltha-discuss@onf

    • Forwarded to slack:votlha-devs & voltha-discuss@opencord

    • A new voltha-discuss newsgroup will likely be created during the transition.

  • OpenOLT Agent

    • Completely manual process, building, testing and OS packages

    • Build instructions: BUILDING.md

    • Proprietary SDKs are used to build the low level agent.

    • Tedious and complex are the adjectives used to describe the process.

    • Output: debian packages.

    • Some uploads and downloads to vendor sites.

    • In the interim repository will be tagged for v2.12 but artifacts/packages are not yet available.

    • Need to figure out all the moving parts.

    • My wiki writeup for retrieval lost during the website attack.

    • Encountered auth/access problems last download attempt.

    • May need to revisit/debug & document

  •  

    • Background for Olt Adapter build:

    • BAL updated during v2.11 release and a few months ago by BISDN for Berlin:

Items:

  • Linux foundation updates:

    • All gerrit repositories and orgs (opencord.org and onosproject.org) will need to migrate in bulk, migrating individually is not an option.

    • A maintenance window will be scheduled to limit disruption.

    • To reduce potential for problems with the Gerrit migration jenkins server will be updated and transitioned first.

    • Jenkins Job Builder & friends will upgrade from v2.8 to v5 & V6:

    •  

      • We are running a very old version of JJB.

      • Start by upgrading global-jjb, lf-ansible and packer for all pipelines.

  •  

    • Berlin lab: map out VPN for transition ONF colo VPN to LF VPN.

    • Gerrit server migration will be a big job:

    • Jenkins server migration:

  • Bbsim testing failures

  •  

  • Release

  • All repositories are tagged and branched

  •  

    • [TODO] Including openolt-agent

  • Repo:voltha-helm-charts

  •  

    • Chart:voltha-onos upgrade blocked by a hang in volthaStackDeploy

Pending Items:

  • OpenOLT agent - variant hardware builds

  • Testing

  •  

    • Bbsim testing failures

    • Remove ‘|| true’ from all builds and test jobs to flush out corruption sources.

  • Triage old jira tickets

  • Triage jenkins jobs, remove duplicate views and job entries.

  • Update 3rd party tools and software libraries

  •  

    • VOL-5220 - VGC - upgrade golang, docker image, so repository can build.

  • VOL-5283 - Consolidate installers/tool-versions: bbsim, kail, kind, voltctl

Code review requests

The Floor is Open for discussion

  • Voltha-go-controller planned for next release to replace onos.

  • Verify TT dependencies to include

  • Verify VGC dependencies are all working correctly.

  •  

    • Verify VGC is being tested for TT workflows.

 

 

2024-02-20

Announcements:

  • V2.12 techinar:

    • Mahir will also be attending

  •  

    • Volunteer(s) – Amit will begin working on the V2.12 Techinar

    • Ongoing call for helper elves.

    • Scheduling: within the next 30 days

    • Michelle will coordinate with Amit to set things up:

  • Linux Foundation

  •  

  •  

    • LF Slack Channel for github-actions discussion:Updates:

Items:

  • [joey] Question: Release Notes Page

  •  

    • Any preference for layout ?  One page or summary with links ?

Code review requests

The Floor is Open for discussion

  • Followup with Girish for background on manual build steps for openolt agent.  Credentials will also be needed for uploads.

  • Bbsim test triage.

  • Jira tickets – search for tickets with label ‘bbsim’ or ‘Testing’

  • Jenkins views

  • Purge old release views; 2.8, etc

  • Search for duplicate jobs

 

2024-02-13

Announcements:

  • V2.12 techinar:

    • Mahir will also be attending

  •  

    • Volunteer(s) – Amit will begin working on the V2.12 Techinar

    • Ongoing call for helper elves.

    • Scheduling: within the next 30 days

    • Michelle will coordinate with Amit to set things up:

The Linux Foundation: Release Engineering

  • Jessica Gonzalez: Github Actions

    • Walkthrough example: ORAN(s) use of actions.

    • Schedule  n / presentation: 3-sessions.

    •  

      • Sessions to be scheduled

      • https://github.com/marketplace

      • Questions always welcome.

      • OSS - Open Source Summit conference coming up in April, may also be able to discuss github-actions at OSS if people are attending.

      • Also consider announcing to the Aether community as well.

  •  

  • Linux Foundation (lfn-gha) slack channel for github actions

  • Correction:

  •  

    • Project name: Broadband   (not “LF Broadband”)

    • Org Name: voltha.org

  • Copyright notice in source:

  •  

    • OLD: Copyright {range} the Open Networking Foundation (ONF) and the ONF Contributors

    • NEW:  Copyright {range}  the Open Networking Foundation Contributors

    • Bulk Editing: REPO:copyright :: gather-and-update.ss

Updates:

Items:

[Votlha-Disuss] - Jaeger Pod fails with status ‘InvalidImageName’

I am trying to install VOLTHA Infra with logging and tracing enabled:

“”” helm upgrade --install --create-namespace -n infra voltha-infra onf/voltha-infra \ --set voltha-tracing.enabled=true \ --set efk.enabled=true ""

When I checked the pod status, Jaeger pod fails with status "InvalidImageName". 

Code review requests

The Floor is Open for discussion

  • Linux Foundation (lfn-gha) slack channel for github actions

 

 

2024-02-06

Announcements:

Updates:

  • [joey] Release Status:

    • repo:bbsim sanity test reported an OLT failure in simulator testing.

    • Repo:voltha-onos failure reported by VolthaStackDeploy

    • [TODO] Add bbsim & jira ticket for voltha-onos

    • HOWTO: Create a patch on the release branch

    • While fixing problems, consider if they need to be backported into release.

    •  

      • Naturally TST approval is required to submit a fix.

    • Stage and merge a patch in one branch then propagate to the other.

    • Per-repository tagging & branching

    •  

      • Scripted and is almost fully automated.

      • Can be added as a jenkins UI job with prompts.

      • Only tag & branch, script unaware of build hierarchy order.

    • Voltha-go-controller

    •  

      • Include repository in the release.

    • device-management-interface:

    • Release REPO:voltha-docs

    •  

      • Most component versions updated, final review & edits needed.

    • Re-test all repos after all early repos updated with final version strings:

    •  

      • Ci-management

      • Voltha-docs

      • Voltha-helm-charts

      • Voltha-system-tests

  •  

    • All repositories tagged and branched so the flood gates are open.

    • Some lingering problems

    • Double updates may be needed for a while (branch={master, release})

    • Release automation

    • Repositories: unreleased (VHC::Chart.yaml), should they be:

    • Post release cleanup:

    • [TODO] Revisit tag names – VERSION file tagging has potential to conflict with voltha-x.y release/branch tagging.

Items:

Linux Foundation Stuff

  • JJB/Jenkins

  •  

    • The Linux Foundation is leaning heavily toward use of Github Actions.

    • From reading through docs, they seem very interesting.

    • Potential clean path forward for some upgrades (ie repo:voltha-docs VS Sphinx).

    • Subject to change but an alpha setup could use GA triggers as a dispatcher using GA as thin wrappers.  Actions invoke makefile targets as they are used today.  Devs have the option of working locally with access to verbatim behavior from a jenkins pipeline:

  • Commit message validation / format:

    • Line[1]: 80cpl summary for the patch.

    • Detect token ‘recheck’ in body to re-start a job.

    • Validation checks (#1 - require a jira ticket #)

    • Format, fields, spelling.

    • Tokens that trigger extra actions:

    •  

      • Perform lint checks disabled due to volume errors.

      • Extra credit: limit checking patch sources.

  •  

    • There have been references to commit message content validation.

    • Our commit hooks currently:

    • Think about options that would be beneficial:

Silly Stats (last 90 days)

  • These will likely change with migration to LF but some interesting stats can be found:

    • 5th most active project

    • 76 commits

    • 7 authors

    • 18 repositories

    • Mahir: 35 commits, 1486 files, lines: added[278], removed[191]

    • Nikesh: 16 commits, lines: added[232], removed[200]

    • Most active repos:

    •  

      • voltha-docs, voltha-go, voltha-openonu-adapter-go

    • Elapsed time and failures

    • Summary by group

    •  

      • 7673 builds

      • 181 jobs

      • 1300 nodes

      • 54 job categories (summary)

    • 25 Docker Images

    • 335,462 pulls

  •  

Code review requests

The Floor is Open for discussion

  • V2.12 techinar:

  •  

    • Volunteer(s) – Amit will begin working on it.

    • Scheduling: Within the next 30 days.

 

 

2024-01-30

Announcements:

Updates:

[Joey] Release

  •  

    • v2.12 Release notes

    •  

      • Release matrix versions and URLs mostly updated.

      • Now includes dated resources (docker, pypi) labeled with state ‘stale’.

      • Component Tracking: Components, VOLTHA

    • HOWTO: Create a release patch

    •  

      • More cookbook detail added.

    • ONOS Components:

    •  

      • Published: aaa, bng, dhcpl2relay, igmpproxy, kafka, maclearner, mcast, olt, olttopology, ppoeagent, sadis

      • Kafka: strange

      • Long publishing times – research/debugging needed:

    • Onos-dependencies in pom.xml - a mix of versions have surfaced:

      • Values need to be corrected and parameterized.

    •  

      • V2.5.5 - general onos-dependency version.

      • V2.5.8 - repo:votlha-onos, Docker image

      • V2.5.9 - Pending merge of Gustavo(s) path

      • Version normalizing will be a post-release activity.  Updating has restrictive overhead, introduction of uncertainty and painfully long artifact deployment times.

      • Also some pom.xml(s) contain hardcoded API version 1.0.0,

    • VOLTHA Resources

      • Voltha-onos w/Gustavo(s) patch

      • VOL-526 - Couldn't find any revision to build.

    •  

      • Published: voltha-protos, voltha-lib-go

      • Pending:

      • Problems: voltha-go, voltha-openolt-adapter

    • TODO

    •  

      • Review helm chart versions

      • Review voltha-system-tests

      • Review and bulk close jira tickets.

    • ONOS: inconsistent naming in pom.xml:

      • Repo:mac-learning, component:maclearner

      • Repo:kafka-onos, component:kafka

    •  

      • Naming convention not always 1-to-1

      • VOLTHA depends on ONOS component igmpproxy.

      • Kafka-onos depends on “igmp.api”

      • Confusing due to existence of repository:igmp and following dependency chain, dep is actually igmpproxy.

Items:

  • [Mahir] Linux Foundation Migration:

  •  

    • What domain name should be used for resources: voltha.org ?

  • [Mahir] REPO:device-management-interface

  •  

    • Repository is not part of the release, should it be ?

    • Plan to add this in the v2.13 release.

  • [Joey] Python module upgrade idea (non-invasive!):

  •  

    • Last week I had an idea for an approach we can use to selectively upgrade python modules, on a per-repository basis, w/o potential for massive fallout.

    • Module Sphinx* can be a guineapig (v5.0+ needed) isolated in repo:voltha-docs.

    • Robot test framework & modules can follow along afterward.

Jira Queries

Code review requests:

 

 

2024-01-23 - cancelled

 

2024-01-16

Announcements:

Updates:

ONOS Component sadis:

  •  

    • Using two different versions of com.fasterxml.jackson in use.

    • 15+ hrs for maven central publish (w/jackson) v2.15

    • Mahir: Testing continues to fail, revert to jackson v2.10.2

    • maven central: waiting for v5.12.1 to publish

  • Release Matrix

  •  

    • Stale v2.12 branches and tags removed: voltha-2.12-beta, v2.12.0

  • Release Notes: docs.voltha.org, Components: VOLTHA, ONOS

Items:

Jira Queries

Code review requests:

The floor is open:

  • [jan] zyxel olt Debugging is ongoing.

  •  

    • Hardware/cabling required.

    • Swapping transceiver(s) can eventually get all devices to show loss of signal, so broken transceivers is unlikely.

    • ONL python environment too old on the devices to run inventory scrips.

    • Several options to pursue, underlying suspicion – not sure if the setup worked.

    • A debugging slack channel was setup for discussion.

    • Let Jan know of any other debugging suggestions to pursue.

    • Release will not be blocked on these tests.

  • HW Certification: Need a point person familiar with individual OLT HW/config brought into the labs to avoid wide area, brute force debugging to diagnose problems.

  • [Mahir] Open jira tickets / document HW flow/debugging to get more people involved.

 

 

2024-01-09

Announcements:

Updates:

Release:

  •  

    • Repo:voltha-protos published

    • % git checkout -b voltha-2.12 tags/v2.12.0

    • % git log –graph –decorate –oneline | grep “2.12”

    •  

      • 25589d8 (HEAD -> voltha-2.12, tag: v2.12.0, origin/voltha-2.12, …)

ONOS Component sadis:

  •  

    • Using two different versions of com.fasterxml.jackson in use.

    • Jackson-databind (2.10.2) and jackson-core (2.15.2)

    •  

    • Rebuild and deployment(?-s-?) needed but automated job failing in a non-obvious way due to missing ‘Commit-ID:’ string.

    • Debugging ongoing.

  • VOL-5099 - verify_voltha_onos_sanity_test failure

  •  

    • The repository ‘... stable Release' does not have a Release file.

    • Known problem with certificate installer from base docker OS.

  • Gerrit:34880 - https://docs.voltha.org  (make linkcheck)

  •  

    • Altered pages (~release) to no longer reference missing wiki URLs.

  • VGC

  •  

    • Docker image: update golang to v1.17.10.

    • Github reports rate limiting for image downloads while job running.

    • Future work:

  • Testing Pipelines:

    • Multicast tests need to be fixed. The culprit seemed to be the ONU, since the counters in the OLT NNI and GEM ports were both increasing when the multicast traffic was being sent, but the RG was not receiving the packets. We tried to run the tests with the GPON ONUs SCOM00001c82 and SCOM00001C89, both of them connected to the same EdgeCore OLT box. Since both ONUs were failing the mcast tests, we needed to find other ONUs to run the tests with:

      • Cabling checked and documented.

      • Spare equipment parts(?): yes the lab likely has some.

      • Passive cabling needs to be documented.

      • Setup meetings / allocate time to try out cabling swaps.

      • [Mahir] proposes releasing independent of a problem fix.

      •  

        • But identify as problem and validate post release.

    •  

      • The other SCOM ONUs connected to the OLT were probably the same model/version, so no point in testing with them.

      • The Adtran ONUs and OLTs have never been tested with the TT workflow and Mahir did not think it would work.

      • The other EdgeCore XGS-PON OLT did not have any config file nor pipeline ready, so again, we could not test with it.

      • The only option was to try with the Zyxel Combo OLT S210Z14009247. This OLT has 2 Zyxel ONUs connected to it: a PM7300-T0 (serial num ZYXE8CAC752C) and a PMG1005-T20C (serial num ZYXE8cabe30b). The first one seemed to have all the cabling ok, but is not showing in the openolt logs, so it's not being discovered, for whatever reason (openolt build maybe?). The second one is showing in the openolt logs, but has the LOS led light in red (also showing LOS OMCI messages in the openonu adapter logs). The only two routes to follow seem to be the following: try to find an updated openolt build for the OLT box and/or replace the PON cabling/connectors between the OLT and the ONU with LOS errors.

  •  

    • DT pipeline: 27/32 pass. Remaining failing tests are bandwidth guarantee tests.

    • TT pipeline: 16/16 pass (with multicast tests disabled).

Items:

Jira Queries

Code reviews needed:

VOL-5239 - Fix latent bbsim linting failures (gofmt -s -e -w) and trigger a build.Gerrit:34866 - lint fixes

[abhilash] reviews needs closure

https://gerrit.opencord.org/c/voltha-helm-charts/+/34392

https://gerrit.opencord.org/c/voltha-go/+/34908

https://gerrit.opencord.org/c/voltha-helm-charts/+/34825

https://gerrit.opencord.org/c/voltha-helm-charts/+/34930

[Mahir] - are any patches critical ?  If not defer nice-to-have until after release.

The floor is open:

Joey away on the 2024-01-23, Mahir will emcee the TST meeting.

 

 

2024-01-02

Announcements:

Updates:

Items:

Jira Queries

https://gerrit.opencord.org/q/(project:voltha-system-tests+OR+project:voltha-helm-charts+OR+project:voltha-onos+OR+project:ofagent-go+OR+project:openolt+OR+project:voltha-lib-go+OR+project:+voltha-go+OR+project:+voltha-openonu-adapter-go+OR+project:voltha-protos+OR+project:voltha-openolt-adapter)+AND+status:open+AND+-is:wip+AND+-owner:joey%2540opennetworking.org

  • Open Tasks:

https://jira.opencord.org/issues/?jql=project%20in%20(SEBA%2C%20VOL)%20AND%20status%20in%20(%22In%20Progress%22%2C%20%22To%20Do%22)%20AND%20fixVersion%20%3D%20%22VOLTHA%20v2.12%22

The floor is open:

 

 

2024-01-02

Announcements:

  • January freeze date today.

  • Robot unit tests and linting:

  •  

    • Nice static analysis tools available to complement pep, pyflakes, etc.

    • Not yet available as makefile targets but will be soon.

Updates:

  • VOL-4746 - release EAPOL/openolt, etc

  •  

    • Mahir reported a problem with repo:voltha-onos testing, docker images, etc.

    • Problem triage: review expired certificate

    • Errors reported in collectArtifacts but job:311 did not fail early.

  • Status update on VGC pipelines

    • 34868 - will patch docker image to use a new golang version.

    • Basic dependency for standing up VGC tests and pipeline jobs.

  •  

    • Go version update? - No updates given. Check at the next meeting.

    • VOL-5222, gerrit: 34868  34870 (Submitted: 12/12), patches still unreviewed 

Items:

  • Open: Patches, Tasks

  •  

    • NOTE: Type ctrl-K to insert abbreviated URLs

  • Open Patches:

https://gerrit.opencord.org/q/(project:voltha-system-tests+OR+project:voltha-helm-charts+OR+project:voltha-onos+OR+project:ofagent-go+OR+project:openolt+OR+project:voltha-lib-go+OR+project:+voltha-go+OR+project:+voltha-openonu-adapter-go+OR+project:voltha-protos+OR+project:voltha-openolt-adapter)+AND+status:open+AND+-is:wip+AND+-owner:joey%2540opennetworking.org

  • Open Tasks:

https://jira.opencord.org/issues/?jql=project%20in%20(SEBA%2C%20VOL)%20AND%20status%20in%20(%22In%20Progress%22%2C%20%22To%20Do%22)%20AND%20fixVersion%20%3D%20%22VOLTHA%20v2.12%22

The floor is open:

[abhilash] patches need another set of reviews , which needs to go in before the release

https://gerrit.opencord.org/c/voltha-openonu-adapter-go/+/34734https://gerrit.opencord.org/c/voltha-openonu-adapter-go/+/34492 gerrit:34492https://gerrit.opencord.org/c/voltha-openonu-adapter-go/+/34909 https://gerrit.opencord.org/c/voltha-helm-charts/+/34392https://gerrit.opencord.org/c/voltha-go/+/34908 Gerrit: 34908https://gerrit.opencord.org/c/voltha-helm-charts/+/34825

[Timon]

  • Re-visit announcement - ONF will be merging with the Linux Foundation.

  • Transition to LF will be minimally invasive to the VOLTHA project.

  •  

    • Highlight any transition problems early for escalation.

  • Upcoming registration for a Webinar to be hosted on Jan 9th.