Package testing for Rolling and Rock branches

This is a Work_In_Progress

2022-02-28T23:00:00Z

2022-03-01 - Purpose of this thread is to get something written about how we currently do package testing.

In the contributor group we all know the situation regarding package testing is far from ideal. This is largely due to lack of resources. Simply put, not enough people testing and not enough different hardware being tested on.

What we do has evolved by some process similar to osmosis to something that works to a surprising degree. However it is always possible to make improvements in the process even considering the lack of resources.

First the simple explanation of packaging workflow is:

Cooker>Rolling>Rock

Cooker is development platform where developers and package maintainers upgrade or add packages and where most package fixing is done. Developers are a most valuable source of testing due to their greater technical knowledge and awareness. Devs move packages from Cooker to Rolling/testing when they decide a package is ready.

Rolling is as the name implies a Rolling release of OMLx. Rolling is to be maintained by QA. There is additional testing done in Rolling first as packages move from Cooker to Rolling/testing repositories in Rolling and then the ultimate testing by users when packages move to Rolling/release repositories.

Rock is designed to be used by users that don’t like so much change and to be the most stable branch of OMLx. Rock is also to be maintained by QA.

Who or what is QA? QA or Quality Assurance in most organizations is a team of people devoted to testing the product and seeing to it that issues are acknowledged, and corrected to the extent possible. Functionally in OMLx QA is the entire contributor group. Currently this is thought to be the best and wisest use of our resources.

The Contributor Group and by extension QA consists of all part-time, unpaid, volunteers. There are no employees.

The actual Package Testing Process for Rolling and Rock:

When packages are moved from Cooker to Rolling/testing and from Rolling to Rock/testing the packages in testing repos are listed in the QA application Kahinah. To login to Kahinah requires a Github username and password.

The tester enables either the main/testing repo or testing repos for any other repos that tester normally uses. This can be easily done in the Software Repository Selector application (om-repo-picker).

Tester upgrades system on a regular basis. Rolling testers should use distro-sync rather than upgrade command.

Tester looks at commonly used applications and checks to see that they at least open and have basic function. Tester will pay attention to any loss of function or glitches that may indicate some problem with system or tool-chain packages that were updated.

Testers also check applications that they use regularly.

Ideally if tester has time they will check any applications that were upgraded.

Testers also test things when specifically asked to do so by Cooker devs. This happens often in the context of trying to solve issues users report in forum or on issue tracker. This also may happen when a dev has specific concerns about something.

If tester believes they have encountered some new issue typically they will first report this in OpenMandriva Cooker IRC channel and then if issue is at all serious they will file a bug report when time allows. Filing bug reports will also require a Github username and password.

Tester reporting such an issue then will likely be asked to help test any solutions or workaround that my be forthcoming.

Packages in testing repos are voted on by testers in Kahinah. One may vote to accept or reject a package. Ideally 3 accept votes are required to move a package to Rolling/release or Rock/updates repo for users. 3 reject votes means the package is rejected for publishing for user access. In addition if a package does not get the required 3 votes either way then the packages is to be moved after 7 days to Rolling/release or Rock/updates repo.

In reality sometimes packages are simultaneously moved to Rolling/testing and Rock/testing.

We have a need for more people and hardware for testing in both Rolling and Rock.

Suggestions to improve this process or the this document are welcome. Please make such suggestions in the English/Development Testing forum.
In the Resources forum we lock threads to prevent pollution, off topic comments, spam, ect.

1 Like