Coupling test execution to CI/CD pipelines has several drawbacks that become apparent as the complexity and scale of your application or deployment infrastructure increases.
Test execution is a specific step in the STLC which consists of several other steps such as requirements analysis, test planning, test environment setup, and test cycle closure.
Executing tests and managing execution results in a scalable and efficient manner turns out to be a complex undertaking as the number of testing tools, CI/CD systems, engineers and applications grows.
Test execution can be broken down into five steps namely Define, Trigger, Scale, Troubleshoot and Report, which can help execute tests in a way that can grow correspondingly.
Apart from the tactical approach to test execution, we can define a number of criteria that need to be assessed and planned for to scale accordingly with the needs of your team and application.
Testkube, an orchestration platform for test execution, is an example of how to execute tests in line with many, though not all, aspects listed above.
Testkube provides a grounded starting point to execute tests, provide results and aid in troubleshooting. There are both open source and cloud versions available.