Automated testing
Testing seems to be a developer-thing at first. However, as a non-technical user, you still want to know whether things are working. Humans make mistakes. Because of this, automated testing (ie: having tests run automatically all the time) are a very reliable way to guarantee software quality.
Why testing?
Not every seems convinced that testing is worth the effort. Yes, maybe on paper. But in practice, testing falls behind. We (Yireo) have been on the barricades numerous times to advocate testing. And with the LokiCheckout, we have pushed our limits.
Why? Because it reduces risk: More testing means catching more bugs beforehand, which means less support time spent with you, because it simply works. The stability of the LokiCheckout, guaranteed with tests, is not only a good thing for you - it is good for us too.
Testing types with Loki
We take proud in taking testing with our extensions up a notch. Manual testing is part of the development process. However, whenever changes are committed and pushed to our GitLab server, various tests are performed:
- PHP Compatibility with future PHP versions
- Unit tests based on PHPUnit
- Integration tests based on PHPUnit
- Functional tests based on Playwright (browser-based)
- PHPCS and PHPStan static analysis
- Yireo ExtensionChecker validation
Automatic test runs
All tests (except Playwright tests) are run with every git push made to our GitLab server. Similarly, all tests (except Playwright tests) are run with every single module release. Playwright tests are run in parallel on a separate server continuously.
Real-life module updates
In the developer section of our documentation, a listing of all modules can be found. This includes details like compatibility with Magento versions and PHP versions. But also how many tests are included: Unit tests, integration tests, functional tests.
This listing is updated in real-time (well, after a small delay of possibly a few hours), once a module its source code is updated with new tests.
The demo sites
In addition to the regular tests per module, we also run Playwright tests on our demo sites. While the per-module tests include functional tests for various module combinations and various settings, the demo tests are purely end-to-end tests. This includes the testing for basic functionality. But it also includes a test to guarantee that the Lighthouse scores don't drop below 100 (for all 4 groups: performance, accessibility, best practices and SEO). It also includes an AXE accessibility test to guarantee WCAG 2.1 AAA compliance.
