Harmonia deployment validation tests for Node.js application codebases are designed to identify issues that could prevent application code from building, deploying, or working as expected before that code is deployed to a production environment.
Two methods of pre-deploy testing and validation powered by Harmonia are available:
- Pull requests: When enabled, an automated suite of tests is run by Harmonia against each pull request made to a branch that deploys to a Node.js production environment, or when a new commit is pushed to a non-production branch.
To opt in, customers must submit a VIP Support request and indicate which environment(s) should have Harmonia enabled.
- VIP-CLI: Validate a Node.js codebase on a user’s local machine. The VIP-CLI command
vip validate preflightcan be run from within the local codebase directory, or by passing the location of the directory on the local machine with the
--pathoption. By targeting a specific application and environment in the command, the tests will run with settings that are identical to the targeted environment.
Settings for Harmonia are not customizable. There are no available methods at this time to configure Harmonia to ignore failing tests that report false positives for an application.
Execution of tests and the Harmonia build process will not work as expected in pull requests and with VIP-CLI for:
- Applications that depend on a containerized service (e.g., Redis). Harmonia currently only instantiates the main Node.js application container.
- Applications that rely on a third-party service that might limit incoming requests on a specific subnet or IP address.
Execution of tests and the Harmonia build process will not work as expected in pull requests for:
- Applications with private dependencies (e.g., private npm modules or private git submodules).
- Deploying branches that have continuous integration and deployment (CI/CD) enabled. By default, when enabled, Harmonia automatically tests pull requests made to a deploying branch. The deploying branch of an environment with automated build and deploy enabled is the
*-builtversion of the main branch (e.g.,
main-built). Harmonia tests will run in the
*-builtbranch instead of the main branch, and feedback will only be visible in the Github commit status.
Individual Harmonia tests are run against code to check for specific issues. Issues identified by each test are reported based on severity level.
Issues reported as Notices by tests are intended to provide helpful insights into the test flow and opportunities for code quality improvement.
Issues that contribute to a lower quality of performance for an application, and may cause additional problems if left unchecked. Warning issue types found by tests will not cause tests to fail, but the test will be marked as a partial success.
Examples of issues that are reported as Warnings:
- Redirects (
3XX) and client errors (
4XX) on homepage health checks
- Cache health check has a delayed response (>100ms but <200ms)
npm serveinstead of
Issues that impact an application’s functionality, but do not prevent a successful deploy and start-up of the application. Any test that throws at least one Error will cause the final test result to be Failed.
Examples of issues that are reported as Errors:
/cache-healthcheck?endpoint takes too long to respond (>200ms)
5xxstatus codes returned for homepage health checks
- Timeouts during homepage health checks
An issue that is severe enough to prevent an application from deploying, such as a build error, a starting error, or not responding to the health-check endpoint. Tests that identify an issue of this type will immediately be aborted, and no further tests will be executed.
Examples of issues that are reported as Blockers:
buildscripts not found in package.json
- Docker build fails
- Application does not respond to the
A Test Suite is a group of individual tests. Any issue identified by an individual test (e.g., Notice, Warning, Error, Blocker) will affect the final result of the Test Suite it belongs to.
package.jsonTest Suite: Tests basic settings such as the presence of
npm buildscripts in application code, and some package.json settings that could cause issues.
- Docker Test Suite: Builds and executes the Docker image, and performs a basic health-check to the required
- Health Check Test Suite: Executes a broader and extendible health-check against a site’s homepage, the
/cache-healthcheck?endpoint, and a few URLs scraped from the homepage.
Final test results
Harmonia’s final test result is based on the summary of issues reported by all tests and Test Suites that completed execution. Application code that receives a final test result of Pass or Pass with warnings is considered to be ready to deploy to a production environment.
- Passed: The application passed all tests, and no issues were reported as Warnings, Errors, or Blockers.
- Passed with warnings: The application passed all tests, however at least one issue was reported as a Warning that should be taken into consideration.
- Failed: The application failed at least one test, or one test has been aborted because of an issue reported as a Blocker.
Last updated: August 03, 2023