Skip to content

Deployment validation for Node.js applications

Automated pre-deploy testing and feedback for Node.js applications on the VIP Platform is powered by Harmonia. When enabled, Harmonia runs an automated suite of tests against each pull request made to a branch that deploys to a Node.js production environment, or when a new commit is pushed to a non-production branch. Tests are designed to identify issues that could prevent application code from building, deploying, or working as expected before that code is deployed to a production environment.

To opt in, submit a VIP Support request and indicate which environment(s) should have Harmonia enabled.

Limitations

Settings for Harmonia are not customizable. There are no available methods at this time to configure Harmonia to ignore failing tests that report false positives for an application.

Execution of tests and the Harmonia build process will not work as expected for:

  • Applications that depend on a containerized service (e.g., Redis). Harmonia currently only instantiates the main Node.js application container.
  • Applications that rely on a third-party service that might limit incoming requests on a specific subnet or IP address.
  • Applications with private dependencies (e.g., private npm modules or private git submodules).
  • Deploying branches that have automated build and deploy enabled. By default, Harmonia automatically tests pull requests made to a deploying branch. The deploying branch of an environment with automated build and deploy enabled is the *-built version of the main branch (e.g., main, main-built). Harmonia tests will run in the *-built branch instead of the main branch, and feedback will only be visible in the Github commit status.
Example screenshot of a Harmonia test result in the GitHub commit status view

Individual tests

Individual Harmonia tests are run against code to check for specific issues. Issues identified by each test are reported based on severity level.

Notice

Issues reported as Notices by tests are intended to provide helpful insights into the test flow and opportunities for code quality improvement.

Warning

Issues that contribute to a lower quality of performance for an application, and may cause additional problems if left unchecked. Warning issue types found by tests will not cause tests to fail, but the test will be marked as a partial success.

Examples of issues that are reported as Warnings:

  • Redirects (3XX) and client errors (4XX) on homepage health checks
  • Cache health check has a delayed response (>100ms but <200ms)
  • Wrong node.engines version
  • Using npm serve instead of npm start

Error

Issues that impact an application’s functionality, but do not prevent a successful deploy and start-up of the application. Any test that throws at least one Error will cause the final test result to be Failed.

Examples of issues that are reported as Errors:

  • /cache-healthcheck? endpoint takes too long to respond (>200ms)
  • 5xx status codes returned for homepage health checks
  • Timeouts during homepage health checks

Blocker

An issue that is severe enough to prevent an application from deploying, such as a build error, a starting error, or not responding to the health-check endpoint. Tests that identify an issue of this type will immediately be aborted, and no further tests will be executed. 

Examples of issues that are reported as Blockers:

  • start and build scripts not found in package.json
  • Docker build fails
  • Application does not respond to the /cache-healthcheck? endpoint

Test Suites

A Test Suite is a group of individual tests. Any issue identified by an individual test (e.g., Notice, Warning, Error, Blocker) will affect the final result of the Test Suite it belongs to.

  • NPM package.json Test Suite: Tests basic settings such as the presence of npm start and npm build scripts in application code, and some package.json settings that could cause issues.
  • Docker Test Suite: Builds and executes the Docker image, and performs a basic health-check to the required /cache-healthcheck? endpoint.
  • Health Check Test Suite: Executes a broader and extendible health-check against a site’s homepage, the /cache-healthcheck? endpoint, and a few URLs scraped from the homepage.
Example screenshot of an issue reported as a Warning in the third test of the Docker Test Suite

Final test results

Harmonia’s final test result is based on the summary of issues reported by all tests and Test Suites that completed execution. Application code that receives a final test result of Pass or Pass with warnings is considered to be ready to deploy to a production environment.

  • Passed: The application passed all tests, and no issues were reported as Warnings, Errors, or Blockers.
  • Passed with warnings: The application passed all tests, however at least one issue was reported as a Warning that should be taken into consideration.
  • Failed: The application failed at least one test, or one test has been aborted because of an issue reported as a Blocker.
Example screenshot of Harmonia feedback in a Github pull request with a final test result reported as Passed
Example screenshot of a final test result reported as Passed with warnings
Example screenshot of a final test result reported as Failed

Last updated: July 01, 2022