Custom Tests
If you do not see this feature enabled in your Vanta instance, please refer to our pricing plans or reach out to your Account Manager or Customer Success Manager for more information!
Overview
Vanta receives resource data through integrations, which are then validated by logical tests that run on top of those data sets. This include Vanta built integrations and any custom integrations you build yourself!
Custom Tests expands Vanta's automation capabilities beyond the test suite that is provided out of the box allowing you to create new validations on the resources received from integration environments.
The following example details the steps to create a Custom Test for any connected Integration.
Steps
There are two areas where custom tests can be created:
Tests Page
Log into Vanta, access the Tests page and select the + Create Custom Test button on the top right corner.
OR
Developer Console
- In the developer console, select your Application, and then click on the Custom Tests tab.
- Select the + Create test button.
The Create a Custom Test form will open. Complete the required fields in the form and click on Create
- Name of the test as it will appear in the Tests page.
- Description of the test as it will appear in the Tests page.
- Integration is the Integration containing the Test Resources that this test will validate.
- Test Resource is the resource that contains the properties that this test will validate.
- Resource Property is the actual property from the resource on which the custom test criteria will be applied.
- Comparison Operator and Comparison Value make up the conditions applied to the Resource Property for the test to PASS. Resources that fail to meet this criteria will cause the Test to fail.
Notice the dependency of the Resource Property on the Test Resource, and of the Resource Property on the Integration, therefore these fields will be updated depending on your selection.
It’s very important to define the PASSING condition of your test. Resources that do not match the criteria, will cause the Test to FAIL.
When all resources satisfy the condition criteria or if there are no resources, the TEST will be in an OK status.
Tests Logic
| Property Type | Comparison Operator | Logic | Comparison Value |
|---|---|---|---|
| Boolean | = | is | True / False |
| String | = | equals | Your Value |
| String | != | not equals | Your Value |
| String | INCLUDES | contains | Your Value |
| Integer | = | equals | Your Value |
| Integer | != | not equals | Your Value |
| Integer | BETWEEN | within range | Your Range |
| Integer | > | greater than | Your Value |
| Integer | > = | greater or equal than | Your Value |
| Integer | < | less than | Your Value |
| Integer | <= | less or equal than | Your Value |
Example
Create a custom test as follows to ensure all AWS Load Balancers are in US East regions
Once Created, a popup with a link to the newly created test will be displayed.
Test Identification
The test will then be available in the Tests Page in Vanta under the Custom Category.
Test Details
Clicking into the test will provide details of the items to remediate, last refresh, and due dates as expected for any Vanta Test.
In our example, the tests is showing as Needing Attention because we have 2 instances of Load Balancers located in us-west-2, which do not meet the condition we have defined for our Custom Test.
Congratulations! You have created your first Custom Test!
Updated about 2 months ago