Skip to content
  • There are no suggestions because the search field is empty.

How to work with KPI evidence

Overview

KPI evidence is a very powerful tool to implement continuous monitoring of your controls. It allows you to retrieve data (evidence) from 3rd party systems and validate this data against an approved baseline. When the evidence matches, it proves that the control (or part of a control) is implemented correctly. The result is stored as KPI data. When the evidence does not match, an event can be created. Typically, an incident management process is started to analyse and fix the situation.
For example, when a policy requires that all users have MFA enabled, a workflow can be created to retrieve the MFA status of all users in Microsoft Entra ID by calling a Microsoft Graph API. The resulting JSON file can be submitted as the reference evidence. An annual plan task can be created that that is linked to the KPI which should retrieve the results. When the annual plan task is scheduled with a recurring pattern of your choice, the workflow is run and the results checked against the baseline. When a user has MFA disabled, an incident is created and routed to the responsible employee.

ISOPlanner has implemented a ‘low code’ solution using Power Automate which makes Continuous Monitoring:

  1. Very safe
  2. Fairly easy to implement
  3. Greatly adaptable to your business

 

Authorization

Unlike other systems like Vanta, Drata or Sprinto, ISOPlanner implements continuous monitoring in such a way that you retain control of the authorizations to other systems. Continuous monitoring always involves ‘background’ tasks that are executed automatically at specific intervals. This means also that the application that performs these tasks must have authorization to read data from your systems. To prevent having a system that has access to your Hosting systems, ERP systems, HR systems, Source control systems and communication platforms (to name some), ISOPlanner is designed in such a way that it does not need to know the credentials to those systems.

ISOPlanner leverages proven Microsoft technologies to authenticate and authorize within your own Microsoft tenant. This also means that evidence gathered from various applications never flows through ISOPlanner.

 

High level steps

In short, the following steps are needed to implement 1 continuous monitoring task.

  1. Create a KPI in ISOPlanner to store the evidence result. This can be of type Success/Error or Number.
  2. Create a Power Automate workflow that gathers the evidence in JSON format.
  3. In the workflow, use the ISOPlanner action ‘Submit evidence to a KPI‘ to send the evidence to ISOPlanner.
  4. In ISOPlanner, go to the KPI and select tab Evidence. Follow the steps to approve the reference evidence.

When the workflow runs again, ISOPlanner will compare the reference evidence to newly gathered evidence and store the result in the KPI over time. Optionally, you can configure which event must be created when the evidence does not match. You can also re-process any evidence that was not accepted due to test scenarios or after fixing the problem causing the faulty evidence.

We are working with the community and partners to create an elaborate store of Power Automate templates for evidence collection for various compliance frameworks and applications.

 

Trigger the continuous monitoring

There are 2 ways of triggering the workflow:

  1. Use an Time interval trigger in Power Automate and send the results directly to a KPI.
  2. Use ISOPlanner to trigger the workflow based on the recurrence pattern of an annual plan task.

Both ways are valid for different use cases.

Time interval

This type of trigger is useful for scenarios where you want to gather data that is not directly related to a control of a compliance framework. For example, you may want to gather NPS scores and show the average NPS score on the dashboard of your Objective that states that the NPS score must be greater than 8.5. In this case, the NPS score is a KPI supporting that you listen to your customers or other stakeholders. There is no need to register a task or event for this.

Annual plan task

This type of trigger is useful for scenarios where you:

  1. Have a mix of automatic an manual tests. You may want to check if MFA is enabled for users in Microsoft, Github and Slack. You can start with 1 flow that checks Microsoft but you do Github and Slack manually. You can implement this by creating a Form with 3 KPI’s for each check and submit evidence to 1 of them. ISOPlanner will automatically detect this and show the gathered evidence in the form, while the user can still enter the 2 remaining KPIs manually. You need an annual plan task for this to send the task with the form to the user at the recurrence pattern of your choice.
  2. You want to present the evidence to the auditor. In this case, an annual plan task that is linked to the compliance framework requirement or control is needed. This task should contain a form with the KPI that receives the evidence on it. This way, during audit, you can simply navigate to the requirement or control, click tab Tasks & Events, observe all the tasks created and show all or a sample. When opening the task you can see the results in the form. To get the overview, navigate to the KPI itself and show all the data for the audit year with a chart.

 

For more information see working with KPIs.

Approving reference evidence

To let ISOPlanner know what the correct result for the KPI must be, you should create a Power Automate workflow and use the workflow action Submit evidence to a KPI. In the evidence field, provide a valid result that should be approved as reference evidence.

Follow these best practice detailed steps:

  1. Create a workflow with the trigger of your choice
  2. Think about what the result will be: a number or a succes/error value.
  3. Create a KPI for this type.
  4. Add the step Submit evidence to a KPI to your workflow and select the KPI.
  5. In the evidence field of this step, create a simple JSON like: { “AllUsersHaveMFA”: true }
  6. Run the workflow.
  7. Navigate to the KPI in ISOPlanner and select tab Evidence.
  8. This tab will indicate that newly submitted evidence is available.
  9. Click on Select to see the JSON content and select the part that must be approved. Click on a JSON node to:
    1. Select the node (for KPIs of type success/error)
    2. Ignore the node
    3. Select the Value of the node (for KPIs of type number)
  10. When you do not have a library category to store the reference evidence yet, create it.
    1. Navigate to the ISOPlanner library and choose tab Categories. Create a new one and choose Document Library.
    2. Select a SharePoint location and create a new folder “ISOPlanner reference evidence“. Make sure only authorized people have access.
    3. Select this location and set the System type of the new category to Reference evidence and click Save.
    4. Navigate back to the KPI on tab Evidence.
  11. In the next step, choose if you want to approve yourself or send the approval task to someone else. To test, choose to approval yourself.
  12. An approval task is opened (or send to someone else). Click Approve and Save.
  13. Now implement the workflow to gather the real life evidence, for example by using an HTTP action to get the MFA status of users in Github.
  14. Replace the ‘true’ in the dummy result of step 5 with the actual result variable.
  15. Add other meta data to the result as needed. This data is send along with the automatically created events when evidence is not approved for analysis.

To learn this procedure quickly, follow our tutorial on how to check if all user have their MFA enabled (coming soon!) If you have any trouble getting started, please contact our Servicedesk. We are happy to get you started or introduce you to one of our technology partners to help you.

When you submitted evidence for the first time and you’re not happy with the format, just change the format in Power Automate and submit (run) it again. Until you approve the reference evidence, you can submit new reference evidence.

Configuring incident response

Navigate to the KPI, select tab Evidence and click Not accepted evidence event.

Select an existing event template or create a new one (follow link Manage event templates).

When an event is created based on the selected template, the following extra information is send along with the event.

  • When the first subtask of the event is unassigned, this task is assigned to the owner of the reference evidence as set in the Library.
  • On the context tab: the controls and objectives that are related to this KPI so that the event is shown in the ‘Tasks & Events‘ tab of each control and objective.
  • On the context tab: the related task that triggered the evidence request, if any.
  • On the library tab: a link to the reference evidence document.
  • A description which KPI is involved and a technical explanation why the evidence was not accepted, including the submitted JSON.
  • You can add the placeholder ‘‘ in the name of the event and subtasks. The placeholder will be replaced with the name of the KPI.

 

Resources