AnalyticsAI configuration
This feature is only available with CaseWare AnalyticsAI, part of the CaseWare Cloud suite.
To refine the tests, select the CONFIGURATION tab.
From here you can see all the tests that are run in AnalyticsAI. You can select and deselect tests to run, configure test parameters and change the emphasis (Risk Score) on specific tests.
You can also refine the test parameters.
(missing or bad snippet)Deselect a test’s check box if you don’t want it to run.
You can also use this page to change the emphasis (Risk Score) on specific tests and their parameters.
To change a test's Risk Score:
- Select the check box of a specific test, for example, Ends in 999.
- Enter a new Risk Score, for example, 15.
- Select RUN TESTS to see the updated results.
Charts and graphs
Tests include a pie chart that shows the percentage and quantity of the flagged transactions for that test. Pie charts appear when a test has flagged transactions.
Some tests, such as High Amounts include a pie chart and an additional graph that helps you configure parameters.
The test-specific bar graphs show the data distribution relative to the test, so that an auditor can have a clear picture of where to set the parameter. For example, the High Amounts bar graph shows the transactions line amounts distribution. You can adjust the Percentile to flag amounts to the right of the orange bar.
In this example, the default percentile value has been changed to 99.7. Notice that for the default percentile value of 99.1 there are 21 transactions flagged. When we change the Percentile to 99.7 and rerun the tests, the number of transactions flagged drops to five.
Outlier detection
AnalyticsAI has a number of machine learning (ML) based outlier detection tests.
These tests are turned off by default. Use the CONFIGURATION page to turn them on as desired.
Test status
The CONFIGURATION page also shows the status of individual tests as follows:
- A Running tests indicator (a spinner)
- Not run icon (
) - Precondition failed icon (
) - Pending run indicator (a spinner)
- Failed run icon (
) - Successful run icon (
)
Risk scores
The default base Risk Scores on the CONFIGURATION page have been selected to reflect the importance of a transaction that has been flagged by a given test.
For example, Complex Account Combinations has a higher base score of 50. This is because the more accounts a transaction affects, the more advisable it may be for the auditor to investigate those transactions further.
Conversely, Unusual Users has a base score of 1, as a user infrequently posting to an account is not always an indication of a red flag.
This is an area where professional judgement and knowledge of the entity are valuable. If you are concerned that one particular test may be indicative of a potential problem, you can increase the base score for that test to bring those transactions to the top of the list.
Risk levels
AnalyticsAI calculates risk level relative to the overall amount of risk in the data set. It sorts all of the transactions in order of descending risk score. It then takes the top 5% of scored transactions and labels those transactions as high risk. AnalyticsAI uses exclusive threshold logic here, so that all transactions with the same score have the same risk level. Similarly, the next 10% of transactions are labeled as medium risk. All other transactions are considered low risk.
For example, assume there are 1,000 transactions in a data set, of which five have five risk points, seven have four risk points, and so on as shown below:
| Score | Number of transactions |
| 5 | 5 |
| 4 | 7 |
| 3 | 30 |
| 2 | 40 |
| 1 | 53 |
| 0 | 865 |
There are a total of 135 scored transactions. The top 5% will be deemed as high risk. Five percent of 135 is about seven transactions, and our exclusive threshold logic will ensure that transactions with the same score will all be included at the same risk level - meaning in this case all transactions with a score of five will be labeled as high risk.
The next cutoff is for medium risk. This looks at the next 10% of transactions, which is about 14 (135 * 10%). As this would split the transactions with a risk score of three, all the transactions with a risk score of four will end up being labeled as medium risk and the transactions with three or fewer points will be deemed as low risk.
This model balances being able to categorize large sets of transactions, with varied risk levels, with being flexible enough for different distributions of risk across different transaction sets.
