Analyzing Your Aws Api Calls for Fun and Profit (And Security)

Posted on Mar 27, 2020

AWS has a very powerful API. With time, as the organization grows, you query that API quite a bit.

At [Globality])(https://www.globality.com), we query that API from multiple sources:

Cluster management
Monitoring
UI
Terraform Enterprise
Deployment pipeline

We also have very strict security requirements. Working with enterprise customers, we are required to analyze and provide reporting for everything that happens on our AWS accounts.

To begin with, not a lot of people have access to that account, for production, that number is even smaller.

The challenge is analyzing it, alerting based on it and reporting it. In this post, I will review our solution for it.

Logging Every API Call

AWS offers a solution called CloudTrail. It logs every single API call made to your account. It saves the results to an S3 bucket.

However, those results are not easily accessible or analyzed

Querying API Calls

AWS offers another solution called Athena. Athena can create a table based on an input. That table is then queryable.

From your AWS CloudTrail, you have a link to connect to athena, it will create the table with sane defaults and allow you to query it.

Sample Query

Let’s check how many throttling exceptions we have, what is the source of them.

SELECT eventname,
         errorcode,
         eventsource,
         awsregion,
         useragent,
         useridentity.principalid,
         COUNT(*) count
FROM {table_name}
WHERE errorcode = 'ThrottlingException'
        AND eventtime
    BETWEEN '2020-03-01T00:00:00Z'
        AND '2020-03-27T00:00:00Z'
GROUP BY  useridentity.principalid, errorcode,awsregion, eventsource, useragent, eventname
ORDER BY  count desc

Combining Athena with CloudTrail is extremely powerful. We can check user actions, analyze those and alert. We use that on our accounts and alert if an action was taken from the UI for example.

Summing up

Pairing least-access policy along with reporting and alerting is the use-case for us. However, even if you have less strict requirements, you can still use it, control what system accesses your AWS account and make sure you do periodic cleanup.

Hack on!