Who Holds the Keys to the Kingdom? (Part 2 of 2)

Part 2 takes a look at sensitive AWS Resources – secret strings and keys used in AWS.

Ermetic Team By Ermetic Team
Who Holds the Keys to the Kingdom? (Part 2 of 2)

This is the second of a two part series discussing access control to secrets and encryption keys in AWS. By the end of this post, you will understand how it works, how to identify risks, and how to remediate them.

In the first part, we explored the different types of resources and what access control mechanisms are used to enable various entities to access them and perform certain functions. We concluded the post with the following diagram:

Figure 1 - Relationship Map of the Access Control Objects We Previously Reviewed
Figure 1 - Relationship Map of the Access Control Objects We Previously Reviewed

“Identity Provider” and “AWS Account” are marked red since they allow external entities access via assumption of a Role. “IAM Policy” is a generalization for AWS-managed, Customer-managed and inline policies. Note that resource-based policies are indicated in the diagram in rectangles. When it comes to assuming a Role, an IAM Policy can only delegate what the Role’s Trust Relationships has allowed.

Any combination of relationships leading to a resource could allow an entity to gain access to it, so if you’re looking to review it manually, you must follow all possible paths that could eventually lead to a Resource, which could be a staggering number. Also remember that what we presented here and in the last post is not a complete description of how access to the resources is managed, so the reality could get a lot more complicated.

If while reading the previous post, you were tempted to think it’s simple to review your AWS environment to control access to the resources we discussed, we hope that by now you realized that’s hardly the case. Even though we can define how it theoretically can be done, real life poses many challenges which are extremely difficult to mitigate. In this post, we will take a closer look at these challenges and present a way in which automated analysis of configuration and logs could help you mitigate them.

Manual Access Control Auditing Limitations

Role Assumption Tracking

Let’s say that while reviewing an IAM policy for an SSM parameter, or a resource policy for a Secrets Manager Secret, you realize it grants an IAM Role access to the Resource in question. If the trust relationship is not defined in a very specific way, many other IAM Roles and IAM Users could potentially assume that Role, and by doing so, use the permissions assigned to it.

This, in fact, can create a major gap in your analysis. To bridge it, you would have to (among other things) review IAM and resource-based policies on many Principals looking for the sts:AssumeRole Action for said Role and look for all Principals that can pass the Role using iam:PassRole to an AWS service included in the trust relationship of the Role. This is nearly impossible to do manually and extremely difficult to manage if done programmatically.

Inline Policy Tracking

As we mentioned before, inline policies defined for IAM Principals are extremely difficult to track and they can grant permissions just as easily and effectively as IAM policies.

Digest the Information and Respond to it

The amount of relevant information that should be compiled from the review activity we described in this article could potentially be mind boggling. You have to gather information about all the KMS CMKs, Secrets Manager Secrets and SSM Secured Parameters. For each of those, you also need information about the IAM Principals that have access. In the case they are Groups or Roles, then you have to also consider the information about the IAM Principals that are included in them or can assume them, respectively.

While collecting the information is a difficult task in its own right, it is not possible to manually organize it in a way to generate meaningful insights and respond accordingly.

Real-Time Auditing to Rightsize Permissions

Another task that simply can’t be completed using manual tools and / or by solely using the AWS console, is auditing the actual usage of permissions granted to use the secrets. It’s very hard to determine whether a permission granted to a User or a Role is justifiably granted and the Principal is required to use it on a regular basis. To do so, you must audit the usage of the resources and make sure that any permission granted to a Principal is, in fact, in use.

Ensuring Full Coverage

Finally, no matter how much time and effort you spend manually combing through configurations and logs, you will never have the assurance that you’ve covered everything and made all the relevant considerations. Even if you did, how will you know it won’t be changed the next day by someone else?

The assurance of governing control to your secrets is, in fact, quite elusive, and this assurance is the main reason why we chose to go on this journey. Isn’t it?

The Case for Automatic Analysis

Luckily, software for automatic analysis of cloud environment configuration and logs can mitigate the limitations of manual analysis we described. Specifically, Ermetic can help you gain visibility to the access that entities have to your secrets, assess risks which apply to said secrets, and even remediate these risks to significantly improve your security posture, all with very little effort.

First, let’s look at a simulated example of how complicated access control can actually get:

Figure 2 - Bird’s-eye view of an access graph for an example KMS key
Figure 2 - Bird’s-eye view of an access graph for an example KMS key

Here we see a graph illustrating all the entities who have access to a KMS key. On the right, we see an indication of where the Resource (the KMS CMK) is. To the left, we see the IAM Policies, Roles and the Users which are allowed access to it. They are connected to one another so we can understand the paths that lead Principals to gain access to the CMK (e.g. a Federated User can assume a Role which is assigned a Policy that allows using the key for decryption). Just imagine having to manually comb through policies and resource configurations to eventually compile this graph; it’s a paralyzing thought. In fact, even going over this simple diagram might take a while, so let’s filter it based on the permissions we really care about. You may recall from a few sections ago, we want to focus on kms:Decrypt, kms:CreateGrant and kms:Sign:

Figure 3 - Filtering the access graph to track only the most sensitive actions
Figure 3 - Filtering the access graph to track only the most sensitive actions

After selecting specific permissions, the graph is automatically filtered and now looks like this:

Figure 4 - The filtered access graph
Figure 4 - The filtered access graph

As this graph is much easier to examine, we easily notice an indication of two IAM Roles with excessive access to the AdministratorAccess policy:

Figure 5 - Two IAM Roles have excessive permissions to the AdministratorAccess Policy
Figure 5 - Two IAM Roles have excessive permissions to the AdministratorAccess Policy

Zooming in allows us to see that it’s an external account with two Roles with an AdministrativeAccess (!) Policy attached to them, one of which is the instance profile for an Elastic Beanstalk Environment. The external account is allowed access via the Key Policy of the Key. This is obviously excessive. If a malicious actor were to leverage a vulnerability that allows it to assume this Role, the KMS CMK could be unnecessarily compromised (and in this case - much more than the KMS CMK).

Figure 6 - Zooming in on the IAM Roles with the AdministratorAccess policy attached
Figure 6 - Zooming in on the IAM Roles with the AdministratorAccess policy attached

Additionally, Ermetic also does the leg work for you by automatically querying and presenting the grants currently available for each KMS CMK:

Figure 7 - Information on a KMS key including Current Grants
Figure 7 - Information on a KMS key including Current Grants

Finally, Ermetic goes the extra mile of auditing the logs to review the actual access and usage of resources by entities so it can provide a more exact analysis of which of the permissions are, in fact, excessive.

When we choose a specific Role or User, or a Group of Users / Roles which are clustered on the graph, we highlight the path through which they have access to the resource and can better understand whether the permissions are all excessive, only some are excessive or none are excessive (with a red, yellow or black line making the connection, respectively).

For example, if we look at a group of 4 IAM Users who are clustered together and assigned to the AdministratorAccess Policy, we can see it provides permissions to the KMS CMK, all of which are excessive permissions.

Figure 8 - 4 IAM Users with all excessive permissions to the KMS CMK
Figure 8 - 4 IAM Users with all excessive permissions to the KMS CMK

Right next to it, there’s a clustered group of IAM Roles which also have access to the KMS CMK through the exact same Policy. If we select it, we can see that its permissions are marked with yellow, which implies that only some of the permissions granted by the Policy are, in fact, excessive.

Figure 9 - IAM Roles with some Excessive Permissions to the KMS CMK
Figure 9 - IAM Roles with some Excessive Permissions to the KMS CMK

This distinction between different Principals accessing the exact same policy is possible because Ermetic is able to track all the activity of the Principals and assess exactly which permissions are needed and which are not. We will see a more detailed demonstration of this ability in our next example.

When we look at a Principal in Ermetic, we also see and understand risks the system is exposed to due to the excessive permissions assigned to it. In the example below, we see an IAM Role called “ProjectsSecretsReader” which Ermetic indicates has permissions to 10 services while it only uses two of them. Not only that, but within those two services it only uses some the permissions it has to them. Let’s see how deeply we can dive into this with Ermetic.

Figure 10 - Ermetic indicating a verbal plain language risk for a Role
Figure 10 - Ermetic indicating a verbal plain language risk for a Role

In Figure 11, we can examine the risk and analyze logs showing activity by Principals assuming this Role. We see that out of the 20 secrets the SecretsManagerReadWrite Policy (AWS managed policy) has granted them access to, they’ve only accessed four secrets, and out of the 19 permissions they have for each, they only use five. All the rest are excessive!

Figure 11 - Specific indication of excessive permissions for Secrets
Figure 11 - Specific indication of excessive permissions for Secrets

Ermetic provides details of exactly which secrets can be accessed and which permissions should be removed. But that’s not the most interesting part! Ermetic takes you to the natural next step of remediating this situation and provides a step-by-step guide explaining how to do it.

Figure 12 - Step-by-step guide for remediating an Excessive Permissions scenario
Figure 12 - Step-by-step guide for remediating an Excessive Permissions scenario

As you may have noticed, the first step instructing you to create a new policy to replace SecretsManagerReadWrite has action buttons on the right. These buttons give you access to a readymade policy to fulfil the step using a simple copy-paste into the AWS console. You also have the option to get a Terraform or CloudFormation script to do it even faster if you practice managing your infrastructure as code.

Figure 13 - Example of an Ermetic policy suggestion to right-size permissions for a Role
Figure 13 - Example of an Ermetic policy suggestion to right-size permissions for a Role

Figure 13 shows an example of an auto-generated document for creating a policy which provides access to the four secrets the Role actually consumes. For each secret, we see only the Actions the Role actually applies to it based on the log analysis. It’s that granular.

In addition, you can also view the differences between the current policy and the one recommended to you (see Figure 14):

Figure 14 - The difference between an existing and a recommended policy
Figure 14 - The difference between an existing and a recommended policy

Finally, automate the remediation procedure with Ermetic’s step-by-step wizard by simply pressing the “Automate” button in the remediation panel:

Figure 15 - Automating the remediation process directly from the Ermetic console
Figure 15 - Automating the remediation process directly from the Ermetic console

In addition to indicating issues, Ermetic sets you on the path to remediate them with as little effort as possible and as much confidence as possible, to achieve the best result possible.

Ready to Find Out More?

Hopefully this document has helped you understand how important and challenging access control for sensitive resources is in AWS. If you want to learn more about how the Ermetic’s solution can help your organization mitigate this issue, please contact us for a demo.