AWS Security Automation - Access Key Rotation Compliance Management using AWS Lambda

Hi folks!!!

In the previous post, we discussed about control and enforcement over Public Ingress Rules in EC2 Security Groups. Feel free to have a look at it here.

Now, we will be proceeding with maintenance of IAM User Access Key hygiene. But first lets understand about the same.

AWS Access Keys are the credentials used by IAM User or the AWS account root user. They are used to sign programmatic requests to the AWS CLI or AWS API (directly or using the AWS SDK). Access Keys consist of two parts: an access key ID and a secret access key. Like a username and password, we have to provide both together to authenticate the requests. A security recommendation suggests that we should only use IAM roles instead of access keys but still in some cases we need to have the access keys handy. These access keys will provide a user complete access to a set of resources defined by the IAM User Policy. So, if the access keys are lost or went into wrong hands, your resources are at risk.

There are a set of IAM Best Practices defined by AWS to help secure your AWS resources. Related to Access Keys, this include,

1. Do Not Share Access Keys

We clearly understood the fact that Access Keys must not be shared in public and should be kept at secure place.

2. Rotate Credentials Regularly

Rotate your credentials either password/access keys regularly s.t. if a password or access key is compromised without your knowledge, you can limit how long the credentials can be used to access your resources.

3. Remove Unnecessary Credentials, etc.

It is important to remove the unused passwords and access keys not in current utilization of business requirements. For this, you can look for the users who have not used their passwords/credentials since a pre-decided particular time.

If you have plenty of users in your AWS account, it will be a tedious task to keep an eye on every user to meet the access key compliance. And manual management, can result in severe consequences.

Compliance:

According to CIS Amazon Web Services Foundations v1.2.0 - 05-23-2018, the compliance benchmark says,

Ensure credentials unused for 90 days or greater are disabled.

Ensure access keys are rotated every 90 days or less.

Strategy:

To meet the following compliance requirements, here we will be implementing an intelligent Lambda function which will take care of the following things,

-  Sanitation: If Access Key is not used since last 45 days being "Active", disable the Access Key and publish the message to Security/SecOps Team.

-  Rotation: If Access Key creation has passed 85 days, generate a new Access Key for the user and publish the message to Security/SecOps Team. Further SecOps/Security Team will inform the user to update all its apps with New Access Key. Delete the older Access Key automatically after 90 days.

-  Audit: Check the compliance everyday and generate a compliance list for all users with relevant information, scope of action required and action recommendation.

Setup:

1. Go to AWS Lambda Console using "Services" drop-down and search for "Lambda". Inside Lambda Console, Go to "Functions" and hit "Create Function". Fill the basic information required for the AWS Lambda function and move forward.

2. Post successful Lambda creation, Click on Amazon CloudWatch Logs and go down to hit "Manage These Permissions".

3. Now we will be attaching an inline policy for IAM Access Key Management permissions. Use "Edit JSON" to write the policy and review it giving a name to hit "Create Policy" button.

4. I have one more inline policy made for SNS publish. Going to attach the same now.

5. To add a trigger, go to our Lambda function, hit "Add Trigger" and fill the details as given below. Please note that we will be setting up a "Schedule expression" which will trigger the Lambda check every day at 10 am UTC. And do not check upon the "Enable trigger" for now.  

"Lambda Core Logic"

There are three parts in this Lambda function:

1. lambda_handler - It is the controller of the function which passes the usernames to check for access key compliance.

2. sns_publish - To send the notification to Security/SecOps team about the compliance status.

3. access_key_remediator - accepts the usernames and check compliance benchmark for every key attached to a username.

Compliance Check:

To check for the compliance, here I have created a set of users in this AWS account. 

When we ran the Lambda, we got a result in return as well as some log information. If we can see the log information along with the result, we can conclude,

- There are 5 users.

- User "4hathacker" has two access keys and both are disabled.

- User "LegitUser3" has one access key and it is disabled.

- User "LegitUser4" have one access key and it isn't used in any of the API call yet.

- User "LegitUser4" has  one access key which is disabled.

If you are wondering about "LegitUser1" and "LegitUser2", then I would like to tell you that, I do not have created any access key for them. Hence, they didn't appear in the logs as well as the compliance report result.

Advantages:

1. Automatic and timely Creation and Deletion of Keys.

2. Notification of Disabled Keys to SecOps/Security Team.

3. Daily Audit for compliance.

4. No manual overhead and zero maintenance.

Scope Of Improvement:

1. SecOps/Security Team needs to convey to the user about Creation of New Access Keys with complete details.

2. Can subscribe to more SNS topics with an arrangement to reach end user with appropriate notification.

This is how we have automated one more compliance check. Stay Tuned, for more posts like this...!!!

PS: I have cleared all the users and keys shared here... ;)

AWS Security Automation - Public Ingress Compliance Auto-Remediation (AWS Config and Lambda) - Part 2

Hi folks!!!

In the previous post, we discussed about SSH Compliance in EC2 Security Groups. Feel free to have a look at it here.

We saw, there were some limitations like, AWS Managed Rule is for SSH only, no sync between AWS Config and AWS Lambda and many more. In this post, we will try to resolve them  while implementing a setup for Custom Config Rule and few tweaks around AWS Lambda for Public Ingress remediation to all the ports instead of SSH only.

Public Ingress Compliance: If the access to instance via any port/protocol is restricted to public IP's of organization behind the firewall/proxy, this is COMPLIANT, else NON-COMPLAINT. 

Strategy:

Using AWS Config Custom Rules with AWS Lambda fucntion, we can check for the Public Ingress associated to the Security Groups in a regional aspect. Here the compliance check will be triggered only when there is a configuration change occurred. So, no need to configure AWS CloudWatch Scheduled Rule. The rule allowing "0.0.0.0/0" or "::/0" for any port/protocol for ingress via EC2 Security Groups must be removed.

Setup:

1. From "Services" drop down, select "Config" and go to AWS Config Console. Select "Rules" from the left pane.

2. From the Rules panel, select "Add rule" and then click on "Add custom rule".

3. In the panel appeared, add the "Name" and "Description" and then Click on "Create AWS Lambda function" with CTRL Key, which is present below the AWS Lambda function ARN* field.

4. Here, we will be following the same approach as earlier to create a Lambda function as "PublicIngressRemovalSecopsSGLambda" provided some basic information like, Name, Runtime, Execution role,  etc.

5. For execution role, we will be creating a new role with name as - "PIRSecOpsSGLambdaRole" with two policy templates viz. AWS Config Rules permissions and Amazon SNS publish policy. 

6. When the function is created, copy the ARN from the top left and go to the Config Console window. Paste the Lambda function here. Add trigger type as "Configuration changes", Scope as "Resources" and Resources as "EC2:SecurityGroup". Click "Save" at the bottom and proceed.

7. You can see the new rule with compliance as "Evaluating".

8. Now, go back to Lambda console, Click on SNS policy and select "Manage these permissions".

9. We will be adding an inline policy to our "PIRSecOpsSGLambdaRole" so click on add inline policy and write the policy as below. Click "Review" policy.

10. Add the name as "PIRSecopsLambdaSGPolicy" and hit "CreatePolicy".

11. You can see your policy is attached here.

"Lambda Core Logic (LCL)"

LCL-1. This is our "evaluate_compliance" function which will check around three cases:

a. If the resource is in APPLICABLE_RESOURCES, for example if you are monitoring a lot of resources via your custom config rules and you specifically want to filter "EC2:SecurityGroups" resources.

b. Check if the resource is present or deleted.
c. If not deleted, get the security group details and proceed to evaluate and remediate as we have done previously for both IPv4 and IPv6 check.

LCL-2. This is our "sns_publish" function which will report the security team about the remediation activity.

LCL-3. This is our main function - "lambda_handler" which acts as a controller here. The interesting part to note here is we will be putting the result after config evaluation to inform AWS Config. So, this Lambda function on its own is evaluating the compliance and then updating the AWS Config about the same.

Compliance Check:

1. To check if its working fine, create two security groups, 4hathackerSecOpsSG and 4hathackerSecOpsSG2 as below.

2. Post 5-6 mins, the Lambda will be automatically triggered by AWS Config, evaluate the compliance and update the security groups.

3. To analyze what happened in due course of time, check for the AWS CloudWatch Logs Log Group. It will show you how the modifications have been made by Lambda.

Advantages:

1. As compared to our previous, blog post where Lambda is being triggered every 2 mins this is much more efficient.

2. The sync of Lambda and AWS Config is functional here.

3. No dependency over AWS Config Managed rules. Write your own custom rule, do evaluation and update the AWS Config.

Scope Of Improvement:

1. Can use CloudFormation Templates to have an easy deployment approach.

2. Filter the ports/protocol as per your requirement. Here it is rigid for every port not allowing public ingress at all.

This is all about Security Groups and compliance management. There are a lot of things, one can try and learn for security groups compliance management. I found a visual guide for best practices from Internet2 Confluence site which is available here.

Feel free to have a look and Stay tuned for more exciting things related to AWS, Security,  and Automation...

Wondering if I will start writing with Python and Core Security... Please share your views about it in comments section...

AWS Security Automation - SSH Compliance Auto-Remediation (AWS Config and Lambda) - Part 1

Hi folks!!!

In the previous series of 3 posts about MFA Compliance, we went through a set of meaningful aspects related to security compliance, how to follow along with compliance documents and how to setup a compliance for MFA in AWS. Feel free to have a look at them here. 

In this post, we will be looking at compliance management for access via SSH to the EC2 instances using control implementation and enforcement over Security Groups.

SSH Compliance: If the access to instance via SSH is restricted to public IP's of organization behind the firewall/proxy, this is COMPLIANT, else NON-COMPLIANT. 

Strategy:

Using AWS Config Managed Config Rules with AWS Lambda function, we can check for the compliance. This compliance check will be triggered within a specified interval with the help of AWS CloudWatch Scheduled Rule. 

Setup:

A. Configure an AWS Managed Config Rule for EC2:SecurityGroup resources using Config Management Console. For the first time, you need to setup AWS Config service as below. Otherwise, create a Config Rule directly.

1. Click on "Services" drop down, type "Config" and select "AWS Config" . This will provide you with a screen to setup AWS Config. In case if below image appears, then AWS Config setup is already there. If not, follow from below steps.

2. From the left panel, select "Settings" and select resources in "Specific types" as   "EC2:SecurityGroup". Please note that AWS Config is a regional construct, and this will consider only the Security Groups present in this region. Select "Create a bucket" option and fill the desired bucket name you want. Also select AWS Config Service linked role and move forward.

3. In the rules tab, search for restricted-ssh rule  and select it. Move forward then.

4. Click on "Confirm" in the bottom of Review panel.

B. Write a Lambda function with a CloudWatch Events Scheduled Rule to trigger the Lambda Function which will check the compliance in a specified interval.

"Core Logic for SecOpsRemediateSGLambda function" 

1. Go to AWS Lambda from "Services" drop down and select "Create Function". Select the following as mentioned in the below images.

 Note: Remember to select the policy templates attached to role as: AWS Config Rules permissions and AWS SNS Publish policy.

2. After selecting, "Create Function" you can see the following "SecOpsRemediateSGLambda".

3. For remediation action, we need to attach an inline policy with permissions as below.

Note: Do not attach the AWS Managed Policy for EC2FullAccess to the Lambda function. Its always best practice to attach only those permissions to the policy which are required to perform an action.

4. Now, we need to add a CloudWatch Scheduled Rule trigger to this Lambda. Click on "Add Trigger" option from the Lambda function console for "SecOpsRemediateSGLambda". Fill the details as mentioned below in the image and move forward.

Keep the state of trigger disabled for now and enable it while testing.

C. Setup an SNS Topic for SecOps Team to notify SecOps team about Lambda Remediation in case

of NON-COMPLIANCE. We already have created this with previous blog posts. Feel free to take a look back.

D. Setup a demo Security Group (4hathackerSecOpsSG) to test if the flow and remediation action to remove the SG Rules is working.

Check:

1. Add 4 rules to the demo security group - 4hathackerSecOpsSG

 2. The rules attached as below have 3 non compliant and one compliant rule.

3. After sometime, you can see in the AWS Config Rules that this resource has been evaluated as NON-COMPLIANT. Upon checking this our, Lambda will take remediation action.

4. You can see the log events to analyze the actions taken by Lambda function for maintaining the compliance. Please note that the notification will only be sent whenever Lambda will do any remediation action.

5. Post Lambda Remediation, come back to Security Group and observe that non-compliant rules have been removed.

Advantage:

No manual action required by SecOps/Security team to check for non compliant security groups having SSH open to world. The remediation action will take only 3-5 minutes to process.

Scope of Improvement:

1. There is no synchronization between Lambda and Config rule to detect for changes in the Security Group configuration. Both are acting like separate entities. There could be a case, when Lambda will automatically detect for AWS Config changes for Security Group and take remediation action.

2. There is only remediation for IPv4 viz. "0.0.0.0/0" and not for IPv6 i.e. "::/0".

3. There is a dependency over AWS Managed Config Rule for restricted-ssh which is fine in this case. But what about other admin ports for protocols like, FTP, SMTP, RDP, etc. There is no rule present in AWS Config for all these ports. So, the use case needs to be generic.

4. Being a regional construct, we are monitoring the Security Group resources for only a single AWS region. So, we need to deploy this Lambda function to all the regions where our applications are present and this require a generic CloudFormation Template like what we did for MFA Compliance. Feel free to create your own template for the same.

In the next post, we will see how we can generalize this compliance management for other admin ports. Till then, stay tuned!!!