Skip to content

Day 2 Management Tools Week at the AWS Pop-up Loft

Pre-Reqs Turn on CloudTrail and AWS-Config

At the Start or Day 2 if you can follow these steps, so you can have some data to play with during the next two days.

Create a Trail in CloudTrail

AWS CloudTrail is an AWS service that helps you enable governance, compliance, risk auditing and operational auditing of your AWS Account. Actions taken by a Principal (User, Role or AWS Service) are recorded as events in CloudTrail. To learn more about AWS CloudTrail you can click on this link. Documentation on creating a Trail via the Console is located here. We will highlight the steps below.

  1. Search for the CloudTrail Service under the Management Tools Section in the console and click on Cloudtrail.

    Get to CloudTrail Console

  2. Click on getting Started if presented with that screen. Once in the CloudTrail Console, click on Trails on the Left Side of the screen.

  3. Then Click on Create Trail, to create our trail for this lab.

    Create Trail

  4. Apply the following settings and create the trail

  5. Trail name: management-tools-week
  6. Apply trail to all regions: Yes
  7. Read/Write Events: All
  8. Data Events (Provide Insights into the resource operations)
    • Check the Box on Select all S3 Buckets in your account
    • Click on the Lambda Tab, and check the Box Log all Current and Future Functions
  9. Create a new S3 Bucket: Yes
    • S3 bucket: management-tools-week-(Today'sDate)-(yourcellnumber)-
    • We are using Cell Phone Number at the End to ensure that we create a Uniquie Bucket per user. For more information on Bucket Restrictions and Limitations click here
  10. Click on Create
  11. Lets setup our trail to send logs to CloudWatch so we can search through them a bit easier.
  12. Click back into the Trail, Go to CloudWatch Logs Section and click on Configure

    Add CloudWatch Log to Trail

  13. In the New or exisiting log group, Type the Following name ManagementToolsWeek/CloudTrail and then click continue

  14. Next Screen Click Allow, this Gives Cloudtrail the ability to assume a role to write to CloudWatch Logs.

We now have a trail capturing activity in our AWS Account. Later on, we will search through our trail.

Turn on AWS Config

AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.

  1. Search for the Config Service under the Management Tools Section in the console and click on Config.

    Get to AWS Config Console

  2. Click on Getting Started, lets follow the Setup Wizard

  3. Keep all Defaults and Click Next – This will Create an S3 Bucket, a Role for the Config Service, and will record all resources supported by Config within the region. For a list of supported Service click here.
  4. Click Next on the Next Screen, we will setup Config Rules later in the day.
  5. On the Last Screen Click on Confirm.

We now have AWS Config recording changes for supported resources.

You Can Stop Here Until Labs Later in the Day. However, with these services enabled we will be capturing activity we can search through later in the day.

Service Actions with Systems Manager

Prerequisites

In order to configure and successfully test Service Catalog Self Service Actions, the Service Catalog should be provisioned in the participant’s account using the steps listed in https://tinyurl.com/yasu52vh

In addition, the EC2 instance should be provisioned from the Service Catalog Products List. This instance will be the target of Self-Service actions created in the lab.

If Service Catalog is already provisioned in your account, go straight to Step 1. If Service Catalog is not provisioned and the Product List is not created follow the instructions below: Log into your AWS account as Administrator.

  1. Open https://tinyurl.com/yasu52vh in your browser.
  2. Select Deploy Demo under Service Catalog.
  3. On Create Stack click Next
  4. On Specify Stack Details page click Next
  5. On the Options page click Next
  6. On the Review page check both boxes for acknowledgement and click Create.

The CloudFormation will launch and provision Service Catalog and sample Portfolio and Products List in your account. Wait for the CREATE_COMPLETE status for all nested stacks before proceeding to next step.

When logged into AWS account with administrative permissions:

  1. Select service Catalog from Services list
  2. Under AWS Service Catalog select Products list
  3. Select Amazon Elastic Compute Cloud (EC2) Windows product
  4. Select Launch Product button
  5. Provide the name for the provisioned product and click Next
  6. Specify required parameters:
    • VPC – click on the  to select the VPC
    • Key Pair – click on the  to select key pair name
    • Remote Access CIDR – 0.0.0.0/0
    • Windows Instance type – t2.micro
    • Windows Subnet – any subnet from the VPC
    • Windows Port – 3389
    • SSM path for latest Windows AMI - /aws/service/ami-windows-latest/Windows_Server-2019-English-Full-Base
  7. On TagOptions screen provide
    • Key - Name
    • Value EC2Win
  8. Click Next 2 times and click Launch on the Review page.

Wait until the provisioning process ends. You can now log into the provisioned Windows instance with public IP. Continue with the Step 1 below.

Step1: Create End User and assign User to Group 1. Open the AWS Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/. 2. Select Users – Add User 3. Provide Name and select AWS Management Console access check box for Access Type. Click Next 4. On the Add User page select checkbox for ServiceCatalogEndusers Group 5. Click Next:Tags – Next:Review – Create User

We have created IAM user and assigned him to the ServiceCatalogEndUsers group.

Step 2: Configure End-User Permissions End-user accounts must have the necessary permissions to view and perform specific service actions. In this example, the end user needs permission to access the AWS Service Catalog service actions feature and to perform an Amazon EC2 restart.

To update permissions

  1. Open the AWS Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/.
  2. From the menu, choose Groups.
  3. On the Groups page, select the ServiceCatalogEndUsers group.
  4. On the Permissions tab of your group’s detail page, select Create Group Policy tab.
  5. On the Set Permissions page select Custom Policy – Select
  6. Use the JSON editor to add the permissions. Add the following permissions to the policy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1536341175150",
            "Action": [
                "servicecatalog:ListServiceActionsForProvisioningArtifact",
                "servicecatalog:ExecuteprovisionedProductServiceAction",
                "ssm:DescribeDocument",
                "ssm:GetAutomationExecution",
                "ssm:StartAutomationExecution",
                "ssm:StopAutomationExecution",
                "cloudformation:ListStackResources",
                "ec2:DescribeInstanceStatus",
                "ec2:StartInstances",
                "ec2:StopInstances"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Note: Make sure there is no space on top of the JSON text in Policy Document 7. Provide the policy name SCUserSelfService

Step 3: Create a Self-Service Action

Next, you create a self-service action to restart Amazon EC2 instances.

  1. Open the AWS Service Catalog console at https://console.aws.amazon.com/sc/.
  2. From the menu, choose Service actions.
  3. On the self-service actions page, choose Create new action.
  4. On the Action creation page, choose an AWS Systems Manager document to define the self-service action. The Amazon EC2 Instance Restart action is defined by an AWS Systems Manager document, so we keep the default option on the drop-down menu, Amazon documents.
  5. Choose the AWS-RestartEC2Instance action, and then choose Next.
  6. On the Configure page, keep the default configuration values for the purposes of this tutorial. Note that you can define a name and description for the action that make sense for your environment and team. The end user will see this description, so choose something that helps them understand what the action does. We are also using default permissions for the self-service action. Other permission configurations are possible and are defined on this page.
  7. After you have reviewed the configuration, choose Create action.
  8. On the next page, a confirmation appears when the action has been created and is ready to use.

Step 4: Associate the Self-Service Action with a Product Version

After you define an action, you must associate a product with that action.

  1. On the self-service actions page, choose AWS-RestartEC2instance, and then choose Associate action.
  2. On the Associate action page, choose the product that you want your end users to take the self-service action on. In this example, we choose Amazon Elastic Compute Cloud (EC2) Windows.
  3. Select a product version. Note that you can use the topmost check box to select all versions.
  4. Choose Associate action.
  5. On the next page, a confirmation message appears. You have now created the self-service action in AWS Service Catalog. The next step is to use the service action as an end user.

Step 5: Test the End-User Experience

End users can perform self-service actions on provisioned products. For the purposes of this tutorial, the end user must have at least one provisioned product. The provisioned product should be launched from the product version that you associated with the self-service action in the previous step.

To access the self-service action as an end user

  1. Log in to the AWS Service Catalog console as an end user that was created in Step 1.
  2. On the AWS Service Catalog dashboard, in the navigation pane, choose Provisioned products list. The list shows the products that are provisioned for the end-user's account.
  3. On the Provisioned products list page, choose the instance that is provisioned.
  4. On the Provisioned product details page, choose Actions in the upper right side, and then choose the AWS-RestartEC2instance action.
  5. Confirm that you want to execute the custom action. You receive confirmation that the action has been sent.

Inventory and Patch Mangement

Click Here to Go the Guide for Patching Labs

Click here To Deploy Lab into your Account

Level 100: Inventory and Patch Management: Lab Guide

In the cloud, you can apply the same engineering discipline that you use for application code to your entire environment. You can define your entire workload (applications, infrastructure, etc.) as code and update it with code. You can script your operations procedures and automate their execution by triggering them in response to events. By performing operations as code, you limit human error and enable consistent execution of operations activities.

In this lab you will apply the concepts of Infrastructure as Code and Operations as Code to the following activities: * Deployment of Infrastructure * Inventory Management * Patch Management

Included in the lab guide are bonus sections that can be completed if you have time or later if interested. * Creating Maintenance Windows and Scheduling Automated Operations Activities * Create and Subscribe to a Simple Notification Service Topic

Important You will be billed for any applicable AWS resources used in this lab that are not covered in the AWS Free Tier. At the end of the lab guide there is an additional section on how to remove all the resources you have created. * Removing Lab Resources

Configuration Management with AWS Config and AWS Systems Manager

In this lab, we will enforce compliance by creating Config Rules, and create State Manager Associations to ensure we have the components we need in place.

Deploy Components for Lab

  1. Click here To Deploy Lab into your Account

Note : Make sure to Specify the Same S3 Bucket Name in the Bucket Parameter as is assigned to the CloudTrail Trail.

AWS Config is a Regional Service, Please make sure to deploy the Lab in the Same Region you enabled AWS Config)

Creating a Config Rule to Alert on SSM Agent

In this step we will create a config rule using an AWS Managed rule that will evaluate if Instances have a working AWS Systems Manager Agent.

  1. Let's go to the AWS Config Console, once there click on Rules on the left side of the console.
  2. Click on Add Rule
  3. In the Add Rule Screen in the Filter section type ec2-instance-managed-by-systems-manager, click on the ec2-instance-managed-by-systems-manager rule.
  4. Under the Trigger Section take notice of the Trigger Type, it is a configuration change. Leave these settings as is.
  5. Click Save

You can create config Rules to monitor a number of items within your infrastructure. Beside utilizing AWS managed Config rules you can also create custom rules using Lambda Functions. Located here in Github are same sample config rules you can create and implement in AWS Lambda.

Deploy an EC2 Instance

Let's Deploy and EC2 Instance to test our AWS Confifg rule. - You Can Deploy via Web Console or you can run the following Command from the AWS CLI. Notice that we are not assigning an IAM role to the instance.

aws ec2 run-instances --image-id $(aws ssm get-parameters --names /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 --query 'Parameters[0].[Value]' --output text) --count 1 --instance-type t3.large --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=configruletest}]'

Go back to the AWS Config Rule and click into the Rule and run Re-Evaluate after the EC2 is deployed and Running. You will have wait for the result and refresh the web page. However, the instance we deployed should be flagged as non-compliant.

Let's Remediate this by adding a remediation action

  1. Go back to AWS Config, and edit the ec2-instance-managed-by-systems-manager config rule. Lets set a Remediation Action to attach the needed IAM Role. Under Choose remediation Action do the following:
    • Remediation action: AWS-AttachIAMToInstance
    • Resource ID parameter: InstanceId
    • This passes the Non-Compliant Instance ID to the Remediation Action
    • Get the Role Name from the output of the Cloudformation stack Named EC2RoleName and paste it here.
  2. Click Save
  3. Go back into the AWS Config rule and take a look at Noncompliant resources. Select the Instance we deployed and then Click on Remediate.
  4. Go to AWS Systems Manager, Click on Automation on the left side. You should see a Automation Task Kick off, that will attach the Role to the Instance.
  5. Once Completed, Reboot the Instance to quicken the process
  6. Go Back AWS Systems Manager, and check under Managed Instances. When the instance shows up as a managed Instance, re-evaluate the rule AWS-Config Rule.

What did we learn?

  • How to create an AWS Config Rule to evaluate if Instances are managed by SSM
  • Importance of having the right IAM Role assigned for the Instance to Report to SSM.
  • How to use AWS Systems Manager Automation Documents to Remediate Non-Compliant Instances

Creating a Config Rule To make sure CloudTrail is Enabled

In this step we will create a config rule using an AWS Managed rule that will evaluate whether Cloudtrail is enabled within your AWS Account.

  1. Let's go to the AWS Config Console, once there click on Rules in the left side of the console.
  2. Click on Add Rule
  3. In the Add Rule Screen in the Filter section type cloudtrail-enabled, click on the cloudtrail-enabled rule.
  4. Under Trigger the Trigger Section, notice the Trigger type is Periodic.
    • Change the Frequency to 1 hour
  5. Under Rule Parameters
    • S3BucketName: management-tools-week-(Today'sDate)-(yourcellnumber)-
    • cloudWatchLogsLogGroupArn: arn:aws:logs:<Region>:<AccountID>:log-group:ManagementToolsWeek/CloudTrail:*
  6. Click Save

Set Triggers for Lambda Functions

Now we will create the triggers for the Lambda Function deployed by the Cfn. Note: This step needs to be done correctly for the Lambda to Trigger.

  1. Go to CloudWatch Console, and Under Events on the left side click on Rules
    • Click Create rule
    • Under Event Source
    • Select the radio button next to Event Pattern
    • Service Name: Config
    • Event Type: Config Rules Compliance Change
    • Select the radio button next to Specific message type
      • From the Drop Down Select ComplianceChangeNotification
    • Select radio button next to Specific rule name
      • Type cloudtrail-enabled
    • Under Targets
    • Select the ConfigSSMLab-EnforceCloudTrailFunction* Lambda Function, which is the function deployed by the CloudFormation. Feel Free to take a look at the function code in Lambda.
  2. Click Configure details
    • Configure rule details
    • Name: CloudTrailChange
    • State: Check Enabled Box
    • Click Create Rule

Testing our enforce cloudtrail Lambda

  1. Go to the trail we setup in the first lab, and remove the CloudWatch Logs Configuration by clicking on the trash can as the next screenshot points to.
  2. Go to our Config rule for Cloudtrail, and re-evaluate the rule. Refresh the screen and make sure it comes up as Noncompliant.
  3. Go Back to Cloudtrail, Did the CloudWatch Log Configuration return? Did you Get an E-mail?

What did we learn?

  • How to use CloudWatch Events to automatically Trigger Lambda Functions to automatically Remediate Non-Compliant Resources
  • Multiple ways to Automate and Remediate Resources that Drift within AWS.

Ensure CloudWatch is Installed on Instances

In this step we will create a State Manager job that will run on a schedule to make sure the latest version of the CloudWatch Agent is installed on our instance. System Manager State Manager is a secure and scalable configuration management service that ensures your Amazon EC2 and hybrid infrastructure is in an intended or consistent state, which you define.

  1. Go to the AWS System Manager Console, on the left side under Actions click on State Manager.
  2. Click on Create Association
  3. Provide a name for your association – CloudWatchAgentInstall
  4. Under Command Document
    • Click the radio button next to AWS-ConfigureAWSPackage
  5. Under Parameters
    • Action: Install
    • Name: AmazonCloudWatchAgent
    • Version: latest
  6. Under Targets, Manually check the Instance deployed earlier.
  7. Under Specify Schedule
    • Select Radio Button next to CRON schedule builder
    • Every Day at 22:30
  8. Click on Create Association

This association will run every day at 10:30 PM and make sure the latest version of the CloudWatch Agent is Installed. We can then run another Association to Pull down the Configuration from the Parameter Store

What did we learn?

  • How to use AWS Systems Manager State Manager to make sure that CloudWatch Agent is installed and up to date.
  • We can use AWS Systems Manager State Manager to ensure a certain state of our EC2 Instances

Observer Configuration Timeline and Compliance Timeline

Observe Instance Configuration Timeline

  1. Go to AWS Config, click on Resources
  2. Select EC2: Instance, and then click Lookup
  3. Click into an Instance we deployed in this lab, then Click on Configuration timeline
  4. Observe the Timeline, and the changes that occurred
  5. Then click on Managed instance information in the right hand corner to see the integration data between AWS Config and AWS Systems Manager.

Observe CloudTrail Compliance Timeline

  1. Go to AWS Config, click on Resources
    • If you are already within the Instance Click on Compliance timeline
  2. Select EC2: Instance, and then click Lookup
  3. Click into an EC2 Instance we deployed in this lab, then Click on Compliance timeline
  4. Observe the Timeline, and the compliance changes that occurred

ManagementToolsWeek/CloudTrail in CloudWatch Logs Insight

  1. Navigate to the CloudWatch Logs Console, on the left side Click on Insight
  2. Choose the Log Group you want to work with from the pull down, at the top right of the grey field. Choose ManagementToolsWeek/CloudTrail
  3. Choose the pull down in the time window definition box, at the top right of the grey field.
  4. Choose to define a Relative time for the past two days.
  5. Select Sample Queries --> CloudTrail --> Number of log entries by service, event type, and region
stats count(*) by eventSource, eventName, awsRegion
  1. Run Query and Observe and play around with the Data and Graph. CW Insights ManagementToolsWeek/CloudTrail

End of Lab Exercises

Thank you for using this lab.