Forward CodeDeploy Agent logs to Cloudwatch Logs

While working with CodeDeploy on EC2, one of the issues was examining the logs to debug a failed deployment. By default, the codedeploy agent logs to /var/log/aws/codedeploy-agent/codedeploy-agent.log and one could login via SSM to the EC2 instance to examine the log file. A better approach would be to stream the CodeDeploy agent logs to CloudWatch Logs via the CloudWatch agent.

Note that there are different approaches on installing the CodeDeploy and Cloudwatch agent from the docs. To simplify the process, I’m documenting the same steps here I took for installing the Cloudwatch agent, which is via this documentation: Install Cloudwatch agent via SSM

The first step is to install the CodeDeploy agent. This could be achieved via SSM > RunCommand and selecting the AWS-ConfigureAWSPackage command document. Under the name field we specify the name of the software package which is AWSCodeDeployAgent. Specify the name of the cloudwatch log group to monitor the progress of the install. Once completed, verify the agent is running from the EC2 instance

The second step is to install the CloudWatch agent. We are going to use the same process as above except we are replacing the name field with AmazonCloudWatchAgent

By default, the cloudwatch agent will not start unless a valid configuration file is provided. Checking on the status of the cloudwatch agent will show the following:

sh-5.2$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
{
  "status": "stopped",
  "starttime": "",
  "configstatus": "not configured",
  "version": "1.300055.0b1095"
}

Cloudwatch Agent Configuration File specifies the format for the config file. This is in the form of a JSON document and consists of four main sections:

• agent

  For overall configuration of the agent

• metrics

  Specify custom metrics for collection and publish to Cloudwatch logs. This can be omitted if only using the agent to collect logs.

• logs

  Specifies which logs files are published to cloudwatch logs.

• traces

  Specifies sources for traces that are sent to AWS X-Ray

The example config file we used below specifies the CodeDeploy agent logs, its location on disk, and the log group and stream to send to:

{
    "logs": {
        "logs_collected": {
            "files": {
                "collect_list": [
                    {
                        "file_path": "/var/log/aws/codedeploy-agent/codedeploy-agent.log",
                        "log_group_name": "testpipeline",
                        "log_stream_name": "{instance_id}-agent-log"
                    },
                    {
                        "file_path": "/opt/codedeploy-agent/deployment-root/deployment-logs/codedeploy-agent-deployments.log",
                        "log_group_name": "testpipeline",
                        "log_stream_name": "{instance_id}-codedeploy-agent-deployment-log"
                    },
                    {
                        "file_path": "/tmp/codedeploy-agent.update.log",
                        "log_group_name": "testpipeline",
                        "log_stream_name": "{instance_id}-codedeploy-agent-updater-log"
                    }
                ]
            }
        }
    }
}

The {instance-id} is a placeholder which will be replaced with the actual instance ID on startup. The configuration is forwarding the codedeploy agent logs to a cloudwatch log group name of testpipeline into its individual log streams, depending on the type of logs. For example, any update to the agent goes to the {instance_id}-codedeploy-agent-updater-log whereas any deployments go to {instance_id}-codedeploy-agent-deployment-log.

To apply the above config to the Cloudwatch agent, we need to store is as a STRING type in SSM Parameter Store. This can be achieved via this CLI command:

aws ssm put-parameter --name "testagentcfg" --type "String" --value file://cloudwatch.json

Assuming that we saved the above config file as cloudwatch.json we can use the put-parameter CLI command to save it.

The next step is to run the AmazonCloudWatch-ManageAgent Run Command, which will take as input the SSM parameter value for the configuration file and configure the agent. Under Optional Configuration Store we select ssm and provie the name of the parameter under Optional Configuration Location:

If applied successfully, the cloudwatch agent should be active:

sh-5.2$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
{
  "status": "running",
  "starttime": "2025-05-18T14:57:35+00:00",
  "configstatus": "configured",
  "version": "1.300055.0b1095"
}

We should also see the log streams collated under the log group:

Any deployments via CodeDeploy and CodePipeline can be viewed in the log stream {instance_id}-codedeploy-agent-deployment-log: