Edit Pipelines to Use Push Events

AWS CodePipeline supports full, end-to-end continuous delivery, which includes starting
your
pipeline whenever there is a code change. There are two supported ways to start your
pipeline upon a code change:

Events (either Amazon CloudWatch Events or
webhooks)

Polling (checking periodically)

Initially, only polling was supported. Events are now the default and recommended
way to start your pipeline when there’s a code change.

Important

You must explicitly set the PollForSourceChanges parameter to false within your Source action’s configuration to stop a
pipeline from polling. As a result, it is possible to erroneously configure a pipeline
with both event-based change detection and polling
by, for example, configuring a CloudWatch Events rule and also omitting the
PollForSourceChanges parameter. This results in duplicate pipeline
executions, and the pipeline is counted toward the limit on total number of polling
pipelines, which by default is much lower than event-based pipelines.

There are some important advantages to using push events instead of polling:

On average, events are significantly faster. Events should start your pipeline
almost immediately, as opposed to polling, which requires waiting for the next
periodic check.

Higher limits. Compared to pipelines that poll for changes, CodePipeline can support
far
more event-based pipelines.

Better experience with many pipelines. Some customers might experience throttling
or higher costs by having many pipelines continuously polling their repository for
code changes. You can avoid this by using events.

When you use the CodePipeline console or AWS CodeStar to create a pipeline, events
are enabled by
default. For backward compatibility, new pipelines created through the API, AWS CLI,
or AWS CloudFormation
use the original polling functionality. We strongly recommend that you use events
instead.
To opt in, use the AWS CLI or AWS CloudFormation to create the CloudWatch event or
webhook and disable polling. Use
the instructions in the following table.

You should also use events on pipelines that were created before the new console was
launched. To opt in, use the CodePipeline console to create the CloudWatch event
or webhook and disable
polling. Use the instructions in the following table .

Update Pipelines for Push Events
(Console)

You can use the CodePipeline console to update your pipeline to use Amazon CloudWatch
Events to detect
changes in your CodeCommit source repository or your Amazon S3 source bucket.

Note

When you use the console to edit a pipeline that has a CodeCommit source
repository or an Amazon S3 source bucket, the rule and IAM role are created for you.
If you use the AWS CLI to edit the pipeline, you must create the Amazon CloudWatch
Events rule
and IAM role yourself. For more information, see Use CloudWatch Events to Start a Pipeline (CodeCommit Source).

A message is displayed to advise that CodePipeline creates a webhook in GitHub to
detect source changes. Choose Update. In addtion to the webhook, CodePipeline creates the
following:

A secret, randomly generated and used to authorize the connection
to GitHub.

The webhook URL, generated using the public endpoint for the
region.

CodePipeline registers the
webhook
with GitHub. This subscribes the URL to receive repository
events.

When you have finished editing your pipeline, choose Save
pipeline changes to return to the summary page.

A message displays the name of the webhook to be created for your
pipeline. Choose Save and continue.

To test your action, release a change by using the AWS CLI to commit a
change to the source specified in the source stage of the pipeline.

Update Pipelines for Push Events
(CLI)

You can use the CLI to update your pipeline to use the recommended change detection
method.

Update Pipelines for Push
Events (CodeCommit Source) (CLI)

Follow these steps to edit a pipeline that is using polling (periodic checks) to
use a CloudWatch Events rule to start the pipeline. If you want to create a pipeline,
see Create a Pipeline in CodePipeline.

To build an event-driven pipeline with CodeCommit, you edit the PollForSourceChanges parameter of your pipeline
and then create the following
resources:

Amazon CloudWatch Events event

IAM role to allow this event to start your pipeline

To edit your pipeline's PollForSourceChanges
parameter

Important

When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it
to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Default Settings for the
PollForSourceChanges Parameter.

Run the get-pipeline command to copy the pipeline structure into a
JSON file. For example, for a pipeline named
MyFirstPipeline, run the following command:

aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

This command returns nothing, but the file you created should appear in the directory
where you ran the command.

Open the JSON file in any plain-text editor and edit the source stage by changing
the
PollForSourceChanges parameter to false, as shown in this
example.

Why am I making this change? Changing this parameter
to false turns off periodic checks so you can use event-based change
detection only.

If you are working with the pipeline structure retrieved using the
get-pipeline command, remove the metadata lines from
the JSON file. Otherwise, the update-pipeline command cannot use it.
Remove the "metadata": { } lines and the "created",
"pipelineARN", and "updated" fields.

The update-pipeline command stops the pipeline. If a revision
is being run through the pipeline when you run the
update-pipeline command, that run is stopped. You must
manually start the pipeline to run that revision through the updated pipeline. Use
the start-pipeline-execution command to manually
start your pipeline.

To create a CloudWatch Events rule with CodeCommit as the event source and CodePipeline
as the
target

To add CodePipeline as a target, call the put-targets command and include
the following parameters:

The --rule parameter is used with the rule_name you
created by using put-rule.

The --targets parameter is used with the list Id of
the target in the list of targets and the ARN of the target
pipeline.

The following sample command specifies that for the rule called
MyCodeCommitRepoRule, the target Id is composed of the
number one, indicating that in a list of targets for the rule, this is target 1. The
sample command also specifies an example ARN for the pipeline. The pipeline
starts when something changes in the repository.

Call the create-trail command and include the --name
and --s3-bucket-name parameters.

Why am I making this change? This creates the CloudTrail
trail required for your S3 source bucket.

The following command uses --name and --s3-bucket-name to
create a trail named my-trail and a bucket named
myBucket.

aws cloudtrail create-trail --name my-trail --s3-bucket-name myBucket

Call the start-logging command and include the --name
parameter.

Why am I making this change? This command starts the
CloudTrail logging for your source bucket and sends events to CloudWatch Events.

Example:

The following command uses --name to start logging on a trail named
my-trail.

aws cloudtrail start-logging --name my-trail

Call the put-event-selectors command and include the
--trail-name and --event-selectors parameters. Use event
selectors to specify that you want your trail to log data events for your source bucket
and send the events to the Amazon CloudWatch Events rule.

Why am I making this change? This command filters
events.

Example:

The following command uses --trail-name and
--event-selectors to specify data events for a source bucket and prefix
named myBucket/myFolder.

To add CodePipeline as a target, call the put-targets command and include
the --rule and --targets parameters.

The following command specifies that for the rule named
MyS3SourceRule, the target Id is composed of the
number one, indicating that in a list of targets for the rule, this is target 1. The
command also specifies an example ARN for the pipeline. The pipeline starts
when something changes in the repository.

When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it
to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Default Settings for the
PollForSourceChanges Parameter.

Run the get-pipeline command to copy the pipeline structure into a
JSON file. For example, for a pipeline named
MyFirstPipeline, run the following command:

aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

This command returns nothing, but the file you created should appear in the directory
where you ran the command.

Open the JSON file in any plain-text editor and edit the source stage by changing
the
PollForSourceChanges parameter for a bucket named
storage-bucket to false, as shown in this example.

Why am I making this change? Setting this parameter
to false turns off periodic checks so you can use event-based change
detection only.

If you are working with the pipeline structure retrieved using the
get-pipeline command, you must remove the metadata
lines from the JSON file. Otherwise, the update-pipeline command
cannot use it. Remove the "metadata": { } lines and the
"created", "pipelineARN", and "updated"
fields.

The update-pipeline command stops the pipeline. If a revision
is being run through the pipeline when you run the
update-pipeline command, that run is stopped. You must
manually start the pipeline to run that revision through the updated pipeline. Use
the start-pipeline-execution command to manually start your
pipeline.

Update Pipelines for Push
Events (GitHub Source) (CLI)

Follow these steps to edit a pipeline that is using periodic checks to use a
webhook instead. If you want to create a pipeline, see Create a Pipeline in CodePipeline.

To build an event-driven pipeline, you edit the PollForSourceChanges
parameter of your pipeline and then create the following resources manually:

GitHub webhook and authorization parameters

To create and register your webhook

Note

When you use the CLI or AWS CloudFormation to create a pipeline and add a webhook,
you must disable
periodic checks. To disable periodic checks, you must explicitly add the
PollForSourceChanges parameter and set it to false, as detailed in the
final procedure below. Otherwise, the default for a CLI or AWS CloudFormation pipeline
is that
PollForSourceChanges defaults to true and does not display in the
pipeline structure output. For more information about PollForSourceChanges defaults,
see
Default Settings for the
PollForSourceChanges Parameter.

In a text editor, create and save a JSON file for the webhook you want to create.
Use
this sample file for a webhook named my-webhook:

When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it
to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Default Settings for the
PollForSourceChanges Parameter.

Run the get-pipeline command to copy the pipeline structure into a
JSON file. For example, for a pipeline named
MyFirstPipeline, you would type the following command:

aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

This command returns nothing, but the file you created should appear in the directory
where you ran the command.

Open the JSON file in any plain-text editor and edit the source stage by changing
or
adding the PollForSourceChanges parameter. In this example, for a
repository named UserGitHubRepo, the parameter is set to false
.

Why am I making this change? Changing this parameter
turns off periodic checks so you can use event-based change detection only.

If you are working with the pipeline structure retrieved using the
get-pipeline command, you must edit the structure in the JSON
file by removing the metadata lines from the file. Otherwise, the
update-pipeline command cannot use it. Remove the
"metadata" section from the pipeline structure in the JSON file,
including the : { } and the "created",
"pipelineARN", and "updated" fields.

The update-pipeline command stops the pipeline. If a revision
is being run through the pipeline when you run the
update-pipeline command, that run is stopped. You must
manually start the pipeline to run that revision through the updated pipeline. Use
the start-pipeline-execution command to manually start your
pipeline.

Update Pipelines for Push Events (AWS CloudFormation
Template)

You can use AWS CloudFormation to update your pipeline to use the recommended method
to detect
changes in your source.

In the template, under Resources, use the AWS::IAM::Role
AWS CloudFormation resource to configure the IAM role that allows your event to start
your pipeline.
This entry creates a role that uses two policies:

The first policy allows the role to be assumed.

The second policy provides permissions to start the pipeline.

Why am I making this change? Adding the
AWS::IAM::Role resource enables AWS CloudFormation to create permissions for CloudWatch Events.
This resource is added to your AWS CloudFormation stack.

In the template, under Resources, use the AWS::Events::Rule
AWS CloudFormation resource to add a CloudWatch Events rule. This event pattern creates
an event that monitors
push changes to your repository When CloudWatch Events detects a repository state
change, the rule
invokes StartPipelineExecution on your target pipeline.

Why am I making this change? Adding the
AWS::Events::Rule resource enables AWS CloudFormation to create the event. This
resource is added to your AWS CloudFormation stack.

Save the updated template to your local computer, and then open the AWS CloudFormation
console.

Choose your stack, and then choose Create Change Set for Current
Stack.

Upload the template, and then view the changes listed in AWS CloudFormation. These
are the changes
to be made to the stack. You should see your new resources in the list.

Choose Execute.

To edit your pipeline's PollForSourceChanges
parameter

Important

In many cases, the PollForSourceChanges parameter defaults to true when
you create a pipeline. When you add event-based change detection, you must add the
parameter to your output and set it to false to disable polling. Otherwise, your
pipeline starts twice for a single source change. For details, see Default Settings for the
PollForSourceChanges Parameter.

In the template, change PollForSourceChanges to false. If
you did not include PollForSourceChanges in your pipeline definition, add
it and set it to false.

Why am I making this change? Changing this parameter
to false turns off periodic checks so you can use event-based change
detection only.

Use these steps to edit your pipeline with an Amazon S3 source from polling to
event-based change detection.

To build an event-driven pipeline with Amazon S3, you edit the PollForSourceChanges parameter of your pipeline
and then add the following resources
to your template:

Amazon CloudWatch Events requires that all Amazon S3 events must be logged. You must
create an
AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the
events that occur. For more information, see Logging Management and Data Events with AWS CloudTrail.

Amazon CloudWatch Events rule and IAM role to allow this event to start our
pipeline.

If you use AWS CloudFormation to create and manage your pipelines, your template includes
content like the following.

Note

The Configuration property in the source stage called
PollForSourceChanges. If your template doesn't include that
property, then PollForSourceChanges is set to
true by default.

To create a CloudWatch Events rule with Amazon S3 as the event source and CodePipeline
as the target and apply
the permissions policy

In the template, under Resources, use the AWS::IAM::Role
AWS CloudFormation resource to configure the IAM role that allows your event to start
your pipeline.
This entry creates a role that uses two policies:

Use the AWS::Events::Rule AWS CloudFormation resource to add a CloudWatch Events rule. This event
pattern creates an event that monitors CopyObject, PutObject
and CompleteMultipartUpload on your Amazon S3 source bucket. In addition,
include a target of your pipeline. When CopyObject, PutObject,
or CompleteMultipartUpload occurs, this rule invokes
StartPipelineExecution on your target pipeline.

Why am I making this change? Adding the
AWS::Events::Rule resource enables AWS CloudFormation to create the event. This
resource is added to your AWS CloudFormation stack.

Save your updated template to your local computer, and open the AWS CloudFormation
console.

Choose your stack, and then choose Create Change Set for Current Stack.

Upload
your updated template, and then view the changes listed in AWS CloudFormation. These
are the changes
that will be made to the stack. You should see your new resources in the list.

Choose
Execute.

To edit your pipeline's PollForSourceChanges
parameter

Important

When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it
to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Default Settings for the
PollForSourceChanges Parameter.

In the template, change PollForSourceChanges to false. If
you did not include PollForSourceChanges in your pipeline definition, add
it and set it to false.

Why am I making this change? Changing
PollForSourceChanges to false turns off periodic checks so
you can use event-based change detection only.

To create a second template for your Amazon S3
pipeline's CloudTrail resources

In a separate template, under Resources, use the
AWS::S3::Bucket, AWS::S3::BucketPolicy, and
AWS::CloudTrail::Trail AWS CloudFormation resources to provide a simple bucket
definition and trail for CloudTrail.

Why am I making this change? Given the current limit
of five trails per account, the CloudTrail trail must be created and managed separately.
(See
Limits in AWS
CloudTrail.) However, you can include many Amazon S3 buckets on a single trail,
so you can create the trail once and then add Amazon S3 buckets for other pipelines
as
necessary. Paste the following into your second sample template file.

When you use AWS CloudFormation to create these resources, your pipeline is triggered
when
files in your repository are created or updated.

Note

Do not stop here. Although your pipeline is created, you must create a
second AWS CloudFormation template for your Amazon S3 pipeline. If you do not create
the
second template, your pipeline does not have any change detection
functionality.

We strongly recommend that you use AWS Secrets Manager to store your credentials.
If you use
Secrets Manager, you must have already configured and stored your secret parameters
in
Secrets Manager. This example uses dynamic references to AWS Secrets Manager for the
GitHub
credentials for your webhook. For more information, see Using
Dynamic References to Specify Template Values.

Important

When passing secret parameters, do not enter the value directly into the template.
The
value is rendered as plaintext and is therefore readable. For security reasons, do
not
use plaintext in your AWS CloudFormation template to store your credentials.

When you use the CLI or AWS CloudFormation to create a pipeline and add a webhook,
you must disable
periodic checks.

Note

To disable periodic checks, you must explicitly add the
PollForSourceChanges parameter and set it to false, as detailed in the
final procedure below. Otherwise, the default for a CLI or AWS CloudFormation pipeline
is that
PollForSourceChanges defaults to true and does not display in the
pipeline structure output. For more information about PollForSourceChanges defaults,
see
Default Settings for the
PollForSourceChanges Parameter.

Save the updated template to your local computer, and then open the AWS CloudFormation
console.

Choose your stack, and then choose Create Change Set for Current
Stack.

Upload the template, and then view the changes listed in AWS CloudFormation. These
are the changes
to be made to the stack. You should see your new resources in the list.

Choose Execute.

To edit your pipeline's PollForSourceChanges
parameter

Important

When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it
to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Default Settings for the
PollForSourceChanges Parameter.

In the template, change PollForSourceChanges to false. If
you did not include PollForSourceChanges in your pipeline definition, add
it and set it to false.

Why am I making this change? Changing this parameter
to false turns off periodic checks so you can use event-based change
detection only.