How do I integrate an API Gateway REST API with Amazon SQS and resolve common errors?

Last updated: 2022-11-23

I want to integrate an Amazon API Gateway REST API with Amazon Simple Queue Service (Amazon SQS). I also want to troubleshoot integration errors. What steps do I take?

Resolution

Users configure API Gateway REST APIs so that they work with Amazon SQS to create an integrated solution. This article provides steps to integrate API Gateway with Amazon SQS and information to troubleshoot common setup errors.

Set up REST API and an Amazon SQS integration

To integrate an API Gateway REST API with Amazon SQS, follow these steps:

1. Create an SQS queue.

2. Create an AWS Identity and Access Management (IAM) role, and then attach an Amazon SQS policy with a SendMessage permission. This policy allows messages from the API to be published to Amazon SQS. In the policy, replace example-region with your AWS Region, example-account-id with your AWS account ID, and example-sqs-queue-name with your SQS queue name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Resource": [
        "arn:aws:sqs:example-region:example-account-id:example-sqs-queue-name"
      ],
      "Action": [
        "sqs:SendMessage"
      ]
    }
  ]
}

3. Create a REST API in API Gateway.

4. In the API Gateway console, create an Amazon SQS integration for your newly created REST API.

Optionally, create a REST API resource or a REST API method.
Create a POST method.
For Integration type , choose AWS Service .
For AWS Region , choose your AWS Region.
For AWS Service , choose Simple Queue Service (SQS) .
For AWS Subdomain , enter Optional .
For HTTP method , choose POST .
For Action Type , choose Use path override .
For Path override (optional) , enter example-account-id/example-sqs-queue-name where you replace example-account-id with your AWS account ID and example-sqs-queue-name with your SQS queue name. For example, 1234567890/MySQSStandardQueue.
For Execution role , enter the ARN of the IAM role created in step 2 .
For Content Handling , choose the option that fits your setup.
Clear or select Default Timeout . Choose the option that fits your setup.
Save the new POST method.
Continue entering your REST API integration information.
Choose the POST method Integration Request .
Expand HTTP Headers .
Choose Add header .
For Name , enter Content-Type .
For Mapped from , enter 'application/x-www-form-urlencoded' and choose Create .
Expand Mapping Templates .
For Request body passthrough , select the option that fits your requirements.
Choose Add mapping template .
For Content-Type , enter application/json and choose Create .
For the template, enter Action=SendMessage&MessageBody=$input.body

4. Deploy the configured REST API.

5. Test the setup by sending the following request to API Gateway. Replace example-api-id with your API ID, example-region with your AWS Region, example-stage with your testing stage name, and example-resource with your resource name.

curl --location --request POST 'https://example-api-id.execute-api.example-region.amazonaws.com/example-stage/example-resource' \
    --header 'Content-Type: application/json' \
    --data-raw '{
        "message": "Hello World"
    }'

When your integration is successful, your response looks similar to the following:

{
    "SendMessageResponse": {
        "ResponseMetadata": {
            "RequestId": "f879fb11-e736-52c0-bd29-a0f2d09ad90d"
        },
        "SendMessageResult": {
            "MD5OfMessageAttributes": null,
            "MD5OfMessageBody": "3fc759ac1733366f98ec4270c788fcd1",
            "MD5OfMessageSystemAttributes": null,
            "MessageId": "4c360c3c-08f4-4392-bc14-8b0c88e314a2",
            "SequenceNumber": null
        }
    }
}

Resolving common errors

UnknownOperationException error

An UnknownOperationException error occurs when a user fails to configure the Content-Type as "application/x-www-form-urlencoded" in the integration request HTTP header. The UnknownOperationException error also occurs when the SendMessage action isn’t added to the integration request mapping template.

AccessDenied error

The following is an example of an AccessDenied error:

{
  "Error": {
    "Code": "AccessDenied",
    "Message": "Access to the resource https://sqs.example-region.amazonaws.com/example-account-id/example-sqs-queue-name is denied.",
    "Type": "Sender"
  },
  "RequestId": "92aea8b7-47f1-5bd4-b3c4-f3d0688d3809"
}

An AccessDenied error occurs when the API integration execution role doesn’t have the sqs:SendMessage permission set to send messages to the SQS queue. The AccessDenied error can also occur when special characters such as "&" and "%" are passed in the request body payload. The special characters must be encoded to pass. Add the $util.urlEncode() function in the mapping template to convert the request body from a string to an encoded format. The following is an example mapping template:

Action=SendMessage&MessageBody=$util.urlEncode($input.body)

The following example includes the permissions needed to send messages to the SQS queue. Replace example-region with your AWS Region, example-account-id with your AWS account ID, and example-sqs-queue-name with your SQS queue name.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Resource": [
        "arn:aws:sqs:example-region:example-account-id:example-sqs-queue-name"
      ],
      "Action": [
        "sqs:SendMessage"
      ]
    }
  ]
}

KMS.AccessDeniedException error

The following are two examples of KMS.AccessDeniedException errors:

{
  "Error": {
    "Code": "KMS.AccessDeniedException",
    "Message": "User: arn:aws:sts::example-account-number:assumed-role/example-sqs-queue-name/BackplaneAssumeRoleSession is not authorized to perform: kms:GenerateDataKey on resource: arn:aws:kms:example-region:example-account-number:key/example-keyId because no identity-based policy allows the kms:GenerateDataKey action (Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException; Request ID: c58f1eec-6b18-4029-826b-d05d6a715716; Proxy: null)",
    "Type": "Sender"
  },
  "RequestId": "99976a6a-3311-57e9-86e9-310d0654ff80"
}

{
  "Error": {
    "Code": "KMS.AccessDeniedException",
    "Message": "The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access. (Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException; Request ID: a8adea02-c246-49d9-8b3d-ff6b6a43b40f; Proxy: null)",
    "Type": "Sender"
  },
  "RequestId": "9565c451-742c-55f3-a1eb-9f3641fd30aa"
}

A KMS.AccessDeniedException error occurs when the API integration execution role can’t perform operations through AWS Key Management Service (AWS KMS). Permission must be configured to perform operations on the AWS KMS keys that are attached to the Amazon SQS server-side encrypted queue.

The following example includes the permissions needed to perform operations on the KMS keys attached to the SQS queue. Replace example-account-id with your AWS account ID, and example-api-gw-integration-execution-role with your execution role name.

{
  "Sid": "Allow use of the key",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::example-account-id:role/example-api-gw-integration-execution-role"
  },
  "Action": [
    "kms:Encrypt",
    "kms:GenerateDataKey*",
    "kms:Decrypt"
  ],
  "Resource": "*"
}

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support

How can I invoke an API Gateway private API using an Application or Network Load balancer?

Last updated: 2022-11-03

I want to set up my API Gateway private API as a target behind a load balancer. Then, I want to access my private API from an Application or Network Load Balancer using my account or another AWS account. How do I set this up?

Short description

To access your private API using AWS Direct Connect or Amazon Route 53, see How to invoke a private API. You can also access an API Gateway private REST API in another AWS account using an interface VPC endpoint.

In the following set up, the private API is added as a target to the load balancer. This is done using the Amazon Virtual Private Cloud (Amazon VPC) endpoint elastic network interface IP address.

Important: Custom domain names are not supported for private APIs. As a workaround, you can invoke and attach the domain to a load balancer. Then, invoke the private API using the setup described in this article.

Resolution

Create an Amazon VPC endpoint

1. Open the Amazon VPC console, choose Endpoints , and then choose Create endpoint .
Note: If you have any Amazon VPC endpoints already set up in your VPC using the API execute-api, then make sure that private DNS is disabled.

2. For Services , choose com.amazonaws.com.your-region.execute-api .

3. For VPC , choose your Amazon VPC.

4. For Subnets , choose two subnets in different Availability Zones (AZ IDs), and then choose Create endpoint .

5. Choose your endpoint, choose subnets , and copy the IP address . You use this IP address in another step.

For more information, see Create an interface VPC endpoint for API Gateway execute-api.

Create a private REST API and grant the Amazon VPC endpoint permission

1. Open the API Gateway console, and then choose Create API .

2. For REST API , choose Build .

3. In Settings , enter the following:
For API name , enter a name for the API.
For Endpoint Type , choose Private .
For Endpoint IDs , enter and the endpoint ID that you created previously.

4. Choose Create API .

5. In the navigation pane, choose Resource Policy .

6. In the resource policy editor, paste the following policy:
Note: Replace vpce-<id> with your VPC endpoint ID.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "execute-api:Invoke",
      "Resource": "arn:aws:execute-api:<region>:<account :<api-id>/*/*/*",
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-081234d1ad408e"
        }
      }
    },
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "execute-api:Invoke",
      "Resource": "arn:aws:execute-api:<region>:<account-id>:<api-id>/*/*/*"
    }
  ]
}

7. Choose Save .

For more information, see Create a private API using the API Gateway console.

Create or import an AWS Certificate Manager public certificate

If you haven't already done so, do one of the following:

Request a public certificate.

-or-

Import a certificate

Create an Application Load Balancer or a Network Load Balancer

If you haven't already done so, do one of the following:

Create an Application Load Balancer

-or-

Create a Network Load Balancer

Create the target group

1. Open the Amazon Elastic Compute Cloud (Amazon EC2) console.

2. In the navigation pane, in Load Balancing , choose Load Balancers , choose Target Groups , and then choose Create target group .

3. For target type , choose IP addresses .

4. for Target group name , enter a name.

Application Load Balancer

For Protocol , choose HTTPS .
For Port , choose 443 .
For VPC , choose your VPC.
For Health check path, enter 200,403. (This makes sure that the VPC endpoint shows as Healthy in the target group)

Network Load Balancer

For Protocol , choose TLS .
For Port , choose 443 .
For VPC , choose your VPC.

5. Choose Next .

6. In Specify IPs , enter the IP address that you copied in the Create an interface Amazon VPC endpoint section, and choose Add IPv4 address .

7. Choose Create target group .

Configure the load balancer

1. Open the EC2 console.

2. In the navigation pane, choose Load Balancers , and then choose Create Load Balancer .

Application Load Balancer

For Scheme , choose either Internet-facing or Internal depending on your configuration.
For Protocol , choose HTTPS .
For VPC and subnets , choose your VPC and subnets.

Network Load Balancer For Scheme , choose either Internet-facing or Internal depending on your configuration.
For Protocol , choose TLS .
For VPC and subnets , choose your VPC and subnets.
For Security policy , choose the default policy ELBSecurityPolicy-TLS (recommended) .
For Default SSL/TLS certificate , choose From ACM .
Choose the Select a certificate dropdown menu, and then choose your certificate.

3. Choose Create load balancer .

Note: The load balancer targets should be the IP addresses of the elastic network interface that the VPC endpoint created. These were done in step 1. You can find those elastic network interfaces by selecting your VPC endpoint and opening the Subnets tab.

Create a record in an Amazon Route 53 public or private hosted zone

If you haven't already done so, do one of the following:

Create a public hosted zone

-or-

Create a private hosted zone

Then, create a CNAME record and associate it with your Application or Network Load Balancer.

Testing

For public load balancers, you can make a curl request from your local machine.

For private load balancers, launch a new EC2 instance in one of the subnets for your load balancer. Then, make a curl request similar to the following:

curl -v https://{custom-domain-name}/<stage-name>/<resource-path> -H 'Host: <api-id>.execute-api.<region>.amazonaws.com'

-or-

curl -v https://{custom-domain-name}/<stage-name>/<resource-path> -H 'x-apigw-api-id:{api-id}'

A successful request returns a 200 OK response code. An unsuccessful request returns a 403 Forbidden response code or a DNS resolution error. If you encounter any issues, see Troubleshoot your load balancers.

Related information

How do I connect to a private API Gateway over a Direct Connect connection?

How do I troubleshoot issues when connecting to an API Gateway private API endpoint?

Setting up CloudWatch logging for a REST API in API Gateway

Monitoring REST APIs with Amazon CloudWatch metrics

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support

API	Example	Stage	Path	Default URL	Custom Domain URL
abcd3456ef	API 1	dev	(none)	https://abcd3456ef.execute-api.us-east-1.amazonaws.com/dev	https://api.example.com
wxyz1234ab	API 2	test	orders	https://wxyz1234ab.execute-api.us-east-1.amazonaws.com/test	https://api.example.com/orders
mnop5678qr	API 3	prod	customers	https://mnop5678qr.execute-api.us-east-1.amazonaws.com/prod	https://api.example.com/customers

Welcome to Knowledge Base!

KB at your finger tips

Cloud-AWS

Forward host headers for an API Gateway REST API

How can I forward the host header with private integration for an API Gateway REST API?

Short description

Resolution

Create an API Gateway REST API

Configure your API's method request and Integration request

Deploy your API to a new stage

Related information