Integrating AWS API Gateway with AWS Lambda and Configuring Dev/QA/Prod Environments

This article is NOT about introducing the basic usage of AWS API Gateway and AWS Lambda.

In simple terms, with AWS API Gateway and Lambda, plus a database (preferably AWS DynamoDB, or MySQL installed on an AWS EC2 instance. You can use non-AWS databases too, but access speed might be affected since using AWS services allows for faster internal network communication), you can develop highly RESTful APIs without any web application framework. This is simply a godsend for small to medium-sized companies looking to boost development efficiency!

To put it even simpler: API Gateway provides you with a URL. When this URL is accessed, it triggers a specific Lambda function you have configured. Once the function executes, it returns the data you specified. In other words, the “do a bunch of stuff” logic and the Response that used to happen on your server after receiving a request can now all be written within a single Lambda function.

Typically, if the browser uses AJAX to access this URL, cross-origin issues may arise. Fortunately, API Gateway thoughtfully provides an Enable CORS option!

AWS Lambda currently supports Python, Java, and NodeJS. Although called Lambda, this function isn’t necessarily related to functional programming; you write it just as you did before. Interestingly, Lambda functions can call each other. Besides API Gateway, many other events unrelated to API Gateway can also serve as triggers (for example, when a new user record is added to a DynamoDB table, trigger a Lambda function named sendWelcomeEmailToUser() to send a welcome email to the user).

So, how do you configure AWS API Gateway and AWS Lambda to meet the requirements of Development, Testing, and Production environments? Do you really need to create 3 separate APIs for the same functionality and 3 separate Lambdas?

All the following content involves settings made via the AWS Web UI. (Besides UI settings, you can also write Shell scripts to batch execute these settings, though I haven’t learned that part yet…)

Oh, I forgot to mention… First, you need an API, and then you need a Lambda.
Assumptions:

• The API endpoint is /stageTest, and the method is GET.
• The Lambda name is stageTest. (The endpoint name and Lambda name can be different… forgive my lack of naming skills…)

Setting Up DEV / QA / PROD Environments

1. Create different stages for API Gateway^1. Assume we created 3 stages: dev, qa, and prod.
   

   After setting the stages, your API endpoints will look something like this:

   • .execute-api..amazonaws.com/dev
   • .execute-api..amazonaws.com/qa
   • .execute-api..amazonaws.com/prod

2. Set stage variables for each Stage^2. Assume we set a variable named env for each Stage.
   • The value of the env variable in the dev stage is DEV
   • The value of the env variable in the qa stage is QA
   • The value of the env variable in the prod stage is PROD
3. Create different aliases for the Lambda stageTest() function^3. Assume we created 3 aliases: DEV, QA, and PROD, pointing to different versions.
   

   • After every deployment, the version is $LATEST. Click Action > Publish New Version to publish different versions.
   • It is recommended to point DEV to the $LATEST version, while QA and PROD point to stable versions.
   • Once aliases are created for the Lambda, the access names become: stageTest, stageTest:DEV, stageTest:QA, stageTest:PROD. Here, stageTest always points to the $LATEST version, while others point to their respective configured versions.

4. Integrate the GET method of API /stageTest with Lambda stageTest()^5
   • First, in the API Gateway interface, set the Integration Request for each API method: Select Integration Type as Lambda Function; for Lambda Function, enter stageTest:${stageVariables.env}. ${stageVariables.env} is the variable set in step 2.
   • When setting the Lambda Function name, remember to click the checkmark (√) next to it to save. Upon saving, AWS will pop up a dialog requiring you to add permissions to the function to use the stageVariables. When adding permissions, you must execute the command three times, replacing the corresponding position in the AWS-provided command with stageTest:DEV, stageTest:QA, and stageTest:PROD respectively.
   • Before executing the above commands, you must:
     • Install AWS CLI
     • Execute the command aws configure, entering your AWS ID and Key, and setting the default region to the region where your Lambda function resides.

Then, another problem arises…

• If you have already deployed many Lambdas and APIs without using stages… it seems unlikely you would manually set them one by one in the AWS interface.
• Even if you are already using stages, it still seems impractical to manually deploy the latest version and change alias pointers one by one in the AWS interface every time…

At this point, you must study AWS CLI, write shells, and leverage other tools for automated deployment…

Other References:

• http://docs.aws.amazon.com/apigateway/latest/developerguide/stage-variables.html
• http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html#stagevariables-template-reference

AWS API Gateway Mapping Template: Retrieving IP, Headers, etc.

Applicable Scenarios

When building RESTful APIs with API Gateway and Lambda, the payload received by Lambda is not typically a request object as obtained on a standard server. The payload received by Lambda does not contain request headers, but only contains the request body.

Some parts of the header can be retrieved from the mapping template using variables provided by API Gateway. Content that can be retrieved includes IP, HTTP method, route parameters, query strings, etc. For details, see the documentation ^1.

Setup Method

Theoretically, all AWS settings that can be done via the interface can also be done via scripts. Below, we only introduce the setup via the interface:

1. In the API Gateway interface, select the specific API method and click Integration Request on the right.
2. Click Mapping Templates at the bottom.
3. Click Add Mapping Template, enter application/json, and click √ to confirm.
4. In the newly appeared area on the right, click the edit button (pencil icon) and input the required variables. For example, to retrieve the IP:
   

   This table is wide — scroll sideways to see it all

   { “body”: $input.json(‘$’), “source_ip” : “$context.identity.sourceIp”, }

Here, $input.json('$') represents the body sent by the client during the request.

Response Headers and Cookies

When using Lambda and API Gateway together, handling headers and cookies can be quite troublesome… The content returned by Lambda is only the response body, so headers must be set in API Gateway, and there are some limitations:

1. In the API Gateway interface, select the specific API method and click Method Response on the right.
2. Expand HTTP Status and click Add Header to add a header.
   

   Note: You can add multiple headers, but they cannot have duplicate names. This means there can only be one Set-Cookie, allowing only one cookie to be added.

3. After completing the above steps, return to the page from step 1, click Integration Response, expand the triangle, and you will see Header Mappings.
4. Edit the mapping value in the corresponding position.
   • If you want to use data returned by Lambda, set it to integration.response.body.KEY.
   • If you are not using Lambda (i.e., using the gateway as a proxy) and instead using headers returned by the actual server, set it to integration.response.header.KEY.

References

• http://docs.aws.amazon.com/zh_cn/apigateway/latest/developerguide/how-to-method-settings-execution-console.html
• http://docs.aws.amazon.com/zh_cn/apigateway/latest/developerguide/request-response-data-mappings.html