LocalStack – Mock AWS cloud in your local development

AWS is one of leading cloud provider in current times. It provides various cloud services. Only way to develop applications using these services is to create free tier account in AWS. But AWS free tier account is valid for only one year. After one year only limited services are free. After free tier period you always run the risk of using some services which might be charged to your card. With LocalStack software you no longer need to depend on AWS for your experiments.It provides an easy-to-use test/mocking framework for developing cloud applications.
In this blog post, I will explain how to install LocalStack in your development machine and start using it to test your cloud based applications.
Requirements
Installation and Running
You can install LocalStack in your machine in 3 ways.
Using pip
Open your command prompt and run the following command
Code language: Python (python)pip install localstack
After installation, run the following command to start Local stack. By default, its get started inside Docker container
Code language: Python (python)localstack start
Using Docker command
Run the following docker command to install and start LocalStack.
docker run --rm -it -p 4566:4566 -p 4571:4571 localstack/localstack
Code language: Python (python)
Using docker-compose
Download/copy the docker-compose.yml file from here and run the following command to install and start LocalStack.
Code language: Python (python)docker-compose up
LocalStack starts all its services on 4566 port.
You can check the status of LocalStack by pointing browser to http://localhost:4566

You can also check the health of each service by pointing the browser to http://localhost:4566/health

By default, LocalStack starts all the services. If you are looking to start only required services , you have to run the command like below. you have to specify the required service through the SERVICES option.
In Linux run below coomand
Code language: Python (python)SERVICES=s3,kinesis docker-compose up
In Windows run below command
Code language: Python (python)set SERVICES=s3,kinesis && docker-compose up

Creating and Accessing AWS Services
Once LocalStack is up and running, you can use the aws cli to create and access AWS services. Use the below command to install aws CLI
, if not installed already.
Code language: Python (python)pip install awscli
Setting up aws region and credentials to run LocalStack
aws cli requires the region and the credentials to be set in order to run the aws commands. Create the default configuration & the credentials using below command on terminal.
Code language: Python (python)aws configure --profile default
It will ask for ‘AWS Access Key ID’ and ‘AWS Secret Access Key’ enter test
for both of them. For all other options, take default options by pressing Enter key.

The above command creates config & credential files under <User-home>/.aws folder
Let’s create s3 buckets in LocalStack using aws cli. We need to point aws cli to LocalStack url to create infrastructure.
aws --endpoint-url=http://localhost:4566 s3 mb s3://mytestbucket
Code language: Java (java)

Now let’s check whether s3 bucket is created or not using below command.
aws --endpoint-url=http://localhost:4566 s3api list-buckets
Code language: Java (java)

Using awslocal
If you observe above commands , with every command you need to specify the endpoint-url which is annoying. Fortunately there is way to avoid it using package called `awslocal`
awslocal
is a thin CLI wrapper that runs commands directly against LocalStack (no need to specify --endpoint-url
anymore).
you can run the s3 bucket list command like below
Code language: Java (java)awslocal s3api list-buckets

Now lets look at the complex example to demonstrate the mocking of AWS infrastructure.

In this example I will demonstrate the vvent publshed to Kinesis stream is processed by AWS Lambda and logging the results to logs.
For AWS Lambda code which processes events from Kinesis stream , you can look at the one of my previous article and you can down load project code from GitHub
Check out the aws-kinesis-lambda
code from GitHub and setup the project.
The project contains template.yaml
SAM file which deploys and configures the lambda in AWS.
To deploy AWS lambda using SAM files , you need to install AWS SAM CLI. Please visit the link for download and install instructions.
Just like awslocal , we have wrapper around AWS SAM CLI called aws-sam-cli-local
. install samlocal with below command.
Code language: Python (python)pip install aws-sam-cli-local
Note
While installing aws-sam-cli-local
package in Windows 10, I faced some errors due to file path exceeding 260 chars. I followed instructions from link to overcome the problem
Deploying the AWS lambda.
Create the jar file by running following command from root of the Lambda project on commandline
Code language: Python (python)mvn clean package
To deploy the lambda run below command from root of the project.
samlocal deploy --stack-name aws-kinesis-lambda --capabilities CAPABILITY_IAM --guided
Code language: Python (python)
The command will ask for some inputs, you can opt for default values. It will create all the required resources for Lambda.


Next we can confirm Lambda function deployment by running the below command.
Code language: Java (java)awslocal lambda list-functions

Note down the function name highlighted in red colour#1.We will use this later.
Now we will create the Kinesis stream.
awslocal kinesis create-stream --stream-name lambda-stream --shard-count 3
Code language: Python (python)

Now verify the stream creation
Code language: Python (python)awslocal kinesis list-streams

We can see the stream details .

Note down the details highlighted in the red#2. We will use this later
Now we need to create event source mapping between Kinesis stream and Lambda to give events as input to Lambda for processing.
awslocal lambda create-event-source-mapping --function-name aws-kinesis-lambda-lambda-e385e9de --batch-size 100 --starting-position AT_TIMESTAMP --starting-position-timestamp 1541139109 --event-source-arn arn:aws:kinesis:us-east-1:000000000000:stream/lambda-stream
Code language: Python (python)
Note
–function-name aws-kinesis-lambda-lambda-e385e9de – is the lambda name ( Ref : #1)
–event-source-arn arn:aws:kinesis:us-east-1:000000000000:stream/lambda-stream – Is the arn of the of kinesis stream ( Ref : #2)
Let’s put a event in the Kinesis stream.
awslocal kinesis put-record --stream-name lambda-stream --partition-key 000 --data "[{\"id\" : 1, \"name\" : \"Suresh\" , \"address\" : \"Hyderabad\" , \"salary\": 20} , {\"id\" : 2, \"name\" : \"Alex\" , \"address\" : \"Auckland\" , \"salary\": 40}]"
Code language: Python (python)

In our Lambda class after receiving the records we are just logging them. So lets check the logs and confirm that our Lambda received records.
First find out the available log groups.
Code language: Python (python)awslocal logs describe-log-groups

Note down the logGroupName name from the output and use it in the below command to list all log streams for it.
awslocal logs describe-log-streams --log-group-name /aws/lambda/aws-kinesis-lambda-lambda-e385e9de
Code language: Python (python)

Take the latest logStreamName from the output and use it in the below command to view the log statements.
awslocal logs get-log-events --log-group-name /aws/lambda/aws-kinesis-lambda-lambda-e385e9de --log-stream-name 2021/06/07/[LATEST]1a08613e
Code language: Python (python)


In the above output we can view log statements from Lambda and the values are matching with data put into Kinesis stream.
If you want to practice AWS CLI commands , LocalStack is the best option available in your local development.
We can also use LocalStack along with Java in integration testing of cloud application. I will cover few examples on this topic in the coming weeks.
References
https://lobster1234.github.io/2017/04/05/working-with-localstack-command-line
https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html https://medium.com/@schogini/aws-kinesis-data-streams-a-tiny-cli-demo-9a20813bd082 https://github.com/localstack/localstack