Create an AWS Lambda Layer Containing Python Project Dependencies using Terraform

Here I will discuss how to develop the infrastructure for an AWS Lambda Layer with Terraform.

Let’s assume the following directory structure for some project where the code for the Lambda function lives:

my_project
├── lib
│   └── my_helper.py
├── my_lambda
│   └── lambda.py
├── terraform
│   └── main.tf
└── requirements.txt

To makes things a bit easier to manage, let’s define the following local variables:

locals {
  ...
  path_to_python_lib = "${path.root}/../lib"
  path_to_requirements_txt = "${path.root}/../requirements.txt"
}

Let’s start by defining a temporary_directory to contain our Python libraries and dependencies. It’s important to name the directory “python” as Lambda runtimes will look for that directory to discover our Python modules

data "temporary_directory" "python" {
  name = "python"
}

Next we need to fill in the directory with content. To do this we can use a null_resource resource to execute a pip command for us and copy local Python libraries:

resource "null_resource" "pip_install" {
  triggers = {
    everytime = timestamp()
  }

  provisioner local-exec {
    command = "pip install -r ${local.path_to_requirements_txt} -t ${data.temporary_directory.build.id} --isolated --quiet"
  }

  provisioner local-exec {
    command = "cp -Ra ${local.path_to_python_lib} ${data.temporary_directory.build.id}"
  }

  depends_on = [ data.temporary_directory.python ]
}

The options used for the pip install command are:

• -r - install via a .txt file listing package names and version numbers if any.
• -t - output the each package to the target location.
• --isolated - isolated mode, which bypasses environment variables and configuration. This is handy when the system requires a virtual environment to be activated prior to running pip install.
• --quite - output less install information to the console.

We should end up with the following contained in the temporary “python” folder:

python
├── bin
├── certifi
├── certifi-2024.2.2.dist.info
├── charset_normalizer
├── charset_normalizer-3.2.2.dist-info
├── idna
├── idna-3.6.dist.info
├── lib
├── requests
├── requests-2.31.0.dist.info
├── urllib3
└──…