AWS Adventures: Adding a Dynamic List of IP Addresses to a Security Group Using Terraform
My current test automation infrastructure requires that I allow traffic from my build server to reach my internal test servers. However my build server has a changing set of IP addresses it uses for its build machines. As a result I periodically get connection timeouts because new build machine IP’s are not in the current security group, while decommissioned build machine IP’s are.
Initially I updated this list manually and then re-applied Terraform code to update the security group. However I discovered that I could get a list of the build machine IP addresses through an API in JSON format. Additionally Terraform has a built-in HTTP client that can be used to get this list.
The HTTP data block below sets the URL and headers. The local variable stores the processed IP address list. In this example I need to transform each IP into CIDR format for the AWS Security Group. Then in the security group I can reference the local variable containing the new list. This updates each time terraform apply is run.
data "http" "build_machine_ips" {
url = "https://dnsjson.com/DNS_NAME/A.json"
request_headers = {
Accept = "application/json"
}
}locals { # Excessive formatting is only for post. the following code
# should all be one line.
build_machine_ip_list = sort([
for ip in jsondecode(
data.http.build_machine_ips.body
)["results"]["records"]: # Reformat list into CIDR notation for AWS
"${ip}/32"
])
}
resource "aws_security_group" "build_machine_sg" {
name = "NetworkPorts"
description = "Allow external access"
vpc_id = "xxxxxxxx" # Add your VPC
ingress {
from_port = 80
to_port = 80
protocol = "TCP"
cidr_blocks = local.build_machine_ip_list
description = "Access to internal servers from build machines"
}
}
Now it updates automatically when the infrastructure is applied.
