Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions gateway/perf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# API Platform Gateway Performance Test Results

<!-- PERF_RESULTS_START -->

**4 Cpu Gateway Runtime**

| Scenario | Users | Throughput | Avg Response Time(ms) | Err % | p90 (ms) | p99 (ms) | Samples |
| --- | --- | --- | --- | --- | --- | --- | --- |
| API Gateway JWT GET | 100 | 7205.05 | 13.69 | 0.00 | 20.00 | 27.00 | 5181719 |
| API Gateway JWT GET | 500 | 7629.20 | 65.18 | 0.00 | 85.00 | 110.00 | 5487063 |
| API Gateway JWT GET | 1000 | 7445.64 | 132.50 | 0.00 | 177.00 | 240.00 | 5357118 |
| API Gateway Plain GET | 100 | 9521.54 | 10.33 | 0.00 | 15.00 | 21.00 | 6849061 |
| API Gateway Plain GET | 500 | 9339.11 | 53.15 | 0.00 | 67.00 | 84.00 | 6716685 |
| API Gateway Plain GET | 1000 | 9202.58 | 108.32 | 0.00 | 133.00 | 163.00 | 6621784 |
| API Gateway Header Policy GET | 100 | 8879.21 | 11.09 | 0.00 | 16.00 | 23.00 | 6386256 |
| API Gateway Header Policy GET | 500 | 8706.58 | 57.19 | 0.00 | 74.00 | 93.00 | 6262678 |
| API Gateway Header Policy GET | 1000 | 8667.24 | 114.66 | 0.00 | 143.00 | 177.00 | 6236016 |

**2 Cpu Gateway Runtime**

| Scenario | Users | Throughput | Avg Response Time(ms) | Err % | p90 (ms) | p99 (ms) | Samples |
| --- | --- | --- | --- | --- | --- | --- | --- |
| API Gateway JWT GET | 100 | 5278.32 | 18.74 | 0.00 | 40.00 | 51.00 | 3796087 |
| API Gateway JWT GET | 500 | 4939.69 | 100.40 | 0.00 | 117.00 | 153.00 | 3553591 |
| API Gateway JWT GET | 1000 | 4972.15 | 200.46 | 0.00 | 229.00 | 265.00 | 3577539 |
| API Gateway Plain GET | 100 | 6149.57 | 16.07 | 0.00 | 35.00 | 45.00 | 4423998 |
| API Gateway Plain GET | 500 | 5707.35 | 87.12 | 0.00 | 105.00 | 137.00 | 4105722 |
| API Gateway Plain GET | 1000 | 5598.03 | 178.28 | 0.00 | 203.00 | 245.00 | 4027863 |
| API Gateway Header Policy GET | 100 | 5751.46 | 17.19 | 0.00 | 37.00 | 47.00 | 4136573 |
| API Gateway Header Policy GET | 500 | 5345.05 | 93.27 | 0.00 | 108.00 | 143.00 | 3845566 |
| API Gateway Header Policy GET | 1000 | 5312.74 | 186.91 | 0.00 | 213.00 | 257.00 | 3822285 |

<!-- PERF_RESULTS_END -->
16 changes: 16 additions & 0 deletions gateway/perf/api-gateway-eks-perf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# `api-gateway-eks-perf` — Jenkins slave orchestrator

This directory lives on the **Jenkins slave** perf workspace. It is the job entrypoint; it does **not** contain JMeter scenarios or RestApi deploy logic.

Those live in **`../performance-test-scripts/`** (in git: `gateway/perf/performance-test-scripts`), which this orchestrator sparse-clones via `PERF_SCRIPTS` on every run.

| File | Role |
|------|------|
| `run-api-gateway-eks-tests.sh` | Full pipeline (EKS → gateway → JMeter → results → cleanup) |
| `create-jmeter-ec2s.sh` | 1 client + 2 server EC2s in the EKS VPC |
| `eks-cluster-perf.yaml.template` | eksctl cluster template |
| `cleanup.sh` | EXIT trap: terminate EC2s, delete cluster |

**To extend scenarios / RestApis / JMX:** edit `performance-test-scripts` and point `PERF_SCRIPTS` at your branch. See that folder’s [README](../performance-test-scripts/README.md).

**Change this folder only when:** job infra changes (EC2 counts, VPC wiring, clone paths, `env.eks` generation defaults, cleanup).
67 changes: 67 additions & 0 deletions gateway/perf/api-gateway-eks-perf/cleanup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/bin/bash
# Clean up all resources created by run-api-gateway-eks-tests.sh.
# Called from the EXIT trap; must not fail hard (|| true guards).
#
# Required env vars:
# EKS_CLUSTER_NAME, AWS_REGION
# STATE_FILE path to the jmeter state file written by create-jmeter-ec2s.sh

set -uo pipefail

AWS="aws --region ${AWS_REGION:-us-east-1}"

echo "==> Cleanup: EKS cluster ${EKS_CLUSTER_NAME:-<not set>}, region ${AWS_REGION:-us-east-1}"

# Load JMeter EC2 state if it exists.
if [[ -f "${STATE_FILE:-}" ]]; then
# shellcheck source=/dev/null
source "${STATE_FILE}"
fi

# Terminate JMeter EC2s first.
INSTANCE_IDS=()
for var in CLIENT_ID SERVER1_ID SERVER2_ID; do
id="${!var:-}"
[[ -n "$id" && "$id" != "None" ]] && INSTANCE_IDS+=("$id")
done

if [[ ${#INSTANCE_IDS[@]} -gt 0 ]]; then
echo "==> Terminating JMeter EC2s: ${INSTANCE_IDS[*]}"
${AWS} ec2 terminate-instances --instance-ids "${INSTANCE_IDS[@]}" >/dev/null 2>&1 || true
echo " Waiting for termination..."
${AWS} ec2 wait instance-terminated --instance-ids "${INSTANCE_IDS[@]}" 2>/dev/null || true
echo " JMeter EC2s terminated."
fi

# Remove JMeter SG rule from EKS node SG.
if [[ -n "${NODE_SG:-}" && -n "${JMETER_SG_ID:-}" ]]; then
echo "==> Removing JMeter SG rule from EKS node SG ${NODE_SG}"
${AWS} ec2 revoke-security-group-ingress \
--group-id "${NODE_SG}" \
--ip-permissions \
"IpProtocol=tcp,FromPort=0,ToPort=65535,UserIdGroupPairs=[{GroupId=${JMETER_SG_ID}}]" \
2>/dev/null || true
fi

# Delete JMeter security group (must wait for EC2s to terminate first).
if [[ -n "${JMETER_SG_ID:-}" ]]; then
echo "==> Deleting JMeter security group ${JMETER_SG_ID}"
${AWS} ec2 delete-security-group --group-id "${JMETER_SG_ID}" 2>/dev/null || \
echo " (SG delete failed — may have dependent resources still detaching; manual cleanup needed)"
fi

# Delete EKS cluster.
if [[ -n "${EKS_CLUSTER_NAME:-}" ]]; then
echo "==> Deleting EKS cluster ${EKS_CLUSTER_NAME} (this takes 10-20 min, runs async)..."
eksctl delete cluster --name "${EKS_CLUSTER_NAME}" --region "${AWS_REGION:-us-east-1}" \
--wait 2>/dev/null || true
echo " EKS cluster deletion initiated."
Comment thread
Milanka00 marked this conversation as resolved.
fi

# Remove per-run state file.
if [[ -f "${STATE_FILE:-}" ]]; then
rm -f "${STATE_FILE}"
echo " Removed state file ${STATE_FILE}"
fi

echo "==> Cleanup complete."
134 changes: 134 additions & 0 deletions gateway/perf/api-gateway-eks-perf/create-jmeter-ec2s.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
#!/bin/bash -e
# Create 3 JMeter EC2s (1 client + 2 servers) in the EKS VPC and write state to a file.
#
# Required env vars (set by run-api-gateway-eks-tests.sh):
# EKS_CLUSTER_NAME, AWS_REGION, JMETER_SG_TAG, JMETER_KEY_NAME,
# JMETER_CLIENT_EC2_INSTANCE_TYPE, JMETER_SERVER_EC2_INSTANCE_TYPE,
# STATE_FILE (path to write instance IDs and IPs)

set -euo pipefail

: "${EKS_CLUSTER_NAME:?}"
: "${AWS_REGION:?}"
: "${JMETER_KEY_NAME:?}"
: "${JMETER_CLIENT_EC2_INSTANCE_TYPE:?}"
: "${JMETER_SERVER_EC2_INSTANCE_TYPE:?}"
: "${STATE_FILE:?}"

AWS="aws --region ${AWS_REGION}"

echo "==> Getting EKS VPC info for cluster ${EKS_CLUSTER_NAME}"
VPC_ID=$(${AWS} eks describe-cluster --name "${EKS_CLUSTER_NAME}" \
--query "cluster.resourcesVpcConfig.vpcId" --output text)
echo " VPC: ${VPC_ID}"

# Get a public subnet in the EKS VPC (tagged by eksctl for public load balancer).
PUBLIC_SUBNET_ID=$(${AWS} ec2 describe-subnets \
--filters "Name=vpc-id,Values=${VPC_ID}" \
"Name=tag:kubernetes.io/role/elb,Values=1" \
"Name=state,Values=available" \
--query "Subnets[0].SubnetId" --output text)

if [[ -z "${PUBLIC_SUBNET_ID}" || "${PUBLIC_SUBNET_ID}" == "None" ]]; then
echo "ERROR: No public subnet (tag kubernetes.io/role/elb=1) found in VPC ${VPC_ID}." >&2
echo " eksctl usually creates these; check VPC subnet tags." >&2
exit 1
fi
echo " Subnet: ${PUBLIC_SUBNET_ID}"

# Create security group for JMeter EC2s.
JMETER_SG_NAME="${JMETER_SG_TAG:-jmeter-perf}"
echo "==> Creating JMeter security group: ${JMETER_SG_NAME}"
JMETER_SG_ID=$(${AWS} ec2 create-security-group \
--group-name "${JMETER_SG_NAME}" \
--description "JMeter perf test instances for ${EKS_CLUSTER_NAME}" \
--vpc-id "${VPC_ID}" \
--query "GroupId" --output text)
echo " JMeter SG: ${JMETER_SG_ID}"

# Allow SSH from anywhere (Jenkins slave can SSH to JMeter EC2s).
${AWS} ec2 authorize-security-group-ingress \
--group-id "${JMETER_SG_ID}" \
--protocol tcp --port 22 --cidr 0.0.0.0/0
Comment thread
Milanka00 marked this conversation as resolved.

# Allow all TCP between JMeter instances themselves (RMI + results callbacks).
${AWS} ec2 authorize-security-group-ingress \
--group-id "${JMETER_SG_ID}" \
--ip-permissions \
"IpProtocol=tcp,FromPort=0,ToPort=65535,UserIdGroupPairs=[{GroupId=${JMETER_SG_ID}}]"

# Get EKS cluster security group and allow JMeter access on NodePort range.
NODE_SG=$(${AWS} eks describe-cluster --name "${EKS_CLUSTER_NAME}" \
--query "cluster.resourcesVpcConfig.clusterSecurityGroupId" --output text)
echo " EKS cluster SG: ${NODE_SG}"

${AWS} ec2 authorize-security-group-ingress \
--group-id "${NODE_SG}" \
--ip-permissions \
"IpProtocol=tcp,FromPort=0,ToPort=65535,UserIdGroupPairs=[{GroupId=${JMETER_SG_ID}}]"

# Get latest Amazon Linux 2023 AMI.
AMI_ID=$(${AWS} ssm get-parameter \
--name /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--query "Parameter.Value" --output text)
echo "==> Using AMI: ${AMI_ID}"

launch_ec2() {
local role="$1" instance_type="$2"
${AWS} ec2 run-instances \
--image-id "${AMI_ID}" \
--instance-type "${instance_type}" \
--key-name "${JMETER_KEY_NAME}" \
--security-group-ids "${JMETER_SG_ID}" \
--subnet-id "${PUBLIC_SUBNET_ID}" \
--associate-public-ip-address \
--block-device-mappings "DeviceName=/dev/xvda,Ebs={VolumeSize=50,VolumeType=gp3,DeleteOnTermination=true}" \
--tag-specifications \
"ResourceType=instance,Tags=[{Key=Name,Value=${JMETER_SG_NAME}-${role}},{Key=project,Value=${EKS_CLUSTER_NAME}},{Key=jmeter-perf-tag,Value=${JMETER_SG_NAME}}]" \
--query "Instances[0].InstanceId" --output text
}

echo "==> Launching JMeter EC2s"
CLIENT_ID=$(launch_ec2 "client" "${JMETER_CLIENT_EC2_INSTANCE_TYPE}")
SERVER1_ID=$(launch_ec2 "server-1" "${JMETER_SERVER_EC2_INSTANCE_TYPE}")
SERVER2_ID=$(launch_ec2 "server-2" "${JMETER_SERVER_EC2_INSTANCE_TYPE}")
echo " client: ${CLIENT_ID}"
echo " server-1: ${SERVER1_ID}"
echo " server-2: ${SERVER2_ID}"

echo "==> Waiting for instances to be running (~60s)..."
${AWS} ec2 wait instance-running --instance-ids "${CLIENT_ID}" "${SERVER1_ID}" "${SERVER2_ID}"

# Get public + private IPs.
get_ip() {
local id="$1" type="$2"
${AWS} ec2 describe-instances --instance-ids "${id}" \
--query "Reservations[0].Instances[0].${type}IpAddress" --output text
}

CLIENT_PUBLIC_IP=$(get_ip "${CLIENT_ID}" Public)
SERVER1_PUBLIC_IP=$(get_ip "${SERVER1_ID}" Public)
SERVER2_PUBLIC_IP=$(get_ip "${SERVER2_ID}" Public)
SERVER1_PRIVATE_IP=$(get_ip "${SERVER1_ID}" Private)
SERVER2_PRIVATE_IP=$(get_ip "${SERVER2_ID}" Private)

echo ""
echo " client ${CLIENT_ID} public=${CLIENT_PUBLIC_IP}"
echo " server-1 ${SERVER1_ID} public=${SERVER1_PUBLIC_IP} private=${SERVER1_PRIVATE_IP}"
echo " server-2 ${SERVER2_ID} public=${SERVER2_PUBLIC_IP} private=${SERVER2_PRIVATE_IP}"

# Write state file for use by main script and cleanup.
cat >"${STATE_FILE}" <<EOF
JMETER_SG_ID=${JMETER_SG_ID}
NODE_SG=${NODE_SG}
VPC_ID=${VPC_ID}
CLIENT_ID=${CLIENT_ID}
SERVER1_ID=${SERVER1_ID}
SERVER2_ID=${SERVER2_ID}
CLIENT_PUBLIC_IP=${CLIENT_PUBLIC_IP}
SERVER1_PUBLIC_IP=${SERVER1_PUBLIC_IP}
SERVER2_PUBLIC_IP=${SERVER2_PUBLIC_IP}
SERVER1_PRIVATE_IP=${SERVER1_PRIVATE_IP}
SERVER2_PRIVATE_IP=${SERVER2_PRIVATE_IP}
EOF
Comment thread
Milanka00 marked this conversation as resolved.
echo "==> State written to ${STATE_FILE}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: ${EKS_CLUSTER_NAME}
region: ${AWS_REGION}
version: "${EKS_K8S_VERSION}"

# OIDC is required for IRSA (EBS CSI driver service account).
iam:
withOIDC: true

addons:
- name: aws-ebs-csi-driver
wellKnownPolicies:
ebsCSIController: true

managedNodeGroups:
- name: gateway-ng
instanceType: ${EKS_NODE_INSTANCE_TYPE}
desiredCapacity: ${EKS_GATEWAY_NODE_DESIRED}
minSize: 1
maxSize: 8
volumeSize: 50
privateNetworking: true
labels:
role: gateway-perf
tags:
project: ${EKS_CLUSTER_NAME}
Comment thread
Milanka00 marked this conversation as resolved.

- name: backend-ng
instanceType: ${EKS_BACKEND_NODE_INSTANCE_TYPE}
desiredCapacity: 1
minSize: 1
maxSize: 2
volumeSize: 30
privateNetworking: true
labels:
workload: backend
tags:
project: ${EKS_CLUSTER_NAME}
Loading
Loading