Description
We are experiencing intermittent execution failures when managing Pulsar resources (Tenants, Namespaces, Topics) using Terraform with the streamnative/pulsar provider. The issues occur in about 1% of our pipeline runs and appear to be related to eventual consistency or state synchronization.
We encounter two main variations of this issue:
1. Creation Conflict (Topic Already Exists)
Terraform runs a plan, confirms the topic does not exist in the state, and attempts to create it. The API returns a 409 Conflict, implying the topic actually exists in the cluster but was not detected during the refresh/plan phase.
2. Deletion Race Condition (Namespace Not Empty)
During a destroy or when removing a namespace, Terraform successfully reports the deletion of the pulsar_topic resources. However, when it immediately attempts to delete the parent pulsar_namespace, the operations fail with a "Namespace not empty" error. This suggests the provider does not wait for the topic deletion to fully propagate or be confirmed before attempting to remove the namespace.
Terraform Version
Terraform v1.x (Provider streamnative/pulsar version ~> 0.4.6)
Affected Resource(s)
pulsar_topic
pulsar_namespace
Debug Output / Error Logs
Scenario A: Creation Failure
The plan shows the resource will be created:
Terraform will perform the following actions:
# module.topics["...-TransmissionRequestIntegrationEvent"].pulsar_topic.topic[0] will be created
+ resource "pulsar_topic" "topic" {
+ id = (known after apply)
+ topic_name = "TransmissionRequestIntegrationEvent"
+ topic_type = "persistent"
...
}
The apply fails immediately:
module.topics["viteglobal-18870-ayo-locus-2026-0401-TransmissionRequestIntegrationEvent"].pulsar_topic.topic[0]: Creating...
Error: ERROR_CREATE_TOPIC: code: 409 reason: This topic already exists
on modules/topic/main.tf line 10, in resource "pulsar_topic" "topic":
10: resource "pulsar_topic" "topic" {
Scenario B: Deletion Failure (Description)
No log provided, but behavior observed:
Terraform output will show pulsar_topic.topic: Destruction complete.
Immediately following, pulsar_namespace.namespace: Destroying... fails with an error stating the namespace cannot be deleted because it still contains topics.
Steps to Reproduce
- Run terraform apply to create a hierarchy of Tenant -> Namespace -> Topic. (Intermittently fails with 409).
- Run terraform destroy on an existing hierarchy. (Intermittently fails with Namespace not empty).
Important Context & Environment
We are running Pulsar in Kubernetes.
Setup: We initially pointed web_service_url to the standard K8s Service (..svc).
Optimization: Suspecting load balancing/caching issues, we switched the web_service_url to point directly to a specific broker pod via a Headless Service (...svc.cluster.local).
Result: While this reduced the frequency of errors, the issue persists (approx. 1 failure in every 100 runs).
Current Workarounds
For Creation: We run terraform import to verify the state against the real infrastructure or manually delete the "phantom" topic.
For Deletion: We have to manually force delete the topics/namespace in the cluster and re-run the pipeline.
Request
Creation: Can the provider implement a check-before-create or handle 409s gracefully by ensuring the resource properties match?
Deletion: Can the provider implement a retry mechanism or a "wait for empty" check when deleting namespaces to handle propagation delays?
Description
We are experiencing intermittent execution failures when managing Pulsar resources (Tenants, Namespaces, Topics) using Terraform with the streamnative/pulsar provider. The issues occur in about 1% of our pipeline runs and appear to be related to eventual consistency or state synchronization.
We encounter two main variations of this issue:
1. Creation Conflict (Topic Already Exists)
Terraform runs a plan, confirms the topic does not exist in the state, and attempts to create it. The API returns a 409 Conflict, implying the topic actually exists in the cluster but was not detected during the refresh/plan phase.
2. Deletion Race Condition (Namespace Not Empty)
During a destroy or when removing a namespace, Terraform successfully reports the deletion of the pulsar_topic resources. However, when it immediately attempts to delete the parent pulsar_namespace, the operations fail with a "Namespace not empty" error. This suggests the provider does not wait for the topic deletion to fully propagate or be confirmed before attempting to remove the namespace.
Terraform Version
Terraform v1.x (Provider streamnative/pulsar version ~> 0.4.6)
Affected Resource(s)
pulsar_topic
pulsar_namespace
Debug Output / Error Logs
Scenario A: Creation Failure
The plan shows the resource will be created:
The apply fails immediately:
Scenario B: Deletion Failure (Description)
No log provided, but behavior observed:
Terraform output will show pulsar_topic.topic: Destruction complete.
Immediately following, pulsar_namespace.namespace: Destroying... fails with an error stating the namespace cannot be deleted because it still contains topics.
Steps to Reproduce
Important Context & Environment
We are running Pulsar in Kubernetes.
Setup: We initially pointed web_service_url to the standard K8s Service (..svc).
Optimization: Suspecting load balancing/caching issues, we switched the web_service_url to point directly to a specific broker pod via a Headless Service (...svc.cluster.local).
Result: While this reduced the frequency of errors, the issue persists (approx. 1 failure in every 100 runs).
Current Workarounds
For Creation: We run terraform import to verify the state against the real infrastructure or manually delete the "phantom" topic.
For Deletion: We have to manually force delete the topics/namespace in the cluster and re-run the pipeline.
Request
Creation: Can the provider implement a check-before-create or handle 409s gracefully by ensuring the resource properties match?
Deletion: Can the provider implement a retry mechanism or a "wait for empty" check when deleting namespaces to handle propagation delays?