Skip to content

[Bug]: Intermittent 409 Topic already exists on Create & "Namespace not empty" on Delete #179

Description

@Akzz123

Description

We are experiencing intermittent execution failures when managing Pulsar resources (Tenants, Namespaces, Topics) using Terraform with the streamnative/pulsar provider. The issues occur in about 1% of our pipeline runs and appear to be related to eventual consistency or state synchronization.

We encounter two main variations of this issue:

1. Creation Conflict (Topic Already Exists)

Terraform runs a plan, confirms the topic does not exist in the state, and attempts to create it. The API returns a 409 Conflict, implying the topic actually exists in the cluster but was not detected during the refresh/plan phase.

2. Deletion Race Condition (Namespace Not Empty)

During a destroy or when removing a namespace, Terraform successfully reports the deletion of the pulsar_topic resources. However, when it immediately attempts to delete the parent pulsar_namespace, the operations fail with a "Namespace not empty" error. This suggests the provider does not wait for the topic deletion to fully propagate or be confirmed before attempting to remove the namespace.

Terraform Version

Terraform v1.x (Provider streamnative/pulsar version ~> 0.4.6)

Affected Resource(s)

pulsar_topic
pulsar_namespace
Debug Output / Error Logs

Scenario A: Creation Failure

The plan shows the resource will be created:

Terraform will perform the following actions:
  # module.topics["...-TransmissionRequestIntegrationEvent"].pulsar_topic.topic[0] will be created
  + resource "pulsar_topic" "topic" {
      + id         = (known after apply)
      + topic_name = "TransmissionRequestIntegrationEvent"
      + topic_type = "persistent"
      ...
    }

The apply fails immediately:

module.topics["viteglobal-18870-ayo-locus-2026-0401-TransmissionRequestIntegrationEvent"].pulsar_topic.topic[0]: Creating...
 
Error: ERROR_CREATE_TOPIC: code: 409 reason: This topic already exists
 
  on modules/topic/main.tf line 10, in resource "pulsar_topic" "topic":
  10: resource "pulsar_topic" "topic" {

Scenario B: Deletion Failure (Description)

No log provided, but behavior observed:
Terraform output will show pulsar_topic.topic: Destruction complete.
Immediately following, pulsar_namespace.namespace: Destroying... fails with an error stating the namespace cannot be deleted because it still contains topics.

Steps to Reproduce

  1. Run terraform apply to create a hierarchy of Tenant -> Namespace -> Topic. (Intermittently fails with 409).
  2. Run terraform destroy on an existing hierarchy. (Intermittently fails with Namespace not empty).

Important Context & Environment

We are running Pulsar in Kubernetes.

Setup: We initially pointed web_service_url to the standard K8s Service (..svc).
Optimization: Suspecting load balancing/caching issues, we switched the web_service_url to point directly to a specific broker pod via a Headless Service (...svc.cluster.local).
Result: While this reduced the frequency of errors, the issue persists (approx. 1 failure in every 100 runs).

Current Workarounds

For Creation: We run terraform import to verify the state against the real infrastructure or manually delete the "phantom" topic.
For Deletion: We have to manually force delete the topics/namespace in the cluster and re-run the pipeline.

Request

Creation: Can the provider implement a check-before-create or handle 409s gracefully by ensuring the resource properties match?
Deletion: Can the provider implement a retry mechanism or a "wait for empty" check when deleting namespaces to handle propagation delays?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions