Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
263 changes: 183 additions & 80 deletions docs/platforms/python/integrations/celery/index.mdx
Original file line number Diff line number Diff line change
@@ -1,26 +1,39 @@
---
title: Celery
description: "Learn about using Sentry with Celery."
description: "Learn how to set up Sentry in your Celery app, capture your first errors and traces, and view them in Sentry."
---

The Celery integration adds support for the [Celery Task Queue System](https://docs.celeryq.dev/).

<Include name="python-stream-mode-general-callout.mdx" />

## Install
## Prerequisites

Install `sentry-sdk` from PyPI:
You need:

```bash {tabTitle:pip}
pip install sentry-sdk
```
- A Sentry [account](https://sentry.io/signup/) and [project](/product/projects/)
- Your application up and running
- Celery `4.4.7+`
- Python `3.6+`

```bash {tabTitle:uv}
uv add sentry-sdk
```
<StepConnector selector="h2" showNumbers={true}>

## Install

<PlatformContent includePath="getting-started-install" />

## Configure

Choose the features you want to configure, and this guide will show you how:

<OnboardingOptionButtons
options={["error-monitoring", "performance", "profiling", "logs", "metrics"]}
/>

<Include name="quick-start-features-expandable" />
Comment thread
sentry[bot] marked this conversation as resolved.

Comment thread
inventarSarah marked this conversation as resolved.
### Initialize the Sentry SDK

Configuration should happen as **early as possible** in your application's lifecycle.

If you have the `celery` package in your dependencies, the Celery integration will be enabled automatically when you initialize the Sentry SDK.

<Alert>
Expand All @@ -33,17 +46,14 @@ If you have the `celery` package in your dependencies, the Celery integration wi

When using Celery without Django, you'll need to initialize the Sentry SDK in both your application and the Celery worker processes spawned by the Celery daemon.

In addition to capturing errors, you can use Sentry for [distributed tracing](/concepts/key-terms/tracing/) and [profiling](/product/profiling/). Select what you'd like to install to get the corresponding installation and configuration instructions below.

#### Set up Sentry in Celery Daemon or Worker Processes

<OnboardingOptionButtons
options={["error-monitoring", "performance", "profiling"]}
/>

```python {filename:tasks.py}
from celery import Celery, signals
import sentry_sdk
# ___PRODUCT_OPTION_START___ metrics
from sentry_sdk import metrics
# ___PRODUCT_OPTION_END___ metrics

# Initializing Celery
app = Celery("tasks", broker="...")
Expand Down Expand Up @@ -81,13 +91,12 @@ The [`celeryd_init`](https://docs.celeryq.dev/en/stable/userguide/signals.html?#

#### Set up Sentry in Your Application

<OnboardingOptionButtons
options={["error-monitoring", "performance", "profiling"]}
/>

```python {filename:main.py}
from tasks import add
import sentry_sdk
# ___PRODUCT_OPTION_START___ metrics
from sentry_sdk import metrics
# ___PRODUCT_OPTION_END___ metrics

def main():
# Initializing Sentry SDK in our process
Expand Down Expand Up @@ -124,11 +133,100 @@ if __name__ == "__main__":

If you're using Celery with Django in a typical setup, have initialized the SDK in your `settings.py` file (as described in the [Django integration documentation](/platforms/python/integrations/django/#configure)), and have your Celery configured to use the same settings as [`config_from_object`](https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html), there's no need to initialize the Celery SDK separately.

## Verify
To further customize your setup, review the [Options section](#options) below.

To confirm that your SDK is initialized on worker start, pass `debug=True` to `sentry_sdk.init()`. This will add extra output to your Celery logs when the SDK is initialized. If you see the output during worker startup, and not just after a task has started, then it's working correctly.
### Capturing Errors

Sentry automatically captures errors and exceptions raised in your Celery tasks and reports them as issues.

To learn how to manually report issues, see <PlatformLink to="/usage/">Capturing Errors</PlatformLink>.

<OnboardingOption optionId="performance">
### Instrumenting Your App

The Sentry SDK automatically creates spans for your Celery tasks, and propagates the trace from the code that enqueues a task to the worker that runs it.

You can also manually capture performance data – see <PlatformLink to="/tracing/instrumentation/custom-instrumentation/">Custom Instrumentation</PlatformLink> to learn more.

Comment thread
inventarSarah marked this conversation as resolved.
#### Distributed Traces

Distributed tracing extends the trace from the code running your Celery task to include the code that initiated the task.

You can disable this globally with the `propagate_traces` option, documented in the [options](#options) below. If you set `propagate_traces` to `False`, all Celery tasks will start their own trace.

<SplitLayout>
<SplitSection>
<SplitSectionText>

The snippet below includes an intentional `ZeroDivisionError` in the Celery task that will be captured by Sentry. To trigger the error call `debug_sentry.delay()`:
If you want to have more fine-grained control over trace distribution, you can override the `propagate_traces` option by passing the `sentry-propagate-traces` header when starting the Celery task:

<Alert>

The `CeleryIntegration` does not utilize the `traces_sample_rate` config option for deciding if a trace should be propagated into a Celery task.

</Alert>

</SplitSectionText>
<SplitSectionCode>

```python
import sentry_sdk

# Enable global distributed traces (this is the default, just to be explicit)
sentry_sdk.init(
# same as above
integrations=[
CeleryIntegration(
propagate_traces=True
),
],
)

# This will propagate the trace:
my_task_a.delay("some parameter")

# This will propagate the trace:
my_task_b.apply_async(
args=("some_parameter", )
)

# This will NOT propagate the trace. The task will start its own trace:
my_task_b.apply_async(
args=("some_parameter", ),
headers={"sentry-propagate-traces": False},
)

# Note: overriding the tracing behaviour using `task_x.delay()` is not possible.
```

</SplitSectionCode>
</SplitSection>
</SplitLayout>

<Alert level="warning" title="Note on distributed tracing for Celery versions 4.x">

Sentry uses custom message headers for distributed tracing. For Celery versions 4.x, with [message protocol version 1](https://docs.celeryq.dev/en/stable/internals/protocol.html#version-1), this functionality is broken, and Celery fails to propagate custom headers to the worker. Protocol version 2, the default since Celery 4.0, is not affected.

The fix for the custom headers propagation issue was introduced to the Celery project ([PR](https://github.com/celery/celery/pull/6374)) starting with version 5.0.1. However, the fix was not backported to versions 4.x.

</Alert>

</OnboardingOption>

## Verify Your Setup

Let's test your setup and confirm that data reaches your Sentry project.

### Issues

<SplitLayout>
<SplitSection>
<SplitSectionText>

To verify that Sentry captures errors and creates issues in your Sentry project, add this intentional error to your application:

</SplitSectionText>
<SplitSectionCode>

```python {filename:tasks.py}
from celery import Celery, signals
Expand All @@ -145,16 +243,42 @@ def debug_sentry():
1/0
```

<Alert title="Note on distributed tracing">
</SplitSectionCode>
</SplitSection>
</SplitLayout>

Trigger the error by calling `debug_sentry.delay()`.

Sentry uses custom message headers for distributed tracing. For Celery versions 4.x, with [message protocol of version 1](https://docs.celeryq.dev/en/stable/internals/protocol.html#version-1), this functionality is broken, and Celery fails to propagate custom headers to the worker. Protocol version 2, which is the default since Celery version 4.0, is not affected.
To confirm that your SDK is initialized on worker start, pass `debug=True` to `sentry_sdk.init()`. This will add extra output to your Celery logs when the SDK is initialized. If you see the output during worker startup, and not just after a task has started, then it's working correctly.

The fix for the custom headers propagation issue was introduced to Celery project ([PR](https://github.com/celery/celery/pull/6374)) starting with version 5.0.1. However, the fix was not backported to versions 4.x.
<OnboardingOption optionId="performance">

</Alert>
<Include name="tracing/python-quick-start-verify-tracing-splitlayout.mdx" />

</OnboardingOption>

<OnboardingOption optionId="logs">

<Include name="logs/python-quick-start-verify-logs-splitlayout.mdx" />

</OnboardingOption>

<OnboardingOption optionId="metrics">

<Include name="metrics/python-quick-start-verify-metrics-splitlayout.mdx" />

</OnboardingOption>

### View Captured Data in Sentry

Now, head over to your project on [Sentry.io](https://sentry.io) to view the collected data (it takes a couple of moments for the data to appear).

<Include name="quick-start-locate-data-expandable" />

## Options

<Include name="python-stream-mode-integration-option-callout.mdx" />

To set options on `CeleryIntegration` to change its behavior, add it explicitly to your `sentry_sdk.init()`:

```python
Expand All @@ -177,79 +301,58 @@ sentry_sdk.init(

You can pass the following keyword arguments to `CeleryIntegration()`:

- `propagate_traces`

Propagate Sentry tracing information to the Celery task. This makes it possible to link Celery task errors to the function that triggered the task.

If this is set to `False`:
- errors in Celery tasks won't be matched to the triggering function.
- your Celery tasks will start a new trace and won't be connected to the trace in the calling function.
<SdkOption name='propagate_traces' type='bool' defaultValue='True'>

The default is `True`.
Propagate Sentry tracing information to the Celery task. This makes it possible to link Celery task errors to the function that triggered the task.

See [Distributed Traces](#distributed-traces) below to learn how to get more fine grained control over distributed tracing in Celery tasks.
If this is set to `False`:

- `monitor_beat_tasks`:
- errors in Celery tasks won't be matched to the triggering function.
- your Celery tasks will start a new trace and won't be connected to the trace in the calling function.

Turn auto-instrumentation on or off for Celery Beat tasks using Sentry Crons.
<OnboardingOption optionId="performance">

See <PlatformLink to="/crons/#celery-beat-auto-discovery">Celery Beat Auto Discovery</PlatformLink> to learn more.
See [Distributed Traces](#distributed-traces) to learn how to get more fine-grained control over distributed tracing in Celery tasks.

The default is `False`.
</OnboardingOption>

- `exclude_beat_tasks`:
</SdkOption>

A list of Celery Beat tasks that should be excluded from auto-instrumentation using Sentry Crons. Only applied if `monitor_beat_tasks` is set to `True`.
<SdkOption name='monitor_beat_tasks' type='bool' defaultValue='False'>

The list can contain strings with the names of tasks in the Celery Beat schedule to be excluded. It can also include regular expressions to match multiple tasks. For example, if you include `"payment-check-.*"` every task starting with `payment-check-` will be excluded from auto-instrumentation.
Turn auto-instrumentation on or off for Celery Beat tasks using Sentry Crons.

See <PlatformLink to="/crons/#celery-beat-auto-discovery">Celery Beat Auto Discovery</PlatformLink> to learn more.
See <PlatformLink to="/integrations/celery/crons/">Celery Beat Auto Discovery</PlatformLink> to learn more.

The default is `None`.
</SdkOption>

## Distributed Traces
<SdkOption name='exclude_beat_tasks' type='list[str]' defaultValue='None'>

Distributed tracing extends the trace from the code that's running your Celery task so that it includes the code that initiated the task.
A list of Celery Beat tasks that should be excluded from auto-instrumentation using Sentry Crons. Only applied if `monitor_beat_tasks` is set to `True`.

You can disable this globally with the `propagate_traces` parameter, documented above. If you set `propagate_traces` to `False`, all Celery tasks will start their own trace.
The list can contain strings with the names of tasks in the Celery Beat schedule to be excluded. It can also include regular expressions to match multiple tasks. For example, if you include `"payment-check-.*"` every task starting with `payment-check-` will be excluded from auto-instrumentation.

If you want to have more fine-grained control over trace distribution, you can override the `propagate_traces` option by passing the `sentry-propagate-traces` header when starting the Celery task:

**Note:** The `CeleryIntegration` does not utilize the `traces_sample_rate` config option for deciding if a trace should be propagated into a Celery task.
See <PlatformLink to="/integrations/celery/crons/">Celery Beat Auto Discovery</PlatformLink> to learn more.

```python
import sentry_sdk
</SdkOption>

# Enable global distributed traces (this is the default, just to be explicit)
sentry_sdk.init(
# same as above
integrations=[
CeleryIntegration(
propagate_traces=True
),
],
)
## Next Steps

# This will propagate the trace:
my_task_a.delay("some parameter")
At this point, you should have integrated Sentry into your Celery application and should already be sending data to your Sentry project.

# This will propagate the trace:
my_task_b.apply_async(
args=("some_parameter", )
)
Now's a good time to customize your setup and look into more advanced topics.
Our next recommended steps for you are:

# This will NOT propagate the trace. The task will start its own trace:
my_task_b.apply_async(
args=("some_parameter", ),
headers={"sentry-propagate-traces": False},
)
- Explore [practical guides](/guides/) on what to monitor, log, track, and investigate after setup
- Continue to <PlatformLink to="/configuration/">customize your configuration</PlatformLink>
- Learn more about <PlatformLink to="/usage/">manually capturing errors or messages</PlatformLink>
- Dive straight into the API with our [API docs](https://getsentry.github.io/sentry-python/)

# Note: overriding the tracing behaviour using `task_x.delay()` is not possible.
```
<Expandable permalink={false} title="Are you having problems setting up the SDK?">

## Supported Versions
- Find various topics in <PlatformLink to="/troubleshooting/">Troubleshooting</PlatformLink>
- [Get support](https://www.sentry.help/en/)

- Celery: 4.4.7+
- Python: 3.6+
</Expandable>

<Include name="python-use-older-sdk-for-legacy-support.mdx" />
</StepConnector>
Loading