How to Design a Task Queue for Slow Operations in PHP

An operation that takes several seconds can leave an HTTP request open, consume web server resources, and create an uncertain experience for the user if the connection is interrupted. Background processes in PHP let you accept work and run it outside the lifecycle of that request. For the solution to be useful, adding a queue is not enough: you need to define what the user receives, how they check the status, and what happens if processing is interrupted.

Identify which operations should run in the background

Assess an operation’s impact rather than applying a rule based solely on how long it takes. A task may be a good candidate if its duration varies, it needs to wait for an external system, or it processes many items. Examples include generating a large report, importing a file, transforming documents, or synchronizing data with an API.

By contrast, if the user needs an immediate decision to continue—such as validating essential data before saving—it may be better to keep that part in the request and move only the subsequent work out of it. Some slow tasks also require a synchronous response for business reasons. The architecture should meet the needs of the workflow, not turn every operation into an asynchronous job.

Before changing the design, identify how long the operation takes, which part consumes that time, and what happens if it fails. A queue does not by itself fix inefficient queries, misconfigured limits, or unstable external dependencies; it can shift those problems to background workers.

Respond with an accepted task, not an invented result

When work is accepted for later execution, the application’s response should clearly distinguish between accepted and completed. It can return a task identifier and indicate that processing is pending. It should not present a result that does not yet exist as complete.

An estimate of the duration is appropriate only if there is enough information to calculate it and communicate its uncertainty. In many systems, it is more honest to show a status such as “queued” or “processing” than to promise a completion time. If the task may finish quickly, the interface can still check its status and display the result when it is available, without blocking the original request.

Also define what happens if the task cannot be submitted to the queue: the application must not confirm acceptance if it cannot reliably register the work. The specific mechanism depends on the architecture, but the response and persisted status must be consistent.

Define valid statuses and transitions

A small, explicit lifecycle makes the user experience and operations easier to manage. As a starting point, consider pending, in_progress, cancellation_requested, completed, failed, and cancelled. The names can vary; the transitions, however, must be valid and verifiable, not an unrestricted combination of statuses.

• Normal transitions: pending → in_progress and, from in_progress, completed or failed.
• Cancellation before work begins: pending → cancelled if it is confirmed that the worker has not yet started the work.
• Cancellation during execution: in_progress → cancellation_requested. From this status, the task moves to cancelled if it is stopped safely; it may move to completed if it finished before the request was handled, or to failed if an error occurred.
• Terminal statuses: completed, failed, and cancelled should not return to active statuses. If a task can be repeated, create a new execution or explicitly define a different lifecycle.

The system must prevent invalid transitions and record when each change occurred. If a worker is interrupted and leaves a task in in_progress, do not automatically mark it as completed. A recovery policy can return it to pending when it is safe to resume, or mark it as failed for review. The decision depends on whether the work can be resumed without duplicating effects. Retry coordination and idempotency require their own decisions; they should not be confused with the user-visible status model.

Communicate progress, results, and errors

Progress should be based on measurable milestones. If it is not possible to calculate a real percentage, it is preferable to report the current phase (“validating data,” “generating report”) or keep the status as “in progress.” A percentage that remains frozen or moves backward without explanation undermines trust.

On completion, provide the business result—for example, a summary of processed items or a reference to the generated resource, subject to permissions and availability. The task status does not have to store the entire result. It can store a reference to data managed by the application’s domain.

On failure, display a message that helps the user decide what to do next, such as correcting a file or trying again where appropriate. Log the technical error needed for diagnosis separately, with restricted access. Do not expose stack traces, credentials, internal paths, or sensitive data in the interface or a public API.

Treat cancellation as a request with limits

Canceling a task does not always mean it can be stopped immediately. A worker may be making an external call, writing data, or executing a phase that cannot be safely interrupted. For this reason, the interface should distinguish between cancellation_requested and cancelled: the first indicates that the request was received; the second, that the system confirmed the task was stopped.

The implementation can check for a cancellation signal at safe points and stop starting subsequent steps. If an effect has already been confirmed in an external system, do not claim that it was undone. Explain to the user what outcome to expect, and record who requested the cancellation when that information is relevant for auditing.

Separate the task record from the business result

A task record is used to coordinate execution and status checks: it can include an identifier, status, dates, requester, and diagnostic references. The business result—for example, a report or imported records—belongs in the data model that represents that result.

Keeping them separate prevents a task containing temporary information from becoming the business’s only source of truth. It also lets you define retention: how long the status is kept, when a file is deleted, and what information must be retained for operational or regulatory obligations. Old statuses should not accumulate without a cleanup policy.

Protect status checks and limit resource consumption

Checking a task requires authorization, just like checking any other resource. The identifier does not replace access control: verify that the requesting person or system can view the task and its result. Also limit the frequency and volume of checks to prevent excessive polling, and consider notifications when the use case does not need constant updates.

Execution should have concurrency and resource limits appropriate to the type of work. Too many workers can overwhelm the database, external services, or storage; too few can cause tasks to pile up and delay results. Monitor the pending workload, task age, and durations, in addition to errors. These signals help distinguish a slow dependency from a capacity problem.

Test the full lifecycle before deployment

Acceptance tests should cover user-visible behavior and persistence, not just whether the worker invokes a function. At a minimum, check these scenarios:

• The request creates a task and returns a reference without claiming that it has already finished.
• A completed task shows its final status and allows access to the result with the correct permissions.
• A failed task displays a safe message and leaves diagnostic information accessible to the authorized team.
• An abandoned or interrupted task does not remain “in progress” indefinitely; there is an operational way to detect and handle it.
• A cancellation request respects safe phases and does not promise to reverse effects that have already occurred.

In an existing system, introduce the change for a narrowly scoped operation and also validate the experience when the queue is delayed or unavailable. The application needs a clear response for these cases: report that it could not accept the work, offer a safe alternative, or let the user try again. The decision depends on the operation’s impact and the guarantees the business can uphold.

Checklist for introducing a queue into an existing system

Before enabling background processing, review these points with the teams responsible for the application and its operations:

• Scope: A specific operation has been selected, and it is clear which part remains synchronous and which work runs in the background.
• Persistence: Task acceptance and its status are stored consistently; a policy is in place for interruptions, recovery, and status changes.
• User contract: The initial response distinguishes acceptance from completion, and the interface explains how to check progress, results, and errors.
• Permissions and data: Authorization is required to check a task or its result; stored data is minimized, and access to diagnostics is restricted.
• Resources: Concurrency, execution time, memory, and polling frequency limits have been set appropriately for the operation and its dependencies.
• Observability: Backlogs, old tasks in progress, failures, and unusual durations can be detected without including secrets in logs.
• Retention: Decisions have been made about how long to retain statuses, results, files, and diagnostic data, and how to delete them in a controlled way.
• Testing and operations: Completion, failure, cancellation, and interruption paths have been tested, and the team knows what to do if the queue cannot accept work or is delayed.

Deploying the code is not the same as immediately enabling every workflow: you can introduce the new execution path gradually and check its signals before expanding it. Keep an appropriate operational alternative in place if the queue or its workers are unavailable; do not confirm work that the system could not register.