Background Jobs

Long-running operations - full-site data imports and exports, bulk machine translation of posts and strings, WPML / Polylang / TranslatePress migrations, full-site string scans - can take minutes on a real site. PerfLocale's background-jobs layer moves them off the request path so the admin UI stays responsive and PHP-FPM timeouts on shared hosting never decide whether your import succeeds.

Dedicated state table. Job state lives in a single custom table ({$wpdb->prefix}perflocale_jobs) with typed columns and indexes on uuid, (status, updated_at), (type, status), and created_by. The table is garbage-collected daily and contributes no overhead on frontend requests - visitors never touch it. Concurrency locks live in wp_options via atomic INSERT IGNORE (race-free row-level mutex), which is the right shape for short-lived advisory locks.

How it works

Nine user-triggered operations are wrapped as "tier-2" jobs (the table below enumerates them). When you trigger one:

1. Dispatch - the plugin chooses sync (inline) or async based on settings + payload size. Small inputs still run inline so you see the result immediately; big ones queue and the admin redirects to the Jobs page.
2. Engine - async jobs run on Action Scheduler when WooCommerce or the standalone AS plugin is loaded, or WP-Cron otherwise. The choice is automatic; an operator can force WP-Cron from the settings page (escape hatch when AS misbehaves).
3. Worker - the worker re-checks the job type's required capability against the dispatching user recorded in the row's created_by column (defense-in-depth against role downgrades between dispatch and execution), takes an atomic lock so a job can't run twice, and calls the operation's execute().
4. Retry - unhandled exceptions are caught; the worker reschedules with exponential backoff up to 5 attempts (configurable).

Bulk translation runs in slices. When the background worker runs a bulk_translate job, it works through at most 20 seconds or 500 source-target pairs, whichever comes first, per background request. It then stores its progress on the job row, and the next worker run continues the same job from there, so a host’s request time limit does not stop it part-way. The job stays running between slices, its progress counts grow slice by slice, and perflocale/jobs/completed fires once, after the last slice. When the worker runs in a WP-CLI process, a slice has no time or pair limit and the job frees its runtime caches every 500 pairs. The slice size is set with perflocale/jobs/bulk_translate/slice_seconds and perflocale/jobs/bulk_translate/slice_max_pairs. If the database refuses twice to store a slice’s progress, the job is marked failed with a message saying so; Retry runs it again, and pairs that already have a translation are skipped without a provider call.

A worker that stops on a PHP fatal error. When PHP stops inside a job - a fatal error such as running out of memory or time, or an exit - PerfLocale releases the job’s locks and marks it failed at once, with the error message (the directories in its file paths replaced by <path>) or “The worker stopped before finishing.”, so it can be retried right away. This holds in a WP-Cron or Action Scheduler request and in a WP-CLI process that runs them (such as wp cron event run): PerfLocale does it before WordPress shows its fatal error page, which ends a WP-CLI process. A wp-content/php-error.php drop-in that ends the process itself, and a worker the host kills from outside PHP (for example the kernel’s out-of-memory killer), leave the job as it is: its locks expire on their own (30 minutes for bulk translation), and the job is then marked failed as described under Recovery.

Each job is one row in {$wpdb->prefix}perflocale_jobs with typed columns: uuid (CHAR(36), UNIQUE), type, hook, engine (action_scheduler / wp_cron), status, progress, total, processed, attempts, created_by (indexed, used for cap re-validation), blog_id, version (optimistic-concurrency CAS column), and the timestamps. Larger payloads - args, result (truncated to 64 KB) and a log ring buffer (20 entries) - live in LONGTEXT columns on the same row, alongside a VARCHAR(2000) error column. Status and stuck-job sweeps use the (status, updated_at) index; admin filters use (type, status).

updated_at is written by PHP, in UTC. The column deliberately carries no ON UPDATE CURRENT_TIMESTAMP - that would stamp the MySQL server's clock, while every sibling datetime is written with current_time( 'mysql', true ) and every consumer (stuck-job watchdog, GC sweeps, job hydration) compares against gmdate(). On a database whose timezone isn't UTC the two clocks disagree, and a row touched a second ago can look hours stale - enough for the watchdog to force-fail a job that is still running. JobState writes updated_at explicitly on every state change instead.

Two short-lived advisory lock options remain in wp_options: perflocale_job_lock_<uuid> (per-job worker mutex) and perflocale_type_lock_<type> (per-type concurrency cap). They use atomic INSERT IGNORE on the UNIQUE option_name key so the DB enforces mutual exclusion at the row-lock layer. Stored expiry timestamps let a daily sweep clean up dead locks left behind by crashed workers.

On PerfLocale → Jobs, a user without perflocale_manage_translations sees only the jobs they dispatched, as in GET /jobs. A running WPML, Polylang or TranslatePress import shows its current stage under the progress bar with that stage’s items done and in total, for example “Posts: 120 of 500”, and screen readers hear the same text with the overall percent. The row updates while the page is open. A finished bulk or site-wide machine translation shows how many created translations kept the source-language content because a “Do not translate” section was lost, with the first warning.

All of the above is per-blog on multisite - each subsite has its own jobs table view (each subsite gets its own table via $wpdb->prefix), settings, and Jobs page.

Job types

Nine operations are background-capable:

What's not on the unified Jobs queue

Some operations have their own scheduler and are not visible on PerfLocale → Jobs:

• Exchange-rate sync (WooCommerce) - runs on a separate daily perflocale_exchange_rate_sync WP-Cron event.
• Webhook delivery + retry - per-delivery one-shot events routed through BackgroundEvents, not Dispatcher.
• Concurrency-lock cleanup - daily cron for the standalone Lock subsystem.

All three schedule their own events via the same engine selector (Action Scheduler when loaded, WP-Cron otherwise), so the engine setting affects them too - they just aren't tracked on the JobState index.

Settings (Performance tab)

Background Processing

• Auto (default) - small operations run inline; large ones queue. The threshold is per-job, with sensible defaults (e.g. 1000 files for string scan, 1000 rows for data import).
• Always - every operation queues. Use on large sites where even "small" operations can spike during peak hours.
• Never - everything runs inline. Not recommended on a real site - long imports will hit PHP-FPM timeouts.

Background Engine (only shown when Action Scheduler is loaded)

• Auto - use Action Scheduler when available (recommended). AS has its own admin UI under Tools → Scheduled Actions filtered to the perflocale group, persists actions in its own tables, and handles concurrency / claim semantics natively.
• Force WP-Cron - skip AS and use WordPress's built-in cron. Escape hatch for sites where AS is misbehaving (rare).

Background Thresholds

Per-job override of the default item-count threshold for Auto mode. Leave blank to use the default. Programmatic equivalent: perflocale/jobs/threshold/<type> filter.

Pause Queue

Operator brake. New dispatches are accepted but workers immediately re-queue them every 5 minutes instead of running. Use as an emergency stop when something is mis-dispatching; uncheck to resume. Already-in-flight jobs finish naturally; only future ticks defer.

Recovery and garbage collection

Three automatic recovery paths run on the daily perflocale_jobs_gc cron:

1. Stuck-job sweep. Any job in queued or running whose updated_at hasn't moved in 6 hours (configurable) is marked failed with a clear message. Catches PHP-FPM kills, OOM crashes, host reboots, or broken cron environments where dispatched events never fire.
2. Terminal-state prune. Jobs in complete, failed, or canceled older than 24 hours have their row deleted from the jobs table. A finished export’s file is deleted together with its row; a recent export that is still offered for download keeps both.
3. Stale lock sweep. Per-job and per-type lock rows (perflocale_job_lock_* and perflocale_type_lock_*) whose stored expiry timestamp is in the past are deleted. Without this, every dispatched job would leave a dead lock row behind forever. The expired lock of a job that is still marked running is kept until the job is marked failed, because the dead-worker check below reads it; the next sweep removes it.

Dead workers. The hourly stuck-job watchdog also marks a running job failed when its worker is gone: the job row has had no update for 5 minutes and the job’s lock has expired. The job can then be retried. Action Scheduler applies the same rule: while one of PerfLocale’s jobs runs with a live worker, PerfLocale lengthens Action Scheduler’s failure period so the long job is not reclaimed mid-run, and a job whose worker was killed stops counting once its lock expires, so the rest of the queue, WooCommerce’s actions included, carries on. When Action Scheduler times out a PerfLocale action, the job is marked failed only when its worker is gone, or, for a queued job, when no other attempt is scheduled for it. After Cancel and then Retry on a stuck job, the retried job therefore stays queued until the stopped worker’s lock expires, and then runs.

One additional recovery path runs on plugin activation (not on the daily cron):

• Reactivation resume. The deactivation hook unschedules every worker event so a deactivated plugin doesn't leave cron firing against missing callbacks. The JobState rows survive deactivation, though - the operator's history shouldn't vanish. On the next activate, a one-shot perflocale_resume_jobs event runs the Resumer handler: it scans the index, re-enqueues each queued/running survivor via the configured runner, flips any running rows back to queued (without bumping attempts - the previous run didn't fail on its own merits), and logs "Resumed after plugin reactivation." on each job. A running job whose worker lock is still held is skipped, because the lock may belong to a live worker; the job’s log says so once. The sweep then schedules one follow-up check for shortly after the earliest of those locks expires, so a job whose worker died holding its lock is resumed then instead of waiting for the stuck-job sweep. A job that has not moved for the whole stuck window (perflocale/jobs/stuck_timeout_seconds) is left to that sweep.

WP-CLI

Full subcommand surface mirroring the REST endpoints. See WP-CLI → Jobs for the complete reference. Quick examples:

# List all active jobs (queued or running)
wp perflocale jobs list

# Filter by status
wp perflocale jobs list --status=failed --format=json

# Inspect, cancel, retry, delete
wp perflocale jobs get    <uuid>
wp perflocale jobs cancel <uuid>
wp perflocale jobs retry  <uuid>
wp perflocale jobs delete <uuid>

# Operator brake
wp perflocale jobs pause
wp perflocale jobs unpause

# Manual maintenance
wp perflocale jobs gc        # Run garbage collection now
wp perflocale jobs resume    # Re-enqueue queued/running jobs (post-reactivation recovery)

REST API

Five endpoints under /wp-json/perflocale/v1/jobs/. See REST API → Jobs for the complete reference with response shapes.

Permissions

The bg-jobs layer integrates with PerfLocale's standard capability system. Two angles:

Who can dispatch a job

Each job type declares the cap it requires (via get_required_capability()). The Dispatcher checks current_user_can() against that cap before doing anything. The worker re-checks the cap at execution time against the originating user - if a user's role was downgraded between dispatch and run, the job fails with "Permission revoked or dispatching user no longer has access."

The mapping is in the job types table above.

Who can cancel / retry / delete a job

The REST and CLI mutation endpoints (cancel, retry, delete) require BOTH:

• perflocale_translate - coarse "you can touch the jobs API at all" gate.
• You must either be the user who originally dispatched the job, OR hold the perflocale_manage_translations supervisor cap.

This prevents a translator who can dispatch their own string-scan from cancelling another translator's in-flight migration. Supervisors and administrators (who naturally have perflocale_manage_translations) can manage any job.

A retry on the Jobs page or through POST /jobs/{id}/retry also needs the capability of the job’s type (the job types table). The worker runs a job as the user who dispatched it, so retrying another user’s job also needs every capability that user holds on the site, and on a multisite network super admin rights when that user is a super admin. Otherwise the retry is refused, and the message asks you to start the job again yourself.

See the Permissions & Roles doc for the full cap matrix.

Hooks

perflocale/jobs/threshold/<type> (filter)

Per-type override of the Auto-mode threshold. The default for each job comes from get_default_threshold() on the job class (1000 for string_scan and data_import, 500 for wpml_migration / polylang_migration, 200 for translatepress_migration, 5000 for data_export, 50 for bulk_string_translate, 1 for site_translate, and 25 for bulk_translate - counted as source posts × target languages).

// Force string scans to ALWAYS run async, regardless of file count.
add_filter( 'perflocale/jobs/threshold/string_scan', static fn(): int => 0 );

// Per-args dynamic threshold for the data importer: bigger files
// queue, small ones stay inline.
add_filter( 'perflocale/jobs/threshold/data_import', static function ( int $base, array $args ): int {
	$file = $args['file_path'] ?? '';
	return file_exists( $file ) && filesize( $file ) > 5 * MB_IN_BYTES ? 0 : 999999;
}, 10, 2 );

Args: int $base (resolved from settings or default), array $args (the dispatch args).
Returns: int. When 0, every dispatch goes async. When PHP_INT_MAX, every dispatch goes sync.

perflocale/jobs/max_attempts (filter)

Maximum retry count before a failed job stops being rescheduled. Default 5 (initial attempt + 4 retries).

// One-shot only - never retry.
add_filter( 'perflocale/jobs/max_attempts', static fn(): int => 1 );

Args: int $max.
Returns: int.

perflocale/jobs/retry_delay (filter)

Seconds to wait before the next retry attempt. Default: exponential backoff capped at 1 hour - min( 3600, 60 * 2^(attempts-1) ).

// Linear backoff: 60, 120, 180, ...
add_filter( 'perflocale/jobs/retry_delay', static function ( int $delay, int $attempts ): int {
	return 60 * max( 1, $attempts );
}, 10, 2 );

Args: int $delay (default), int $attempts (current attempt count).
Returns: int (seconds).

perflocale/jobs/max_concurrent/<type> (filter)

Per-type serialisation switch. Default 1 - only one worker of each type runs at a time, enforced by a per-type lock. In practice it behaves as a boolean rather than a counter: any value above 1 removes the lock entirely instead of capping parallelism at that number, so use it only where a job type is safe to run unbounded in parallel.

// Let string scans run in parallel (drops the per-type lock).
add_filter( 'perflocale/jobs/max_concurrent/string_scan', static fn(): int => PHP_INT_MAX );

// Same for data_export.
add_filter( 'perflocale/jobs/max_concurrent/data_export', static fn(): int => PHP_INT_MAX );

Args: int $max (default 1).
Returns: int. Values ≤ 1 keep the lock; anything > 1 removes it.

perflocale/jobs/deduplicate_admission (filter)

Whether an identical job that is still queued or running blocks a new dispatch. Default true. A double-clicked button, a retried request or the same --async command run twice returns the job already in flight (its ID, with duplicate => true) rather than queueing a second one.

“Identical” is the logical operation, not the type: same job type, byte-identical arguments, same site. Unrelated work of the same type still runs in parallel, and the chunked site-translation chain - whose cursor advances on every link - is never mistaken for a repeat of itself. This matters most if you have raised max_concurrent for a type: with overlap enabled, two identical translation jobs call the provider twice for every string and you pay twice.

// Let a custom job type be dispatched repeatedly with the same args.
add_filter( 'perflocale/jobs/deduplicate_admission', static function ( bool $on, string $type ): bool {
	return 'my_addon_reindex' === $type ? false : $on;
}, 10, 2 );

Args: bool $enabled, string $type, array $args.
Returns: bool.

perflocale/jobs/max_args_bytes (filter)

Maximum JSON-encoded size of the args payload accepted by Dispatcher::enqueue(). Default 100 KB. Caller-side hardening against bloating the args LONGTEXT column.

// Bump to 500 KB for custom jobs that legitimately carry larger payloads.
add_filter( 'perflocale/jobs/max_args_bytes', static fn(): int => 500 * 1024 );

Args: int $bytes.
Returns: int. Clamped to a minimum of 1024 bytes.

perflocale/jobs/active_index_max (filter)

Cap on how many rows appear on PerfLocale → Jobs at once. Default 50. The cap drives a LIMIT on the listing query plus a daily-GC eviction pass: oldest terminal rows (complete/failed/canceled) are evicted first; queued and running rows are never evicted. Each row is ~1 KB on disk (mostly the LONGTEXT log + result columns) so even a cap of 250 stays well under typical InnoDB row-overhead noise.

// High-throughput site that dispatches hundreds of jobs/day.
add_filter( 'perflocale/jobs/active_index_max', static fn(): int => 250 );

Args: int $max.
Returns: int (floor-clamped to 10).

Per-job throughput filters

The unified dispatch decision (sync vs async, retry count, threshold) is the same for every job, but each job has its own internal batch size that controls throughput once it's running. These are independent of the bg-jobs threshold filters and live with the job's underlying class. All clamped to safe ranges so a filter typo can't break the operation.

perflocale/jobs/stuck_timeout_seconds (filter)

How long a job may sit in queued or running without its updated_at being bumped before the daily GC or the stuck-job watchdog declares it stuck and marks it failed. Default 6 hours. The reactivation resume uses the same window for its follow-up check.

// More aggressive sweep - 1 hour.
add_filter( 'perflocale/jobs/stuck_timeout_seconds', static fn(): int => HOUR_IN_SECONDS );

Args: int $seconds.
Returns: int.

perflocale/jobs/pause_recheck_seconds (filter)

When the queue is paused, a worker that picks up a job immediately re-schedules it for this many seconds later instead of running. Default 300 (5 minutes). Lower values poll the pause flag more often at the cost of more cron / AS churn.

add_filter( 'perflocale/jobs/pause_recheck_seconds', static fn(): int => 60 );

Args: int $seconds.
Returns: int.

perflocale/jobs/type_busy_retry_seconds (filter)

When the per-type concurrency lock is held by another worker, the new attempt re-queues this many seconds later. Default 60.

add_filter( 'perflocale/jobs/type_busy_retry_seconds', static fn(): int => 10 );

Args: int $seconds.
Returns: int.

perflocale/jobs/runner (filter)

Globally override the runner instance. Mostly for tests and custom deployments backed by an external queue (Sidekiq, SQS, etc.). Returning a JobRunnerInterface instance from this filter bypasses the engine setting and Action Scheduler detection.

add_filter( 'perflocale/jobs/runner', static function ( $default ) {
	return new MyCustomQueueRunner();
}, 99 );

Args: JobRunnerInterface|null $override.
Returns: JobRunnerInterface|null.

perflocale/jobs/enqueued (action)

Fires after a job has been successfully enqueued for async execution. Useful for hooking external monitoring / observability.

add_action( 'perflocale/jobs/enqueued', static function ( string $job_id, string $type, string $engine, array $args ): void {
	error_log( "PerfLocale enqueued $type job $job_id on $engine" );
}, 10, 4 );

Args:

• string $job_id - UUID v4 of the new job.
• string $type - job type slug (e.g. string_scan).
• string $engine - runner engine: action_scheduler or wp_cron.
• array $args - the dispatch args.

perflocale/jobs/completed (action)

Fires after a worker finishes a job successfully.

add_action( 'perflocale/jobs/completed', static function ( string $job_id, string $type, array $result ): void {
	// Push to your metrics pipeline...
}, 10, 3 );

Args: string $job_id, string $type, array $result (worker's return value, already stored on the job row, truncated to MAX_RESULT_BYTES = 64 KB).

perflocale/jobs/failed (action)

Fires when a background job fails in its worker: the worker threw an exception and the retry-with-backoff is about to be scheduled (or skipped because the attempt cap has been hit); the job reported a run that did nothing but fail, as a bulk string translation or an import can, and $e is a RuntimeException carrying its reason; or the progress of a job that runs in slices could not be stored. It does not fire when PHP itself stops inside the worker, nor when the watchdog or Action Scheduler marks failed a job whose worker is gone.

add_action( 'perflocale/jobs/failed', static function ( string $job_id, string $type, \Throwable $e ): void {
	// Send to Sentry / Bugsnag / etc.
	sentry_capture_exception( $e, [
		'tags' => [ 'perflocale_job_id' => $job_id, 'perflocale_job_type' => $type ],
	] );
}, 10, 3 );

Args: string $job_id, string $type, \Throwable $e (the original exception, untruncated - useful for monitoring; the version stored on the job row is path-redacted and truncated).

perflocale/jobs/canceled (action)

Fires when a long-running worker cooperatively aborts itself in response to an operator cancel mid-flight, and when the cancel is found only after the worker’s last progress tick or between two slices of a job that runs in slices (distinct from failed because it isn't an error). Useful for distinguishing operator-canceled from worker-errored in monitoring dashboards.

add_action( 'perflocale/jobs/canceled', static function ( string $job_id, string $type ): void {
	// Record cancellation without alerting...
}, 10, 2 );

Args: string $job_id, string $type.

perflocale/strings/after_scan (action)

Fires after the all-mode string scan finishes iterating every theme + plugin directory. Addons hook here to register their non-gettext strings (e.g. attribute labels, email subjects).

add_action( 'perflocale/strings/after_scan', static function (): void {
	// Register any strings your code generates dynamically...
} );

Multisite

The bg-jobs system is fully per-blog. On both subdir and subdomain installs, each subsite:

• Keeps its rows in a per-subsite table - $wpdb->prefix resolves to the blog's own prefix, so wp_2_perflocale_jobs and so on - with the dispatching blog_id also stored as a column for cross-blog worker handoffs.
• Sees and manages only its own queue via the Jobs admin page and CLI (wp perflocale jobs list --url=<blog>).
• Has its own settings (processing mode, engine, thresholds, paused flag) stored as per-subsite options.
• Runs its own GC + Resumer (per-blog daily cron + per-blog activation handler).

Network activation iterates all sites in chunks of 100 (filterable via perflocale/activation/chunk_size) and runs Activator::activate() per blog. The wp_initialize_site hook auto-provisions a brand-new subsite - it runs Activator::activate() (all nine perflocale_* tables) plus Bootstrap::ensure_recurring_schedules(), so a subsite that is never opened in wp-admin still gets its GC + watchdog crons. The wp_uninitialize_site hook (priority 5, ahead of core’s own table-dropper at priority 10) runs SiteCleanup::purge_current_site() when a subsite is permanently deleted - and it forces a full purge regardless of the delete_data_on_uninstall setting, because blog deletion is irreversible and the plugin’s per-blog tables would just be orphans. That is the one place the “keep my data” preference does not apply; plugin uninstall (uninstall.php) does respect each subsite’s own setting.

Performance

The dedicated jobs table is touched only by the admin Jobs page, the daily GC + stuck-job watchdog crons, and the worker code paths. No code path on a public pageview ever queries it. Loading the homepage, a single post, or a category archive runs zero queries against perflocale_jobs regardless of how many jobs are queued.

The two short-lived lock options (perflocale_job_lock_* and perflocale_type_lock_*) live in wp_options with autoload=no, so even on the small subset of admin requests that exercise the locking path, the rows are not pulled into the alloptions blob.

Row size on the jobs table is dominated by the args, result, and log LONGTEXT columns; the typed scalar columns (status, attempts, timestamps, indexes) are ~150 bytes. A typical 50-row backlog occupies well under 1 MB on disk.

Action Scheduler integration

When Action Scheduler ≥ 3.4 is loaded (WooCommerce ships it; the standalone Action Scheduler plugin works too), PerfLocale enqueues async jobs via as_enqueue_async_action() / as_schedule_single_action() in the perflocale group. The Jobs admin page shows a link to Tools → Scheduled Actions (filtered to that group) so you can use the full AS history UI alongside the PerfLocale-native view.

On sites without AS, the runner falls back to wp_schedule_single_event(). Same code path, same retry semantics, same UI; just a different scheduler.

Concurrency safety & self-healing

Workers can run in parallel under Action Scheduler. PerfLocale guarantees correctness under contention without relying on the operator to size cron concurrency:

• Atomic per-job + per-type locks. Backed by raw INSERT IGNORE INTO wp_options via the InnoDB UNIQUE constraint - not add_option(), which is not atomic across concurrent forks. Token-guarded release prevents a hung holder's late release() from deleting another worker's lock row after a TTL takeover. Verified at 1000-way parallelism.
• Per-(source, target) translation locks. Two concurrent create_translation() calls for the same post + language serialise at the plugin layer instead of relying on the database's UNIQUE KEY to catch one of two already-inserted wp_posts rows, so no orphan post is left behind.
• Self-healing worker re-schedule. When a worker finds the per-job lock already held (typically a leaked row from a crashed sibling), instead of silently returning it re-queues itself with exponential backoff + jitter, then marks the job failed after 20 retries with an actionable diagnostic. Cap configurable via perflocale/jobs/lock_busy_max_retries.
• Per-provider circuit breakers. Every MT provider call is wrapped in a breaker that trips on the first auth error and after five transient errors in a five-minute window, so a bad key or a provider outage can't burn the monthly character budget one retry at a time. Site Health surfaces the breaker state.
• Multisite isolation. Per-blog locks live in wp_<id>_options - same lock NAME on two subsites doesn't collide. switch_to_blog() mid-request is honoured. Verified at 100-blog scale.

See the Reliability & Circuit Breakers section of the hooks reference for all tuning filters.

See also

• WP-CLI: wp perflocale jobs reference
• REST API: Jobs endpoints
• Hooks reference: Background Jobs section
• Permissions & Roles
• Features: Performance → Background Processing