Debugging AWS Glue Job Bookmarks Not Working as Expected

Job bookmarks are supposed to make Glue jobs incremental: each run processes only what’s new since the last successful run. When they break, they break silently. The job succeeds, and you either reprocess everything or skip data you needed.

This is the checklist I go through.

How bookmarks actually work

A bookmark is state stored per job name and per transformation_ctx. For S3 sources, Glue tracks which objects (by last-modified time) it already processed. For JDBC sources, it tracks the last value of one or more bookmark keys.

That state is persisted only when job.commit() runs at the end of a successful run. Everything below follows from these three facts.

1. Bookmarks aren’t enabled on the job

The default is job-bookmark-disable. Check:

$ aws glue get-job --job-name my-job \
    --query 'Job.DefaultArguments."--job-bookmark-option"'

It must be job-bookmark-enable. Also check the run itself: arguments passed to start-job-run override the job defaults.

2. Missing job.init() / job.commit()

The minimal skeleton:

import sys
from awsglue.utils import getResolvedOptions
from awsglue.context import GlueContext
from awsglue.job import Job
from pyspark.context import SparkContext

args = getResolvedOptions(sys.argv, ["JOB_NAME"])
glue_ctx = GlueContext(SparkContext.getOrCreate())
job = Job(glue_ctx)
job.init(args["JOB_NAME"], args)

# ... read, transform, write ...

job.commit()

No job.commit() means state is never saved, so every run starts from scratch. If the script raises before commit(), that’s correct behavior: the run failed, so the bookmark shouldn’t advance.

3. Missing or changing transformation_ctx

src = glue_ctx.create_dynamic_frame.from_catalog(
    database="raw",
    table_name="events",
    transformation_ctx="src_events",  # bookmark key
)

Without transformation_ctx, there’s no key to store state under. Renaming it (src_events → events_src) is equivalent to a reset: the new key has no state.

4. You’re reading with Spark, not DynamicFrames

This is the one that catches most people. Bookmarks only apply to reads through Glue APIs (create_dynamic_frame.from_catalog, from_options). This bypasses them entirely:

df = spark.read.parquet("s3://my-bucket/events/")  # no bookmark

If you need a DataFrame, read as a DynamicFrame and call .toDF().

5. Unsupported source or format

Bookmarks support S3 and JDBC sources. For S3, check that your format is in the supported list (JSON, CSV, Avro, XML, Parquet, ORC per the docs). Other connectors may ignore bookmark state.

6. JDBC: non-monotonic bookmark keys

For JDBC, Glue uses the primary key by default. Bookmarks assume the key only increases (or only decreases). A UUID primary key, or an updated_at column that gets backfilled, breaks this.

Set explicit keys:

src = glue_ctx.create_dynamic_frame.from_catalog(
    database="ops",
    table_name="orders",
    transformation_ctx="src_orders",
    additional_options={
        "jobBookmarkKeys": ["order_id"],
        "jobBookmarkKeysSortOrder": "asc",
    },
)

Note the trade-off: a monotonic insert key catches new rows, not updates to existing ones. If you need updates, bookmarks are the wrong tool. Use CDC or a watermark you manage yourself.

7. S3 objects overwritten in place

S3 bookmarks rely on object last-modified timestamps. If an upstream process overwrites data.parquet under the same key, behavior depends on timing relative to the stored state. Append-only writes with unique object names keep bookmarks predictable.

8. Concurrent runs

With MaxConcurrentRuns > 1, two runs can read the same starting state and process overlapping data, and the last one to commit wins. For bookmarked jobs, keep concurrency at 1.

Resetting and replaying

Full reset:

$ aws glue reset-job-bookmark --job-name my-job

Replay a window without moving the bookmark: run with --job-bookmark-option job-bookmark-pause plus --job-bookmark-from <run-id> and --job-bookmark-to <run-id>. Useful for reprocessing a bad day without losing your position.

Quick checklist

• --job-bookmark-option job-bookmark-enable on the run
• job.init() at the top, job.commit() at the end
• Stable transformation_ctx on every source
• Reads via DynamicFrame APIs, not spark.read
• Supported source and format
• Monotonic jobBookmarkKeys for JDBC
• Append-only S3 writes upstream
• MaxConcurrentRuns = 1

References

• Tracking processed data using job bookmarks
• Using job bookmarks
• AWS Glue job parameters
• ResetJobBookmark API