Tag: Glue
All the articles with the tag "Glue".
-
Handling Boolean vs IntegerType Mismatches Between MySQL and Spark (Glue JDBC)
When integrating MySQL data into Apache Spark (for example, through AWS Glue), you might encounter schema mismatches caused by how MySQL represents TINYINT(1) fields. This issue often surfaces when
-
Estimating the Cost of an AWS Glue Workflow
When working with AWS Glue, one of the most common questions data engineers ask is: How much will this job cost me? If you have a workflow that runs for 13 minutes, understanding the cost model of AWS
-
Orchestrating Multiple AWS Glue Workflows: A Practical Guide
AWS Glue provides a robust environment for building and managing ETL pipelines, but many data engineers face the challenge of chaining or coordinating multiple workflows . This article explores
-
How to Disable an AWS Glue Trigger from the CLI
When working with AWS Glue , triggers are an important mechanism to orchestrate jobs or workflows. Sometimes, however, you may need to temporarily disable a trigger without deleting it—for example, to
-
Orchestrating Multiple AWS Glue Workflows with Step Functions
In modern data architectures, it is common to manage multiple ETL pipelines that must run in sequence or in parallel. AWS Glue provides a robust framework for building workflows, but when we need to
-
Choosing Between saveAsTable and Iceberg’s writeTo in AWS Glue and Athena
When working with Spark on AWS Glue , there are multiple ways to persist DataFrames as tables and make them queryable in Amazon Athena . Two common approaches are: Using Spark’s Hive-style saveAsTable
-
AWS Glue Workflow vs Apache Airflow: A Professional Comparison
While both serve the common purpose of managing and automating data workflows, they differ significantly in architecture, flexibility, integration capabilities, and operational control. This article
-
EMR vs AWS Glue: Choosing the Right Data Processing Tool on AWS
When working with big data on AWS, two commonly used services for data processing are Amazon EMR and AWS Glue . Although both support scalable data transformation and analytics, they differ