Cloud & infra

The ground everything runs on: AWS accounts and IAM, networking and SSH, Linux and the shell, Docker, Terraform, CI/CD.

Tech

All posts (84)

• Pulling Brevo email logs into your backoffice without sharing credentials

  12 Nov, 2026

  Read Brevo transactional email events from your own backend with one API key: the endpoint, date filters, three header mistakes and when to use webhooks.

• De-identifying FHIR data: HIPAA Safe Harbor, limited data sets and leaks

  3 Nov, 2026

  Dropping name and SSN is not de-identification. Apply HIPAA Safe Harbor to FHIR NDJSON with an allowlist, surrogate keys and a leak check, in Python.

• From SSH keys to SMART Backend Services: auth for system-to-system FHIR

  29 Oct, 2026

  Replace long-lived SFTP keys with SMART Backend Services: JWKS, signed JWT assertions, short-lived scoped tokens. Client and FastAPI server in Python.

• Tracking S3 file changes with ETags and LastModified

  16 Sep, 2024 · Updated 15 Oct, 2026

  Detect changed S3 objects without versioning: read ETag and LastModified with boto3, export an ETag CSV per date prefix, and handle multipart ETags.

• Greenfield vs Brownfield Projects: What Actually Changes in Engineering

  11 Oct, 2026

  Greenfield vs brownfield software projects: constraints, risks, architecture decisions, and the patterns that work for each — from blank repo to legacy system.

• S3 Eventual Consistency Edge Cases in Data Pipelines

  8 Oct, 2026

  S3 has been strongly consistent since 2020. The edge cases that still break data pipelines: no atomic rename, concurrent writers, LastModified, events.

• How to Get the Status of an AWS Glue Workflow Using boto3

  30 Sep, 2026

  A Glue workflow can report COMPLETED while its jobs failed. How to poll workflow runs with boto3 and read per-node status from the run graph.

• AWS Glue Job Fails Due to Dependency Version Conflict: How to Fix It

  29 Sep, 2026

  Your Glue job works locally but fails with ImportError or numpy binary incompatibility. Why it happens and how to pin dependencies against the Glue runtime.

• Triggering AWS Glue Jobs from EventBridge: Workflows, Scheduler, and the IAM You Actually Need

  28 Sep, 2026

  A Glue job isn't a native EventBridge rule target. Three working patterns (workflow, Scheduler, Step Functions) with the exact IAM each one needs.

• Debugging AWS Glue Job Bookmarks Not Working as Expected

  27 Sep, 2026

  Glue job bookmarks fail silently: reprocessed data, skipped files, or no state at all. A checklist of the root causes and how to verify each one.

• Why Athena Returns No Rows on Partitioned Tables (and How to Fix It)

  26 Sep, 2026

  Your data is in S3, the table exists, and Athena returns zero rows. Six root causes behind partition metadata desync and how to diagnose each one.

• Why apt upgrade Didn’t Update VS Code (and What Actually Happened)

  26 Apr, 2026

  Problem Statement sudo apt update sudo apt upgrade But Visual Studio Code (Visual Studio Code) remains outdated. sudo apt install --only-upgrade code it updates successfully. This behavior is not

• Why Terraform Does Not Deploy Your Lambda Container Image

  24 Apr, 2026

  When teams start packaging AWS Lambda functions as container images, a common misunderstanding appears quickly: “I created the Lambda with Terraform, so why is AWS saying the image does not exist?”

• Can an AWS VPC Have Two Peering Connections? Yes. But Should It?

  24 Mar, 2026

  When teams begin structuring cloud networks in AWS, one of the first connectivity mechanisms they encounter is VPC Peering . It is simple, direct, and usually easy to implement for small environments.

• Lambda vs n8n: A Simple Explanation for Data Workflows

  23 Mar, 2026

  Introduction When building data systems or integrating APIs, a common question appears: should we use AWS Lambda or n8n? Both tools can automate processes, call APIs, and move data between systems,

• Should You Use AWS Lambda or AWS Glue to Update Records in HubSpot?

  23 Mar, 2026

  When integrating HubSpot with a data platform on AWS, a common architectural decision appears quickly: Should updates to HubSpot be executed from AWS Lambda or AWS Glue? The correct choice depends on

• Can You Know the Location of an IPv6 Address?

  28 Feb, 2026

  Example IPv6: 2600:100e:b0c7:7403:f88c:92d0:bc41:46ff Short answer: only approximately , and with significant limitations. This article explains what can and cannot be inferred from an IPv6 address,

• Running Scheduled GitHub Actions Locally for Safer Debugging

  10 Jan, 2026

  Overview When working with scheduled automation jobs in GitHub Actions, it is common to face a simple but critical question: Can this workflow be executed locally before pushing to production? The

• Designing a Scalable Course Progress Service on AWS

  19 Dec, 2025

  EC2, Lambda, DynamoDB, and RDS Cost and Architecture Trade-offs Context In a multi-platform learning environment where users can advance through courses using both Web and Mobile applications ,

• Handling Boolean vs IntegerType Mismatches Between MySQL and Spark (Glue JDBC)

  12 Nov, 2025

  When integrating MySQL data into Apache Spark (for example, through AWS Glue), you might encounter schema mismatches caused by how MySQL represents TINYINT(1) fields. This issue often surfaces when

• Controlling Branch Deployments and Redirects in Vercel: A Practical Guide

  10 Nov, 2025

  Continuous deployment platforms simplify the release process, but they can easily become noisy when every branch triggers a build. Teams working with multiple development environments often need finer

• AWS EventBridge Rules vs EventBridge Scheduler: Which One Should You Use?

  2 Oct, 2025

  In the AWS ecosystem, there are two main ways to schedule and automate tasks: EventBridge Rules (scheduled rules) and the newer EventBridge Scheduler , which introduces Schedule Groups . While both

• Estimating the Cost of an AWS Glue Workflow

  2 Oct, 2025

  When working with AWS Glue, one of the most common questions data engineers ask is: How much will this job cost me? If you have a workflow that runs for 13 minutes, understanding the cost model of AWS

• Running Production Servers on AWS: EC2 vs RDS Cost Breakdown

  1 Oct, 2025

  When planning to run production workloads in the cloud, cost is one of the most important considerations. In this post, we will explore the monthly expenses of running two application servers and a

• How to Keep a Docker Container Running Persistently

  17 Sep, 2025

  When working with Docker, you may have noticed that some containers stop as soon as you exit the shell. This is because Docker considers the container's main process to have finished. In this post, we

• Orchestrating Multiple AWS Glue Workflows: A Practical Guide

  16 Sep, 2025

  AWS Glue provides a robust environment for building and managing ETL pipelines, but many data engineers face the challenge of chaining or coordinating multiple workflows . This article explores

• Secure Ways to Share Private Data on AWS: Beyond Public Buckets

  16 Sep, 2025

  When building data platforms in the cloud, it is common to share data with partners, clients, or internal teams outside your own. AWS provides several mechanisms to grant secure, granular access — far

• How to Set CloudWatch Log Retention Policies with Terraform

  7 Sep, 2025

  AWS CloudWatch is a powerful service for monitoring applications and infrastructure. However, by default, CloudWatch Logs are configured to never expire . This can lead to excessive storage costs and

• How to Disable an AWS Glue Trigger from the CLI

  6 Sep, 2025

  When working with AWS Glue , triggers are an important mechanism to orchestrate jobs or workflows. Sometimes, however, you may need to temporarily disable a trigger without deleting it—for example, to

• Orchestrating Multiple AWS Glue Workflows with Step Functions

  6 Sep, 2025

  In modern data architectures, it is common to manage multiple ETL pipelines that must run in sequence or in parallel. AWS Glue provides a robust framework for building workflows, but when we need to

• Optimizing Amazon Athena Queries with Partitions: A Practical Example

  1 Sep, 2025

  When working with Amazon Athena, one of the most effective strategies to improve query performance and reduce costs is partitioning your data . Partitions allow Athena to scan only the relevant

• Versioning Terraform Resources to Meet CIS Security Standards

  19 Jul, 2025

  Infrastructure as Code (IaC) has become a foundational practice for modern DevOps and cloud-native teams. Terraform, as one of the most widely adopted IaC tools, enables infrastructure automation,

• Reducing AWS Costs: How to Temporarily Stop an Aurora Serverless v2 Cluster

  9 Jul, 2025

  When managing cloud infrastructure, minimizing costs without compromising data integrity is a continuous priority. Amazon Aurora Serverless v2 offers scalability and high availability, but unlike

• EMR vs AWS Glue: Choosing the Right Data Processing Tool on AWS

  6 Jul, 2025

  When working with big data on AWS, two commonly used services for data processing are Amazon EMR and AWS Glue . Although both support scalable data transformation and analytics, they differ

• When Should You Use Iceberg with Athena? Partitioning Strategies and Best Practices

  5 Jul, 2025

  As data lakes grow in size and complexity, tools like Amazon Athena combined with table formats like Apache Iceberg become essential for scalability, data governance, and performance. In this post,

• Why You Should Use the -out Option with terraform plan

  5 Jul, 2025

  When working with Terraform, a common workflow involves running terraform plan followed by terraform apply . However, you may have come across the following warning: "You didn't use the -out option to

• ecure Database Access in AWS Using SSH Tunneling

  2 Jul, 2025

  Accessing databases located in private subnets within AWS Virtual Private Clouds (VPCs) is a common requirement in enterprise architectures. To ensure secure connectivity without exposing the database

• Mastering the Linux find Command: A Practical Introduction

  1 Jul, 2025

  When working with Linux, one of the most powerful tools at your disposal is the find command. Whether you're managing a personal machine or maintaining a production server, being able to locate files

• How Network Topology Shapes Distributed Computing and Big Data Systems

  19 Jun, 2025

  When discussing distributed systems and Big Data, people often focus on storage, processing frameworks, and scalability—but one foundational concept underlies it all: network topology . It’s the

• Fixing Spark Ivy Error in Docker: "basedir must be absolute"

  9 May, 2025

  If you're running Apache Spark inside Docker using Bitnami's images and suddenly encounter an Ivy error that says: Exception in thread "main" java.lang.IllegalArgumentException: basedir must be

• How Dynamo Reshaped the Internal Architecture of Amazon S3

  9 May, 2025

  Introduction Amazon S3 launched in 2006 as a scalable, durable object storage system. It avoided hierarchical file systems and used flat key-based addressing from day one. However, early versions of

• What’s Behind Amazon S3?

  9 May, 2025

  When you upload a file to the cloud using an app or service, there's a good chance it's being stored on Amazon S3 (Simple Storage Service). But what powers it under the hood? What is Amazon S3? Amazon

• From HDFS to S3: The Evolution of Data Lakes in the Cloud

  29 Apr, 2025

  For years, HDFS (Hadoop Distributed File System) was the default choice for building data lakes in on-premises and Hadoop-based environments. But as cloud computing gained momentum, a new player took

• The History and Evolution of Amazon S3: Was It Ever Based on HDFS?

  28 Apr, 2025

  When discussing cloud storage today, Amazon S3 is almost synonymous with scalable, reliable object storage. However, a common question among those familiar with big data technologies like Hadoop is:

• Enabling Internet Access for Resources in a Public Subnet

  3 Apr, 2025

  When deploying resources in a public subnet within an AWS Virtual Private Cloud (VPC), you need to configure several components to allow them to communicate with the internet. Below are the essential

• Network Address Translation (NAT): Overcoming IPv4 Shortages

  3 Apr, 2025

  Introduction Network Address Translation (NAT) is a technology designed to mitigate the shortage of IPv4 addresses by allowing multiple devices on a private network to share a limited number of public

• Understanding Subnets, Gateways, and Route Tables in AWS

  3 Apr, 2025

  When designing applications in AWS, it's crucial to understand how networking components interact within a Virtual Private Cloud (VPC). This post will cover subnets, gateways, and route tables,

• Understanding the Difference Between hostname -i and hostname -I in Linux

  1 Mar, 2025

  When working with Linux, you might come across the commands hostname -i and hostname -I , both of which return IP addresses. At first glance, they seem similar, but they serve different purposes. In

• Understanding Docker Engine Components

  2 Feb, 2025

  Docker Engine is an open-source platform that has revolutionized how applications are developed, deployed, and executed using container technology. By encapsulating applications and their dependencies

• Exploring Free Resources to Learn AWS and Azure Cloud Platforms

  18 Jan, 2025

  Cloud computing is an essential skill in today’s tech landscape. Among the major players, AWS and Azure stand out as leading cloud platforms, offering a wealth of free resources to help individuals

• Understanding Stateful vs. Stateless Firewalls in AWS

  16 Oct, 2024

  When working with network security, it's crucial to understand the difference between stateful and stateless firewalls. In AWS, this understanding is particularly important when configuring security

• Handling shutil.SameFileError When Copying Files in Python

  18 Sep, 2024

  When using Python’s shutil.copy() or shutil.copy2() to copy files, you might run into a shutil.SameFileError if you mistakenly attempt to copy a file onto itself. This error occurs when the source and

• Preserving Directory Structure While Copying Files in Python - version 2

  17 Sep, 2024

  When copying files from one directory to another in Python, it's important to maintain the original directory structure, especially when dealing with nested directories. In this post, we'll explore

• Avoiding Duplicate File Copies Based on Content in Python on AWS

  16 Sep, 2024

  When working with large file systems, copying files can often lead to unintentional duplication, especially if files with the same content are repeatedly copied into different directories. While

• Handling NoneType Errors When Extending Lists in Python

  16 Sep, 2024

  When working with Python, especially with functions that return lists or other iterable objects, you might encounter a TypeError that says something like: TypeError: 'NoneType' object is not iterable

• Efficiently Listing and Filtering S3 Objects by Date

  7 Sep, 2024

  When working with AWS S3 buckets, it’s common to have a large number of objects stored, and you might need to filter them based on certain criteria like dates. This blog post will guide you on how to

• Inserting Data from a CSV File into SQL Server Using PowerShell with Windows Authentication

  14 Aug, 2024

  Introduction In many data integration scenarios, you'll need to import data from a CSV file into a SQL Server database. PowerShell is a powerful tool that allows you to automate this process

• Resolving PostgreSQL Authentication Errors in Docker Compose for Redash

  12 Aug, 2024

  Introduction When setting up Redash with Docker Compose, one of the common errors users might encounter is related to PostgreSQL authentication. Specifically, the psycopg2.OperationalError:

• Effortlessly Count Lines Starting with "DE" in a Text File Using PowerShell

  7 Aug, 2024

  PowerShell is a powerful scripting language that can automate various tasks, including file manipulation and data processing. In this post, we'll demonstrate how to count the number of lines in a text

• Exploring Key Services for the AWS Solution Architect Exam

  1 Jul, 2024

  Key AWS Services for the Solution Architect Exam Amazon RDS (Relational Database Service) Amazon RDS makes it easy to set up, operate, and scale a relational database in the cloud. It provides

• Understanding MAC Addresses: Hexadecimal, Binary, and Decimal Representations

  29 Jun, 2024

  In this post, we'll explore what a MAC address is, how it's represented in hexadecimal notation, and how to convert it to binary and decimal formats. We'll use the MAC address 88-B2-2F-54-1A-0F as an

• Route Summarization and Subnetting

  24 Jun, 2024

  We will walk through the process of subnetting a network and performing route summarization using an example. Subnetting Example Let's consider the following four subnets: 192.168.0.0/22

• Minimizing Operational Overhead of EC2 Fleet OS Security Governance in AWS: Recommendations for DevOps Teams

  19 May, 2024

  Minimizing the operational overhead of EC2 fleet OS security governance is essential for maintaining a secure and efficient AWS environment. In this blog post, we'll explore the challenges faced by

• Implementing Resilient Architectures in AWS: Strategies for Automated Recovery and Testing

  17 May, 2024

  Implementing resilient architectures in AWS is essential for ensuring high availability and reliability of your applications. In this blog post, we'll explore strategies for automating recovery and

• Enabling Traceability and Auditing Security Events in AWS: Best Practices and Tools

  16 May, 2024

  Traceability and auditing of security events are crucial for maintaining the security and compliance of your AWS environment. In this blog post, we'll explore how to enable traceability and auditing

• Data Protection and Security Events in AWS: Best Practices for Ensuring Data Security

  13 May, 2024

  Protecting data in transit and at rest is critical for maintaining the security and compliance of your AWS environment. In this blog post, we'll explore best practices for classifying and protecting

• Automating Security Best Practices in AWS: A Guide to Efficient and Secure Operations

  12 May, 2024

  Automating security best practices in AWS is essential for ensuring the security, scalability, and efficiency of your cloud environment. In this blog post, we'll explore the benefits of automation,

• Authentication and Federation in AWS: Best Practices and Implementation Strategies

  10 May, 2024

  Authentication and federation are critical components of any AWS environment, ensuring secure access to resources and services. In this blog post, we'll explore the different types of identity in AWS,

• Applying Security at All Layers in AWS: A Comprehensive Approach

  2 May, 2024

  Security is paramount in any cloud environment, and AWS offers a range of tools and services to help you apply security at all layers of your infrastructure. In this blog post, we'll explore the

• Implementing a Strong Identity Foundation in AWS: Best Practices and Implementation Patterns

  30 Apr, 2024

  In any cloud environment, ensuring a strong identity foundation is paramount for maintaining security and compliance. AWS offers a range of tools and services to help you implement the principle of

• Elements in the JSON Policy Structure in IAM

  22 Apr, 2024

  Identities in AWS In AWS you manage access by creating policies and attaching them to an identity. The way that AWS thinks of the elements which interact with them is through IDENTITIES or AWS

• The AWS Well Architected Framework

  18 Apr, 2024

  Discover how to effectively design, utilize, and manage workloads in the cloud by translating requirements into architecture and operations while adhering to best practices. The Six Pillars:

• Data Encryption at AWS S3

  17 Apr, 2024

  What is Encryption at rest? Encryption works by using an algorithm to convert plain text into ciphertext. This new ciphertext will be unreadable if it falls into the wrong hands. There are many

• Introduction to AWS Identity and Access Management (IAM)

  12 Apr, 2024

  Theory Users must be authenticated before they can access AWS services and Resources. AWS services can be accessed via AWS CLI AWS SDKs AWS Management Console You can create: Individual IAM users:

• Distinctions Between AWS EC2 and ECS

  18 Dec, 2023

  Introduction Embarking on the cloud computing journey often involves deciphering the nuanced offerings of platforms like Amazon Web Services (AWS). In this exploration, we'll unravel the seemingly

• FTP and SFTP - Running through a Container

  13 Nov, 2023

  Running an FTP server using docker is really easy. In fact, you can use it running the following image: atmoz/sftp - But, at the end we want to know what is and FTP and why is it worth to know a

• What $ in shell scripts means?

  7 Feb, 2023

  In this video: https://www.youtube.com/watch?v=o9THkT5ZPi4&t=308s I saw the weird symbol: echo $? When I'm start looking about it on the internet I discovered that a lot of people has asked, and

• Basic concepts about Amazon Redshift

  30 Jan, 2023

  One of the first things you will know when you do the course Getting Started with Amazon Redshift are the following Redshift is based on PostgreSQL, and there are four key concepts to understand about

• Using Presing in AWS

  12 Dec, 2022

  Presing is a command you can use in the AWS CLI that allows anyone to have the pre-signed URL to make and HTTP get request to retrieve the data that is inside the bucket pre-signed. In the CLI you

• UNIX: A History and a Memoir

  3 Nov, 2022

  In the era of bright consultancy, where all things are opinionated, it’s difficult to find some refreshing ideas. For real it exists, but it is difficult to find. We are talking also about some

• What is a bastion host?

  24 Aug, 2022

  Definition of Bastion Host A bastion host is a specific computer in a network that has the objective of not affecting another part of the system by the attack from outside the network. For Example,

• Are SSH and Bash the same? (Spoiler: No)

  17 Aug, 2022

  The thing is: when you start to run some console commands you notice that all the things you write in that place are not the same. Simple to understand, difficult to order each part in your head. I

• Connect Ubuntu in Virtual Box with SSH

  10 Aug, 2022

  After understanding the importance of a well understanding of ssh . It’s time to make our first practice connecting from our windows to a Ubuntu installed in a virtual machine in Virtual Box. Download

• SSH: A Brave new world

  3 Aug, 2022

  When you make your first steps as a developer, you realize that one of the first activities you have to do when your code is ready is to deploy it. In that case, generally, the senior dev or someone