Tag: HDFS
All the articles with the tag "HDFS".
-
How HDFS Achieves Fault Tolerance Through Replication
One of the core strengths of the Hadoop Distributed File System (HDFS) is its fault tolerance . In a world of distributed computing, failures are not rare—they're expected. HDFS tackles this by using
-
Summary: Teaching HDFS Concepts to New Learners
Introducing Hadoop Distributed File System (HDFS) to newcomers can be both exciting and challenging. To make the learning experience structured and impactful, it’s helpful to break down the core
-
How Clients Know Where to Read or Write in HDFS
Hadoop Distributed File System (HDFS) is designed to decouple metadata management from actual data storage . But how does a client—like a Spark job or command-line tool—know where to read or write the
-
How HDFS Avoids Understanding File Content
One of the defining features of Hadoop Distributed File System (HDFS) is that it doesn’t understand the contents of the files it stores . This is not a limitation—it's an intentional design choice
-
How Spark and MapReduce Handle Partial Records in HDFS
When working with large-scale data processing frameworks like Apache Spark or Hadoop MapReduce, one common question arises: What happens when a record (e.g., a line of text or a JSON object) is split
-
How HDFS Tracks Block Size and File Boundaries
When dealing with massive files, Hadoop Distributed File System (HDFS) doesn't read or store them as a whole. Instead, it splits them into large, fixed-size blocks . But how does it know where each
-
How Metadata Works in HDFS and What It Stores
HDFS stores metadata separately from the actual file content to optimize performance and scalability. This metadata is managed entirely by the NameNode , which allows clients to quickly locate and
-
The Architecture of HDFS: NameNode, DataNodes, and Metadata
HDFS (Hadoop Distributed File System) was built to support the reliable storage and access of large datasets distributed across commodity hardware. To make this possible, HDFS relies on a master/slave