File Descriptors Deep Dive: open flags, offsets, CLOEXEC, and dup()

iofilesunixlinuxfile-descriptorssyscallsczigrust

This article covers File Descriptors Deep Dive: open flags, offsets, CLOEXEC, and dup(). File descriptors are the universal I/O handle on Unix. Learn how fd state works (offsets, flags), how dup/dup2 behave, why CLOEXEC matters, and how ...

On Unix-like systems, a file descriptor (fd) is the basic currency of I/O.

  • regular files
  • pipes
  • sockets
  • eventfd/timerfd/signalfd
  • epoll fds

They’re all “just fds”. That simplicity is powerful, but correctness and security require understanding what state is attached to an fd.

1) fd vs open file description (the part people miss)

There are two layers:

  • File descriptor (per-process): a small integer index into a table.
  • Open file description (kernel object): contains the open state.

Multiple fds can point to the same open file description.

That matters because the open file description contains:

  • current file offset (for read/write)
  • status flags (e.g., O_APPEND, O_NONBLOCK)

So, if you dup() an fd, both fds share the same offset.

2) open flags you should actually care about

O_CLOEXEC

Prevents a file descriptor from leaking into a child process after execve.

Why it matters:

  • security (accidental access)
  • correctness (keeps sockets/files open unexpectedly)
  • resource leaks (child inherits fds)

O_APPEND

Every write appends at end-of-file atomically (with respect to file offset updates).

O_NONBLOCK

Crucial for event loops.

O_TRUNC, O_CREAT, O_EXCL

Use O_EXCL|O_CREAT when you need “create new file or fail”.

3) dup() vs open() vs pread()

  • dup(fd) → new fd, same open file description (shared offset)
  • open(path) twice → two independent open file descriptions (independent offsets)
  • pread(fd, buf, n, off) → reads from a specific offset without changing current offset

4) C: demonstrate shared offsets with dup()

#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char **argv) {
    if (argc != 2) {
        fprintf(stderr, "usage: %s <file>\n", argv[0]);
        return 2;
    }

    int fd = open(argv[1], O_RDONLY | O_CLOEXEC);
    if (fd < 0) {
        fprintf(stderr, "open: %s\n", strerror(errno));
        return 1;
    }

    int fd2 = dup(fd);
    if (fd2 < 0) {
        fprintf(stderr, "dup: %s\n", strerror(errno));
        return 1;
    }

    char a[4] = {0};
    char b[4] = {0};

    // Read 3 bytes from fd.
    if (read(fd, a, 3) != 3) {
        fprintf(stderr, "read a failed\n");
        return 1;
    }

    // Read next 3 bytes from fd2. Because fd2 shares the offset, this continues.
    if (read(fd2, b, 3) != 3) {
        fprintf(stderr, "read b failed\n");
        return 1;
    }

    printf("a=%s\n", a);
    printf("b=%s\n", b);

    close(fd2);
    close(fd);
    return 0;
}

Try it on a file containing abcdef... and you’ll see:

  • first read returns abc
  • second returns def

5) Zig: open with CLOEXEC and use pread

const std = @import("std");
const os = std.os;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const a = gpa.allocator();

    var args = try std.process.argsAlloc(a);
    defer std.process.argsFree(a, args);
    if (args.len != 2) return error.InvalidArgs;

    const path = args[1];

    // Prefer openFile via std.fs when possible.
    var f = try std.fs.cwd().openFile(path, .{});
    defer f.close();

    var buf: [3]u8 = undefined;

    // pread-like behavior:
    const n0 = try f.preadAll(&buf, 0);
    std.debug.print("pread@0: {s}\n", .{buf[0..n0]});

    const n3 = try f.preadAll(&buf, 3);
    std.debug.print("pread@3: {s}\n", .{buf[0..n3]});
}

6) Rust: CLOEXEC, dup, and pread

Rust’s std::fs::File is built on fds on Unix.

use std::fs::File;
use std::io::{self, Read};
use std::os::unix::io::{AsRawFd, FromRawFd};

fn main() -> io::Result<()> {
    let path = std::env::args().nth(1).expect("path");
    let mut f = File::open(path)?;

    // Demonstrate that reading advances the offset:
    let mut buf = [0u8; 3];
    f.read_exact(&mut buf)?;
    println!("read: {}", String::from_utf8_lossy(&buf));

    // Duplicate fd (shares offset):
    unsafe {
        let fd2 = libc::dup(f.as_raw_fd());
        if fd2 < 0 { return Err(io::Error::last_os_error()); }
        let mut f2 = File::from_raw_fd(fd2);

        let mut buf2 = [0u8; 3];
        f2.read_exact(&mut buf2)?;
        println!("read via dup: {}", String::from_utf8_lossy(&buf2));
    }

    Ok(())
}

7) Robustness patterns

  • Always set O_CLOEXEC (or call fcntl(FD_CLOEXEC)) for server programs.
  • Prefer pread/pwrite when multiple threads access one file.
  • Use openat and directory fds to avoid path races.

References