The count-allocate-fill pattern

> Canonical sources: BJP §6.4 (count-allocate-fill case study) at pp 425–432; lecture sources/14-w26-lectures/week-07/arrays-lecture-notes.tex §"Two-pass file reading"; Lab 11 (Typed File Stats) SPEC.md.

A composite file-reading pattern that appears verbatim in Lab 11, in CSCD 240's lab-4 (file statistics in C), and in BJP chapter 7's Benford's-law case study. The pattern combines a hasNextLine loop that reads until the end of the file, an array allocation sized with new, and a counted-for fill. No single one of those three owns the composition, so this material names it explicitly.

The pattern answers the question: "How do I read N values from a file into an array when N is not known until the file has been examined?" The answer is two passes:

Pass 1 opens the file, counts the data lines, closes the file, and returns the count.
The caller allocates new int[count] (or double[count], String[count], etc.) of exactly the right size.
Pass 2 opens the file again (a fresh Scanner), fills the array in a counted for loop using sc.nextInt() (or nextDouble, nextLine), closes the file, and returns the array.

The crucial property: each pass opens its own Scanner. A Scanner cannot rewind; once Pass 1 has walked to EOF, the only way for Pass 2 to start from the beginning is to construct a new Scanner on the same file.

Why this is a concept area, not a single leaf

Each component (EOF loop, array allocation, counted fill) is already documented in its own leaf elsewhere. The pattern's pedagogical value is in the composition: the discipline of "two separate methods, each opening its own Scanner, with the caller allocating between them." That discipline is what gets tested on Lab 11; teaching the components alone leaves the composition implicit and students assemble it incorrectly.

The CSCD 211 alternative (ArrayList<Integer> plus .add()) collapses the two passes into one and is more idiomatic in industrial code, but it autoboxes for primitive types, which is the cost of choosing ArrayList over an array, and obscures the array-allocation semantics that week 6 has just introduced. CSCD 210 sticks with two passes; CSCD 211 revisits.

Why two passes: `Scanner` cannot rewind

Student can explain why java.util.Scanner does not support rewinding, identify two valid responses to "read N unknown-size values into an array" (two-pass with fresh Scanner each pass; ArrayList<E> with .add()), and justify why CSCD 210 chooses the two-pass route.

A Scanner reads forward through its input stream. Once a method has called sc.hasNextLine() and sc.nextLine() until the stream is exhausted, there is no language-level way to return the Scanner to the beginning of the file. The Scanner has no rewind, reset, or seek method: those operations would require buffering the entire file in memory, which Java avoids by design.

final Scanner sc = new Scanner(new File("data.txt"));
while (sc.hasNextLine()) {                   //  walk to EOF, counting
    sc.nextLine();
    count++;
}
//  At this point, sc is exhausted. Calling sc.nextLine() here throws NoSuchElementException.
//  There is no way to make sc start over.

This forces a design choice when the size of the data is not known up front. To read N values from a file into an array sized exactly N (no slack, no overflow):

Option A: Two passes, two Scanners. Pass 1 counts the lines without storing them. The caller allocates new int[count] between passes. Pass 2 opens a fresh Scanner, reads each value, and fills the array. The pattern this concept area documents.
Option B: One pass with ArrayList<E> and .add(). Allocate a resizable list, append each value, then convert to an array if needed. More idiomatic in production code, but autoboxes for primitive types (wraps each int in an Integer object on the heap, ~16 bytes overhead per value) and obscures the explicit allocation that week 6 just taught.

CSCD 210 chooses Option A. The choice is pedagogical: the two-pass form makes the array-allocation step visible and lets students compose primitives they already know (EOF loop, counted loop, array literal). Option B is revisited in CSCD 211 once ArrayList<E> and generics are fully covered.

What "rewind" looks like in C

C's fseek(fp, 0L, SEEK_SET) does exactly what students wish Java's Scanner could do: return the stream to the start. Java's FileInputStream has getChannel().position(0) for the underlying byte stream, but Scanner's buffered abstraction does not expose it. The deliberate choice keeps the Scanner API simple at the cost of forcing the two-pass discipline for unknown-size data.

The "just guess a big size" anti-pattern

A tempting alternative: allocate an oversized array, fill what you can, return it with the unused slack. The problems:

Wastes memory (proportional to the slack).
The caller now has to know how many slots are actually used, requiring a separate count.
The caller's loops over the array iterate past the real data into garbage.

Two passes plus exact allocation is the clean answer.

In other languages

Python: the with open(f) as fp: block reads forward; fp.seek(0) rewinds. The two-pass approach works in Python but is less idiomatic: Python programmers use list.append in one pass.
C: fseek(fp, 0L, SEEK_SET) rewinds; the two-pass approach is also valid (and sometimes faster than dynamic allocation for primitive arrays).
Go: bufio.Scanner is forward-only like Java's; the two-pass approach applies. Go's append to a slice is the one-pass alternative.

Pass 1: the counting loop

**Student can produce a Pass-1 counting method that takes a filename, opens a fresh Scanner, walks to EOF using a hasNextLine (or hasNextX) guard, increments a counter while discarding the read value, closes the Scanner, and returns the count.**

The first pass walks the file from start to end, increments a counter for each data line (or token), and returns the count. It does not store the values themselves: the values come back in Pass 2. The signature:

public static int countLines(final String filename) throws FileNotFoundException {
    final Scanner sc = new Scanner(new File(filename));
    int count = 0;
    while (sc.hasNextLine()) {
        sc.nextLine();       //  consume the line; throw it away
        count++;
    }
    sc.close();
    return count;
}

Four design commitments visible in this shape:

1. The method takes a String filename, not a Scanner. The short version of the discipline is that each pass opens its own Scanner. 2. sc.nextLine() is called, not just sc.hasNextLine(). The hasNextLine guard only peeks at the buffer; without a corresponding nextLine call inside the loop, the cursor never advances and the loop runs forever. The two methods always pair: hasNextLine to test, nextLine to consume. 3. The result of sc.nextLine() is discarded. Pass 1 cares about the count, not the values. Discarding by sc.nextLine(); (no assignment) is idiomatic; the JDK does the same in its own counting utilities. 4. The Scanner is closed before the return. The OS file descriptor is released. The rule that each pass opens its own Scanner covers the close discipline, and the try-with-resources form is the modern idiom that avoids the manual close.

Variant: counting tokens, not lines

When the data is whitespace-delimited rather than line-delimited:

public static int countInts(final String filename) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {
        int count = 0;
        while (sc.hasNextInt()) {
            sc.nextInt();        //  consume the token, discard the value
            count++;
        }
        return count;
    }
}

The shape is the same; only the guard and the read change. hasNextInt paired with nextInt for typed tokens; hasNext paired with next for generic tokens.

Variant: skipping a header line before counting

Lab 11's typed-file format puts a type tag on the first line and the data on subsequent lines. The count should exclude the tag:

public static int countDataLines(final String filename) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {
        sc.nextLine();           //  discard the type tag
        int count = 0;
        while (sc.hasNextLine()) {
            sc.nextLine();
            count++;
        }
        return count;
    }
}

The discard happens before the loop, not inside it. Putting it inside requires a "first iteration is special" branch and is harder to read.

In other languages

Python: with open(filename) as f: count = sum(1 for _ in f): one-liner using generator counting.
C: loop with fgets returning non-NULL; increment a counter; same shape as Java's.
Unix shell: wc -l filename does exactly this from the command line; the wc source code is the canonical reference for "how to count lines in a file."

Pass 2: allocate the array, fill with a counted `for`

Student can produce a Pass-2 method that takes a filename and a count, opens a fresh Scanner, allocates an array of exactly count slots, fills it with a counted for loop using the appropriate typed read, closes the Scanner, and returns the array.

Pass 2 receives the count from Pass 1, opens a fresh Scanner on the same file, allocates the array of exactly that size, fills it with a counted for loop, closes the Scanner, and returns the array.

public static int[] readInts(final String filename, final int count) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {
        final int[] arr = new int[count];               //  allocate exactly count slots
        for (int i = 0; i < count; i++) {               //  counted for, not EOF loop
            arr[i] = sc.nextInt();
        }
        return arr;
    }
}

Four design commitments visible in this shape:

1. The method takes both filename AND count. Pass 1 returned the count; the caller supplies both pieces to Pass 2. Alternative designs (Pass 2 calls Pass 1 internally) couple the two passes and make testing harder. 2. The allocation uses the exact count, not a "safe overestimate." new int[count] produces an array sized to fit the data exactly: no slack, no overflow. The reason two passes are needed covered why this matters. 3. The fill loop is a counted for, not an EOF loop. Now that the size is known, the loop bound is the array's length. Using while (sc.hasNextInt()) here would also work if the count was correct, but the counted for makes the contract explicit ("exactly count reads"). If the file has fewer values than count claims (a Pass-1 bug), the counted-for fails loudly with NoSuchElementException at the right slot. 4. The cross-pass type-tag handling. If Pass 1 skipped a header tag, Pass 2 must do the same. The skip happens before the counted loop starts: sc.nextLine(); once on entry, then the counted for. Both passes must agree on the header structure.

Variant: reading whitespace-delimited tokens

public static double[] readDoubles(final String filename, final int count) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {
        final double[] arr = new double[count];
        for (int i = 0; i < count; i++) {
            arr[i] = sc.nextDouble();
        }
        return arr;
    }
}

Same shape with nextDouble replacing nextInt. The element type of the array (double[]) matches the typed read; mixing them produces compile errors.

Variant: reading String lines

public static String[] readLines(final String filename, final int count) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {
        final String[] arr = new String[count];
        for (int i = 0; i < count; i++) {
            arr[i] = sc.nextLine();
        }
        return arr;
    }
}

The Scanner's mode (token vs line) is set by which next is called. Each Pass 2 variant matches its Pass 1 counterpart: nextLine counts → nextLine reads; nextInt counts → nextInt reads.

The contract between Pass 1 and Pass 2

The two passes must agree on three things:

1. The file format. Both passes open the same file; both must read it the same way (line-mode or token-mode). 2. The header convention. If Pass 1 skips the header, Pass 2 does too. If Pass 1 counts the header as a data line, Pass 2 reads it. 3. The count's meaning. Pass 1's count must equal Pass 2's iteration count. A Pass-1 bug (off by one, wrong skip rule) produces a Pass-2 crash or silent data corruption.

The pair is fragile because the agreement is informal: Java's type system does not enforce that Pass 1 and Pass 2 share a contract. The CSCD 210 discipline: write the two as paired helpers (countLines + readLines), keep them in the same class, and review them together when either changes.

In other languages

Python: with open(filename) as f: lines = [next(f) for _ in range(count)]: list comprehension with explicit count. More commonly, lines = list(f) (one-pass, dynamic-size).
C: identical pattern: fopen → fscanf count times → fclose. The C version is the conceptual source.
Java streams: Files.lines(Path.of(filename)).limit(count).toArray(String[]::new): one-pass, Stream-based. CSCD 211 territory.

Each pass opens its own Scanner

Student can recognize the "each pass opens its own Scanner" discipline as the correct count-allocate-fill convention, identify three common antipatterns (Scanner-as-parameter, Scanner-as-field, sc.reset() between passes) and explain why each fails, and articulate that opening a fresh Scanner reopens the underlying OS file at byte position 0.

The discipline that ties the count-allocate-fill pattern together: Pass 1 and Pass 2 each construct, use, and close their own Scanner. Neither pass receives a Scanner as a parameter; neither pass leaves a Scanner open for the other to consume.

public static int countLines(final String filename) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {     //  Pass 1's own Scanner
        // ... count loop ...
        return count;
    }   //  Scanner closed here
}

public static int[] readInts(final String filename, final int count) throws FileNotFoundException {
    try (Scanner sc = new Scanner(new File(filename))) {     //  Pass 2's own Scanner: fresh
        // ... fill loop ...
        return arr;
    }   //  Scanner closed here
}

Three properties of the per-pass discipline:

1. Each method takes a String filename, not a Scanner. The function signature advertises "I open my own file." Callers cannot accidentally share a partially-consumed Scanner between the passes. 2. Each Scanner is constructed inside its method. The construction is what reopens the underlying OS file at byte position 0. There is no language-level "Scanner rewind"; constructing a new one is the only way. 3. Each Scanner is closed before the method returns. Try-with-resources handles this automatically. Without try-with-resources, the explicit sc.close() after the loop body (paired with a try/finally cleanup) does the same.

What students want the pattern to be (but is not)

Three antipatterns students reach for, each rejected:

//  Antipattern 1: share the Scanner via parameter
public static int countLines(Scanner sc) { ... }       //  WRONG
public static int[] readInts(Scanner sc, int count) { ... }     //  WRONG
//  After Pass 1 walks to EOF, Pass 2 gets an exhausted Scanner.

//  Antipattern 2: store the Scanner as a static field
private static Scanner sc;                              //  WRONG
//  Same problem: Pass 1 exhausts; Pass 2 sees EOF.

//  Antipattern 3: call sc.reset() between passes
sc.reset();                                             //  WRONG
//  Scanner.reset() resets configuration (delimiters, locale).
//  It does NOT rewind the stream. The Scanner is still at EOF.

The correct pattern is the verbose one: two methods, each opening its own Scanner.

What "opening a fresh Scanner on the same file" actually does

At the OS level, each new Scanner(new File(filename)) call:

1. Opens the file via FileInputStream (allocates a new file descriptor at the OS level). 2. Wraps it in an InputStreamReader (character decoding). 3. The Scanner reads from that Readable and maintains its own internal CharBuffer (Scanner does its own buffering; it does not wrap a separate BufferedReader). 4. Returns a Scanner whose cursor is at byte position 0.

The two passes therefore use two independent OS file descriptors. There is no shared state between them. If the file is modified between passes (rare in CSCD 210, common in production), the second pass sees the modified version.

Why this works for CSCD 210 inputs but is suspect at scale

The two-passes-with-fresh-Scanner pattern is correct for files that do not change between passes, which is true for every CSCD 210 input. In production code, the assumption fails: a log file actively being written to may have more lines on the second pass than the first did. The mitigations (file locking, snapshot-and-process, atomic append patterns) are out of scope for CSCD 210 but worth knowing exist.

In other languages

C: fopen is called twice on the same path; each call returns an independent FILE*. Same discipline.
Python: with open(filename) as f1: and with open(filename) as f2: open independently. The pattern is identical.
Bash: wc -l file.txt > tmp; <count from tmp>; <process file.txt>: the count is captured between two reads of the file.

The "fresh resource per pass" discipline is universal in stream-based file processing.

Why this is a concept area, not a single leaf

Why two passes: Scanner cannot rewind

What "rewind" looks like in C

The "just guess a big size" anti-pattern

In other languages

Pass 1: the counting loop

Variant: counting tokens, not lines

Variant: skipping a header line before counting

In other languages

Pass 2: allocate the array, fill with a counted for

Variant: reading whitespace-delimited tokens

Variant: reading String lines

The contract between Pass 1 and Pass 2

In other languages

Each pass opens its own Scanner

What students want the pattern to be (but is not)

What "opening a fresh Scanner on the same file" actually does

Why this works for CSCD 210 inputs but is suspect at scale

In other languages

Why two passes: `Scanner` cannot rewind

Pass 2: allocate the array, fill with a counted `for`