Streams and Backpressure

Detailed answer

Logic: Readable pushes data; Writable signals backpressure when internal buffer exceeds highWaterMark. Respect drain event before resuming writes to avoid memory blowup. Complexity notes included in code section when present. Pros: Constant memory for large files; composable pipeline with pipe(). Cons: Error handling across piped streams needs explicit destroy/abort logic. Core: Readable pushes data; Writable signals backpressure when internal buffer exceeds highWaterMark. Respect drain event before resuming writes to avoid memory blowup. Real-time example: ETL job pipes HTTP download → gzip transform → S3 upload without loading entire file into RAM. Pros: Constant memory for large files; composable pipeline with pipe(). Cons: Error handling across piped streams needs explicit destroy/abort logic. Common mistakes: Ignoring write() return false; not awaiting 'drain' on backpressure. Best practices: Use pipeline() from stream/promises for automatic cleanup on error. Audience level: Fresher.

Full explanation

Readable pushes data; Writable signals backpressure when internal buffer exceeds highWaterMark. Respect drain event before resuming writes to avoid memory blowup.

Code example

const { Readable, Writable } = require('stream');

function writeWithBackpressure(source, dest) {
  source.on('data', (chunk) => {
    const ok = dest.write(chunk);
    if (!ok) {
      source.pause();
      dest.once('drain', () => source.resume());
    }
  });
  source.on('end', () => dest.end());
}

Practice code · javascript (view only · no execution)

const { Readable, Writable } = require('stream');

function writeWithBackpressure(source, dest) {
  source.on('data', (chunk) => {
    const ok = dest.write(chunk);
    if (!ok) {
      source.pause();
      dest.once('drain', () => source.resume());
    }
  });
  source.on('end', () => dest.end());
}