Streams, Buffers and Backpressure
Know how Buffers share memory and encode text, how streams buffer and apply backpressure, and why `pipeline` beats chained `.pipe()` calls in production.
Key points
- 1
Buffer.alloczero-fills;Buffer.allocUnsafeis faster but may contain old memory.subarrayandBuffer.from(arrayBuffer)share memory, whileBuffer.from(buffer | typedArray)copies. - 2
String length is UTF-16 code units;
Buffer.byteLengthis encoded bytes. Decode chunked text withsetEncodingorStringDecoderso multi-byte characters split across chunks survive. - 3
write()always queues the chunk and returnsfalseonce the buffer reacheshighWaterMark. Stop producing and wait fordrain, or letpipelinedo it for you. - 4
pipeline(a, b, c, cb)forwards errors, destroys every stream on failure or premature close, and calls back once..pipe()handles backpressure but not errors or cleanup. - 5
Async iteration (
for await) is pull-based and respects backpressure. Breaking out of the loop destroys the stream. Async generators can act as transforms insidepipeline. - 6
A Transform must call its callback exactly once per chunk. Trailing output belongs in
flush, and a missing callback stalls the whole pipeline without an error. - 7
Readables start paused and keep data until someone reads it. A
readablelistener takes precedence overdata.fetchbodies are web streams; bridge them withReadable.fromWeb.
Common traps
Buffer#sliceshares memory, unlikeArrayandTypedArrayslice, which copy.An unhandled
errorevent on any piped stream (for example gunzip on bad input) crashes the whole process.Readable.from('abc')emits the whole string as one chunk; it does not iterate characters.
Read the source
Test yourself on Streams, Buffers and Backpressure
Ten questions, with the answer and explanation after each one.