{"thread":{"id":"58954","subject":"Prematurely-closed stdin using async NodeJS to smudge large file on Mac","startedAt":"2022-12-15T03:59:21Z","lastAt":"2022-12-15T03:59:21Z","messageCount":1,"participants":["Stephen Hicks"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"469065","messageId":"CAKNkOnNAvV0tTkQXyRXjTiiUqPpu2cJSg5emrODu3AoGm79v+A@mail.gmail.com","threadId":"58954","inReplyTo":null,"subject":"Prematurely-closed stdin using async NodeJS to smudge large file on Mac","fromName":"Stephen Hicks","fromEmail":"stephenhicks@gmail.com","sentAt":"2022-12-15T03:59:05Z","receivedAt":"2022-12-15T03:59:21Z","isPatch":false,"sender":{"key":"stephenhicks@gmail.com","avatar":null},"body":"I recently observed some odd behavior when setting up a repo on a new\ncomputer.  I isolated it to some weird behavior where git seems to be\nclosing stdin prematurely.\n\nRepro steps:\n```\n# Make a new repo\nmkdir smudge-test\ncd smudge-test\ngit init\n\n# Grab the first directory on the path (NOTE: we'll write to $BIN/smudge)\nBIN=${PATH%%:*}\n\n# Set up the smudge filter\necho 'file filter=smudge' > .gitattributes\ncat >> .git/config <<EOF\n[filter \"smudge\"]\n  clean = smudge\n  smudge = smudge\n  required\nEOF\n\n# Write the smudge script into $BIN/smudge\ncat > $BIN/smudge <<EOF\n#!/usr/bin/env node\nconst fs = require('fs');\nfs.promises.readFile('/dev/stdin').then(src => {\n  console.error('Read ' + src.length + ' bytes');\n  process.exit(1);\n});\nEOF\nchmod +x $BIN/smudge\n\n# Make a 2mb file to smudge\nhead -c 2000000 /dev/random > file\n\n# Add it to the repo\ngit add .\n```\n\nWhat I see is that every time I run `git add .`, it quotes me a\ndifferent number of bytes read.  If I truncate the file down to (say)\n1mb, it works consistently every time (note: the add still fails\nbecause smudge exits 1, but the bytes read is consistent, which is\nwhat I'm looking for).  The problems seem to start somewhere around\n1.2mb or 1.3mb for me.\n\nChanging the NodeJS script to use `fs.readFileSync` seems to fix it,\nso it appears to be something peculiar to how NodeJS handles the main\ntask exiting before all of its async work is done (FWIW this is my\ncurrent workaround).  Piping the file to smudge directly (rather than\nvia git) works consistently as well, so it's apparently an interaction\nwith how git is handling the pipe.  I also put together a quick shell\nscript as a wrapper.  If the shell script `cat`s stdin to a temp file\nfirst and redirects the temp file to NodeJS, it works consistently.\nIf it just redirects /dev/stdin directly to NodeJS, it's inconsistent.\n\nI'm running git 2.39.0 freshly-installed via MacPorts on macOS\nMonterey 12.6.1, Darwin Kernel 21.6.0, Node v18.12.1.\n"}]}