Re: [RFC PATCH] Introduce "precious" file concept

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Nov 28, 2018, 01:21 UTC
Message-ID: <20181128012142.GT890086@genre.crustytoothpaste.net>
In-Reply-To: <56ffc52f-644c-2c1d-6a16-c1005b064385@hibox.tv>

On Tue, Nov 27, 2018 at 02:50:34PM +0000, Per Lundberg wrote:

Show 12 quoted lines

> I agree strongly with this personally; if we must choose between "might
> break automation" and "might delete non-garbage files", I would say the
> former is the lesser evil of the two.
> 
> But, if I had 10 000 000 servers set up using automated scripts that
> would break because of this, I might think differently. Quite likely so,
> in fact.
> 
> What are these automation scenarios _more specifically_? Junio or Brian,
> would you care to elaborate? Is it for build servers where you want "git
> clean -dfx" to always reset the working copy to a pristine state or are
> we talking about some other scenarios?

We had long-running CI servers, since bootstrapping a new system took an hour. These would check out the branch to test and run some process (essentially, a "make" and "make test"). Then, another branch would be tested, and so on. The old branch would likely not be merged at this point.

The scenario I'm thinking of is when a file (say a CSS file) became built instead of stored in the repository. Then the file would be added to .gitignore in the new commit, and it would be generated as part of the make step. It would be important to blow away that file away when checking out a new commit, because not doing so would mean that the CI system would simply fail to work and require manual intervention.

Moreover, a CI job might fail, due to either a flaky test or a legitimate failures, so the job might need to be re-run multiple times. Requiring human intervention, especially when such jobs might be running at odd hours, would be undesirable.

Another thing we did was to use a specially named gitignore file in our build step. We created a new repository, copied the special gitignore file in as .gitignore, copied in the source and build products, ran git add and git commit, and then ran git clean -dfx to remove proprietary source code, packaging the result. A change to the behavior of git clean -dfx would be devastating here.

I point this out to underscore how fundamental this change is. People overwhelmingly do not read the release notes, so expecting people to realize that a change has been made, especially when many people only upgrade Git because of a security issue, may result in unexpected consequences. Just because we don't think of this use of Git as normal or desirable doesn't mean people don't do it and don't expect it to keep working. People do read and rely on our documentation.

I think any change we make here has to be opt-in, at least until Git 3.0. A config knob would probably be the right way to go. I realize that may not provide all the benefits we want out of the box, but it lets people turn the option on once and forget about it. It also lets people who don't desire this new behavior explicitly turn it off.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204