threads / discuss / 62983

Deleting first commits; maintaining last commits

Subject: Deleting first commits; maintaining last commits

## tl;dr

6 messages between Feb 20, 2025 and Feb 21, 2025.

replies: 5people: 5as markdown or json

Jamenson Espindula· Feb 20, 2025, 22:53 UTC · lore
Hi all.
My Git repository on GitHub <https://github.com/espindula/br-blfs> has
about 23,500 commits. However, there are several old (before Feb, 28
2022) commits I would like to delete and maintain the newer ones
(after Feb, 28 2022). So, Is there any Git command (or combined
commands) I could use?
Any help will be appreciated.
Thank you in advance.

Jamenson Ferreira Espindula de Almeida Melo Jaboatão dos Guararapes, Pernambuco, Brazil GNU/Linux user #166197; LFS ID 24492 Key fingerprint: 234D 1914 4224 7C53 BD13 6855 2AE0 25C0 08A8 6180

brian m. carlson· Feb 21, 2025, 00:18 UTC · re: Jamenson Espindula · lore

Re: Deleting first commits; maintaining last commits

On 2025-02-20 at 22:53:06, Jamenson Espindula wrote:
> Hi all.
Hi,
Show 5 quoted lines
> My Git repository on GitHub <https://github.com/espindula/br-blfs> has
> about 23,500 commits. However, there are several old (before Feb, 28
> 2022) commits I would like to delete and maintain the newer ones
> (after Feb, 28 2022). So, Is there any Git command (or combined
> commands) I could use?

No, Git doesn't offer such a thing. Due to the use of cryptographic hashes used, it would be impossible to verify the integrity of the repository if it could just be truncated like that. In addition, the goal of Git as a version control system is to track history, not to destroy it.

However, if the concern is size and not something else (like removing personal information), then you could use a shallow clone to just download a certain number of revisions and work on that. The full history would remain on the server, and you could still push newer changes, but the size on your local machine would be smaller. If you need more history, you could use a partial clone instead if you're willing to be online to work.

I'll note that 23,500 commits is not that many. Git itself has 76,212 commits in my local copy, Linux has over 1,335,000, and I routinely work on a work project with over 500,000. Git should scale much farther than that provided you don't have a really misshapen repository.

-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA
Konstantin Ryabitsev· Feb 21, 2025, 15:17 UTC · re: brian m. carlson · lore

Re: Deleting first commits; maintaining last commits

On Fri, Feb 21, 2025 at 12:18:09AM +0000, brian m. carlson wrote:
Show 19 quoted lines
> > My Git repository on GitHub <https://github.com/espindula/br-blfs> has
> > about 23,500 commits. However, there are several old (before Feb, 28
> > 2022) commits I would like to delete and maintain the newer ones
> > (after Feb, 28 2022). So, Is there any Git command (or combined
> > commands) I could use?
> 
> No, Git doesn't offer such a thing.  Due to the use of cryptographic
> hashes used, it would be impossible to verify the integrity of the
> repository if it could just be truncated like that.  In addition, the
> goal of Git as a version control system is to track history, not to
> destroy it.
> 
> However, if the concern is size and not something else (like removing
> personal information), then you could use a shallow clone to just
> download a certain number of revisions and work on that.  The full
> history would remain on the server, and you could still push newer
> changes, but the size on your local machine would be smaller.  If you
> need more history, you could use a partial clone instead if you're
> willing to be online to work.

Another approach is to create a new repository and use a graft/replacement commit to indicate that history continues in a different repository, right? I do sometimes wish this was a bit easier/more accessible to perform, because that would allow creating "epochs" for very large repos. Unfortunately, shallow clones tend to be very heavy on the server-side.

-K
Emily Shaffer· Feb 21, 2025, 16:05 UTC · re: Konstantin Ryabitsev · lore

Re: Deleting first commits; maintaining last commits

On Fri, Feb 21, 2025 at 7:18 AM Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

Show 27 quoted lines
>
> On Fri, Feb 21, 2025 at 12:18:09AM +0000, brian m. carlson wrote:
> > > My Git repository on GitHub <https://github.com/espindula/br-blfs> has
> > > about 23,500 commits. However, there are several old (before Feb, 28
> > > 2022) commits I would like to delete and maintain the newer ones
> > > (after Feb, 28 2022). So, Is there any Git command (or combined
> > > commands) I could use?
> >
> > No, Git doesn't offer such a thing.  Due to the use of cryptographic
> > hashes used, it would be impossible to verify the integrity of the
> > repository if it could just be truncated like that.  In addition, the
> > goal of Git as a version control system is to track history, not to
> > destroy it.
> >
> > However, if the concern is size and not something else (like removing
> > personal information), then you could use a shallow clone to just
> > download a certain number of revisions and work on that.  The full
> > history would remain on the server, and you could still push newer
> > changes, but the size on your local machine would be smaller.  If you
> > need more history, you could use a partial clone instead if you're
> > willing to be online to work.
>
> Another approach is to create a new repository and use a graft/replacement
> commit to indicate that history continues in a different repository, right? I
> do sometimes wish this was a bit easier/more accessible to perform, because
> that would allow creating "epochs" for very large repos. Unfortunately,
> shallow clones tend to be very heavy on the server-side.

For hosts which support it - which I believe includes GitHub - partial clone is generally easier on the server and a little bit less bug-prone than shallow clone.

>
> -K
>
brian m. carlson· Feb 21, 2025, 22:42 UTC · re: Emily Shaffer · lore

Re: Deleting first commits; maintaining last commits

On 2025-02-21 at 16:05:59, Emily Shaffer wrote:
> For hosts which support it - which I believe includes GitHub - partial
> clone is generally easier on the server and a little bit less
> bug-prone than shallow clone.

The really expensive thing is fetching into a shallow clone. A simple shallow clone itself is not very expensive, and at $DAYJOB we encourage large-scale users (such as CI systems) to use shallow clones for that reason, since they're cheaper to serve than full clones, provided that they never fetch into them.

Now, I agree that the particular use case here is probably going to be fetching into a shallow clone, and that's okay if it's one particular user doing that occasionally, which it sounds like it is. It's definitely a problem if it's thousands of CI jobs, though.

It may be that a partial clone does perform better overall, but it has the downside that it effectively requires you to be online during usage, whereas a shallow clone does not. Whether that is a problem depends on your use case: for work, I am effectively always online, so that's not a problem, but for personal work, I want to be able to work on an airplane (where there may be no Internet and the Wi-Fi, if any, is very slow), in a hotel room with bad connectivity, or even on a retreat in the middle of nowhere without Internet at all.

So that's why I mentioned both: both options have some upsides and some downsides, and there's no one-size-fits-all solution.

-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA
Elijah Newren· Feb 21, 2025, 21:50 UTC · re: Konstantin Ryabitsev · lore

Re: Deleting first commits; maintaining last commits

On Fri, Feb 21, 2025 at 7:18 AM Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

Show 27 quoted lines
>
> On Fri, Feb 21, 2025 at 12:18:09AM +0000, brian m. carlson wrote:
> > > My Git repository on GitHub <https://github.com/espindula/br-blfs> has
> > > about 23,500 commits. However, there are several old (before Feb, 28
> > > 2022) commits I would like to delete and maintain the newer ones
> > > (after Feb, 28 2022). So, Is there any Git command (or combined
> > > commands) I could use?
> >
> > No, Git doesn't offer such a thing.  Due to the use of cryptographic
> > hashes used, it would be impossible to verify the integrity of the
> > repository if it could just be truncated like that.  In addition, the
> > goal of Git as a version control system is to track history, not to
> > destroy it.
> >
> > However, if the concern is size and not something else (like removing
> > personal information), then you could use a shallow clone to just
> > download a certain number of revisions and work on that.  The full
> > history would remain on the server, and you could still push newer
> > changes, but the size on your local machine would be smaller.  If you
> > need more history, you could use a partial clone instead if you're
> > willing to be online to work.
>
> Another approach is to create a new repository and use a graft/replacement
> commit to indicate that history continues in a different repository, right? I
> do sometimes wish this was a bit easier/more accessible to perform, because
> that would allow creating "epochs" for very large repos. Unfortunately,
> shallow clones tend to be very heavy on the server-side.

I'm totally in support of the "friends don't let friends use shallow-clones" point of view, even if I've had rather less success than I would have liked at promoting it. But throwing shade at shallow clones is just an impulse, not the real reason I'm responding here.

Yes, if you're willing to rewrite history, invalidating all hashes and rewriting the recent-enough-ones-that-you-still-want-to-keep, you can go that route too. In fact, doing so is as easy as creating a replace ref graft (or grafts) which make the oldest commit(s) you want to keep look parentless, and then rewriting history to make that graft permanent and remove the graft. (Afterwards, you can make a separate graft from that new root commit to the original real commit if you want.)

It turns out this is an example from user filed issues that comes up occasionally in git-filter-repo. While it's an example I documented with the commands to run, just so I can link to it if it's asked again, I really don't like the strategy of destroying history, especially since most users who ask for this kind of capability (Konstantin, you're a notable exception here) don't understand the points that brian brought up in his first paragraph. And I make sure to say as much right before I provide the example in my docs. But, yes, this is a possibility for those that don't mind destroying history.

← back to recent threads