# [PATCH] Dissociating a repository from its alternates 9 messages from 2007-02-15 to 2007-10-23. Participants: Junio C Hamano, Johannes Schindelin, James Cloos, Michael Hendricks. Thread: https://gitlist.dev/t/6829 ## Junio C Hamano, 2007-02-15 07:51 Subject: [PATCH] Dissociating a repository from its alternates Message-ID: <7vabzfhn9q.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vabzfhn9q.fsf%40assigned-by-dhcp.cox.net ``` People often start out with "clone -l -s" (or --reference) and later wish to dissociate the repository from its alternates. The new option -A to "git repack" lets you do so. What it does: - git-pack-objects gets a new option --ignore-alternate-pack. The command used in git-repack is told to pack only unpacked objects in general, but also is told to pretend objects in packs in the local repository are unpacked (this is necessary to keep historical huge packs marked with .keep files without repacking). This new option tells it to also pretend that objects from packs in alternate repositories are unpacked, to make them subject to packing. - code to parse revision traversal options gets an additional bit, ignore_alternate_pack. - has_sha1_pack() function gets an additional argument for that bit. Existing callers pass 0 to it (they continue to behave as before). - git-repack obviously understands the new -A option. When it is given, it passes --ignore-alternate-pack option to git-pack-objects to include objects from borrowed packs in the result, and $GIT_OBJECT_DIRECTORY/info/alternates file is removed at the end. The patch was tested very lightly, and I haven't verified the interactions between various options git-repack takes, and options it passes to git-pack-objects (especially, I think interaction with --local and --incremental might be dubious). Because a bug in git-repack has real risk of corrupting repository in a bad way, somebody else needs to double check this patch, if we were to do this. Signed-off-by: Junio C Hamano --- builtin-count-objects.c | 2 +- builtin-pack-objects.c | 1 + builtin-prune-packed.c | 2 +- cache.h | 2 +- diff.c | 2 +- git-repack.sh | 30 ++++++++++++++++++++++-------- revision.c | 12 +++++++++--- revision.h | 1 + sha1_file.c | 16 +++++++++------- 9 files changed, 46 insertions(+), 22 deletions(-) diff --git a/builtin-count-objects.c b/builtin-count-objects.c index f5b22bb..10c8860 100644 --- a/builtin-count-objects.c +++ b/builtin-count-objects.c @@ -62,7 +62,7 @@ static void count_objects(DIR *d, char *path, int len, int verbose, hex[40] = 0; if (get_sha1_hex(hex, sha1)) die("internal error"); - if (has_sha1_pack(sha1, NULL)) + if (has_sha1_pack(sha1, NULL, 0)) (*packed_loose)++; } } diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c index 3824ee3..0f5d72c 100644 --- a/builtin-pack-objects.c +++ b/builtin-pack-objects.c @@ -1623,6 +1623,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) } if (!strcmp("--unpacked", arg) || !strncmp("--unpacked=", arg, 11) || + !strcmp("--ignore-alternate-pack", arg) || !strcmp("--reflog", arg) || !strcmp("--all", arg)) { use_internal_rev_list = 1; diff --git a/builtin-prune-packed.c b/builtin-prune-packed.c index 9777300..eb55c58 100644 --- a/builtin-prune-packed.c +++ b/builtin-prune-packed.c @@ -20,7 +20,7 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts) memcpy(hex+2, de->d_name, 38); if (get_sha1_hex(hex, sha1)) continue; - if (!has_sha1_pack(sha1, NULL)) + if (!has_sha1_pack(sha1, NULL, 0)) continue; memcpy(pathname + len, de->d_name, 38); if (opts & DRY_RUN) diff --git a/cache.h b/cache.h index c62b0b0..0c24318 100644 --- a/cache.h +++ b/cache.h @@ -266,7 +266,7 @@ extern int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer, extern int write_sha1_to_fd(int fd, const unsigned char *sha1); extern int move_temp_to_file(const char *tmpfile, const char *filename); -extern int has_sha1_pack(const unsigned char *sha1, const char **ignore); +extern int has_sha1_pack(const unsigned char *sha1, const char **ignore, int ignore_alternates); extern int has_sha1_file(const unsigned char *sha1); extern void *map_sha1_file(const unsigned char *sha1, unsigned long *); extern int legacy_loose_object(unsigned char *); diff --git a/diff.c b/diff.c index 12c8b2b..ee46960 100644 --- a/diff.c +++ b/diff.c @@ -1254,7 +1254,7 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int * objects however would tend to be slower as they need * to be individually opened and inflated. */ - if (!FAST_WORKING_DIRECTORY && !want_file && has_sha1_pack(sha1, NULL)) + if (!FAST_WORKING_DIRECTORY && !want_file && has_sha1_pack(sha1, NULL, 0)) return 0; len = strlen(name); diff --git a/git-repack.sh b/git-repack.sh index ddfa8b4..774286e 100755 --- a/git-repack.sh +++ b/git-repack.sh @@ -3,17 +3,18 @@ # Copyright (c) 2005 Linus Torvalds # -USAGE='[-a] [-d] [-f] [-l] [-n] [-q] [--window=N] [--depth=N]' +USAGE='[-a] [-A] [-d] [-f] [-l] [-n] [-q] [--window=N] [--depth=N]' SUBDIRECTORY_OK='Yes' . git-sh-setup -no_update_info= all_into_one= remove_redundant= +no_update_info= all_into_one= remove_redundant= nuke_alternates= local= quiet= no_reuse_delta= extra= while case "$#" in 0) break ;; esac do case "$1" in -n) no_update_info=t ;; -a) all_into_one=t ;; + -A) nuke_alternates=t ;; -d) remove_redundant=t ;; -q) quiet=-q ;; -f) no_reuse_delta=--no-reuse-delta ;; @@ -28,8 +29,7 @@ done # Later we will default repack.UseDeltaBaseOffset to true default_dbo=false -case "`git config --bool repack.usedeltabaseoffset || - echo $default_dbo`" in +case "`git config --bool repack.usedeltabaseoffset || echo $default_dbo`" in true) extra="$extra --delta-base-offset" ;; esac @@ -40,11 +40,14 @@ rm -f "$PACKTMP"-* trap 'rm -f "$PACKTMP"-*' 0 1 2 3 15 # There will be more repacking strategies to come... -case ",$all_into_one," in -,,) +case ",$all_into_one,$nuke_alternates," in +,,,) args='--unpacked --incremental' ;; -,t,) +,,t,) + args='--incremental --ignore-alternate-pack' + ;; +,t,*) if [ -d "$PACKDIR" ]; then for e in `cd "$PACKDIR" && find . -type f -name '*.pack' \ | sed -e 's/^\.\///' -e 's/\.pack$//'` @@ -57,7 +60,12 @@ case ",$all_into_one," in fi done fi - [ -z "$args" ] && args='--unpacked --incremental' + case "$nuke_alternates" in + '') + [ -z "$args" ] && args='--unpacked --incremental' ;; + t) + args="$args --ignore-alternate-pack" ;; + esac ;; esac @@ -113,6 +121,12 @@ then git-prune-packed $quiet fi +case "$nuke_alternates" in +t) + rm -f "$GIT_OBJECT_DIRECTORY/info/alternates" + ;; +esac + case "$no_update_info" in t) : ;; *) git-update-server-info ;; diff --git a/revision.c b/revision.c index 5b1794b..b9e33c3 100644 --- a/revision.c +++ b/revision.c @@ -949,6 +949,10 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, const ch add_ignore_packed(revs, arg+11); continue; } + if (!strcmp(arg, "--ignore-alternate-pack")) { + revs->ignore_alternate_pack = 1; + continue; + } if (!strcmp(arg, "-r")) { revs->diff = 1; revs->diffopt.recursive = 1; @@ -1240,9 +1244,11 @@ static struct commit *get_revision_1(struct rev_info *revs) if (commit->object.flags & SHOWN) continue; - if (revs->unpacked && has_sha1_pack(commit->object.sha1, - revs->ignore_packed)) - continue; + if ((revs->unpacked || revs->ignore_alternate_pack) && + has_sha1_pack(commit->object.sha1, + revs->ignore_packed, + revs->ignore_alternate_pack)) + continue; /* We want to show boundary commits only when their * children are shown. When path-limiter is in effect, diff --git a/revision.h b/revision.h index 5fec184..ca6affd 100644 --- a/revision.h +++ b/revision.h @@ -40,6 +40,7 @@ struct rev_info { edge_hint:1, limited:1, unpacked:1, /* see also ignore_packed below */ + ignore_alternate_pack:1, boundary:1, left_right:1, parents:1, diff --git a/sha1_file.c b/sha1_file.c index b83f59f..2d8c0f2 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -1403,7 +1403,7 @@ static int matches_pack_name(struct packed_git *p, const char *ig) return 1; } -static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, const char **ignore_packed) +static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, const char **ignore_packed, int ignore_alternates) { struct packed_git *p; unsigned long offset; @@ -1419,6 +1419,8 @@ static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, cons if (*ig) continue; } + if (ignore_alternates && !p->pack_local) + continue; offset = find_pack_entry_one(sha1, p); if (offset) { /* @@ -1485,9 +1487,9 @@ int sha1_object_info(const unsigned char *sha1, char *type, unsigned long *sizep { struct pack_entry e; - if (!find_pack_entry(sha1, &e, NULL)) { + if (!find_pack_entry(sha1, &e, NULL, 0)) { reprepare_packed_git(); - if (!find_pack_entry(sha1, &e, NULL)) + if (!find_pack_entry(sha1, &e, NULL, 0)) return sha1_loose_object_info(sha1, type, sizep); } return packed_object_info(e.p, e.offset, type, sizep); @@ -1497,7 +1499,7 @@ static void *read_packed_sha1(const unsigned char *sha1, char *type, unsigned lo { struct pack_entry e; - if (!find_pack_entry(sha1, &e, NULL)) + if (!find_pack_entry(sha1, &e, NULL, 0)) return NULL; else return unpack_entry(e.p, e.offset, type, size); @@ -2004,10 +2006,10 @@ int has_pack_file(const unsigned char *sha1) return 1; } -int has_sha1_pack(const unsigned char *sha1, const char **ignore_packed) +int has_sha1_pack(const unsigned char *sha1, const char **ignore_packed, int ignore_alternates) { struct pack_entry e; - return find_pack_entry(sha1, &e, ignore_packed); + return find_pack_entry(sha1, &e, ignore_packed, ignore_alternates); } int has_sha1_file(const unsigned char *sha1) @@ -2015,7 +2017,7 @@ int has_sha1_file(const unsigned char *sha1) struct stat st; struct pack_entry e; - if (find_pack_entry(sha1, &e, NULL)) + if (find_pack_entry(sha1, &e, NULL, 0)) return 1; return find_sha1_file(sha1, &st) ? 1 : 0; } ``` ## Junio C Hamano, 2007-02-15 09:37 Subject: [PATCH +1] Dissociating a repository from its alternates Message-ID: <7vsld7g3sz.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vsld7g3sz.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <7vabzfhn9q.fsf@assigned-by-dhcp.cox.net> ``` On top of the previous patch. This is needed to: - reject "repack -A -l", which does not make any sense; you want to repack the objects you borrow from the alternates. - make "repack -A" without "-a" to work. These operations corrupted the repository with the previous patch alone. Signed-off-by: Junio C Hamano --- git-repack.sh | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/git-repack.sh b/git-repack.sh index 774286e..66b5039 100755 --- a/git-repack.sh +++ b/git-repack.sh @@ -25,6 +25,10 @@ do esac shift done +if test "$nuke_alternates,$local" = "t,--local" +then + die "-A and -l are incompatible" +fi # Later we will default repack.UseDeltaBaseOffset to true default_dbo=false @@ -45,7 +49,7 @@ case ",$all_into_one,$nuke_alternates," in args='--unpacked --incremental' ;; ,,t,) - args='--incremental --ignore-alternate-pack' + args='--ignore-alternate-pack' ;; ,t,*) if [ -d "$PACKDIR" ]; then ``` ## Johannes Schindelin, 2007-02-15 15:39 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.63.0702151638130.22628%40wbgn013.biozentrum.uni-wuerzburg.de In-Reply-To: <7vabzfhn9q.fsf@assigned-by-dhcp.cox.net> ``` Hi, On Wed, 14 Feb 2007, Junio C Hamano wrote: > People often start out with "clone -l -s" (or --reference) and > later wish to dissociate the repository from its alternates. Why not make a really dumb script which just _copies_ (or maybe optionally hard link) the objects from the alternate repo, and then kills the alternatives file? That should be easier. You always can repack after that. Ciao, Dscho ``` ## James Cloos, 2007-02-16 19:11 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: URL: https://gitlist.dev/e/m3wt2h3ojt.fsf%40lugabout.jhcloos.org In-Reply-To: <7vabzfhn9q.fsf@assigned-by-dhcp.cox.net> ``` [ARGH; hit r rather than f; cc-ing the list. -JimC] Wierd. I was /just/ about to inquire on what the best method was for disassociating. And then I read this thread. Effective mind reading! :) WRT Johannes's followup, I presume it will re-use the alternate packs, such that the computation is minimal? -JimC -- James Cloos OpenPGP: 1024D/ED7DAEA6 ``` ## Junio C Hamano, 2007-02-16 20:13 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: <7v8xex50aa.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7v8xex50aa.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` Johannes Schindelin writes: > On Wed, 14 Feb 2007, Junio C Hamano wrote: > >> People often start out with "clone -l -s" (or --reference) and >> later wish to dissociate the repository from its alternates. > > Why not make a really dumb script which just _copies_ (or maybe > optionally hard link) the objects from the alternate repo, and then kills > the alternatives file? That should be easier. You always can repack after > that. I think that is much safer. ``` ## Michael Hendricks, 2007-10-22 18:04 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: <20071022180424.GA17429@ginosko.local> URL: https://gitlist.dev/e/20071022180424.GA17429%40ginosko.local In-Reply-To: <7v8xex50aa.fsf@assigned-by-dhcp.cox.net> ``` On Fri, Feb 16, 2007 at 12:13:01PM -0800, Junio C Hamano wrote: > Johannes Schindelin writes: > > > On Wed, 14 Feb 2007, Junio C Hamano wrote: > > > >> People often start out with "clone -l -s" (or --reference) and > >> later wish to dissociate the repository from its alternates. > > > > Why not make a really dumb script which just _copies_ (or maybe > > optionally hard link) the objects from the alternate repo, and then kills > > the alternatives file? That should be easier. You always can repack after > > that. > > I think that is much safer. Was such a script ever incorporated into Git? -- Michael ``` ## Johannes Schindelin, 2007-10-22 19:05 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0710221957360.25221%40racer.site In-Reply-To: <20071022180424.GA17429@ginosko.local> ``` Hi, On Mon, 22 Oct 2007, Michael Hendricks wrote: > On Fri, Feb 16, 2007 at 12:13:01PM -0800, Junio C Hamano wrote: > > Johannes Schindelin writes: > > > > > On Wed, 14 Feb 2007, Junio C Hamano wrote: > > > > > >> People often start out with "clone -l -s" (or --reference) and > > >> later wish to dissociate the repository from its alternates. > > > > > > Why not make a really dumb script which just _copies_ (or maybe > > > optionally hard link) the objects from the alternate repo, and then kills > > > the alternatives file? That should be easier. You always can repack after > > > that. > > > > I think that is much safer. > > Was such a script ever incorporated into Git? Not that I know of, but "git repack -a && rm .git/objects/info/alternates" should do what you want. You can even make a script of it, add some documentation and a test case, and earn git fame by posting a patch ;-) Ciao, Dscho ``` ## Michael Hendricks, 2007-10-22 23:30 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: <20071022233050.GA18430@ginosko.ndrix.org> URL: https://gitlist.dev/e/20071022233050.GA18430%40ginosko.ndrix.org In-Reply-To: ``` On Mon, Oct 22, 2007 at 08:05:58PM +0100, Johannes Schindelin wrote: > On Mon, 22 Oct 2007, Michael Hendricks wrote: > > Was such a script ever incorporated into Git? > > Not that I know of, but "git repack -a && rm > .git/objects/info/alternates" should do what you want. You can even > make a script of it, add some documentation and a test case, and earn > git fame by posting a patch ;-) With 1.5.3.4, it doesn't appear to work: $ git clone -s git git2 Initialized empty Git repository in /Users/michael/src/git2/.git/ $ cd git2 $ git repack -a && rm .git/objects/info/alternates Generating pack... Done counting 0 objects. Nothing new to pack. $ git status # On branch master fatal: bad object HEAD -- Michael ``` ## Johannes Schindelin, 2007-10-23 00:13 Subject: Re: [PATCH] Dissociating a repository from its alternates Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0710230112131.25221%40racer.site In-Reply-To: <20071022233050.GA18430@ginosko.ndrix.org> ``` Hi, On Mon, 22 Oct 2007, Michael Hendricks wrote: > On Mon, Oct 22, 2007 at 08:05:58PM +0100, Johannes Schindelin wrote: > > On Mon, 22 Oct 2007, Michael Hendricks wrote: > > > Was such a script ever incorporated into Git? > > > > Not that I know of, but "git repack -a && rm > > .git/objects/info/alternates" should do what you want. You can even > > make a script of it, add some documentation and a test case, and earn > > git fame by posting a patch ;-) > > With 1.5.3.4, it doesn't appear to work: > > $ git clone -s git git2 > Initialized empty Git repository in /Users/michael/src/git2/.git/ > > $ cd git2 > > $ git repack -a && rm .git/objects/info/alternates > Generating pack... > Done counting 0 objects. > Nothing new to pack. > > $ git status > # On branch master > fatal: bad object HEAD Indeed. Seems that somewhere along the line, repack learnt to imply "-l" by "-a" or something. Try this instead of "git repack -a": git rev-parse --all | git pack-objects --revs .git/objects/pack/pack (And make sure you read the relevant parts of the documentation to understand what you're doing ;-) Hth, Dscho ```