git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git gc & deleted branches

From
Brandon Casey <drafnel@gmail.com>
Date
May 10, 2008, 03:32 UTC
Message-ID
<ee63ef30805092032h18fe7ff7mda4233b08ae431ea@mail.gmail.com>
In-Reply-To
<7v1w4bb291.fsf@gitster.siamese.dyndns.org>

I didn't read your email until now. It may sound strange, but I can't read my personal email from work.

Also, looks like I inadvertantly took this message off-list.
On Fri, May 9, 2008 at 4:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 8 quoted lines
> "Brandon Casey" <drafnel@gmail.com> writes:
>
>> Is this invocation of pack-objects that you commented on necessary
>> though? This is the section that is replacing an existing pack file
>> with a newly created pack file of the same name...
>
> Yuck, I failed to see it in the context.  You are right.  This won't do
> anything I suspect.

Now I'm confused. I hope you realized I said "pack-objects" when I meant "unpack-objects".

It was only the first invocation of unpack-objects that I thought may be unnecessary. This is because pack-objects (yes, pack) created a new pack with the same name as an existing pack which means (if I'm thinking about things correctly) that none of the objects inside the pack are unreachable. So trying to unpack-objects is a waste of time.

There is a second invocation of unpack-objects which is run on packs which will be deleted. This one should unpack any unreferenced objects from each pack.

-brandon
The remainder of your email is below since the list has never seen it.
Show 147 quoted lines
> Perhaps we would want a new option --eject that causes pack-object write
> out unreachable packed objects in the loose format?  This would require a
> minor surgery to write_sha1_file() so that it won't pay attention to the
> return value of has_sha1_file(sha1), perhaps like this.
>
> This is a lunch-time hack and needs more serious work, such as fixing
> horrible "write_sha1_file_1" to something more sensible.
>
> Also it should add --eject as an independent and incompatible option
> instead of dropping --keep-unpacked support.
>
> ---
>
>  builtin-pack-objects.c |   56 ++++++++++++-----------------------------------
>  sha1_file.c            |    9 ++++++-
>  2 files changed, 22 insertions(+), 43 deletions(-)
>
> diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> index 777f272..8240a4b 100644
> --- a/builtin-pack-objects.c
> +++ b/builtin-pack-objects.c
> @@ -1836,38 +1836,25 @@ struct in_pack {
>        struct in_pack_object *array;
>  };
>
> -static void mark_in_pack_object(struct object *object, struct packed_git *p, struct in_pack *in_pack)
> +static void eject_object(struct object *obj)
>  {
> -       in_pack->array[in_pack->nr].offset = find_pack_entry_one(object->sha1, p);
> -       in_pack->array[in_pack->nr].object = object;
> -       in_pack->nr++;
> -}
> -
> -/*
> - * Compare the objects in the offset order, in order to emulate the
> - * "git-rev-list --objects" output that produced the pack originally.
> - */
> -static int ofscmp(const void *a_, const void *b_)
> -{
> -       struct in_pack_object *a = (struct in_pack_object *)a_;
> -       struct in_pack_object *b = (struct in_pack_object *)b_;
> +       enum object_type type;
> +       unsigned long size;
> +       void *data;
> +       unsigned char sha1[20];
>
> -       if (a->offset < b->offset)
> -               return -1;
> -       else if (a->offset > b->offset)
> -               return 1;
> -       else
> -               return hashcmp(a->object->sha1, b->object->sha1);
> +       data = read_sha1_file(obj->sha1, &type, &size);
> +       assert(data && type == obj->type);
> +       if (write_sha1_file_1(data, size, typename(type), sha1, 1))
> +               die("cannot eject packed object %s", sha1_to_hex(obj->sha1));
> +       assert(!memcmp(sha1, obj->sha1));
>  }
>
> -static void add_objects_in_unpacked_packs(struct rev_info *revs)
> +static void eject_objects_in_unpacked_packs(struct rev_info *revs)
>  {
>        struct packed_git *p;
> -       struct in_pack in_pack;
>        uint32_t i;
>
> -       memset(&in_pack, 0, sizeof(in_pack));
> -
>        for (p = packed_git; p; p = p->next) {
>                const unsigned char *sha1;
>                struct object *o;
> @@ -1881,28 +1868,15 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
>                if (open_pack_index(p))
>                        die("cannot open pack index");
>
> -               ALLOC_GROW(in_pack.array,
> -                          in_pack.nr + p->num_objects,
> -                          in_pack.alloc);
> -
>                for (i = 0; i < p->num_objects; i++) {
>                        sha1 = nth_packed_object_sha1(p, i);
>                        o = lookup_unknown_object(sha1);
> -                       if (!(o->flags & OBJECT_ADDED))
> -                               mark_in_pack_object(o, p, &in_pack);
> +                       if (o->flags & OBJECT_ADDED)
> +                               continue;
>                        o->flags |= OBJECT_ADDED;
> +                       eject_object(o);
>                }
>        }
> -
> -       if (in_pack.nr) {
> -               qsort(in_pack.array, in_pack.nr, sizeof(in_pack.array[0]),
> -                     ofscmp);
> -               for (i = 0; i < in_pack.nr; i++) {
> -                       struct object *o = in_pack.array[i].object;
> -                       add_object_entry(o->sha1, o->type, "", 0);
> -               }
> -       }
> -       free(in_pack.array);
>  }
>
>  static void get_object_list(int ac, const char **av)
> @@ -1938,7 +1912,7 @@ static void get_object_list(int ac, const char **av)
>        traverse_commit_list(&revs, show_commit, show_object);
>
>        if (keep_unreachable)
> -               add_objects_in_unpacked_packs(&revs);
> +               eject_objects_in_unpacked_packs(&revs);
>  }
>
>  static int adjust_perm(const char *path, mode_t mode)
> diff --git a/sha1_file.c b/sha1_file.c
> index 3516777..ce3a661 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -2102,7 +2102,7 @@ int hash_sha1_file(const void *buf, unsigned long len, const char *type,
>        return 0;
>  }
>
> -int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *returnsha1)
> +int write_sha1_file_1(void *buf, unsigned long len, const char *type, unsigned char *returnsha1, int refresh)
>  {
>        int size, ret;
>        unsigned char *compressed;
> @@ -2120,7 +2120,7 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha
>        filename = sha1_file_name(sha1);
>        if (returnsha1)
>                hashcpy(returnsha1, sha1);
> -       if (has_sha1_file(sha1))
> +       if (!refresh && has_sha1_file(sha1))
>                return 0;
>        fd = open(filename, O_RDONLY);
>        if (fd >= 0) {
> @@ -2185,6 +2185,11 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha
>        return move_temp_to_file(tmpfile, filename);
>  }
>
> +int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *returnsha1)
> +{
> +       return write_sha1_file_1(buf, len, type, returnsha1, 0);
> +}
> +
>  /*
>  * We need to unpack and recompress the object for writing
>  * it out to a different file.
>
Previous: Jeff KingNext: Brandon Casey
Message 49 of 50 in “git gc & deleted branches”
  1. Guido OstkampMay 8, 2008
  2. Jeff KingMay 8, 2008
  3. Guido OstkampMay 8, 2008
  4. Brandon CaseyMay 8, 2008
  5. Guido OstkampMay 8, 2008
  6. Jeff KingMay 8, 2008
  7. Nicolas PitreMay 8, 2008
  8. Jeff KingMay 8, 2008
  9. Brandon CaseyMay 8, 2008
  10. Jeff KingMay 8, 2008
  11. Brandon CaseyMay 8, 2008
  12. Jeff KingMay 8, 2008
  13. Brandon CaseyMay 8, 2008
  14. Jeff KingMay 8, 2008
  15. Brandon CaseyMay 9, 2008
  16. Junio C HamanoMay 9, 2008
  17. 0/3 leave unreferenced objects unpackeddrafnel@gmail.com, May 10, 2008
  18. 1/3 repack: modify behavior of -A option to leave unreferenced objects unpackeddrafnel@gmail.com, May 10, 2008
  19. Jeff KingMay 10, 2008
  20. Nicolas PitreMay 11, 2008
  21. Junio C HamanoMay 11, 2008
  22. Brandon CaseyMay 11, 2008
  23. Brandon CaseyMay 11, 2008
  24. 2/3 git-gc: always use -A when manually repackingdrafnel@gmail.com, May 10, 2008
  25. 3/3 builtin-gc.c: deprecate --prune, it now really has no effectdrafnel@gmail.com, May 10, 2008
  26. Jeff KingMay 9, 2008
  27. Geert BoschMay 9, 2008
  28. Brandon CaseyMay 9, 2008
  29. Jeff KingMay 9, 2008
  30. Brandon CaseyMay 9, 2008
  31. Nicolas PitreMay 9, 2008
  32. Brandon CaseyMay 9, 2008
  33. Junio C HamanoMay 9, 2008
  34. Updating documentation to match Brandon Casey's proposed git-repack patch.Chris Frey, May 9, 2008
  35. Jeremy Maitin-ShepardMay 10, 2008
  36. Shawn O. PearceMay 10, 2008
  37. Jeremy Maitin-ShepardMay 10, 2008
  38. Junio C HamanoMay 10, 2008
  39. Jeremy Maitin-ShepardMay 10, 2008
  40. Jeff KingMay 10, 2008
  41. Jeremy Maitin-ShepardMay 10, 2008
  42. Johannes SchindelinMay 10, 2008
  43. Jeremy Maitin-ShepardMay 10, 2008
  44. Johannes SchindelinMay 11, 2008
  45. Junio C HamanoMay 11, 2008
  46. Guido OstkampMay 8, 2008
  47. Jeff KingMay 8, 2008
  48. Jeff KingMay 8, 2008
  49. Brandon CaseyMay 10, 2008
  50. Brandon CaseyMay 10, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.