{"thread":{"id":"8217","subject":"[PATCH 1/2] pack-objects: pass fullname down to add_object_entry()","startedAt":"2007-05-19T07:48:10Z","lastAt":"2007-05-22T16:04:59Z","messageCount":6,"participants":["Junio C Hamano","Dana How","Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"42601","messageId":"11795608912129-git-send-email-junkio@cox.net","threadId":"8217","inReplyTo":null,"subject":"[PATCH 1/2] pack-objects: pass fullname down to add_object_entry()","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-05-19T07:48:10Z","receivedAt":"2007-05-19T07:48:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Instead of giving a hash for grouping, pass fullname to add_object_entry().\nI want to add \"do not try deltifying this object\" bit to object_entry based on\nthe settings in .gitattributes, and hashing the name down too early would\ninterfere with that plan.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n builtin-pack-objects.c |   27 ++++++++++++++-------------\n 1 files changed, 14 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 5fa9813..12d9685 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -721,6 +721,9 @@ static unsigned name_hash(const char *name)\n \tunsigned char c;\n \tunsigned hash = 0;\n \n+\tif (!name)\n+\t\treturn 0;\n+\n \t/*\n \t * This effectively just creates a sortable number from the\n \t * last sixteen non-whitespace characters. Last characters\n@@ -735,12 +738,13 @@ static unsigned name_hash(const char *name)\n }\n \n static int add_object_entry(const unsigned char *sha1, enum object_type type,\n-\t\t\t    unsigned hash, int exclude)\n+\t\t\t    const char *name, int exclude)\n {\n \tstruct object_entry *entry;\n \tstruct packed_git *p, *found_pack = NULL;\n \toff_t found_offset = 0;\n \tint ix;\n+\tunsigned hash = name_hash(name);\n \n \tix = nr_objects ? locate_object_entry_hash(sha1) : -1;\n \tif (ix >= 0) {\n@@ -929,10 +933,9 @@ static void add_pbase_object(struct tree_desc *tree,\n \t\tif (cmp < 0)\n \t\t\treturn;\n \t\tif (name[cmplen] != '/') {\n-\t\t\tunsigned hash = name_hash(fullname);\n \t\t\tadd_object_entry(entry.sha1,\n \t\t\t\t\t S_ISDIR(entry.mode) ? OBJ_TREE : OBJ_BLOB,\n-\t\t\t\t\t hash, 1);\n+\t\t\t\t\t fullname, 1);\n \t\t\treturn;\n \t\t}\n \t\tif (S_ISDIR(entry.mode)) {\n@@ -992,10 +995,11 @@ static int check_pbase_path(unsigned hash)\n \treturn 0;\n }\n \n-static void add_preferred_base_object(const char *name, unsigned hash)\n+static void add_preferred_base_object(const char *name)\n {\n \tstruct pbase_tree *it;\n \tint cmplen;\n+\tunsigned hash = name_hash(name);\n \n \tif (!num_preferred_base || check_pbase_path(hash))\n \t\treturn;\n@@ -1003,7 +1007,7 @@ static void add_preferred_base_object(const char *name, unsigned hash)\n \tcmplen = name_cmp_len(name);\n \tfor (it = pbase_tree; it; it = it->next) {\n \t\tif (cmplen == 0) {\n-\t\t\tadd_object_entry(it->pcache.sha1, OBJ_TREE, 0, 1);\n+\t\t\tadd_object_entry(it->pcache.sha1, OBJ_TREE, NULL, 1);\n \t\t}\n \t\telse {\n \t\t\tstruct tree_desc tree;\n@@ -1434,7 +1438,6 @@ static void read_object_list_from_stdin(void)\n {\n \tchar line[40 + 1 + PATH_MAX + 2];\n \tunsigned char sha1[20];\n-\tunsigned hash;\n \n \tfor (;;) {\n \t\tif (!fgets(line, sizeof(line), stdin)) {\n@@ -1457,22 +1460,20 @@ static void read_object_list_from_stdin(void)\n \t\tif (get_sha1_hex(line, sha1))\n \t\t\tdie(\"expected sha1, got garbage:\\n %s\", line);\n \n-\t\thash = name_hash(line+41);\n-\t\tadd_preferred_base_object(line+41, hash);\n-\t\tadd_object_entry(sha1, 0, hash, 0);\n+\t\tadd_preferred_base_object(line+41);\n+\t\tadd_object_entry(sha1, 0, line+41, 0);\n \t}\n }\n \n static void show_commit(struct commit *commit)\n {\n-\tadd_object_entry(commit->object.sha1, OBJ_COMMIT, 0, 0);\n+\tadd_object_entry(commit->object.sha1, OBJ_COMMIT, NULL, 0);\n }\n \n static void show_object(struct object_array_entry *p)\n {\n-\tunsigned hash = name_hash(p->name);\n-\tadd_preferred_base_object(p->name, hash);\n-\tadd_object_entry(p->item->sha1, p->item->type, hash, 0);\n+\tadd_preferred_base_object(p->name);\n+\tadd_object_entry(p->item->sha1, p->item->type, p->name, 0);\n }\n \n static void show_edge(struct commit *commit)\n-- \n1.5.2.rc3.87.g404f\n"},{"id":"42602","messageId":"11795608922961-git-send-email-junkio@cox.net","threadId":"8217","inReplyTo":"11795608912129-git-send-email-junkio@cox.net","subject":"[PATCH 2/2] Teach \"delta\" attribute to pack-objects.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-05-19T07:48:11Z","receivedAt":"2007-05-19T07:48:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This teaches pack-objects to use .gitattributes mechanism so\nthat the user can specify certain blobs are not worth spending\nCPU cycles to attempt deltification.\n\nThe name of the attrbute is \"delta\", and when it is set to\nfalse, like this:\n\n\t== .gitattributes ==\n\t*.jpg\t-delta\n\nthey are always stored in the plain-compressed base object\nrepresentation.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n builtin-pack-objects.c |   35 ++++++++++++++++++++++++++++++++++-\n 1 files changed, 34 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 12d9685..651011c 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"attr.h\"\n #include \"object.h\"\n #include \"blob.h\"\n #include \"commit.h\"\n@@ -40,9 +41,10 @@ struct object_entry {\n \tenum object_type in_pack_type;\t/* could be delta */\n \tunsigned char in_pack_header_size;\n \tunsigned char preferred_base; /* we do not pack this, but is available\n-\t\t\t\t       * to be used as the base objectto delta\n+\t\t\t\t       * to be used as the base object to delta\n \t\t\t\t       * objects against.\n \t\t\t\t       */\n+\tunsigned char no_try_delta;\n };\n \n /*\n@@ -737,6 +739,28 @@ static unsigned name_hash(const char *name)\n \treturn hash;\n }\n \n+static void setup_delta_attr_check(struct git_attr_check *check)\n+{\n+\tstatic struct git_attr *attr_delta;\n+\n+\tif (!attr_delta)\n+\t\tattr_delta = git_attr(\"delta\", 5);\n+\n+\tcheck[0].attr = attr_delta;\n+}\n+\n+static int no_try_delta(const char *path)\n+{\n+\tstruct git_attr_check check[1];\n+\n+\tsetup_delta_attr_check(check);\n+\tif (git_checkattr(path, ARRAY_SIZE(check), check))\n+\t\treturn 0;\n+\tif (ATTR_FALSE(check->value))\n+\t\treturn 1;\n+\treturn 0;\n+}\n+\n static int add_object_entry(const unsigned char *sha1, enum object_type type,\n \t\t\t    const char *name, int exclude)\n {\n@@ -801,6 +825,9 @@ static int add_object_entry(const unsigned char *sha1, enum object_type type,\n \tif (progress)\n \t\tdisplay_progress(&progress_state, nr_objects);\n \n+\tif (name && no_try_delta(name))\n+\t\tentry->no_try_delta = 1;\n+\n \treturn 1;\n }\n \n@@ -1349,6 +1376,10 @@ static void find_deltas(struct object_entry **list, int window, int depth)\n \n \t\tif (entry->size < 50)\n \t\t\tcontinue;\n+\n+\t\tif (entry->no_try_delta)\n+\t\t\tcontinue;\n+\n \t\tfree_delta_index(n->index);\n \t\tn->index = NULL;\n \t\tfree(n->data);\n@@ -1376,6 +1407,8 @@ static void find_deltas(struct object_entry **list, int window, int depth)\n \t\t\tm = array + other_idx;\n \t\t\tif (!m->entry)\n \t\t\t\tbreak;\n+\t\t\tif (m->entry->no_try_delta)\n+\t\t\t\tcontinue;\n \t\t\tif (try_delta(n, m, max_depth) < 0)\n \t\t\t\tbreak;\n \t\t}\n-- \n1.5.2.rc3.87.g404f\n"},{"id":"42627","messageId":"56b7f5510705190910h7e5ed03bnbc5a1b9a1369fc9f@mail.gmail.com","threadId":"8217","inReplyTo":"11795608922961-git-send-email-junkio@cox.net","subject":"Re: [PATCH 2/2] Teach \"delta\" attribute to pack-objects.","fromName":"Dana How","fromEmail":"danahow@gmail.com","sentAt":"2007-05-19T16:10:35Z","receivedAt":"2007-05-19T16:10:35Z","isPatch":true,"sender":{"key":"danahow@gmail.com","avatar":null},"body":"On 5/19/07, Junio C Hamano <junkio@cox.net> wrote:\n> This teaches pack-objects to use .gitattributes mechanism so\n> that the user can specify certain blobs are not worth spending\n> CPU cycles to attempt deltification.\n>\n> The name of the attrbute is \"delta\", and when it is set to\n> false, like this:\n>\n>         == .gitattributes ==\n>         *.jpg   -delta\n>\n> they are always stored in the plain-compressed base object\n> representation.\n\nAnd we could also have an attribute \"repack\" :\n    == .gitattributes ==\n    *.wmv    -repack\nwhich would result in *.wmv files [enormous] not being packed.\nSince add_object_entry() now gets the name and\ncan reject objects (e.g. currently on their current packing status),\na call to \"no_pack\" which mirrors your new \"no_try_delta\"\ncould be inserted there.  But such an attribute should be\nignored when --stdout is in effect -- it only affects on-disk\nrepacking, not packing for transfers, which is why it's named \"repack\".\n\n-- \nDana L. How  danahow@gmail.com  +1 650 804 5991 cell\n"},{"id":"42644","messageId":"7v4pm8wpgc.fsf@assigned-by-dhcp.cox.net","threadId":"8217","inReplyTo":"56b7f5510705190910h7e5ed03bnbc5a1b9a1369fc9f@mail.gmail.com","subject":"Re: [PATCH 2/2] Teach \"delta\" attribute to pack-objects.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-05-19T20:03:47Z","receivedAt":"2007-05-19T20:03:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Dana How\" <danahow@gmail.com> writes:\n\n> ...  But such an attribute should be\n> ignored when --stdout is in effect -- it only affects on-disk\n> repacking, not packing for transfers, which is why it's named \"repack\".\n\nYes -- if we want to have an option to keep objects selectively\nleft out of packs in loose format, you would need 'repack' which\nacts differently between the server-feeding-client case vs\npacking-repository case.\n\nWhich is a bigger change that I did not want to show in the\nquick-and-clean patch, but I would agree we would want both.\n"},{"id":"42659","messageId":"56b7f5510705191656y1a5cdb1cn1b8b1082519f4bc3@mail.gmail.com","threadId":"8217","inReplyTo":"7v4pm8wpgc.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH 2/2] Teach \"delta\" attribute to pack-objects.","fromName":"Dana How","fromEmail":"danahow@gmail.com","sentAt":"2007-05-19T23:56:23Z","receivedAt":"2007-05-19T23:56:23Z","isPatch":true,"sender":{"key":"danahow@gmail.com","avatar":null},"body":"On 5/19/07, Junio C Hamano <junkio@cox.net> wrote:\n> \"Dana How\" <danahow@gmail.com> writes:\n> > ...  But such an attribute should be\n> > ignored when --stdout is in effect -- it only affects on-disk\n> > repacking, not packing for transfers, which is why it's named \"repack\".\n>\n> Yes -- if we want to have an option to keep objects selectively\n> left out of packs in loose format, you would need 'repack' which\n> acts differently between the server-feeding-client case vs\n> packing-repository case.\n>\n> Which is a bigger change that I did not want to show in the\n> quick-and-clean patch, but I would agree we would want both.\n\nOK -- I'll put this on the back-burner with the so-called\n\"ent:relative\" patch and I'll revisit this once your \"delta\"\nattribute shows up in next or master.\n\nAt the moment I'm experimenting on a git repository with\na 4.5GB checkout,  and 18 months of history in 4K commits\ncomprising 100GB (uncompressed) of blobs stored in\n7 packfiles of 2GB or less.  Hopefully I'll be able to say\nmore about tweaking packing shortly.\n\nThanks,\n-- \nDana L. How  danahow@gmail.com  +1 650 804 5991 cell\n"},{"id":"42984","messageId":"alpine.LFD.0.99.0705221200010.3366@xanadu.home","threadId":"8217","inReplyTo":"11795608922961-git-send-email-junkio@cox.net","subject":"Re: [PATCH 2/2] Teach \"delta\" attribute to pack-objects.","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-05-22T16:04:59Z","receivedAt":"2007-05-22T16:04:59Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 19 May 2007, Junio C Hamano wrote:\n\n> This teaches pack-objects to use .gitattributes mechanism so\n> that the user can specify certain blobs are not worth spending\n> CPU cycles to attempt deltification.\n[...]\n> @@ -1349,6 +1376,10 @@ static void find_deltas(struct object_entry **list, int window, int depth)\n>  \n>  \t\tif (entry->size < 50)\n>  \t\t\tcontinue;\n> +\n> +\t\tif (entry->no_try_delta)\n> +\t\t\tcontinue;\n> +\n>  \t\tfree_delta_index(n->index);\n>  \t\tn->index = NULL;\n>  \t\tfree(n->data);\n> @@ -1376,6 +1407,8 @@ static void find_deltas(struct object_entry **list, int window, int depth)\n>  \t\t\tm = array + other_idx;\n>  \t\t\tif (!m->entry)\n>  \t\t\t\tbreak;\n> +\t\t\tif (m->entry->no_try_delta)\n> +\t\t\t\tcontinue;\n>  \t\t\tif (try_delta(n, m, max_depth) < 0)\n>  \t\t\t\tbreak;\n>  \t\t}\n\nThis last hunk is unnecessary.  Because of the other hunk above, the \nno_try_delta objects will never get into the m array.\n\nWell done otherwise.  This attribute thing is really nice.\n\n\nNicolas\n"}]}