{"thread":{"id":"17035","subject":"[RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","startedAt":"2009-01-07T07:43:41Z","lastAt":"2009-01-12T09:50:23Z","messageCount":7,"participants":["Christian Couder","Junio C Hamano","Johannes Schindelin","Jakub Narebski"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"99555","messageId":"20090107084341.1554d8cd.chriscool@tuxfamily.org","threadId":"17035","inReplyTo":null,"subject":"[RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Christian Couder","fromEmail":"chriscool@tuxfamily.org","sentAt":"2009-01-07T07:43:41Z","receivedAt":"2009-01-07T07:43:41Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"The code of this mechanism has been copied from the commit graft code.\n\nCurrently this mechanism is only used from the \"parse_commit_buffer\"\nfunction in \"commit.c\". It should probably be used from \"fsck.c\" too.\n\n(For information, grafts are looked up only from \"parse_commit_buffer\"\nfunction in \"commit.c\" and from \"fsck_commit\" in \"fsck.c\".)\n\nIn \"parse_commit_buffer\", the parent sha1s from the original commit\nor from a commit graft that match a ref name in \"refs/replace/\" are\nreplaced by the commit sha1 that has been read in the ref.\n\nThis means that for example \"git show <original commit sha1>\" will\ndisplay information about the original commit. If the mechanism\nhad been called from \"read_sha1_file\" instead of when parents\nare read, then \"git show <original commit sha1>\" would display\ninformation about the commit that replaces the original one.\nThis may be seen as a feature or as a bug depending on the point\nof view.\n\nAnyway this implementation makes sure that the mechanism is\ntriggered only when commit graft could be triggered, so hopefully the\nobject reachability traverser will ignore this mechanism as it\nignores the graft mechanism.\n\nSigned-off-by: Christian Couder <chriscool@tuxfamily.org>\n---\n Makefile         |    1 +\n commit.c         |    7 +++-\n commit.h         |    2 +\n replace_object.c |  102 ++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 110 insertions(+), 2 deletions(-)\n create mode 100644 replace_object.c\n\ndiff --git a/Makefile b/Makefile\nindex aabf013..f355e63 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -471,6 +471,7 @@ LIB_OBJS += read-cache.o\n LIB_OBJS += reflog-walk.o\n LIB_OBJS += refs.o\n LIB_OBJS += remote.o\n+LIB_OBJS += replace_object.o\n LIB_OBJS += rerere.o\n LIB_OBJS += revision.o\n LIB_OBJS += run-command.o\ndiff --git a/commit.c b/commit.c\nindex c99db16..0014174 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -241,6 +241,7 @@ int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)\n \tchar *tail = buffer;\n \tchar *bufptr = buffer;\n \tunsigned char parent[20];\n+\tconst unsigned char *parent_sha1;\n \tstruct commit_list **pptr;\n \tstruct commit_graft *graft;\n \n@@ -268,7 +269,8 @@ int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)\n \t\tbufptr += 48;\n \t\tif (graft)\n \t\t\tcontinue;\n-\t\tnew_parent = lookup_commit(parent);\n+\t\tparent_sha1 = lookup_replace_object(parent);\n+\t\tnew_parent = lookup_commit(parent_sha1);\n \t\tif (new_parent)\n \t\t\tpptr = &commit_list_insert(new_parent, pptr)->next;\n \t}\n@@ -276,7 +278,8 @@ int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)\n \t\tint i;\n \t\tstruct commit *new_parent;\n \t\tfor (i = 0; i < graft->nr_parent; i++) {\n-\t\t\tnew_parent = lookup_commit(graft->parent[i]);\n+\t\t\tparent_sha1 = lookup_replace_object(graft->parent[i]);\n+\t\t\tnew_parent = lookup_commit(parent_sha1);\n \t\t\tif (!new_parent)\n \t\t\t\tcontinue;\n \t\t\tpptr = &commit_list_insert(new_parent, pptr)->next;\ndiff --git a/commit.h b/commit.h\nindex 3a7b06a..37aa763 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -122,6 +122,8 @@ struct commit_graft *read_graft_line(char *buf, int len);\n int register_commit_graft(struct commit_graft *, int);\n struct commit_graft *lookup_commit_graft(const unsigned char *sha1);\n \n+const unsigned char *lookup_replace_object(const unsigned char *sha1);\n+\n extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2, int cleanup);\n extern struct commit_list *get_merge_bases_many(struct commit *one, int n, struct commit **twos, int cleanup);\n extern struct commit_list *get_octopus_merge_bases(struct commit_list *in);\ndiff --git a/replace_object.c b/replace_object.c\nnew file mode 100644\nindex 0000000..b50890d\n--- /dev/null\n+++ b/replace_object.c\n@@ -0,0 +1,102 @@\n+#include \"cache.h\"\n+#include \"refs.h\"\n+\n+static struct replace_object {\n+\tunsigned char sha1[2][20];\n+} **replace_object;\n+\n+static int replace_object_alloc, replace_object_nr;\n+\n+static int replace_object_pos(const unsigned char *sha1)\n+{\n+\tint lo, hi;\n+\tlo = 0;\n+\thi = replace_object_nr;\n+\twhile (lo < hi) {\n+\t\tint mi = (lo + hi) / 2;\n+\t\tstruct replace_object *rep = replace_object[mi];\n+\t\tint cmp = hashcmp(sha1, rep->sha1[0]);\n+\t\tif (!cmp)\n+\t\t\treturn mi;\n+\t\tif (cmp < 0)\n+\t\t\thi = mi;\n+\t\telse\n+\t\t\tlo = mi + 1;\n+\t}\n+\treturn -lo - 1;\n+}\n+\n+static int register_replace_object(struct replace_object *replace,\n+\t\t\t\t   int ignore_dups)\n+{\n+\tint pos = replace_object_pos(replace->sha1[0]);\n+\n+\tif (0 <= pos) {\n+\t\tif (ignore_dups)\n+\t\t\tfree(replace);\n+\t\telse {\n+\t\t\tfree(replace_object[pos]);\n+\t\t\treplace_object[pos] = replace;\n+\t\t}\n+\t\treturn 1;\n+\t}\n+\tpos = -pos - 1;\n+\tif (replace_object_alloc <= ++replace_object_nr) {\n+\t\treplace_object_alloc = alloc_nr(replace_object_alloc);\n+\t\treplace_object = xrealloc(replace_object,\n+\t\t\t\t\t  sizeof(*replace_object) *\n+\t\t\t\t\t  replace_object_alloc);\n+\t}\n+\tif (pos < replace_object_nr)\n+\t\tmemmove(replace_object + pos + 1,\n+\t\t\treplace_object + pos,\n+\t\t\t(replace_object_nr - pos - 1) *\n+\t\t\tsizeof(*replace_object));\n+\treplace_object[pos] = replace;\n+\treturn 0;\n+}\n+\n+static int register_replace_ref(const char *refname,\n+\t\t\t\tconst unsigned char *sha1,\n+\t\t\t\tint flag, void *cb_data)\n+{\n+\t/* Get sha1 from refname */\n+\tconst char *slash = strrchr(refname, '/');\n+\tconst char *hash = slash ? slash + 1 : refname;\n+\tstruct replace_object * repl_obj = xmalloc(sizeof(*repl_obj));\n+\n+\tif (strlen(hash) != 40 || get_sha1_hex(hash, repl_obj->sha1[0])) {\n+\t\tfree(repl_obj);\n+\t\twarning(\"bad replace ref name: %s\", refname);\n+\t}\n+\n+\t/* Copy sha1 from the read ref */\n+\thashcpy(repl_obj->sha1[1], sha1);\n+\n+\t/* Register new object */\n+\tif (register_replace_object(repl_obj, 1))\n+\t\twarning(\"duplicate replace ref: %s\", refname);\n+\n+\treturn 0;\n+}\n+\n+static void prepare_replace_object(void)\n+{\n+\tstatic int replace_object_prepared;\n+\n+\tif (replace_object_prepared)\n+\t\treturn;\n+\n+\tfor_each_replace_ref(register_replace_ref, NULL);\n+\treplace_object_prepared = 1;\n+}\n+\n+const unsigned char *lookup_replace_object(const unsigned char *sha1)\n+{\n+\tint pos;\n+\n+\tprepare_replace_object();\n+\tpos = replace_object_pos(sha1);\n+\n+\treturn (0 <= pos) ? replace_object[pos]->sha1[1] : sha1;\n+}\n-- \n1.6.1.162.g1cd53\n"},{"id":"99560","messageId":"7vmye3a4pt.fsf@gitster.siamese.dyndns.org","threadId":"17035","inReplyTo":"20090107084341.1554d8cd.chriscool@tuxfamily.org","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-01-07T08:41:02Z","receivedAt":"2009-01-07T08:41:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <chriscool@tuxfamily.org> writes:\n\n> In \"parse_commit_buffer\", the parent sha1s from the original commit\n> or from a commit graft that match a ref name in \"refs/replace/\" are\n> replaced by the commit sha1 that has been read in the ref.\n\nI may be reading this wrong, but if you replace the parent of the commit,\nthat won't improve anything over the graft mechanism, which we already\nhave.\n\nWhat I was hoping to see was to give replacing commit back when a commit\nis asked for by normal callers, while not replacing the objects when\nreachability traversal (prune, pack transfer and fsck) wants to read the\ncommit.  That way:\n\n (1) Normal callers will see the rewritten history (which is the same as\n     graft); and\n\n (2) We will kill the design bug of the current graft mechanism that graft\n     information cannot be transferred to the other end.  By using object\n     replacement, you can fetch objects reachable from refs/replace/ at\n     the other end and place them in the same refs/replace/ hierarchy\n     locally, if you choose to use the same altered history, or you can\n     choose not to use the replacement yourself.  The resulting repository\n     is fully connected either way.\n\n (3) We will also kill the design bug of the current graft mechanism that\n     graft information hides repository corruption to fsck.  The problem\n     with this is that if you and then remove the grafts, you will end up\n     with a corrupt repository, because these operations do look at grafts\n     (this is justified only partly because otherwise you will lose\n     objects that are reachable only via grafts, but is broken at the same\n     time because you can lose the true parents by letting graft hide them\n     from a reachable commit).\n\n     By doing fsck and prune always using the true reachability, and using\n     refs in the refs/replace/ hierarchy for protecting objects that are\n     involved in this new way of grafting, you will ensure the integrity\n     of the repository.  Removal of a ref from the refs/replace/ hierarchy\n     won't result in such a corruption we can have today.\n\nAnd that is exactly the reason why I was hoping the hook will be at\nread_sha1_file() that gives you a rewritten object contents when you ask\nfor the original object, not at parse_commit_buffer which does not give\nyou a rewritten object, but makes you follow to a rewritten object when\nparsing a commit (which itself is not replaced if it is the starting\npoint).\n\nDoing replacement at parse_commit_buffer() time is one step too late.  For\nexample, if you have replacement information for the commit that sits at\nthe tip of 'master' branch, I think your \"git log master\" will ignore that\nreplacement information.  Creating one new commit on top of it and saying\n\"git log master\" then will show the new commit, and suddenly starts\nshowing the replacement commit for the one you used to have at the tip of\nthe branch.\n"},{"id":"99574","messageId":"alpine.DEB.1.00.0901071327481.7496@intel-tinevez-2-302","threadId":"17035","inReplyTo":"20090107084341.1554d8cd.chriscool@tuxfamily.org","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-01-07T12:29:27Z","receivedAt":"2009-01-07T12:29:27Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 7 Jan 2009, Christian Couder wrote:\n\n> diff --git a/replace_object.c b/replace_object.c\n> new file mode 100644\n> index 0000000..b50890d\n> --- /dev/null\n> +++ b/replace_object.c\n> @@ -0,0 +1,102 @@\n> +#include \"cache.h\"\n> +#include \"refs.h\"\n> +\n> +static struct replace_object {\n> +\tunsigned char sha1[2][20];\n> +} **replace_object;\n> +\n> +static int replace_object_alloc, replace_object_nr;\n> +\n> +static int replace_object_pos(const unsigned char *sha1)\n> +{\n> +\tint lo, hi;\n> +\tlo = 0;\n> +\thi = replace_object_nr;\n> +\twhile (lo < hi) {\n> [...]\n\nI suspect that this sorted list should be a hashmap.\n\nCiao,\nDscho\n"},{"id":"99726","messageId":"200901081831.22616.chriscool@tuxfamily.org","threadId":"17035","inReplyTo":"7vmye3a4pt.fsf@gitster.siamese.dyndns.org","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Christian Couder","fromEmail":"chriscool@tuxfamily.org","sentAt":"2009-01-08T17:31:22Z","receivedAt":"2009-01-08T17:31:22Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Le mercredi 7 janvier 2009, Junio C Hamano a écrit :\n> Christian Couder <chriscool@tuxfamily.org> writes:\n> > In \"parse_commit_buffer\", the parent sha1s from the original commit\n> > or from a commit graft that match a ref name in \"refs/replace/\" are\n> > replaced by the commit sha1 that has been read in the ref.\n>\n> I may be reading this wrong, but if you replace the parent of the commit,\n> that won't improve anything over the graft mechanism, which we already\n> have.\n>\n> What I was hoping to see was to give replacing commit back when a commit\n> is asked for by normal callers, \n\nYeah, but read_sha1_file is called to read all object files, not just \ncommits. So putting the hook there will:\n\n\t1) add a lookup overhead when reading any object,\n\t2) make it possible to replace any object,\n\nAnd there is also the following problem:\n\n\t3) this function is often called like this:\n\n\tbuffer = read_sha1_file(sha1, &type, &size);\n\tif (!buffer)\n\t\tdie(\"Cannot read %s\", sha1_to_hex(sha1));\n\n\tso in case of error, it will give an error message with a bad sha1\n\tin it because the sha1 of the file that we cannot read is the sha1\n\tin the replace ref not the one passed to read_sha1_file.\n\nTo avoid the above problems, maybe we can try to also improve what \nread_sha1_file does:\n\n1) allow callers to pass a type in the \"type\" argument and only lookup in \nthe replace refs if we say we want a commit, but this makes calling this \nfunction more error prone\n2) when we say we want an object with a given type, check if the object we \nread has this type (and die if not)\n3) die in read_sha1_file when there is an error and we are replacing so that \ncallers don't need to die themself and so that we can always report an \naccurate sha1 in the error message\n\nSomething like this (totally untested):\n\nvoid *read_sha1_file(const unsigned char *sha1, enum object_type *type,\n\t\t     unsigned long *size)\n{\n\tvoid *data;\n\tenum object_type read_type;\n\tint replacing = (type && *type == OBJ_COMMIT);\n\n\t/* only replace commits when we ask for one */\n\tif (replacing)\n\t\tsha1 = lookup_replace_object(sha1);\n\tdata = read_object(sha1, &read_type, size);\n\t/* legacy behavior is to die on corrupted objects */\n\tif (!data && (has_loose_object(sha1) || has_packed_and_bad(sha1)))\n\t\tdie(\"object %s is corrupted\", sha1_to_hex(sha1));\n\tif (type && *type != OBJ_NONE && *type != read_type)\n\t\tdie(\"type of object %s (%d) is different from the type we want (%d)\",\n\t\t    sha1_to_hex(sha1), read_type, *type);\n\tif (!data && replacing)\n\t\tdie(\"Cannot read commit %s\", sha1_to_hex(sha1));\n\n\treturn data;\n}\n\nOr perhaps it is better to leave read_sha1_file as it is and add a new \nfunction like this:\n\nvoid *read_commit(const unsigned char *sha1, unsigned long *size)\n{\n\tvoid *data;\n\tenum object_type type;\n\n\tsha1 = lookup_replace_object(sha1);\n\tdata = read_sha1_file(sha1, &type, size);\n\n\tif (*type != OBJ_COMMIT)\n\t\tdie(\"read_commit: object %s is not a commit\", sha1_to_hex(sha1));\n\tif (!data)\n\t\tdie(\"read_commit: cannot read commit %s\", sha1_to_hex(sha1));\n\n\treturn data;\n}\n\nThe latter seems better to me. But I haven't looked much at the 60 \nread_sha1_file callers yet.\n\n[...]\n\n> Doing replacement at parse_commit_buffer() time is one step too late. \n> For example, if you have replacement information for the commit that sits\n> at the tip of 'master' branch, I think your \"git log master\" will ignore\n> that replacement information.  Creating one new commit on top of it and\n> saying \"git log master\" then will show the new commit, and suddenly\n> starts showing the replacement commit for the one you used to have at the\n> tip of the branch.\n\nYeah, I agree there are drawbacks to do it at parse_commit_buffer time.\n\nThanks,\nChristian.\n"},{"id":"99749","messageId":"7v8wpl4akr.fsf@gitster.siamese.dyndns.org","threadId":"17035","inReplyTo":"200901081831.22616.chriscool@tuxfamily.org","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-01-08T23:55:32Z","receivedAt":"2009-01-08T23:55:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <chriscool@tuxfamily.org> writes:\n\n> Yeah, but read_sha1_file is called to read all object files, not just \n> commits. So putting the hook there will:\n>\n> \t1) add a lookup overhead when reading any object,\n> \t2) make it possible to replace any object,\n\nI actually see (2) as an improvement, and (1) as an associated cost.\n\n> And there is also the following problem:\n>\n> \t3) this function is often called like this:\n>\n> \tbuffer = read_sha1_file(sha1, &type, &size);\n> \tif (!buffer)\n> \t\tdie(\"Cannot read %s\", sha1_to_hex(sha1));\n>\n> \tso in case of error, it will give an error message with a bad sha1\n> \tin it because the sha1 of the file that we cannot read is the sha1\n> \tin the replace ref not the one passed to read_sha1_file.\n\nYou have refs/replace/$A that records $B, to tell git that the real object\n$A in the history is replaced by another object $B.  The caller feeds $A\nin the above snippet to read_sha1_file(), and your read_sha1_file()\nnotices that it needs to read $B instead, returns the buffer from the\nobject $B, and reports its type and size.  If there is no $B available, it\nmay return NULL and the caller says \"I asked for $A but in this repository\nI cannot get to it\".  That sounds consistent to me, but I agree it would\nbe more helpful to report \"and the reason why I cannot get to it is\nbecause you have replacement defined as $B which you do not have.\"\n\n> To avoid the above problems, maybe we can try to also improve what \n> read_sha1_file does:\n>\n> 1) allow callers to pass a type in the \"type\" argument and only lookup in \n> the replace refs if we say we want a commit, but this makes calling this \n> function more error prone\n\nThis is debatable, but can go either way.\n\n> 2) when we say we want an object with a given type, check if the object we \n> read has this type (and die if not)\n\nThat we already do anyway, don't we?  parse_commit() gets data from\nread_sha1_file() and would complain if it gets a blob, etc.\n\n> 3) die in read_sha1_file when there is an error and we are replacing so that \n> callers don't need to die themself and so that we can always report an \n> accurate sha1 in the error message\n\nI expect the use of graft and object replacement (or if you insist,\n\"commit replacement\") rather rare, and I think it is probably Ok to\ndeclare it a grave repository misconfiguration if somebody claims that $A\nis replaced by $B without actually having $B:\n\n\tvoid *read_sha1_file(sha1, type, size)\n        {\n\t\tvoid *data;\n        \tunsigned char *replacement = lookup_replace_object(sha1);\n                if (replacement) {\n                \tdata = read_sha1_file(replacement, type, size);\n                        if (!data)\n                        \tdie(\"replacement %s not found for %s\",\n                                    get_sha1_hex(replacement),\n                                    get_sha1_hex(sha1));\n\t\t} else {\n\t\t\tdata = read_object(sha1, type, size);\n\t\t}\n                ... existing code ...\n                return data;                \n        }\n\n\nTo disable replacement for connectivity walkers, lookup_replace_object()\ncan look at a some global defined in environment.c, perhaps.\n"},{"id":"99907","messageId":"m3tz87yvh4.fsf@localhost.localdomain","threadId":"17035","inReplyTo":"7v8wpl4akr.fsf@gitster.siamese.dyndns.org","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-01-10T16:30:39Z","receivedAt":"2009-01-10T16:30:39Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n> Christian Couder <chriscool@tuxfamily.org> writes:\n> \n> > Yeah, but read_sha1_file is called to read all object files, not just \n> > commits. So putting the hook there will:\n> >\n> > \t1) add a lookup overhead when reading any object,\n> > \t2) make it possible to replace any object,\n> \n> I actually see (2) as an improvement, and (1) as an associated cost.\n\nI just had an idea: we can use this mechanism to better manage large\nbinary files in Git, by using replacements for _blobs_.\n\nWe want to be able to have two flavours of repository: one with large\nblobs (media files usually), and one without.  We can use stubs in the\nplace of large binary files in 'no-megablobs' flavor, and add contents\nof those files via refs/replace/* for _blobs_ in 'with-megablobs'\nflavour.  We can control which objects we want to have, and which\nobjects to transfer.\n\nWhat do you think about this (abuse of) an idea?\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"100095","messageId":"200901121050.24548.jnareb@gmail.com","threadId":"17035","inReplyTo":"1231727868.6716.155.camel@vaio","subject":"Re: [RFC/PATCH 2/3] replace_object: add mechanism to replace objects found in \"refs/replace/\"","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-01-12T09:50:23Z","receivedAt":"2009-01-12T09:50:23Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Mon, 12 Jan 2009, Tim 'mithro' Ansell wrote:\n\n> > I just had an idea: we can use this mechanism to better manage large\n> > binary files in Git, by using replacements for _blobs_.\n> > \n> > We want to be able to have two flavours of repository: one with large\n> > blobs (media files usually), and one without.  We can use stubs in the\n> > place of large binary files in 'no-megablobs' flavor, and add contents\n> > of those files via refs/replace/* for _blobs_ in 'with-megablobs'\n> > flavour.  We can control which objects we want to have, and which\n> > objects to transfer.\n> > \n> > What do you think about this (abuse of) an idea?\n> \n> I'm not sure I understand. I don't really care much about the\n> implementation details if it's transparent to the user :)\n> \n> I have not really had much time to pursue my idea of getting sha1_file\n> to read download blob on an as-needed basis (work has been hectic).\n\nActually this idea is a bit different from lazy / on demand downloading\nof large blobs.\n\n\nIn \"on demand loading\" solution you have sha-1 of _full_ blob in a tree,\nbut git knows that not having it is not a fatal error (somehow), and you\ncan ask git to download it. This is as far as I understand the solution\nyou proposed and partially implemented.\n\nIn \"replacement blobs\" solution, the (ab)use of object replacement\nmechanism meant originally for easier bisect, you have sha-1 of _stub_\nobject in a tree.  If you want full (large) blob, you add replacement\nin refs/replace/*, replacing stub blob object (must be unique; it can\nfor example contain some header + sha-1 of full object) with full\nobject.  If you want to have large object, you need to transfer \nrefs/replace/*.  This solution means that user needs to be aware of\nthis mechanism, or have some wrapper (script) around it.\n\n\nDifferent solution, and a bit different behavior wrt getting large\nobject for the user.\n\n-- \nJakub Narebski\nPoland\n"}]}