# What's in git.git 14 messages from 2006-05-10 to 2006-05-10. Participants: Junio C Hamano, Linus Torvalds, Randal L. Schwartz, Martin Langhoff, Jakub Narebski, Nicolas Pitre, Alex Riesen. Thread: https://gitlist.dev/t/4115 ## Junio C Hamano, 2006-05-10 03:11 Subject: What's in git.git Message-ID: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7viroezi8s.fsf%40assigned-by-dhcp.cox.net ``` This week's "What's in" is a day early, since I do not expect to be able to do much gitting for the rest of the week. * The 'maint' branch has these fixes since the last announcement. Dmitry V. Levin: Separate object name errors from usage errors Eric Wong: apply: fix infinite loop with multiple patches with --index Johannes Schindelin: repo-config: trim white-space before comment Junio C Hamano: core.prefersymlinkrefs: use symlinks for .git/HEAD repo-config: document what value_regexp does a bit more clearly. Fix repo-config set-multivar error return path. Documentation: {caret} fixes (git-rev-list.txt) Linus Torvalds: Fix "git diff --stat" with long filenames revert/cherry-pick: use aggressive merge. Martin Waitz: clone: keep --reference even with -l -s repack: honor -d even when no new pack was created Matthias Lederhofer: core-tutorial.txt: escape asterisk Pavel Roskin: Release config lock if the regex is invalid Sean Estabrooks: Fix for config file section parsing. Yakov Lerner: read-cache.c: use xcalloc() not calloc() * The 'master' branch has these since the last announcement, in addition to the above. I've flushed topics that have been cooked in "next" long enough and hadn't given me problems. Eric Wong: git-svn: documentation updates git-svn 1.0.0 Johannes Schindelin: builtin-push: --all and --tags _are_ explicit refspecs Fix users of prefix_path() to free() only when necessary Fix users of prefix_path() to free() only when necessary Junio C Hamano: get_sha1(): :path and :[0-3]:path to extract from index. Makefile: do not link rev-list any specially. delta: stricter constness pack-object: squelch eye-candy on non-tty binary patch. binary diff: further updates. update-index --unresolve: work from a subdirectory. checkout-index: plug memory leak from prefix_path() update-index: plug memory leak from prefix_path() update-index --again update-index --again: take optional pathspecs binary diff and apply: testsuite. repo-config: document what value_regexp does a bit more clearly. get_sha1() - fix infinite loop on nonexistent stage. Teach git-clean optional ... parameters. checkout: use --aggressive when running a 3-way merge (-m). Martin Waitz: Transitively read alternatives test case for transitive info/alternates clone: don't clone the info/alternates file Nicolas Pitre: split the diff-delta interface use delta index data when finding best delta matches replace adler32 with Rabin's polynomial in diff-delta tiny optimization to diff-delta improve diff-delta with sparse and/or repetitive data improve base85 generated assembly code Peter Hagervall: Sparse fix for builtin-diff Sean Estabrooks: Several trivial documentation touch ups. Fix up docs where "--" isn't displayed correctly. Update git-unpack-objects documentation. Clarify git-cherry documentation. Another config file parsing fix. t1300-repo-config: two new config parsing tests. * The 'next' branch, in addition, has these. - cvsserver and cvsexportcommit updates (Martin Langhoff and Martyn Smith) This is a new merge but not very new code. Martin may want to comment on how ready they are. - built-in fmt-patch (Johannes Schindelin) I think this is ready, even though it does not have some things we have in format-patch (i.e. --attach, --signoff, --check). If anybody deeply cares please stop me soon or better yet enhance with your patches; otherwise I would like to push this out to "master" sometime next week to supersede the git-format-patch script. - built-in grep (me) I think this is also ready, even though it robs users from having funky "grep" on their $PATH and invoke it. Compared to GNU grep, it lacks -P (pcre), -Z (NUL-terminated output), -q (totally quiet), -z (NUL-terminated input), but all the commonly used ones including -f (from file), -F (fixed), -w (word regexp), -l/-L (files with/without match) and -n (line number) are implemented. The same "stop me or else" comment applies. - use config "remote.$name.url" and friends for fetch/pull (Johannes Schindelin) I think this itself is ready; the only reason I do not plan to do so this week is to wait until the new config format discussion settles, at which time we would need to adjust this to the new format. - cache-tree (me) This has been stalled; I would want to redo it without using out-of-index data structure, but that would need the following steps, and lately I have too many distractions to concentrate on them. - review existing index/working-tree/tree walkers. - prepare a merge-tree style path walker that walks index, working tree, and zero or more trees in parallel. This would probably have an interator interface to return list of either tree or blob to the caller. Because index does not currently have tree entries, a "not up-to-date" tree entry would be returned from the index part of the walker if I base this change on "master". I might base this on "next" and use information from cache-tree. - using the parallel path walker, revamp diff-cache (both cached and non cached) to work out kinks to the walker API and see how well it performs. If I base the work on "next", the code should be able to skip unchanged subtrees. - revamp diff-files (walks index and working tree), although this will not get any benefit from having tree entries in the index. - revamp read-tree (all forms). - if I based the work on "next", rip out the cache-tree dependency and make the walker return "not up-to-date" tree entries for index. - revamp update-index, apply, and write-tree to have tree entries in the index. This probably involves revamping read-cache interface to update the index file contents. - review read-tree to make sure it correctly maintains the tree entries in the cache. * The 'pu' branch, in addition, has the proposed configuration file syntax updates from Linus with a patch from Sean. I haven't had time to really look at it, and it seems to fail a test right now, but I left it as is. This is merged from a throw-away topic branch for now, but when I resume to gitting sometime next week I am hoping we have something ready to be tested (iow, "next" material) based on the list concensus. At that time we might probably do backport to cope with the syntax updates for 1.3.X release and perhaps 1.2.X series just for fun. ``` ## Linus Torvalds, 2006-05-10 03:48 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605092043270.3718%40g5.osdl.org In-Reply-To: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> ``` On Tue, 9 May 2006, Junio C Hamano wrote: > > * The 'pu' branch, in addition, has the proposed configuration > file syntax updates from Linus with a patch from Sean. I > haven't had time to really look at it, and it seems to fail a > test right now, but I left it as is. The reason for the failure is the new syntax for multi-section variables. This patch makes the test succed, by changing [1.2.3] into [1 "2.3"] which is how subsections now end up being shown (you can still _parse_ them the old way, but they get created the new way, which is why the test fails) That's a very strange test-case, and on the face of it the new syntax looks "worse", but if you were to be realistic about this kind of section name, it would more likely explain _what_ that number sequence means, so you would more realistically name your sections something like [version "1.2.3"] which I think everybody agrees looks nicer than [version.1.2.3] or similar. Of course, I don't think we currently actually have any _users_ of any multi-level section names at all, so this is all entirely theoretical until we start actually using them. Linus --- diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh index 7090ea9..54b3394 100755 --- a/t/t1300-repo-config.sh +++ b/t/t1300-repo-config.sh @@ -241,7 +241,7 @@ # empty line NoNewLine = wow2 for me [123456] a123 = 987 -[1.2.3] +[1 "2.3"] alpha = beta EOF ``` ## Linus Torvalds, 2006-05-10 04:21 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605092117240.3718%40g5.osdl.org In-Reply-To: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> ``` On Tue, 9 May 2006, Junio C Hamano wrote: > > Junio C Hamano: > binary patch. > binary diff: further updates. Btw, am I just smoking stuff, or is this going to be fundamentally problematic for "git-apply -R" if we ever want to support that? I think the new binary diff is non-reversible. That's ok right now, since we don't actually support patching in reverse (if you want to get the reverse patch, you need to _diff_ it in reverse and then patch it that way). But at least in theory we might well want to do "-R" eventually. Hmm? Or did I just mis-understand the format? Linus ``` ## Linus Torvalds, 2006-05-10 04:26 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605092123520.3718%40g5.osdl.org In-Reply-To: ``` On Tue, 9 May 2006, Linus Torvalds wrote: > > I think the new binary diff is non-reversible. That's ok right now, since > we don't actually support patching in reverse (if you want to get the > reverse patch, you need to _diff_ it in reverse and then patch it that > way). Btw, I don't actually know why we don't support "-R". The way git-apply is written, it should be totally trivial (just switch old/new around for data and line numbers - since it doesn't actually apply the patch directly line by line or anything like that) for a normal patch. So if I read the binary patch right, the lack of "-R" went from "silly oversight" to "uhhuh, I don't think the patch format supports it". Maybe it's not a big deal. Linus ``` ## Randal L. Schwartz, 2006-05-10 04:36 Subject: Re: What's in git.git Message-ID: <864pzyh4x0.fsf@blue.stonehenge.com> URL: https://gitlist.dev/e/864pzyh4x0.fsf%40blue.stonehenge.com In-Reply-To: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> ``` >>>>> "Junio" == Junio C Hamano writes: Junio> This week's "What's in" is a day early, since I do not expect to Junio> be able to do much gitting for the rest of the week. I just got this with the latest, on the git archive, using git-repack -a: Generating pack... Done counting 19151 objects. Deltifying 19151 objects. Segmentation fault (core dumped) This is on OpenBSD. Is there a secret sabotage afoot? This is repeatable. Is there anything I can try differently? -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training! ``` ## Junio C Hamano, 2006-05-10 04:41 Subject: Re: What's in git.git Message-ID: <7vd5emze3h.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vd5emze3h.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` Linus Torvalds writes: > On Tue, 9 May 2006, Junio C Hamano wrote: >> >> Junio C Hamano: >> binary patch. >> binary diff: further updates. > > Btw, am I just smoking stuff, or is this going to be fundamentally > problematic for "git-apply -R" if we ever want to support that? It is problematic but not more than the current index + "Binary files differ" output. If you have both pre and postimage then you do not need the binary data. The forward application is done assuming you have the preimage (but not necessarily postimage), and when you do not have postimage the binary data is used. When going reverse we should assume you have the postimage (but not necessarily preimage), but the pack-object format xdelta is not reversible so if you do not have preimage that matches the index, with the current output format you lose. If we care enough, we could add a reverse delta from postimage to preimage to the output, but I am not sure if it is worth it. > But at least in theory we might well want to do "-R" eventually. Yes, but even without binary, -R has a funny implication when copy-edit patch is involved. What if a patch copy-edits to create a new file B based on old A, and also modifies A in-place, and somehow the postimages of A and B you already have are not consistent with what that patch does? Application with -R would produce two versions of A and you would get a conflict. I guess showing a combined diff would be a helpful way to resolve that ;-). ``` ## Linus Torvalds, 2006-05-10 04:45 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605092142050.3718%40g5.osdl.org In-Reply-To: <864pzyh4x0.fsf@blue.stonehenge.com> ``` On Tue, 9 May 2006, Randal L. Schwartz wrote: > >>>>> "Junio" == Junio C Hamano writes: > > Junio> This week's "What's in" is a day early, since I do not expect to > Junio> be able to do much gitting for the rest of the week. > > I just got this with the latest, on the git archive, using git-repack -a: > > Generating pack... > Done counting 19151 objects. > Deltifying 19151 objects. > Segmentation fault (core dumped) > > This is on OpenBSD. Is there a secret sabotage afoot? This is repeatable. > Is there anything I can try differently? Can you see what the traceback is with gdb? I'd suspect the deltifier changes, the rabin hash in particular. The core file traceback would probably point right at the culprit if so. I don't see the problem myself, but if it's an access just past the end of an array or something, it would depend on exactly what the delta pattern is (which, without the "-f" flag, in turn depends on what your previous packs looked like) and also on the allocation strategy (which migth explain why it shows on OpenBSD but Linux people hadn't seen it). Linus ``` ## Linus Torvalds, 2006-05-10 04:51 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605092150460.3718%40g5.osdl.org In-Reply-To: <7vd5emze3h.fsf@assigned-by-dhcp.cox.net> ``` On Tue, 9 May 2006, Junio C Hamano wrote: > > It is problematic but not more than the current index + "Binary > files differ" output. If you have both pre and postimage then > you do not need the binary data. Fair enough. > > But at least in theory we might well want to do "-R" eventually. > > Yes, but even without binary, -R has a funny implication when > copy-edit patch is involved. What if a patch copy-edits to > create a new file B based on old A, and also modifies A > in-place, and somehow the postimages of A and B you already have > are not consistent with what that patch does? Yeah, that could get exciting ;) Linus ``` ## Junio C Hamano, 2006-05-10 05:05 Subject: Re: What's in git.git Message-ID: <7v64kezczd.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7v64kezczd.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <864pzyh4x0.fsf@blue.stonehenge.com> ``` merlyn@stonehenge.com (Randal L. Schwartz) writes: >>>>>> "Junio" == Junio C Hamano writes: > > Junio> This week's "What's in" is a day early, since I do not expect to > Junio> be able to do much gitting for the rest of the week. > > I just got this with the latest, on the git archive, using git-repack -a: > > Generating pack... > Done counting 19151 objects. > Deltifying 19151 objects. > Segmentation fault (core dumped) > > This is on OpenBSD. Is there a secret sabotage afoot? This is repeatable. > Is there anything I can try differently? Tonight's latest (f7a3276) merged Nico's delta patches that was in "next" branch for some time, so that is what I would suspect first. commit 06a9f9203570d21f9ef5fe219cdde527dcdf0990 Author: Nicolas Pitre improve diff-delta with sparse and/or repetitive data commit 2d08e5dd730680f7f8645a6326ec653435e032df Author: Nicolas Pitre tiny optimization to diff-delta commit 3dc5a9e4cdcc7124c665a050547d1285d86a421f Author: Nicolas Pitre replace adler32 with Rabin's polynomial in diff-delta commit f6c7081aa97aa67baa06390a1ef36088c33bf010 Author: Nicolas Pitre use delta index data when finding best delta matches commit 08abe669c05521499149dbf84fedefb04a8fa34d Author: Nicolas Pitre split the diff-delta interface Could you revert them (i.e. run "git revert 06a9f9", "git revert 2d08e5", ...) to see where it starts behaving again? Alternatively, you could bisect between 2fc240a (before merging np/delta branch) and master (f7a3276). (1) Reverting recipe. $ git checkout -b testfix master $ make $ GIT_EXEC_PATH=. PATH=.:$PATH \ ./git-repack -a ;# you already know this fails. $ EDITOR=: git revert 06a9f92 $ make $ GIT_EXEC_PATH=. PATH=.:$PATH ./git-repack -a ;# does it? $ EDITOR=: git revert 2d08e5d $ make $ GIT_EXEC_PATH=. PATH=.:$PATH ./git-repack -a ;# does it? $ EDITOR=: git revert 3dc5a9e $ make $ GIT_EXEC_PATH=. PATH=.:$PATH ./git-repack -a ;# does it? ... (2) Bisecting recipe. $ git checkout -b testfix master $ make $ GIT_EXEC_PATH=. PATH=.:$PATH \ ./git-repack -a ;# you already know this fails. $ git reset --hard 2fc240a $ make $ GIT_EXEC_PATH=. PATH=.:$PATH \ ./git-repack -a ;# before merging np/delta, # should hopefully work $ git bisect start $ git bisect good 2fc240a $ git bisect bad master Bisecting: 10 revisions left to test after this [143f4d94c6e2188a6bedfdfa268e66b579e3fbf9] Merge branch 'jc/again' $ make $ GIT_EXEC_PATH=. PATH=.:$PATH ./git-repack -a ;# does this work? $ git bisect good ;# if it works, or $ git bisect bad ;# otherwise Bisecting: 7 revisions left to test after this ... ``` ## Martin Langhoff, 2006-05-10 05:34 Subject: Re: What's in git.git Message-ID: <46a038f90605092234u14d4cd2en73c9cbe698eea8b9@mail.gmail.com> URL: https://gitlist.dev/e/46a038f90605092234u14d4cd2en73c9cbe698eea8b9%40mail.gmail.com In-Reply-To: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> ``` On 5/10/06, Junio C Hamano wrote: > * The 'next' branch, in addition, has these. > > - cvsserver and cvsexportcommit updates (Martin Langhoff and Martyn Smith) > > This is a new merge but not very new code. Martin may want > to comment on how ready they are. They have seen some limited use in-house -- we don't use Eclipse in-house that much, but that has been the test target. I am currently looking for a machine with good bandwidth to a backbone and cycles to spare where we can run anon cvs access to Linus's kernel.git repo. cheers, martin ``` ## Jakub Narebski, 2006-05-10 06:48 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/e3s2b0%24i1m%241%40sea.gmane.org In-Reply-To: <7viroezi8s.fsf@assigned-by-dhcp.cox.net> ``` Junio C Hamano wrote: > - built-in grep (me) > > I think this is also ready, even though it robs users from > having funky "grep" on their $PATH and invoke it. Compared > to GNU grep, it lacks -P (pcre), -Z (NUL-terminated output), > -q (totally quiet), -z (NUL-terminated input), but all the > commonly used ones including -f (from file), -F (fixed), -w > (word regexp), -l/-L (files with/without match) and -n (line > number) are implemented. The same "stop me or else" comment > applies. If there would be possible to use external grep (like one can use external diff), then lack of some options wouldn't matter. -- Jakub Narebski Warsaw, Poland ``` ## Nicolas Pitre, 2006-05-10 14:15 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605101012250.24505%40localhost.localdomain In-Reply-To: ``` On Tue, 9 May 2006, Linus Torvalds wrote: > > > On Tue, 9 May 2006, Randal L. Schwartz wrote: > > > >>>>> "Junio" == Junio C Hamano writes: > > > > Junio> This week's "What's in" is a day early, since I do not expect to > > Junio> be able to do much gitting for the rest of the week. > > > > I just got this with the latest, on the git archive, using git-repack -a: > > > > Generating pack... > > Done counting 19151 objects. > > Deltifying 19151 objects. > > Segmentation fault (core dumped) > > > > This is on OpenBSD. Is there a secret sabotage afoot? This is repeatable. > > Is there anything I can try differently? > > Can you see what the traceback is with gdb? > > I'd suspect the deltifier changes, the rabin hash in particular. The core > file traceback would probably point right at the culprit if so. > > I don't see the problem myself, but if it's an access just past the end of > an array or something, it would depend on exactly what the delta pattern > is (which, without the "-f" flag, in turn depends on what your previous > packs looked like) and also on the allocation strategy (which migth > explain why it shows on OpenBSD but Linux people hadn't seen it). When linking with Electric Fence I can reproduce the segfault on Linux as well. Looking into it now. Nicolas ``` ## Alex Riesen, 2006-05-10 15:00 Subject: Re: What's in git.git Message-ID: <81b0412b0605100800p48fa6b46nb91c103aeee4aa94@mail.gmail.com> URL: https://gitlist.dev/e/81b0412b0605100800p48fa6b46nb91c103aeee4aa94%40mail.gmail.com In-Reply-To: ``` On 5/10/06, Nicolas Pitre wrote: > > > > > > Junio> This week's "What's in" is a day early, since I do not expect to > > > Junio> be able to do much gitting for the rest of the week. > > > > > > I just got this with the latest, on the git archive, using git-repack -a: > > > > > > Generating pack... > > > Done counting 19151 objects. > > > Deltifying 19151 objects. > > > Segmentation fault (core dumped) > > > Probably unrelated but I get something similar in cygwin: $ git pull //pc1/share/dir +ref1:ref2 Generating pack... Done counting 93 objects. Result has 62 objects. Deltifying 62 objects. git-unpack-objects: fatal: early EOF git-fetch-pack: fatal: git-unpack-objects died with error code 128 (The program names come from a patch attached to the message) After I remerged (from git's master) everything but np/delta (I saw that report about crash on openbsd) it worked again. This time I cannot make the repository public but will try to help with testing. ``` ## Linus Torvalds, 2006-05-10 16:48 Subject: Re: What's in git.git Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0605100919250.3718%40g5.osdl.org In-Reply-To: ``` On Wed, 10 May 2006, Nicolas Pitre wrote: > > When linking with Electric Fence I can reproduce the segfault on Linux > as well. > > Looking into it now. Yeah, I get #0 0x1000bfe4 in create_delta (index=0xf7d758a0, trg_buf=0xf7d72eb8, trg_size=327, delta_size=0xffb422b4, max_size=143) at diff-delta.c:308 #1 0x10005734 in try_delta (trg=0xf7fdbfa0, src=0xf7fdbf94, src_index=0xf7d758a0, max_depth=10) at pack-objects.c:1049 #2 0x10005b04 in find_deltas (list=0xf7e32f54, window=11, depth=10) at pack-objects.c:1128 #3 0x10005ca0 in prepare_pack (window=10, depth=10) at pack-objects.c:1161 #4 0x1000677c in main (argc=3, argv=0xffb436e4) at pack-objects.c:1359 ie the "entry" chain seems to be corrupt in create_delta. Actually, it's not even the chain: it's the very first entry: (gdb) p index->hash[i] $2 = (struct index_entry *) 0xf7d7385c (gdb) p entry $3 = (struct index_entry *) 0xf7d7385c and it's not a random value - it looks perfectly valid. As do all the other index hash entries. It's just that the index hash entries seem to all have been freed, so accessing them causes a SIGSEV! Linus ```