{"thread":{"id":"26823","subject":"calculating a git packfile sha1 checksum in java","startedAt":"2011-03-22T09:27:24Z","lastAt":"2011-03-22T21:10:26Z","messageCount":4,"participants":["madmarcos","Nicolas Pitre","Shawn Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"164029","messageId":"1300786044816-6195501.post@n2.nabble.com","threadId":"26823","inReplyTo":null,"subject":"calculating a git packfile sha1 checksum in java","fromName":"madmarcos","fromEmail":"fru574@my.utsa.edu","sentAt":"2011-03-22T09:27:24Z","receivedAt":"2011-03-22T09:27:24Z","isPatch":false,"sender":{"key":"fru574@my.utsa.edu","avatar":null},"body":"I am learning about the Git packfile and currently trying to reproduce (in\nJava) what I believe to be the SHA1 20-byte checksum for the entire\npackfile. I take the byte array from, and including, the \"PACK\" 4-byte\nheader to the end of the last packaged object's compressed data. Everything\nI have read indicates that the next 20 bytes is the SHA1 checksum for the\nentire packfile.\n\nThe 20-byte checksum that is part of the byte array received from Git is:\nB910248BF9B63AC53595E3835CA57BDAF08DA830\n\nI use the following to calculate my own SHA1 checksum: \ncrypt = MessageDigest.getInstance(\"SHA-1\");\ncrypt.reset();\ncrypt.update(testData);\nbyte [] result = crypt.digest();\n\nMy result ends up as: B910248BF9B63AC53595E3835CA57BDAF08DA813\n\nI am baffled at how only the last byte of my result can be different from\nGit's (if I am using the correct part of the byte stream). If the only\nproblem was the range of data passed to digest() then the entire calculated\nchecksum would most likely look different.\n\nAny ideas?\n\nNote: I use the same code to generate test SHA1 ids for each contained\nobject and they match the references in the tree objects. This problem\ncurrently only involves calculating the checksum over the entire packfile.\n\n--\nView this message in context: http://git.661346.n2.nabble.com/calculating-a-git-packfile-sha1-checksum-in-java-tp6195501p6195501.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"164260","messageId":"AANLkTimWt4P3qtR3b6YfRJ=Tx69EL+9+oXhD_2c9O6JX@mail.gmail.com","threadId":"26823","inReplyTo":"1300786044816-6195501.post@n2.nabble.com","subject":"Re: calculating a git packfile sha1 checksum in java","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-03-22T14:01:12Z","receivedAt":"2011-03-22T14:01:12Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Tue, Mar 22, 2011 at 02:27, madmarcos <fru574@my.utsa.edu> wrote:\n> I am learning about the Git packfile and currently trying to reproduce (in\n> Java) what I believe to be the SHA1 20-byte checksum for the entire\n> packfile. I take the byte array from, and including, the \"PACK\" 4-byte\n> header to the end of the last packaged object's compressed data. Everything\n> I have read indicates that the next 20 bytes is the SHA1 checksum for the\n> entire packfile.\n\nThat is correct.\n\n> The 20-byte checksum that is part of the byte array received from Git is:\n> B910248BF9B63AC53595E3835CA57BDAF08DA830\n>\n> I use the following to calculate my own SHA1 checksum:\n> crypt = MessageDigest.getInstance(\"SHA-1\");\n> crypt.reset();\n> crypt.update(testData);\n> byte [] result = crypt.digest();\n\nLooks right, assuming that testData is \"PACK...\" up to but not\nincluding the last 20 bytes. :-)\n\n> My result ends up as: B910248BF9B63AC53595E3835CA57BDAF08DA813\n>\n> I am baffled at how only the last byte of my result can be different from\n> Git's (if I am using the correct part of the byte stream). If the only\n> problem was the range of data passed to digest() then the entire calculated\n> checksum would most likely look different.\n\nI've never seen SHA-1 produce a value this close to the expected value\nbefore. My first guess is that its a problem elsewhere in your code,\nlike your byte[]->hex formatter, or the code that is reading in the 20\nbytes from Git that has the trailer is reading the wrong thing.\n\n> Note: I use the same code to generate test SHA1 ids for each contained\n> object and they match the references in the tree objects. This problem\n> currently only involves calculating the checksum over the entire packfile.\n\nBut aside from just learning about Git, if you want to work with Git\nin Java... use JGit[1].  :-)\n\n[1] http://www.eclipse.org/jgit/\n\n-- \nShawn.\n"},{"id":"164088","messageId":"1300827506173-6197923.post@n2.nabble.com","threadId":"26823","inReplyTo":"AANLkTiniKOzzwZhJ5GN4nf8AWRTsT6bbMeK4s5XqvH42@mail.gmail.com","subject":"Re: calculating a git packfile sha1 checksum in java","fromName":"madmarcos","fromEmail":"fru574@my.utsa.edu","sentAt":"2011-03-22T20:58:26Z","receivedAt":"2011-03-22T20:58:26Z","isPatch":false,"sender":{"key":"fru574@my.utsa.edu","avatar":null},"body":"Hi Shawn,\nThanks for the response!\nI looked at the JGit API. will it allow me to read a packfile, alter its\ncontents, and repack it?\nregards,\nMark\n\n--\nView this message in context: http://git.661346.n2.nabble.com/calculating-a-git-packfile-sha1-checksum-in-java-tp6195501p6197923.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"164089","messageId":"alpine.LFD.2.00.1103221708370.11889@xanadu.home","threadId":"26823","inReplyTo":"1300827506173-6197923.post@n2.nabble.com","subject":"Re: calculating a git packfile sha1 checksum in java","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2011-03-22T21:10:26Z","receivedAt":"2011-03-22T21:10:26Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 22 Mar 2011, madmarcos wrote:\n\n> Hi Shawn,\n> Thanks for the response!\n> I looked at the JGit API. will it allow me to read a packfile, alter its\n> contents, and repack it?\n\nWhy would you want to do that?  Altering a pack file in place is not a \ntrivial affair.  There is also the corresponding pack index which has to \nbe rebuilt, etc.\n\n\nNicolas\n"}]}