{"thread":{"id":"17031","subject":"[JGIT RFC] How read versions of a specific object","startedAt":"2009-01-07T03:44:05Z","lastAt":"2009-01-07T09:23:23Z","messageCount":3,"participants":["Imran M Yousuf","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"99535","messageId":"7bfdc29a0901061944x454a9t1d01e6744f08cf78@mail.gmail.com","threadId":"17031","inReplyTo":null,"subject":"[JGIT RFC] How read versions of a specific object","fromName":"Imran M Yousuf","fromEmail":"imyousuf@gmail.com","sentAt":"2009-01-07T03:44:05Z","receivedAt":"2009-01-07T03:44:05Z","isPatch":false,"sender":{"key":"imyousuf@gmail.com","avatar":"https://gravatar.com/avatar/fda3c870262849d03c7b9c4d288842e128d6d80769fa7bc2d22731b7597928be?d=mp&s=160"},"body":"Hi,\n\nI am trying to read all or n-th version of an object. Currently to do\nthis I am using the following piece of code, which has to walk to\nevery commit is present and from there prepare a set of its object id,\nit is definitely expensive if the commit history is huge, is there a\nfaster/better way to achieve it?\n\nfor (int i = 0; i < App.OBJECT_COUNT;\n ++i) {\n System.out.println(\"INDEX: \" + i);\n String isbn =\n String.valueOf(Integer.parseInt(App.INIT_ID) + i);\n System.out.println(\"ISBN: \" + isbn);\n ObjectWalk objectWalk = new ObjectWalk(repo);\n /*\n * Checks whether the Commit has the tree or not. It does not\n * check whether it has changed or not.\n */\n objectWalk.setTreeFilter(PathFilter.create(isbn));\n RevObject revObject = null;\n objectWalk.markStart(objectWalk.parseCommit(repo.resolve(\n Constants.HEAD)));\n Set revisions =\n new HashSet();\n do {\n if (revObject != null) {\n Commit revision = repo.mapCommit(revObject.getId());\n Tree versionTree = repo.mapTree(revision.getTreeId());\n if (versionTree.existsBlob(isbn)) {\n revisions.add(versionTree.findBlobMember(isbn).getId());\n }\n }\n revObject = objectWalk.next();\n }\n while (revObject != null);\n System.out.println(\"Revisions: \" + revisions);\n }\n\nThe details source code of the project is available @\nhttp://github.com/imyousuf/jgit-usage/tree/master\n\nThank you,\n\n-- \nImran M Yousuf\nEntrepreneur & Software Engineer\nSmart IT Engineering\nDhaka, Bangladesh\nEmail: imran@smartitengineering.com\nBlog: http://imyousuf-tech.blogs.smartitengineering.com/\nMobile: +880-1711402557\n"},{"id":"99536","messageId":"20090107040417.GA10790@spearce.org","threadId":"17031","inReplyTo":"7bfdc29a0901061944x454a9t1d01e6744f08cf78@mail.gmail.com","subject":"Re: [JGIT RFC] How read versions of a specific object","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-01-07T04:04:17Z","receivedAt":"2009-01-07T04:04:17Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Imran M Yousuf wrote:\n> I am trying to read all or n-th version of an object. Currently to do\n> this I am using the following piece of code, which has to walk to\n> every commit is present and from there prepare a set of its object id,\n> it is definitely expensive if the commit history is huge, is there a\n> faster/better way to achieve it?\n\nNot really. You can more efficiently use JGit and reduce some of\nthe overheads, but that's about it.\n\n> for (int i = 0; i < App.OBJECT_COUNT;\n> ++i) {\n> ObjectWalk objectWalk = new ObjectWalk(repo);\n\nDon't use ObjectWalk, use a RevWalk. You don't need it to keep\ntrack of tree or blob identities. The ObjectWalk code has more\noverhead to do that bookkeeping.\n\n> Commit revision = repo.mapCommit(revObject.getId());\n> Tree versionTree = repo.mapTree(revision.getTreeId());\n> if (versionTree.existsBlob(isbn)) {\n> revisions.add(versionTree.findBlobMember(isbn).getId());\n\nUse a TreeWalk to do this. Its quicker because it doesn't\nhave to parse as much data to come up with the same result.\n\nMore specifically there's a static factory method that sets up for\na path limited walk and returns the TreeWalk pointing at that entry.\n\nYou can use the fact that RevWalk.next() returns a RevCommit to get\nyou the RevTree, which is the tree you need to give to the TreeWalk\nconstructor (its the root level tree of the commit).\n\n\nBut if App.OBJECT_COUNT is quite large and covers most of your\nobjects, you are probably better off using a loop over the commits\nand diff'ing against the ancestor:\n\n\tfinal HashMap> versions = ...;\n\tfinal RevWalk rw = new RevWalk(repo);\n\tfinal TreeWalk tw = new TreeWalk(repo);\n\trw.markStart(rw.parseCommit(repo.parse(HEAD)));\n\ttw.setFilter(TreeFilter.ANY_DIFF);\n\n\tRevCommit c;\n\twhile ((c = rw.next()) != null) {\n\t\tfinal ObjectId[] p = new ObjectId[c.getParentCount() + 1];\n\t\tfor (int i = 0; i < c.getParentCount(); i++) {\n\t\t\trw.parse(c.getParent(i));\n\t\t\tp[i] = c.getParent(i).getTree();\n\t\t}\n\t\tfinal int me = p.length -1;\n\t\tp[me] = c.getTree();\n\t\ttw.reset(p);\n\t\twhile (tw.next()) {\n\t\t\tif (tw.getFileMode(me).getObjectType() == Constants.OBJ_BLOB) {\n\t\t\t\t// This path was modified relative to the ancestor(s).\n\t\t\t\t//\n\t\t\t\tString s = tw.getPathString();\n\t\t\t\tSet i = versions.get(s);\n\t\t\t\tif (i == null)\n\t\t\t\t\tversions.put(s, i = new HashSet());\n\t\t\t\ti.add(tw.getObjectId(me));\n\t\t\t}\n\n\t\t\tif (tw.isSubtree()) {\n\t\t\t\t// make sure we recurse into modified directories\n\t\t\t\ttw.enterSubtree();\n\t\t\t}\n\t\t}\n\t}\n\n-- \nShawn.\n"},{"id":"99562","messageId":"7bfdc29a0901070123q6b1b7d0at16d963ec9c9f97af@mail.gmail.com","threadId":"17031","inReplyTo":"20090107040417.GA10790@spearce.org","subject":"Re: [JGIT RFC] How read versions of a specific object","fromName":"Imran M Yousuf","fromEmail":"imyousuf@gmail.com","sentAt":"2009-01-07T09:23:23Z","receivedAt":"2009-01-07T09:23:23Z","isPatch":false,"sender":{"key":"imyousuf@gmail.com","avatar":"https://gravatar.com/avatar/fda3c870262849d03c7b9c4d288842e128d6d80769fa7bc2d22731b7597928be?d=mp&s=160"},"body":"On Wed, Jan 7, 2009 at 10:04 AM, Shawn O. Pearce wrote:\n> Imran M Yousuf wrote:\n>> I am trying to read all or n-th version of an object. Currently to do\n>> this I am using the following piece of code, which has to walk to\n>> every commit is present and from there prepare a set of its object id,\n>> it is definitely expensive if the commit history is huge, is there a\n>> faster/better way to achieve it?\n>\n> Not really. You can more efficiently use JGit and reduce some of\n> the overheads, but that's about it.\n>\n\nThanks Shawn, for pointing it out and it actually does improve the\nperformance, for every lookup its like 200ms.\n\nBest regards,\n\nImran\n\n>> for (int i = 0; i < App.OBJECT_COUNT;\n>> ++i) {\n>> ObjectWalk objectWalk = new ObjectWalk(repo);\n>\n> Don't use ObjectWalk, use a RevWalk. You don't need it to keep\n> track of tree or blob identities. The ObjectWalk code has more\n> overhead to do that bookkeeping.\n>\n>> Commit revision = repo.mapCommit(revObject.getId());\n>> Tree versionTree = repo.mapTree(revision.getTreeId());\n>> if (versionTree.existsBlob(isbn)) {\n>> revisions.add(versionTree.findBlobMember(isbn).getId());\n>\n> Use a TreeWalk to do this. Its quicker because it doesn't\n> have to parse as much data to come up with the same result.\n>\n> More specifically there's a static factory method that sets up for\n> a path limited walk and returns the TreeWalk pointing at that entry.\n>\n> You can use the fact that RevWalk.next() returns a RevCommit to get\n> you the RevTree, which is the tree you need to give to the TreeWalk\n> constructor (its the root level tree of the commit).\n>\n>\n> But if App.OBJECT_COUNT is quite large and covers most of your\n> objects, you are probably better off using a loop over the commits\n> and diff'ing against the ancestor:\n>\n> final HashMap> versions = ...;\n> final RevWalk rw = new RevWalk(repo);\n> final TreeWalk tw = new TreeWalk(repo);\n> rw.markStart(rw.parseCommit(repo.parse(HEAD)));\n> tw.setFilter(TreeFilter.ANY_DIFF);\n>\n> RevCommit c;\n> while ((c = rw.next()) != null) {\n> final ObjectId[] p = new ObjectId[c.getParentCount() + 1];\n> for (int i = 0; i < c.getParentCount(); i++) {\n> rw.parse(c.getParent(i));\n> p[i] = c.getParent(i).getTree();\n> }\n> final int me = p.length -1;\n> p[me] = c.getTree();\n> tw.reset(p);\n> while (tw.next()) {\n> if (tw.getFileMode(me).getObjectType() == Constants.OBJ_BLOB) {\n> // This path was modified relative to the ancestor(s).\n> //\n> String s = tw.getPathString();\n> Set i = versions.get(s);\n> if (i == null)\n> versions.put(s, i = new HashSet());\n> i.add(tw.getObjectId(me));\n> }\n>\n> if (tw.isSubtree()) {\n> // make sure we recurse into modified directories\n> tw.enterSubtree();\n> }\n> }\n> }\n>\n> --\n> Shawn.\n>\n\n\n\n-- \nImran M Yousuf\nEntrepreneur & Software Engineer\nSmart IT Engineering\nDhaka, Bangladesh\nEmail: imran@smartitengineering.com\nBlog: http://imyousuf-tech.blogs.smartitengineering.com/\nMobile: +880-1711402557\n"}]}