threads / discuss / 23190

Enumerating all objects in the Git object store

Subject: Enumerating all objects in the Git object store

## tl;dr

8 messages between Mar 26, 2010 and Mar 27, 2010.

replies: 7people: 5as markdown or json

Paul Richards· Mar 26, 2010, 15:31 UTC · lore

Hi, I'd like to visualize the objects in a Git repository using Graphviz and "dot". I'd like to see commits as circles, trees as triangles, blobs as rectangles, and have arrows linking these objects showing how they reference each other in the Git DAG. For small repositories this may be a useful way to visualize how Git objects work.

I could not find a pre-existing script to do this so I am considering writing one. Is there a way to enumerate all the objects in the Git object store, and for each one figure out its type (commit, tree, or blob) and obtain a list of the objects it references? If not, is there a way to do this for a single object at a time? (I can then simply recurse through the whole structure.)

-- 
Paul Richards
Johannes Sixt· Mar 26, 2010, 15:53 UTC · re: Paul Richards · lore

Re: Enumerating all objects in the Git object store

Am 3/26/2010 16:31, schrieb Paul Richards:
Show 5 quoted lines
> Is there a way to enumerate all the objects in the Git
> object store, and for each one figure out its type (commit, tree, or
> blob) and obtain a list of the objects it references?  If not, is
> there a way to do this for a single object at a time?  (I can then
> simply recurse through the whole structure.)

Use 'git rev-list --objects --all' to find all objects connected to all refs (use something else than --all if you want to). Do *not* look at the path that the tree and blob objects have attached - it is not authorative information.

Use 'git cat-file -t' to find the type.

Use 'git cat-file -p' to pretty-print the content of an object. Parse it depending on the type that you find.

-- Hannes
Jakub Narebski· Mar 26, 2010, 16:13 UTC · re: Johannes Sixt · lore

Re: Enumerating all objects in the Git object store

Johannes Sixt <j.sixt@viscovery.net> writes:
Show 13 quoted lines
> Am 3/26/2010 16:31, schrieb Paul Richards:
> > Is there a way to enumerate all the objects in the Git
> > object store, and for each one figure out its type (commit, tree, or
> > blob) and obtain a list of the objects it references?  If not, is
> > there a way to do this for a single object at a time?  (I can then
> > simply recurse through the whole structure.)
> 
> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.
> 
> Use 'git cat-file -t' to find the type.
For performance it would be better to feed `git rev-list --objects --all`
output to `git cat-file --batch-check` instead.
 
> Use 'git cat-file -p' to pretty-print the content of an object. Parse it
> depending on the type that you find.

For performance it would be better to use `git cat-file --batch`... but on the other hand side you would have to parse "binary" tree objects representation.

-- 
Jakub Narebski
Poland
ShadeHawk on #git
Nicolas Pitre· Mar 26, 2010, 17:25 UTC · re: Johannes Sixt · lore

Re: Enumerating all objects in the Git object store

On Fri, 26 Mar 2010, Johannes Sixt wrote:
> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.
Really??
Nicolas
Johannes Sixt· Mar 26, 2010, 20:11 UTC · re: Nicolas Pitre · lore

Re: Enumerating all objects in the Git object store

On Freitag, 26. März 2010, Nicolas Pitre wrote:
Show 7 quoted lines
> On Fri, 26 Mar 2010, Johannes Sixt wrote:
> > Use 'git rev-list --objects --all' to find all objects connected to all
> > refs (use something else than --all if you want to). Do *not* look at the
> > path that the tree and blob objects have attached - it is not authorative
> > information.
>
> Really??

Yes, really. Consider the same content at two differen paths. Now what? rev-list will only ever list the object once.

-- Hannes
Nicolas Pitre· Mar 26, 2010, 20:24 UTC · re: Johannes Sixt · lore

Re: Enumerating all objects in the Git object store

On Fri, 26 Mar 2010, Johannes Sixt wrote:
Show 11 quoted lines
> On Freitag, 26. März 2010, Nicolas Pitre wrote:
> > On Fri, 26 Mar 2010, Johannes Sixt wrote:
> > > Use 'git rev-list --objects --all' to find all objects connected to all
> > > refs (use something else than --all if you want to). Do *not* look at the
> > > path that the tree and blob objects have attached - it is not authorative
> > > information.
> >
> > Really??
> 
> Yes, really. Consider the same content at two differen paths. Now what? 
> rev-list will only ever list the object once.
Sure. It is still a valid path though.
Nicolas
Paul Richards· Mar 26, 2010, 21:11 UTC · re: Johannes Sixt · lore

Re: Enumerating all objects in the Git object store

On 26 March 2010 15:53, Johannes Sixt <j.sixt@viscovery.net> wrote:
Show 17 quoted lines
> Am 3/26/2010 16:31, schrieb Paul Richards:
>> Is there a way to enumerate all the objects in the Git
>> object store, and for each one figure out its type (commit, tree, or
>> blob) and obtain a list of the objects it references?  If not, is
>> there a way to do this for a single object at a time?  (I can then
>> simply recurse through the whole structure.)
>
> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.
>
> Use 'git cat-file -t' to find the type.
>
> Use 'git cat-file -p' to pretty-print the content of an object. Parse it
> depending on the type that you find.
>
Thank you, this looks perfect!

I'll check this out (and perhaps the optimizations suggested by Jakub if necessary).

-- 
Paul Richards
Michael J Gruber· Mar 27, 2010, 13:33 UTC · re: Paul Richards · lore

Re: Enumerating all objects in the Git object store

Paul Richards venit, vidit, dixit 26.03.2010 16:31:
Show 14 quoted lines
> Hi,
> I'd like to visualize the objects in a Git repository using Graphviz
> and "dot".  I'd like to see commits as circles, trees as triangles,
> blobs as rectangles, and have arrows linking these objects showing how
> they reference each other in the Git DAG.  For small repositories this
> may be a useful way to visualize how Git objects work.
> 
> I could not find a pre-existing script to do this so I am considering
> writing one.  Is there a way to enumerate all the objects in the Git
> object store, and for each one figure out its type (commit, tree, or
> blob) and obtain a list of the objects it references?  If not, is
> there a way to do this for a single object at a time?  (I can then
> simply recurse through the whole structure.)
> 

If you don't care about loose (unreferenced) objects then it's enough to walk through all refs (branches, tags) and, for each ref, look at parent and tree references (using git cat-file) and recurse.

If you don't care about the sha1 of commits you might even want to (ab)use git-fast-export for that!

Michael

← back to recent threads