Re: Any plans to support JTA and XA in jgit?

torsdag 13 november 2008 20:20:49 skrev Farrukh Najmi:

No such plans exist. We do not even have a J2EE Resource Manager yet. I did some toying implementing one. As for XA support, I guess that would not be very hard per se, but my thoughts on JEE support was more in the direction of gitweb-like stuff, i.e. reading.

Trying to involve git in distributed database transactions might be cool, but seriously: Do you need it? As for JEE my ideas are: A nice JSP tag library and a resource manager. When is an entirely different question, as is who. Did you look at my experiment in a reply of mine in another recent jgit thread?

The term "distributed" in XA is not quite the same as in distributed verison control. If it would, then we'd send SQL commands over e-mail (now, /that/ would be cool :).

-- robin

Robin Rosenberg wrote:

I am not an expert on this but what I thought could be done is to enhance jgit so it can serve the role of a JTA compliant transactional resource manager similar to that in JDBC, JMS etc. As part of that it could serve resources as JTA XAResource. Not sure how hard all this could be though.

The problem I am trying to solve is this. In my service I need to store metadata in a relational db and content in git such that both either commit or not in a single transaction. If one commits and the other does not that is a serious integrity issue. Seems to me, two phase commit would be the right solution for that in the long run. This what JDBC + JMS topologies do.

A totally separate issue I have to sort out is how to handle multiple unrelated transactions that are modifying the same git repo. If a transaction needs to be rolled back how do roll back exactly some changes in some files in git that were impacted by the transaction. This is not easy because git (and most VCS) do not have transaction isolation like databases do. Any suggestions?

I am not very well versed in tag libraries myself. My situation is one where everything happens inside a SOAP service endpoint and so I suspect JSP tag libraries are not likely to be useful in that situation. Let me know if I am wrong in this assessment.

That is correct.

+1

Thanks very much for your help and that of other colleagues on the list.

torsdag 13 november 2008 21:27:42 skrev Farrukh Najmi:

Updating the head ref is the hard part, but we have locking already there and it is atomic on any sane file system. There is no waiting implemented so one gets an error if two threads try to update the same head simultaneously. Seems doable I'd say. For server use one could wrap the repo with a real database, and just let the disk based refs mirror the database view, though that is quite a bit off focus for my own needs.

not at all.

-- robin

On Thu, Nov 13, 2008 at 3:27 PM, Farrukh Najmi <farrukh@wellfleetsoftware.com> wrote:

That's really easy! First tweak jgit so that *instead* of using .git/refs, it uses your database to store references and exports them on a routine basis to .git/refs for debugging purposes. Then, for each database update:

(1) Start transaction (2) Commit the change to GIT (adds ref update to the transaction) (3) Make other metadata updates (4) Commit transaction

Then set up periodic garbage collection and you're done! If the transaction is aborted, there will simply be a bunch of random loose objects in the git repository, which will be cleaned up the next time you garbage collect. The ref update will be atomic and conditional with the rest of the transaction, and in git the *only* part that really matters for atomicity is the ref.

Cheers, Kyle Moffett

Kyle Moffett wrote:

Wow! This does sound simple in theory. Of course I have much to learn about jgit code base first. Any pointers on what parts of the code are involved that I should start looking at? I would hate to keep a fork of gjit around. Would this work be of interest to the project if it could be done in a way that a few properties control whether to use database or not?

Thanks Kyle!

I am looking for any available docs that will help me learn the layout of the jgit code base.

In the ideal world here is what would help newbies get productive with the project quickly:

1. Have a clear one line description available in the javadocs for each 
package.
2. Have a clear one line description available in javadocs for each 
class in each package.
3. Have a clear one line description available in javadocs for each 
method in each class.

I would gladly do this and contribute the changes back but its a chicken and egg situation since I do not know the code base.

I realize its a lot to ask but would anyone familiar with code base volunteer to do at least (1) above?

At minimum could anyone tell me what packages and classes to look at in order to make changes to jgit so it uses a database instead of filesystem to references.

Lastly, where can I find information on the structure of the .git directory?

Thanks for your help.

Kyle Moffett wrote:

Farrukh Najmi <farrukh@wellfleetsoftware.com> wrote:

See RefDatabase, created by Repository. You would want to replace RefDatabase with your own implementation, which means abstracting that class to be an abstract base class or an interface.

RefUpdate and RefLogWriter are connected to RefDatabase, so you would likely need to modify those too in order to make the change, especially since RefUpdate is the "INSERT OR UPDATE OR DELETE" logic. :)

BTW, I want to do that RefDatabase abstraction anyway, because I'd like to get an in-memory only version of RefDatabase, to make unit testing easier. So abstracting this code may be useful beyond just your desire to back Git with a JDBC-style database.

Shawn O. Pearce wrote:

This is just what I was hoping for. Thanks Shawn. I will try and do this next week and ask for help as needed. Thank you.