threads / rfc / 17096

RFC patch, 3 partsAdd specification of git-vcs helpers

Subject: [RFC PATCH 2/3] Add specification of git-vcs helpers

## tl;dr

5 messages between Jan 11, 2009 and Jan 12, 2009. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Daniel Barkalow· Jan 11, 2009, 20:12 UTC · lore

So far, I've actually used list and import; I've also implemented an export, but haven't figured out exactly how the user should cause it to be used.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
 Documentation/git-vcs.txt |   93 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 93 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-vcs.txt

Show changes to Documentation/git-vcs.txt +93 −0

diff --git a/Documentation/git-vcs.txt b/Documentation/git-vcs.txt
new file mode 100644
index 0000000..fa02b76
--- /dev/null
+++ b/Documentation/git-vcs.txt
@@ -0,0 +1,93 @@
+git-vcs-*(1)
+============
+
+NAME
+----
+git-vcs-* - Helper programs for interoperation with foreign systems
+
+SYNOPSIS
+--------
+'git vcs-<system>' <command> [options] [arguments]
+
+DESCRIPTION
+-----------
+
+These programs are normally not used directly by end users, but are
+invoked by various git programs that interact with remote repositories
+when the repository they would operate on is part of a foreign
+system.
+
+Each 'git vcs-<system>' is a helper for interoperating with a
+particular version control system. Different helpers have different
+capabilities (limited both be the particular helper and by the
+capabilities of the system they connect to), and they report what
+capabilities they support.
+
+In general, these programs interact with a portion of the refs
+namespace that isn't used by the rest of git. The configuration will
+then (generally) map these refs into the remotes namespace. This
+allows the importer to do whatever it wants with its refs without
+affecting the state visible to normal git programs.
+
+COMMANDS
+--------
+
+'capabilities'::
+	Prints the capabilities of the helper, one per line. These are:
+	 - import: the basic import command
+	 - marks: import should be done with a saved marks file
+	 - find-new-branches: detect new branches
+	 - export: the general export command
+	 - fork: create a new branch and export to it
+	 - anonymous-fork: make commits on a branch without an inherent name
+	 - merge: merge branches (of whatever type the system supports)
+
+	If the helper doesn't support "merge", the default for pull is
+	to rebase instead of merging.
+
+'list'::
+	Takes the remote name, and outputs the names of refs. These
+	may be followed, after a single space, by "changed" or
+	"unchanged", indicating whether the foreign repository has
+	changed from the state in the ref. If the helper doesn't know,
+	it doesn't have to provide a value. (In particular, it
+	shouldn't do expensive operations, such as importing the
+	content, to see whether it matches.)
+
+'import'::
+	Takes the remote name and a list of names of refs, and imports
+	whatever it describes, by outputting it in git-fast-import
+	format.
+
+'export'::
+	Sends the branch to the foreign system and reimports it in
+	fast-import format.
+
+	Reads a list of commits from stdin, where each commit has no
+	parents which were neither produced by an earlier import nor
+	appearing earlier in the list, where some commit has the old
+	value of the branch as a parent, and where all commits listed
+	are ancestors of the last one. Furthermore:
+
+	 - if the system doesn't support merges, each of these commits
+	   has only a single parent;
+
+	 - if the system doesn't support anonymous branches, the first
+	   commit has the old value of the branch as a parent (if the
+	   branch already had a value), and all parents are either the
+	   commit listed immediately before or produced by an earlier
+	   import;
+
+	 - if the system doesn't support many-way merges, each commit
+	   has at most two parents.
+
+	export produces output in fast-import format giving the
+	content after a round-trip through the foreign system. This
+	also contains extra headers to report the mapping of original
+	git commits to reimported git commits (to facilitate rewriting
+	local branches to use the history-as-reimported instead of the
+	git-only version).
+
+	export reports how much it managed to export by producing
+	commits in the fast-import stream that replace the listed
+	items that were successfully exported.

-- 
1.6.0.6

Junio C Hamano· Jan 12, 2009, 01:29 UTC · re: Daniel Barkalow · lore

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

Daniel Barkalow <barkalow@iabervon.org> writes:

Show 16 quoted lines

> So far, I've actually used list and import; I've also implemented an 
> export, but haven't figured out exactly how the user should cause it to be 
> used.
> ...
> +'capabilities'::
> +	Prints the capabilities of the helper, one per line. These are:
> +	 - import: the basic import command
> +	 - marks: import should be done with a saved marks file
> +	 - find-new-branches: detect new branches
> +	 - export: the general export command
> +	 - fork: create a new branch and export to it
> +	 - anonymous-fork: make commits on a branch without an inherent name
> +	 - merge: merge branches (of whatever type the system supports)
> +
> +	If the helper doesn't support "merge", the default for pull is
> +	to rebase instead of merging.

Have you tried formatting this to both html and man? I think you need to dedent the second paragraph, and have '+' as the sole character on a line immediately before, without any blank lines. The formatted output of the description of 'export' is probably more troublesome as it is much longer.

This list feels like overengineered yet without enough thinking behind it.

Your 'list' allows to list what can become branches on the git end, so as long as you have tracking information on the git side, find-new-branches seems unnecessary, for examle. Worse, find-new-branches needs its own tracking mechansim anyway, but that means how the namespace for remote tracking branches is managed is left to the vcs backends, even though that namespace is directly visiblt to git --- my gut feeling is that it would probably better to arrange things similar to what git native transport does already. 'list' would say "The remote has what can become refs on the git end and calls them A, B, C", and the built-in code, using ref mapping specification, maps them to refs/remotes/somename/{A,B,C}. You do it again and if you see the remote has D but does not have B anymore, you know D is a new branch and B was deleted.

What does 'merge' mean? It cannot mean to fetch their changes from foreign vcs and create a merge locally (you only need import from the backend, and merge will be a usual git merge). Perhaps you meant if you can export a merge back? Some foreign systems may support importing merges but not octopus, so it may not be just a black-or-white boolean.

Also "the default is to rebase instead of merge" is probably too weak. You would want to error out if the user tried to merge, wouldn't you?

Show 8 quoted lines

> +'list'::
> +	Takes the remote name, and outputs the names of refs. These
> +	may be followed, after a single space, by "changed" or
> +	"unchanged", indicating whether the foreign repository has
> +	changed from the state in the ref. If the helper doesn't know,
> +	it doesn't have to provide a value. (In particular, it
> +	shouldn't do expensive operations, such as importing the
> +	content, to see whether it matches.)

I am guessing by 'a value' you mean 'changed/unchanged', iow you are not saying if the helper does not know it can omit such refs from the list, but it is unclear.

Daniel Barkalow· Jan 12, 2009, 02:03 UTC · re: Junio C Hamano · lore

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

On Sun, 11 Jan 2009, Junio C Hamano wrote:

Show 23 quoted lines

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > So far, I've actually used list and import; I've also implemented an 
> > export, but haven't figured out exactly how the user should cause it to be 
> > used.
> > ...
> > +'capabilities'::
> > +	Prints the capabilities of the helper, one per line. These are:
> > +	 - import: the basic import command
> > +	 - marks: import should be done with a saved marks file
> > +	 - find-new-branches: detect new branches
> > +	 - export: the general export command
> > +	 - fork: create a new branch and export to it
> > +	 - anonymous-fork: make commits on a branch without an inherent name
> > +	 - merge: merge branches (of whatever type the system supports)
> > +
> > +	If the helper doesn't support "merge", the default for pull is
> > +	to rebase instead of merging.
> 
> Have you tried formatting this to both html and man?  I think you need to
> dedent the second paragraph, and have '+' as the sole character on a line
> immediately before, without any blank lines.  The formatted output of the
> description of 'export' is probably more troublesome as it is much longer.

Nope, I've just been referring to it in the original text, so I've been careless with markup.

> This list feels like overengineered yet without enough thinking behind it.

Yeah, I haven't really nailed this one down; it's hard to get really right without having a bunch of different helpers which can do different amounts of stuff and need different git-side help.

> Your 'list' allows to list what can become branches on the git end, so as
> long as you have tracking information on the git side, find-new-branches
> seems unnecessary, for examle.

I was unclear about what that one meant, I guess. It's supposed to handle systems where it's possible to create things that work like branches but are hard to find. If a helper doesn't report the capability, then the user may need to say, "I also want some/branch/location", or the helper wouldn't know that's a branch.

For example, in order to find all of the branches in Perforce, you have to figure out both directions of integration, in order to find branches that haven't been integrated back into the location you know about, and that's kind of expensive to determine and a bit tricky to parse.

It's not intended to indicate that the helper will tell you which ones are new or not, just that, if the helper doesn't have it, it will only tell you about branches that it's been told about.

Show 5 quoted lines

> What does 'merge' mean?  It cannot mean to fetch their changes from
> foreign vcs and create a merge locally (you only need import from the
> backend, and merge will be a usual git merge).  Perhaps you meant if you
> can export a merge back?  Some foreign systems may support importing
> merges but not octopus, so it may not be just a black-or-white boolean.

If the helper supports "merge", it means that it is able to create a merge in the foreign vcs. You're right about needing to separately list an "octopus" capability, and maybe further refinements.

> Also "the default is to rebase instead of merge" is probably too weak.
> You would want to error out if the user tried to merge, wouldn't you?

I think users should be able to create merges in git if they want to, but they'd have to linearize the history in order to push it back. Hopefully, at some point, we'll have a tool to help with this (which would be helpful for generating patch series, anyway).

Show 12 quoted lines

> > +'list'::
> > +	Takes the remote name, and outputs the names of refs. These
> > +	may be followed, after a single space, by "changed" or
> > +	"unchanged", indicating whether the foreign repository has
> > +	changed from the state in the ref. If the helper doesn't know,
> > +	it doesn't have to provide a value. (In particular, it
> > +	shouldn't do expensive operations, such as importing the
> > +	content, to see whether it matches.)
> 
> I am guessing by 'a value' you mean 'changed/unchanged', iow you are not
> saying if the helper does not know it can omit such refs from the list,
> but it is unclear.

Yeah, that's what I meant.

	-Daniel
*This .sig left intentionally blank*

Johannes Sixt· Jan 12, 2009, 08:13 UTC · re: Daniel Barkalow · lore

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

Daniel Barkalow schrieb:

Show 7 quoted lines

> +NAME
> +----
> +git-vcs-* - Helper programs for interoperation with foreign systems
> +
> +SYNOPSIS
> +--------
> +'git vcs-<system>' <command> [options] [arguments]

The current rule is that helper programs have double-dash in the name: git-rebase--interactive, git-web--browse, etc.

-- Hannes

Daniel Barkalow· Jan 12, 2009, 17:46 UTC · re: Johannes Sixt · lore

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

On Mon, 12 Jan 2009, Johannes Sixt wrote:

Show 11 quoted lines

> Daniel Barkalow schrieb:
> > +NAME
> > +----
> > +git-vcs-* - Helper programs for interoperation with foreign systems
> > +
> > +SYNOPSIS
> > +--------
> > +'git vcs-<system>' <command> [options] [arguments]
> 
> The current rule is that helper programs have double-dash in the name:
> git-rebase--interactive, git-web--browse, etc.

I'm not entirely sure if these are helper programs in that sense, or really what the distinction is between these and regular plumbing. Multiple programs will be calling them, and they'll have a specified API, in any case. (I think the double-dash programs are ones where all of the call sites are internal, so both the API and all the users can be changed together.)

	-Daniel
*This .sig left intentionally blank*