threads / discuss / 44601

[PATCHv1 0/2] git-p4 patches

Subject: [PATCHv1 0/2] git-p4 patches

## tl;dr

9 messages between Dec 2, 2016 and Dec 5, 2016.

replies: 8people: 2as markdown or json

Luke Diamand· Dec 2, 2016, 22:43 UTC · lore
This is a couple of small patches for git-p4.

The first just teaches git-p4 about git workspaces, so that you can do "git p4 submit" from within a workspace (P.S. workspaces are completely *awesome*).

The second follows on from the work by Vinicius for shelving changelists, by letting you update an existing shelved changelist.

This is a bit like using "git commit --amend" only far more painful....
Luke Diamand (2):
  git-p4: support git-workspaces
  git-p4: support updating an existing shelved changelist
 Documentation/git-p4.txt |  4 ++++
 git-p4.py                | 39 +++++++++++++++++++++++++++++++++++----
 t/t9807-git-p4-submit.sh | 38 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 77 insertions(+), 4 deletions(-)
-- 
2.11.0.274.g0ea315c
Luke Diamand· Dec 2, 2016, 22:43 UTC · re: Luke Diamand · lore

[PATCHv1 1/2] git-p4: support git-workspaces

Teach git-p4 about git-workspaces.
Signed-off-by: Luke Diamand <luke@diamand.org>
---
 git-p4.py | 6 ++++++
 1 file changed, 6 insertions(+)
diff --git a/git-p4.py b/git-p4.py
index 0c4f2afd2..5e2db1919 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -566,6 +566,12 @@ def isValidGitDir(path):
     if (os.path.exists(path + "/HEAD")
         and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
         return True;
+
+    # git workspace directory?
+    if (os.path.exists(path + "/HEAD")
+        and os.path.exists(path + "/gitdir")):
+        return True
+
     return False
 
 def parseRevision(ref):
-- 
2.11.0.274.g0ea315c
Junio C Hamano· Dec 5, 2016, 20:53 UTC · re: Luke Diamand · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

Luke Diamand <luke@diamand.org> writes:
> Teach git-p4 about git-workspaces.
Is this what we call "git worktree", or something else?
Show 23 quoted lines
>
> Signed-off-by: Luke Diamand <luke@diamand.org>
> ---
>  git-p4.py | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/git-p4.py b/git-p4.py
> index 0c4f2afd2..5e2db1919 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -566,6 +566,12 @@ def isValidGitDir(path):
>      if (os.path.exists(path + "/HEAD")
>          and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
>          return True;
> +
> +    # git workspace directory?
> +    if (os.path.exists(path + "/HEAD")
> +        and os.path.exists(path + "/gitdir")):
> +        return True
> +
>      return False
>  
>  def parseRevision(ref):
Luke Diamand· Dec 5, 2016, 21:13 UTC · re: Junio C Hamano · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

On 5 December 2016 at 20:53, Junio C Hamano <gitster@pobox.com> wrote:
Show 5 quoted lines
> Luke Diamand <luke@diamand.org> writes:
>
>> Teach git-p4 about git-workspaces.
>
> Is this what we call "git worktree", or something else?
Ah, I think you're right!
Show 24 quoted lines
>
>>
>> Signed-off-by: Luke Diamand <luke@diamand.org>
>> ---
>>  git-p4.py | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/git-p4.py b/git-p4.py
>> index 0c4f2afd2..5e2db1919 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -566,6 +566,12 @@ def isValidGitDir(path):
>>      if (os.path.exists(path + "/HEAD")
>>          and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
>>          return True;
>> +
>> +    # git workspace directory?
>> +    if (os.path.exists(path + "/HEAD")
>> +        and os.path.exists(path + "/gitdir")):
>> +        return True
>> +
>>      return False
>>
>>  def parseRevision(ref):
Junio C Hamano· Dec 5, 2016, 21:24 UTC · re: Luke Diamand · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

Luke Diamand <luke@diamand.org> writes:
Show 8 quoted lines
> On 5 December 2016 at 20:53, Junio C Hamano <gitster@pobox.com> wrote:
>> Luke Diamand <luke@diamand.org> writes:
>>
>>> Teach git-p4 about git-workspaces.
>>
>> Is this what we call "git worktree", or something else?
>
> Ah, I think you're right!
Then I'll queue it like the attached.
HOWEVER.

How fast does isValidGitDir() function need to be? The primary one seems to check HEAD (but it does not notice a non-repository that has a directory with that name), refs and objects (but it does not notice a non-repository that has a non-directory with these names), and this new one uses a test that is even more sloppy.

What I am trying to get at is if we want to use a single command that can be given a path and answer "Yes, that is a repository" here, and that single command should know how the repository should look like. I offhand do not know we already have such a command we can use, e.g. "git rev-parse --is-git-dir $path", but if there isn't perhaps we would want one, so that not just "git p4" but other scripted Porcelains can make use of it?

-- >8 --
From: Luke Diamand <luke@diamand.org>
Date: Fri, 2 Dec 2016 22:43:18 +0000
Subject: [PATCH] git-p4: support secondary working trees managed by "git worktree"
Teach git-p4 about them.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 git-p4.py | 6 ++++++
 1 file changed, 6 insertions(+)
diff --git a/git-p4.py b/git-p4.py
index fd5ca52462..b3c50ae7e5 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -566,6 +566,12 @@ def isValidGitDir(path):
     if (os.path.exists(path + "/HEAD")
         and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
         return True;
+
+    # secondary working tree managed by "git worktree"?
+    if (os.path.exists(path + "/HEAD")
+        and os.path.exists(path + "/gitdir")):
+        return True
+
     return False
 
 def parseRevision(ref):
-- 
2.11.0-222-g22b1346184
Junio C Hamano· Dec 5, 2016, 21:54 UTC · re: Junio C Hamano · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

Junio C Hamano <gitster@pobox.com> writes:
Show 12 quoted lines
> Luke Diamand <luke@diamand.org> writes:
> ...
>      if (os.path.exists(path + "/HEAD")
>          and os.path.exists(path + "/refs") and os.path.exists(path + "/objects")):
>          return True;
> +
> +    # secondary working tree managed by "git worktree"?
> +    if (os.path.exists(path + "/HEAD")
> +        and os.path.exists(path + "/gitdir")):
> +        return True
> +
>      return False
Wait a bit.  How is this expected to work?

I assume "path" is something like "$somewhere/.git" (either absolute or relative), and by concatenating HEAD, refs and objects under it the original checks form paths to expected directories, because "$somewhere/.git" is a directory.

A worktree created with "git worktree add" gets ".git" that is a file, and the contents of the file records the path to a subdirectory "worktrees/$name" of the primary repository the worktree is borrowing from, i.e. "$somewhere/.git/worktrees/$name".

So for this patch to work correctly, the caller, when in a "git worktree" managed secondary working tree, MUST have found the ".git" file at the root level of the working tree, MUST have read its contents to learn that "$somewhere/.git/worktrees/$name" path, and then feeding it to this function. But doesn't such a caller already know that it is a valid repository? After all, it trusted the contents of that ".git" file at the root level.

    ... goes and looks at the callers ...

Between the two call sites of isValidGitDir() in main(), the first one (i.e. if ".git" in the current directory, made into abspath, does not look like a repository, ask "rev-parse --git-dir" for one) looks correct and I think it would grab the correct $GIT_DIR just fine [*1*]. Isn't the real problem the second one, i.e. the one that feeds a correct cmd.gitdir that we just obtained by asking "rev-parse --git-dir" to the sloppy isValidGitDir() again. The latter second guesses "rev-parse --git-dir" and botches.

I have a feeling that adding more to isValidGitDir() like this patch does is a step in the wrong direction. The first fix would be not to do the "if isValidGitDir() says no, then do something else" step if you already asked "rev-parse --git-dir" and got a valid $GIT_DIR, perhaps?

Stepping back even more.  

I wonder why the script needs to do os.environ.get("GIT_DIR") in the first place to initialize cmd.gitdir field. If it instead asked "rev-parse --gitdir", the script would get the right answer that already takes GIT_DIR environment into account.

[Footnote]
*1* This ends up asking "rev-parse --git-dir" only because
    isValidGitDir() is sloppy.  If you are in a secondary working
    tree whose ".git" is a file, the function would say "no", and
    that is the only thing that allows you to make a call to
    "rev-parse --git-dir" to obtain the correct result.
Luke Diamand· Dec 5, 2016, 21:55 UTC · re: Junio C Hamano · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

On 5 December 2016 at 21:24, Junio C Hamano <gitster@pobox.com> wrote:
Show 16 quoted lines
> Luke Diamand <luke@diamand.org> writes:
>
>> On 5 December 2016 at 20:53, Junio C Hamano <gitster@pobox.com> wrote:
>>> Luke Diamand <luke@diamand.org> writes:
>>>
>>>> Teach git-p4 about git-workspaces.
>>>
>>> Is this what we call "git worktree", or something else?
>>
>> Ah, I think you're right!
>
> Then I'll queue it like the attached.
>
> HOWEVER.
>
> How fast does isValidGitDir() function need to be?
It doesn't need to be fast.
Show 13 quoted lines
> The primary one
> seems to check HEAD (but it does not notice a non-repository that
> has a directory with that name), refs and objects (but it does not
> notice a non-repository that has a non-directory with these names),
> and this new one uses a test that is even more sloppy.
>
> What I am trying to get at is if we want to use a single command
> that can be given a path and answer "Yes, that is a repository"
> here, and that single command should know how the repository should
> look like.  I offhand do not know we already have such a command we
> can use, e.g. "git rev-parse --is-git-dir $path", but if there isn't
> perhaps we would want one, so that not just "git p4" but other
> scripted Porcelains can make use of it?

That would be nicer than random ad-hoc rules, certainly. I couldn't find any obvious combination that would work.

Luke
Junio C Hamano· Dec 5, 2016, 23:17 UTC · re: Luke Diamand · lore

Re: [PATCHv1 1/2] git-p4: support git-workspaces

Luke Diamand <luke@diamand.org> writes:
Show 10 quoted lines
>> What I am trying to get at is if we want to use a single command
>> that can be given a path and answer "Yes, that is a repository"
>> here, and that single command should know how the repository should
>> look like.  I offhand do not know we already have such a command we
>> can use, e.g. "git rev-parse --is-git-dir $path", but if there isn't
>> perhaps we would want one, so that not just "git p4" but other
>> scripted Porcelains can make use of it?
>
> That would be nicer than random ad-hoc rules, certainly. I couldn't
> find any obvious combination that would work.

I've already sent an update but my reading of this code is that there is no need for the program to be able to, given a random directory path, ask isValidGitDir() "is this a Git repository?" at all.

What the program wanted to know, IIUC, is "Where is the 'Git repository directory', if there is one?". This is needed primarily because the program wants to error out before doing any operation that requires a Git repository to work on, when the answer is "there is none".

It also wants to know it because (for whatever reason) it wants to export it in GIT_DIR [*1*].

And isValidGitDir() is a helper function used by the handcrafted logic in main() to answer that question, but as far as I can tell, you'd get a more correct answer by asking "rev-parse --git-dir" unconditionally (even when the user of the program exported GIT_DIR to you).

I just was reading Lars's LFS changes, but that one has hardcoded "Is there a '.git/lfs/objects/' directory directly inside the current working directory?" and similar, which I do not think would work well in a secondary working tree where ".git" is merely a file [*2*]. In a "secondary working tree"-aware world, I think you would need to ask locations of --git-dir and --git-common-dir to rev-parse upfront and choose which ones are ought to be shared entities across the family of worktrees and which ones are to be private per worktree. I suspect that LFS objects want to be shared across working trees that share the object store, for example, which would mean that "--git-dir" is not what Lars would want to use.

[Footnote]

*1* I do not think this is necessary. Git tools know how to find the repository by first checking GIT_DIR environment and if it is not set then by walking up the directory hierarchy just fine without the program exporting GIT_DIR for them.

*2* The part that bases this path relative to getcwd() is OK, as the start-up sequence in main() does a cdup to the top before everything else.

Luke Diamand· Dec 2, 2016, 22:43 UTC · re: Luke Diamand · lore

[PATCHv1 2/2] git-p4: support updating an existing shelved changelist

Adds new option "--update-shelve CHANGELIST" which updates an existing shelved changelist.

The original changelist must have been created by the current user.
This allows workflow something like:
   hack hack hack
   git commit
   git p4 submit --shelve
   $mail interested parties about shelved changelist
   make corrections
   git commit --amend
   git p4 submit --update-shelve $CHANGELIST
   $mail interested parties about shelved changelist
   etc
Signed-off-by: Luke Diamand <luke@diamand.org>
---
 Documentation/git-p4.txt |  4 ++++
 git-p4.py                | 33 +++++++++++++++++++++++++++++----
 t/t9807-git-p4-submit.sh | 38 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt
index 1bbf43d15..ce40b9a54 100644
--- a/Documentation/git-p4.txt
+++ b/Documentation/git-p4.txt
@@ -308,6 +308,10 @@ These options can be used to modify 'git p4 submit' behavior.
 	After creating each shelve, the relevant files are reverted/deleted.
 	If you have multiple commits pending multiple shelves will be created.
 
+--update-shelve CHANGELIST::
+	Update an existing shelved changelist with this commit. Implies
+	--shelve.
+
 --conflict=(ask|skip|quit)::
 	Conflicts can occur when applying a commit to p4.  When this
 	happens, the default behavior ("ask") is to prompt whether to
diff --git a/git-p4.py b/git-p4.py
index 5e2db1919..36242d384 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -262,6 +262,10 @@ def p4_revert(f):
 def p4_reopen(type, f):
     p4_system(["reopen", "-t", type, wildcard_encode(f)])
 
+def p4_reopen_in_change(changelist, files):
+    cmd = ["reopen", "-c", str(changelist)] + files
+    p4_system(cmd)
+
 def p4_move(src, dest):
     p4_system(["move", "-k", wildcard_encode(src), wildcard_encode(dest)])
 
@@ -1298,6 +1302,9 @@ class P4Submit(Command, P4UserMap):
                 optparse.make_option("--shelve", dest="shelve", action="store_true",
                                      help="Shelve instead of submit. Shelved files are reverted, "
                                      "restoring the workspace to the state before the shelve"),
+                optparse.make_option("--update-shelve", dest="update_shelve", action="store", type="int",
+                                     metavar="CHANGELIST",
+                                     help="update an existing shelved changelist, implies --shelve")
         ]
         self.description = "Submit changes from git to the perforce depot."
         self.usage += " [name of git branch to submit into perforce depot]"
@@ -1306,6 +1313,7 @@ class P4Submit(Command, P4UserMap):
         self.preserveUser = gitConfigBool("git-p4.preserveUser")
         self.dry_run = False
         self.shelve = False
+        self.update_shelve = None
         self.prepare_p4_only = False
         self.conflict_behavior = None
         self.isWindows = (platform.system() == "Windows")
@@ -1474,7 +1482,7 @@ class P4Submit(Command, P4UserMap):
                     return 1
         return 0
 
-    def prepareSubmitTemplate(self):
+    def prepareSubmitTemplate(self, changelist=None):
         """Run "p4 change -o" to grab a change specification template.
            This does not use "p4 -G", as it is nice to keep the submission
            template in original order, since a human might edit it.
@@ -1486,7 +1494,11 @@ class P4Submit(Command, P4UserMap):
 
         template = ""
         inFilesSection = False
-        for line in p4_read_pipe_lines(['change', '-o']):
+        args = ['change', '-o']
+        if changelist:
+            args.append(str(changelist))
+
+        for line in p4_read_pipe_lines(args):
             if line.endswith("\r\n"):
                 line = line[:-2] + "\n"
             if inFilesSection:
@@ -1585,11 +1597,14 @@ class P4Submit(Command, P4UserMap):
         editedFiles = set()
         pureRenameCopy = set()
         filesToChangeExecBit = {}
+        all_files = list()
 
         for line in diff:
             diff = parseDiffTreeEntry(line)
             modifier = diff['status']
             path = diff['src']
+            all_files.append(path)
+
             if modifier == "M":
                 p4_edit(path)
                 if isModeExecChanged(diff['src_mode'], diff['dst_mode']):
@@ -1715,6 +1730,10 @@ class P4Submit(Command, P4UserMap):
             mode = filesToChangeExecBit[f]
             setP4ExecBit(f, mode)
 
+        if self.update_shelve:
+            print("all_files = %s" % str(all_files))
+            p4_reopen_in_change(self.update_shelve, all_files)
+
         #
         # Build p4 change description, starting with the contents
         # of the git commit message.
@@ -1723,7 +1742,7 @@ class P4Submit(Command, P4UserMap):
         logMessage = logMessage.strip()
         (logMessage, jobs) = self.separate_jobs_from_description(logMessage)
 
-        template = self.prepareSubmitTemplate()
+        template = self.prepareSubmitTemplate(self.update_shelve)
         submitTemplate = self.prepareLogMessage(template, logMessage, jobs)
 
         if self.preserveUser:
@@ -1795,7 +1814,10 @@ class P4Submit(Command, P4UserMap):
                 if self.isWindows:
                     message = message.replace("\r\n", "\n")
                 submitTemplate = message[:message.index(separatorLine)]
-                if self.shelve:
+
+                if self.update_shelve:
+                    p4_write_pipe(['shelve', '-r', '-i'], submitTemplate)
+                elif self.shelve:
                     p4_write_pipe(['shelve', '-i'], submitTemplate)
                 else:
                     p4_write_pipe(['submit', '-i'], submitTemplate)
@@ -1921,6 +1943,9 @@ class P4Submit(Command, P4UserMap):
         if len(self.origin) == 0:
             self.origin = upstream
 
+        if self.update_shelve:
+            self.shelve = True
+
         if self.preserveUser:
             if not self.canChangeChangelists():
                 die("Cannot preserve user names without p4 super-user or admin permissions")
diff --git a/t/t9807-git-p4-submit.sh b/t/t9807-git-p4-submit.sh
index 42a5fada5..e37239e65 100755
--- a/t/t9807-git-p4-submit.sh
+++ b/t/t9807-git-p4-submit.sh
@@ -444,6 +444,44 @@ test_expect_success 'submit --shelve' '
 	)
 '
 
+# Update an existing shelved changelist
+
+test_expect_success 'submit --update-shelve' '
+	test_when_finished cleanup_git &&
+	git p4 clone --dest="$git" //depot &&
+	(
+		cd "$cli" &&
+		p4 revert ... &&
+		cd "$git" &&
+		git config git-p4.skipSubmitEdit true &&
+		test_commit "test-update-shelved-change" &&
+		git p4 submit --origin=HEAD^ --shelve &&
+
+		shelf_cl=$(p4 -G changes -s shelved -m 1 |\
+			marshal_dump change) &&
+		test -n $shelf_cl &&
+		echo "updating shelved change list $shelf_cl" &&
+
+		echo "updated-line" >>shelf.t &&
+		echo added-file.t >added-file.t &&
+		git add shelf.t added-file.t &&
+		git rm -f test-update-shelved-change.t &&
+		git commit --amend -C HEAD &&
+		git show --stat HEAD &&
+		git p4 submit -v --origin HEAD^ --update-shelve $shelf_cl &&
+		echo "done git p4 submit"
+	) &&
+	(
+		cd "$cli" &&
+		change=$(p4 -G changes -s shelved -m 1 //depot/... | \
+			 marshal_dump change) &&
+		p4 unshelve -c $change -s $change &&
+		grep -q updated-line shelf.t &&
+		p4 describe -S $change | grep added-file.t &&
+		test_path_is_missing test-update-shelved-change.t
+	)
+'
+
 test_expect_success 'kill p4d' '
 	kill_p4d
 '
-- 
2.11.0.274.g0ea315c

← back to recent threads