threads / patch / 59584

patchconvert-svnexternals: fix parsing of wrongly transformed SVN revisions

Subject: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

## tl;dr

4 messages between Apr 12, 2023 and Apr 13, 2023. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

Markus Heidelberg· Apr 12, 2023, 13:41 UTC · lore

SVN revision numbers from svn:externals property, which are a multiple of 1024 (2^10), are transformed by SubGit to contain a binary suffix ("k", "m" and "g" have been checked) in .gitsvnextmodules file. These aren't valid revision numbers in SVN either.

Examples:
  1024 -> 1k
  2048 -> 2k
  1048576 -> 1m
  1049600 -> 1025k
  1073741824 -> 1g
This led to the following error:
    svn_rev = int(parsed_config[section]['revision'])
ValueError: invalid literal for int() with base 10: '1k'
Signed-off-by: Markus Heidelberg <markus.heidelberg@web.de>
---
 contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
Show changes to contrib/filter-repo-demos/convert-svnexternals +16 −2
diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals
index 0c81507..39ff288 100644
--- a/contrib/filter-repo-demos/convert-svnexternals
+++ b/contrib/filter-repo-demos/convert-svnexternals
@@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):

   return True, svnext_url

+def parse_revision_value(value):
+  """
+  Parse the value of key 'revision' from a .gitsvnextmodules file and return it
+  as integer.
+
+  Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit
+  instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...
+  """
+  suffix = value[-1]
+  if suffix in "kmg":
+    mult = {"k": 1024, "m": 1024**2, "g": 1024**3}
+    return int(value[0:-1]) * mult[suffix]
+  else:
+    return int(value)
+
 def add_submodule_tree_entry(commit, parsed_config, section):
   """
   Add a submodule entry to the tree of a Git commit.
@@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):

   # Get SVN revision
   if parsed_config.has_option(section, 'revision'):
-    svn_rev = int(parsed_config[section]['revision'])
+    svn_rev = parse_revision_value(parsed_config[section]['revision'])
   else:
     # TODO: revision has to be guessed according to commit timestamp, skip for now
     return False
--
2.40.0
Elijah Newren· Apr 12, 2023, 15:45 UTC · re: Markus Heidelberg · lore

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

On Wed, Apr 12, 2023 at 6:42 AM Markus Heidelberg <markus.heidelberg@web.de> wrote:

Show 59 quoted lines
>
> SVN revision numbers from svn:externals property, which are a multiple
> of 1024 (2^10), are transformed by SubGit to contain a binary suffix
> ("k", "m" and "g" have been checked) in .gitsvnextmodules file.
> These aren't valid revision numbers in SVN either.
>
> Examples:
>   1024 -> 1k
>   2048 -> 2k
>   1048576 -> 1m
>   1049600 -> 1025k
>   1073741824 -> 1g
>
> This led to the following error:
>     svn_rev = int(parsed_config[section]['revision'])
> ValueError: invalid literal for int() with base 10: '1k'
>
> Signed-off-by: Markus Heidelberg <markus.heidelberg@web.de>
> ---
>  contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals
> index 0c81507..39ff288 100644
> --- a/contrib/filter-repo-demos/convert-svnexternals
> +++ b/contrib/filter-repo-demos/convert-svnexternals
> @@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):
>
>    return True, svnext_url
>
> +def parse_revision_value(value):
> +  """
> +  Parse the value of key 'revision' from a .gitsvnextmodules file and return it
> +  as integer.
> +
> +  Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit
> +  instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...
> +  """
> +  suffix = value[-1]
> +  if suffix in "kmg":
> +    mult = {"k": 1024, "m": 1024**2, "g": 1024**3}
> +    return int(value[0:-1]) * mult[suffix]
> +  else:
> +    return int(value)
> +
>  def add_submodule_tree_entry(commit, parsed_config, section):
>    """
>    Add a submodule entry to the tree of a Git commit.
> @@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):
>
>    # Get SVN revision
>    if parsed_config.has_option(section, 'revision'):
> -    svn_rev = int(parsed_config[section]['revision'])
> +    svn_rev = parse_revision_value(parsed_config[section]['revision'])
>    else:
>      # TODO: revision has to be guessed according to commit timestamp, skip for now
>      return False
> --
> 2.40.0
Thanks for sending this in!  Applied.
Junio C Hamano· Apr 12, 2023, 19:26 UTC · re: Elijah Newren · lore

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

Elijah Newren <newren@gmail.com> writes:
> Thanks for sending this in!  Applied.

It may work around the issue, but sounds like it is SubGit that needs fixing, at least seeing the problem description from the sideline.

Markus Heidelberg· Apr 13, 2023, 13:59 UTC · re: Junio C Hamano · lore

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

Am Mittwoch, 12. April 2023, 21:26:32 CEST schrieb Junio C Hamano:
> It may work around the issue, but sounds like it is SubGit that
> needs fixing, at least seeing the problem description from the
> sideline.
Indeed the commit log should have been something like
  "work around wrongly transformed SVN revisions"
to make it clear that it wasn't a bug in the script.

Thanks to your comment I have now reported this SubGit issue in the TMate Support Forum.

← back to recent threads