# [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions 4 messages from 2023-04-12 to 2023-04-13. Participants: Markus Heidelberg, Elijah Newren, Junio C Hamano. Thread: https://gitlist.dev/t/59584 ## Markus Heidelberg, 2023-04-12 13:41 Subject: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions Message-ID: <20230412134119.28257-1-markus.heidelberg@web.de> URL: https://gitlist.dev/e/20230412134119.28257-1-markus.heidelberg%40web.de ``` SVN revision numbers from svn:externals property, which are a multiple of 1024 (2^10), are transformed by SubGit to contain a binary suffix ("k", "m" and "g" have been checked) in .gitsvnextmodules file. These aren't valid revision numbers in SVN either. Examples: 1024 -> 1k 2048 -> 2k 1048576 -> 1m 1049600 -> 1025k 1073741824 -> 1g This led to the following error: svn_rev = int(parsed_config[section]['revision']) ValueError: invalid literal for int() with base 10: '1k' Signed-off-by: Markus Heidelberg --- contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals index 0c81507..39ff288 100644 --- a/contrib/filter-repo-demos/convert-svnexternals +++ b/contrib/filter-repo-demos/convert-svnexternals @@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url): return True, svnext_url +def parse_revision_value(value): + """ + Parse the value of key 'revision' from a .gitsvnextmodules file and return it + as integer. + + Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit + instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ... + """ + suffix = value[-1] + if suffix in "kmg": + mult = {"k": 1024, "m": 1024**2, "g": 1024**3} + return int(value[0:-1]) * mult[suffix] + else: + return int(value) + def add_submodule_tree_entry(commit, parsed_config, section): """ Add a submodule entry to the tree of a Git commit. @@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section): # Get SVN revision if parsed_config.has_option(section, 'revision'): - svn_rev = int(parsed_config[section]['revision']) + svn_rev = parse_revision_value(parsed_config[section]['revision']) else: # TODO: revision has to be guessed according to commit timestamp, skip for now return False -- 2.40.0 ``` ## Elijah Newren, 2023-04-12 15:45 Subject: Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions Message-ID: URL: https://gitlist.dev/e/CABPp-BH1172tfZn5i0PvNm6Fw5OObKEKCqT1gjTr%2Bhux86TexQ%40mail.gmail.com In-Reply-To: <20230412134119.28257-1-markus.heidelberg@web.de> ``` On Wed, Apr 12, 2023 at 6:42 AM Markus Heidelberg wrote: > > SVN revision numbers from svn:externals property, which are a multiple > of 1024 (2^10), are transformed by SubGit to contain a binary suffix > ("k", "m" and "g" have been checked) in .gitsvnextmodules file. > These aren't valid revision numbers in SVN either. > > Examples: > 1024 -> 1k > 2048 -> 2k > 1048576 -> 1m > 1049600 -> 1025k > 1073741824 -> 1g > > This led to the following error: > svn_rev = int(parsed_config[section]['revision']) > ValueError: invalid literal for int() with base 10: '1k' > > Signed-off-by: Markus Heidelberg > --- > contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++- > 1 file changed, 16 insertions(+), 1 deletion(-) > > diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals > index 0c81507..39ff288 100644 > --- a/contrib/filter-repo-demos/convert-svnexternals > +++ b/contrib/filter-repo-demos/convert-svnexternals > @@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url): > > return True, svnext_url > > +def parse_revision_value(value): > + """ > + Parse the value of key 'revision' from a .gitsvnextmodules file and return it > + as integer. > + > + Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit > + instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ... > + """ > + suffix = value[-1] > + if suffix in "kmg": > + mult = {"k": 1024, "m": 1024**2, "g": 1024**3} > + return int(value[0:-1]) * mult[suffix] > + else: > + return int(value) > + > def add_submodule_tree_entry(commit, parsed_config, section): > """ > Add a submodule entry to the tree of a Git commit. > @@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section): > > # Get SVN revision > if parsed_config.has_option(section, 'revision'): > - svn_rev = int(parsed_config[section]['revision']) > + svn_rev = parse_revision_value(parsed_config[section]['revision']) > else: > # TODO: revision has to be guessed according to commit timestamp, skip for now > return False > -- > 2.40.0 Thanks for sending this in! Applied. ``` ## Junio C Hamano, 2023-04-12 19:26 Subject: Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions Message-ID: URL: https://gitlist.dev/e/xmqqbkjs9a1j.fsf%40gitster.g In-Reply-To: ``` Elijah Newren writes: > Thanks for sending this in! Applied. It may work around the issue, but sounds like it is SubGit that needs fixing, at least seeing the problem description from the sideline. ``` ## Markus Heidelberg, 2023-04-13 13:59 Subject: Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions Message-ID: <12097307.TzdMVdA1NY@evo> URL: https://gitlist.dev/e/12097307.TzdMVdA1NY%40evo In-Reply-To: ``` Am Mittwoch, 12. April 2023, 21:26:32 CEST schrieb Junio C Hamano: > It may work around the issue, but sounds like it is SubGit that > needs fixing, at least seeing the problem description from the > sideline. Indeed the commit log should have been something like "work around wrongly transformed SVN revisions" to make it clear that it wasn't a bug in the script. Thanks to your comment I have now reported this SubGit issue in the TMate Support Forum. ```