{"thread":{"id":"59584","subject":"[filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions","startedAt":"2023-04-12T13:42:50Z","lastAt":"2023-04-13T14:00:21Z","messageCount":4,"participants":["Markus Heidelberg","Elijah Newren","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"475216","messageId":"20230412134119.28257-1-markus.heidelberg@web.de","threadId":"59584","inReplyTo":null,"subject":"[filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions","fromName":"Markus Heidelberg","fromEmail":"markus.heidelberg@web.de","sentAt":"2023-04-12T13:41:19Z","receivedAt":"2023-04-12T13:42:50Z","isPatch":true,"sender":{"key":"markus.heidelberg@web.de","avatar":"https://avatars.githubusercontent.com/u/6334512?v=4"},"body":"SVN revision numbers from svn:externals property, which are a multiple\nof 1024 (2^10), are transformed by SubGit to contain a binary suffix\n(\"k\", \"m\" and \"g\" have been checked) in .gitsvnextmodules file.\nThese aren't valid revision numbers in SVN either.\n\nExamples:\n 1024 -> 1k\n 2048 -> 2k\n 1048576 -> 1m\n 1049600 -> 1025k\n 1073741824 -> 1g\n\nThis led to the following error:\n svn_rev = int(parsed_config[section]['revision'])\nValueError: invalid literal for int() with base 10: '1k'\n\nSigned-off-by: Markus Heidelberg \n---\n contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-\n 1 file changed, 16 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals\nindex 0c81507..39ff288 100644\n--- a/contrib/filter-repo-demos/convert-svnexternals\n+++ b/contrib/filter-repo-demos/convert-svnexternals\n@@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):\n\n return True, svnext_url\n\n+def parse_revision_value(value):\n+ \"\"\"\n+ Parse the value of key 'revision' from a .gitsvnextmodules file and return it\n+ as integer.\n+\n+ Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit\n+ instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...\n+ \"\"\"\n+ suffix = value[-1]\n+ if suffix in \"kmg\":\n+ mult = {\"k\": 1024, \"m\": 1024**2, \"g\": 1024**3}\n+ return int(value[0:-1]) * mult[suffix]\n+ else:\n+ return int(value)\n+\n def add_submodule_tree_entry(commit, parsed_config, section):\n \"\"\"\n Add a submodule entry to the tree of a Git commit.\n@@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):\n\n # Get SVN revision\n if parsed_config.has_option(section, 'revision'):\n- svn_rev = int(parsed_config[section]['revision'])\n+ svn_rev = parse_revision_value(parsed_config[section]['revision'])\n else:\n # TODO: revision has to be guessed according to commit timestamp, skip for now\n return False\n--\n2.40.0\n\n"},{"id":"475218","messageId":"CABPp-BH1172tfZn5i0PvNm6Fw5OObKEKCqT1gjTr+hux86TexQ@mail.gmail.com","threadId":"59584","inReplyTo":"20230412134119.28257-1-markus.heidelberg@web.de","subject":"Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-04-12T15:45:04Z","receivedAt":"2023-04-12T15:45:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Apr 12, 2023 at 6:42 AM Markus Heidelberg\n wrote:\n>\n> SVN revision numbers from svn:externals property, which are a multiple\n> of 1024 (2^10), are transformed by SubGit to contain a binary suffix\n> (\"k\", \"m\" and \"g\" have been checked) in .gitsvnextmodules file.\n> These aren't valid revision numbers in SVN either.\n>\n> Examples:\n> 1024 -> 1k\n> 2048 -> 2k\n> 1048576 -> 1m\n> 1049600 -> 1025k\n> 1073741824 -> 1g\n>\n> This led to the following error:\n> svn_rev = int(parsed_config[section]['revision'])\n> ValueError: invalid literal for int() with base 10: '1k'\n>\n> Signed-off-by: Markus Heidelberg \n> ---\n> contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-\n> 1 file changed, 16 insertions(+), 1 deletion(-)\n>\n> diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals\n> index 0c81507..39ff288 100644\n> --- a/contrib/filter-repo-demos/convert-svnexternals\n> +++ b/contrib/filter-repo-demos/convert-svnexternals\n> @@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):\n>\n> return True, svnext_url\n>\n> +def parse_revision_value(value):\n> + \"\"\"\n> + Parse the value of key 'revision' from a .gitsvnextmodules file and return it\n> + as integer.\n> +\n> + Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit\n> + instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...\n> + \"\"\"\n> + suffix = value[-1]\n> + if suffix in \"kmg\":\n> + mult = {\"k\": 1024, \"m\": 1024**2, \"g\": 1024**3}\n> + return int(value[0:-1]) * mult[suffix]\n> + else:\n> + return int(value)\n> +\n> def add_submodule_tree_entry(commit, parsed_config, section):\n> \"\"\"\n> Add a submodule entry to the tree of a Git commit.\n> @@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):\n>\n> # Get SVN revision\n> if parsed_config.has_option(section, 'revision'):\n> - svn_rev = int(parsed_config[section]['revision'])\n> + svn_rev = parse_revision_value(parsed_config[section]['revision'])\n> else:\n> # TODO: revision has to be guessed according to commit timestamp, skip for now\n> return False\n> --\n> 2.40.0\n\nThanks for sending this in! Applied.\n"},{"id":"475226","messageId":"xmqqbkjs9a1j.fsf@gitster.g","threadId":"59584","inReplyTo":"CABPp-BH1172tfZn5i0PvNm6Fw5OObKEKCqT1gjTr+hux86TexQ@mail.gmail.com","subject":"Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-04-12T19:26:32Z","receivedAt":"2023-04-12T19:26:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> Thanks for sending this in! Applied.\n\nIt may work around the issue, but sounds like it is SubGit that\nneeds fixing, at least seeing the problem description from the\nsideline.\n"},{"id":"475303","messageId":"12097307.TzdMVdA1NY@evo","threadId":"59584","inReplyTo":"xmqqbkjs9a1j.fsf@gitster.g","subject":"Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions","fromName":"Markus Heidelberg","fromEmail":"markus.heidelberg@web.de","sentAt":"2023-04-13T13:59:53Z","receivedAt":"2023-04-13T14:00:21Z","isPatch":true,"sender":{"key":"markus.heidelberg@web.de","avatar":"https://avatars.githubusercontent.com/u/6334512?v=4"},"body":"Am Mittwoch, 12. April 2023, 21:26:32 CEST schrieb Junio C Hamano:\n\n> It may work around the issue, but sounds like it is SubGit that\n> needs fixing, at least seeing the problem description from the\n> sideline.\n\nIndeed the commit log should have been something like\n \"work around wrongly transformed SVN revisions\"\nto make it clear that it wasn't a bug in the script.\n\nThanks to your comment I have now reported this SubGit issue in the\nTMate Support Forum.\n\n\n\n"}]}