{"thread":{"id":"55637","subject":"[PATCH] tr2: log parent process name","startedAt":"2021-05-07T00:29:17Z","lastAt":"2021-08-31T00:17:58Z","messageCount":87,"participants":["Emily Shaffer","Bagas Sanjaya","Ævar Arnfjörð Bjarmason","Jeff Hostetler","Junio C Hamano","Randall S. Becker","Eric Sunshine","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"423832","messageId":"20210507002908.1495061-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":null,"subject":"[PATCH] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-07T00:29:08Z","receivedAt":"2021-05-07T00:29:17Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? Knowing where the Git invocation came\nfrom can help with debugging to isolate where the problem came from.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient regardless of platform.\n\nSigned-off-by: Emily Shaffer \n---\nWe briefly discussed hiding this behind a config, internally. However, I\nwanted to include the parent name alongside the cmd_start event, which\nhappens very early (maybe before config gathering?).\n\nMaybe it's better to log the parent_name as its own event, since it\nshouldn't change over the lifetime of the process?\n\nprocfs is very non-portable, though - I think this won't even work on\nMacOS. So I'm curious if anybody has better suggestions for how to do\nthis.\n\n - Emily\n\n Makefile | 1 +\n compat/procinfo.c | 19 +++++++++++++++++++\n git-compat-util.h | 6 ++++++\n t/t0212-trace2-event.sh | 8 ++++++++\n t/t0212/parse_events.perl | 1 +\n trace2/tr2_tgt_event.c | 3 +++\n 6 files changed, 38 insertions(+)\n create mode 100644 compat/procinfo.c\n\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..19f5189c6f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -855,6 +855,7 @@ LIB_OBJS += commit-graph.o\n LIB_OBJS += commit-reach.o\n LIB_OBJS += commit.o\n LIB_OBJS += compat/obstack.o\n+LIB_OBJS += compat/procinfo.o\n LIB_OBJS += compat/terminal.o\n LIB_OBJS += config.o\n LIB_OBJS += connect.o\ndiff --git a/compat/procinfo.c b/compat/procinfo.c\nnew file mode 100644\nindex 0000000000..7f4c8dd284\n--- /dev/null\n+++ b/compat/procinfo.c\n@@ -0,0 +1,19 @@\n+#include \"git-compat-util.h\"\n+\n+#include \"strbuf.h\"\n+\n+char *get_process_name(int pid)\n+{\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf out = STRBUF_INIT;\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/cmdline\", pid);\n+\tif (strbuf_read_file(&out, procfs_path.buf, 0) > 0)\n+\t{\n+\t\tstrbuf_release(&procfs_path);\n+\t\treturn strbuf_detach(&out, NULL);\n+\t}\n+\n+\t/* NEEDSWORK: add non-procfs implementations here. */\n+\treturn NULL;\n+}\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex a508dbe5a3..cc7d5d8a2a 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1382,4 +1382,10 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)\n \n void sleep_millisec(int millisec);\n \n+/*\n+ * Convert PID to process name (as would show in top/task manager). Returns\n+ * NULL if unimplemented - be sure to check for NULL at callsite.\n+ */\n+char *get_process_name(int pid);\n+\n #endif\ndiff --git a/t/t0212-trace2-event.sh b/t/t0212-trace2-event.sh\nindex 1529155cf0..3a2a8a5b5f 100755\n--- a/t/t0212-trace2-event.sh\n+++ b/t/t0212-trace2-event.sh\n@@ -61,6 +61,7 @@ test_expect_success JSON_PP 'event stream, error event' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\n@@ -115,6 +116,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| },\n \t| \"_SID0_/_SID1_\":{\n@@ -143,6 +145,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2/trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"test-tool\",\n \t| \"version\":\"$V\"\n \t| },\n \t| \"_SID0_/_SID1_/_SID2_\":{\n@@ -155,6 +158,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2/trace2/trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"$TEST_DIRECTORY/../t/helper//test-tool\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\n@@ -192,6 +196,7 @@ test_expect_success JSON_PP 'event stream, list config' '\n \t| \"value\":\"hello world\"\n \t| }\n \t| ],\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\n@@ -229,6 +234,7 @@ test_expect_success JSON_PP 'event stream, list env vars' '\n \t| \"value\":\"hello world\"\n \t| }\n \t| ],\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\n@@ -263,6 +269,7 @@ test_expect_success JSON_PP 'basic trace2_data' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\n@@ -295,6 +302,7 @@ test_expect_success JSON_PP 'using global config, event stream, error event' '\n \t| \"exit_code\":0,\n \t| \"hierarchy\":\"trace2\",\n \t| \"name\":\"trace2\",\n+\t| \"parent_name\":\"/bin/sh\",\n \t| \"version\":\"$V\"\n \t| }\n \t|};\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..dd8b1be844 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -99,6 +99,7 @@\n }\n \n elsif ($event eq 'start') {\n+\t$processes->{$sid}->{'parent_name'} = $line->{'parent_name'};\n \t$processes->{$sid}->{'argv'} = $line->{'argv'};\n \t$processes->{$sid}->{'argv'}[0] = \"_EXE_\";\n }\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..d258d5807c 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -145,10 +145,12 @@ static void fn_start_fl(const char *file, int line,\n \tconst char *event_name = \"start\";\n \tstruct json_writer jw = JSON_WRITER_INIT;\n \tdouble t_abs = (double)us_elapsed_absolute / 1000000.0;\n+\tchar *parent_name = get_process_name(getppid());\n \n \tjw_object_begin(&jw, 0);\n \tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n \tjw_object_double(&jw, \"t_abs\", 6, t_abs);\n+\tjw_object_string(&jw, \"parent_name\", parent_name ? parent_name : NULL );\n \tjw_object_inline_begin_array(&jw, \"argv\");\n \tjw_array_argv(&jw, argv);\n \tjw_end(&jw);\n@@ -156,6 +158,7 @@ static void fn_start_fl(const char *file, int line,\n \n \ttr2_dst_write_line(&tr2dst_event, &jw.json);\n \tjw_release(&jw);\n+\tfree(parent_name);\n }\n \n static void fn_exit_fl(const char *file, int line, uint64_t us_elapsed_absolute,\n-- \n2.31.1.607.g51e8a6a459-goog\n\n"},{"id":"423839","messageId":"3bde96f5-2df6-4c4d-7e40-e938bbb31643@gmail.com","threadId":"55637","inReplyTo":"20210507002908.1495061-1-emilyshaffer@google.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-05-07T03:25:56Z","receivedAt":"2021-05-07T03:26:02Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 07/05/21 07.29, Emily Shaffer wrote:\n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? Knowing where the Git invocation came\n> from can help with debugging to isolate where the problem came from.\n> \n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient regardless of platform.\n\nWhat about on Windows?\n\n> Signed-off-by: Emily Shaffer \n> ---\n> We briefly discussed hiding this behind a config, internally. However, I\n> wanted to include the parent name alongside the cmd_start event, which\n> happens very early (maybe before config gathering?).\n> \n> Maybe it's better to log the parent_name as its own event, since it\n> shouldn't change over the lifetime of the process?\n> \n> procfs is very non-portable, though - I think this won't even work on\n> MacOS. So I'm curious if anybody has better suggestions for how to do\n> this.\n\nMaybe we can say that \"currently the method of gathering parent process\nname that Git uses only work on Linux\".\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"423877","messageId":"YJV0YibkFYUe2UcA@google.com","threadId":"55637","inReplyTo":"20210507002908.1495061-1-emilyshaffer@google.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-07T17:09:54Z","receivedAt":"2021-05-07T17:10:05Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, May 06, 2021 at 05:29:08PM -0700, Emily Shaffer wrote:\n> \n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? Knowing where the Git invocation came\n> from can help with debugging to isolate where the problem came from.\n> \n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient regardless of platform.\n> \n> Signed-off-by: Emily Shaffer \n> ---\n> We briefly discussed hiding this behind a config, internally. However, I\n> wanted to include the parent name alongside the cmd_start event, which\n> happens very early (maybe before config gathering?).\n> \n> Maybe it's better to log the parent_name as its own event, since it\n> shouldn't change over the lifetime of the process?\n> \n> procfs is very non-portable, though - I think this won't even work on\n> MacOS. So I'm curious if anybody has better suggestions for how to do\n> this.\n\nI wrote this and then I wrote nonportable tests anyways, bah. Working on\na fix now - the tests break for MacOS and Windows.\n\nThe parent_name needs to be set conditionally below.\n\n> diff --git a/t/t0212-trace2-event.sh b/t/t0212-trace2-event.sh\n> index 1529155cf0..3a2a8a5b5f 100755\n> --- a/t/t0212-trace2-event.sh\n> +++ b/t/t0212-trace2-event.sh\n> @@ -61,6 +61,7 @@ test_expect_success JSON_PP 'event stream, error event' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n> @@ -115,6 +116,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| },\n> \t| \"_SID0_/_SID1_\":{\n> @@ -143,6 +145,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2/trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"test-tool\",\n> \t| \"version\":\"$V\"\n> \t| },\n> \t| \"_SID0_/_SID1_/_SID2_\":{\n> @@ -155,6 +158,7 @@ test_expect_success JSON_PP 'event stream, return code 0' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2/trace2/trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"$TEST_DIRECTORY/../t/helper//test-tool\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n> @@ -192,6 +196,7 @@ test_expect_success JSON_PP 'event stream, list config' '\n> \t| \"value\":\"hello world\"\n> \t| }\n> \t| ],\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n> @@ -229,6 +234,7 @@ test_expect_success JSON_PP 'event stream, list env vars' '\n> \t| \"value\":\"hello world\"\n> \t| }\n> \t| ],\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n> @@ -263,6 +269,7 @@ test_expect_success JSON_PP 'basic trace2_data' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n> @@ -295,6 +302,7 @@ test_expect_success JSON_PP 'using global config, event stream, error event' '\n> \t| \"exit_code\":0,\n> \t| \"hierarchy\":\"trace2\",\n> \t| \"name\":\"trace2\",\n> +\t| \"parent_name\":\"/bin/sh\",\n> \t| \"version\":\"$V\"\n> \t| }\n> \t|};\n\n\n - Emily\n"},{"id":"424030","messageId":"87im3qu4gy.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"20210507002908.1495061-1-emilyshaffer@google.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-05-10T12:29:15Z","receivedAt":"2021-05-10T13:41:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, May 06 2021, Emily Shaffer wrote:\n\n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? Knowing where the Git invocation came\n> from can help with debugging to isolate where the problem came from.\n\nAside from the portability concerns others have raised, I don't really\nsee why you'd need this.\n\nWe already have the nest-level as part of the SID, so isn't it\nsufficient (and portable) at the top-level to log what isatty says + set\nthe initial SID \"root\" in the IDE (which presumably knows about git).\n\nWouldn't this log passwords in cases of e.g.:\n\n some-script --git-password secret # invokes \"git\"\n\nIn older versions of linux reading e.g. smaps from /proc/self would\nstall the kernel while the read was happening, I haven't checked whether\ncmdline is such a thing (probably not), but it's a subtle thing to have\nin mind for this / follow-ups if it's made portable (is that an issue on\nother OS's?).\n\nAll that being said I've got nothing fundamentally against this.\n"},{"id":"424146","messageId":"1cae0717-745b-5e3a-fadc-3966da00d80c@jeffhostetler.com","threadId":"55637","inReplyTo":"20210507002908.1495061-1-emilyshaffer@google.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-05-11T17:28:39Z","receivedAt":"2021-05-11T17:28:41Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 5/6/21 8:29 PM, Emily Shaffer wrote:\n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? Knowing where the Git invocation came\n> from can help with debugging to isolate where the problem came from.\n> \n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient regardless of platform.\n> \n> Signed-off-by: Emily Shaffer \n> ---\n> We briefly discussed hiding this behind a config, internally. However, I\n> wanted to include the parent name alongside the cmd_start event, which\n> happens very early (maybe before config gathering?).\n> \n> Maybe it's better to log the parent_name as its own event, since it\n> shouldn't change over the lifetime of the process?\n> \n> procfs is very non-portable, though - I think this won't even work on\n> MacOS. So I'm curious if anybody has better suggestions for how to do\n> this.\n> \n> - Emily\n\n\nLook at `trace2_collect_process_info()` in `trace2.h` and\n`compat/win32/trace2_win32_process_info.c`.\n\nThat function is designed to let the process call out to\nplatform-specific code to do things like getting the name\nof the invoking process.\n\nIt is called with an enum to indicate when/why it is being\ncalled.\n\nOn Windows, I have platform code to get the process ancestry,\nthe peak VM usage, and whether the process is being debugged.\nTwo of those were easy.... And all are very platform-specific.\n\nThey all generate events of the form:\n\n\ttrace2_data_*(\"process\", \"windows/\", ...)\n\nSome of my `t021[012]/` helper scripts exclude \"process\" category\nmessages from the output to make the t*.sh tests portable.\n\n\nTo implement a /proc lookup as you have suggested, I would\nfixup the Makefile or config.mak.uname to include something\nlike a \"HAVE_...\" feature and then use that at the\nbottom of trace2.h to declare a trace2_collect_process_info()\nfunction and then have your code in compat/procinfo.c hide\nbehind my interface.\n\nYour code should emits events of the form:\n\n\ttrace2_data_*(\"process\", \"linux/\", ...)\n\nI notice there are several `PROCFS_*` symbols currently defined\nin `config.mak.uname` so maybe this should be:\n\n\ttrace2_data_*(\"process\", \"procfs/\", ...)\n\n(I'm not sure how similar code for Linux and the various BSD\nversions would be.)\n\n\nJeff\n\n"},{"id":"424207","messageId":"xmqq8s4lkleu.fsf@gitster.g","threadId":"55637","inReplyTo":"87im3qu4gy.fsf@evledraar.gmail.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-05-11T21:31:05Z","receivedAt":"2021-05-11T21:31:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Thu, May 06 2021, Emily Shaffer wrote:\n>\n>> It can be useful to tell who invoked Git - was it invoked manually by a\n>> user via CLI or script? By an IDE? Knowing where the Git invocation came\n>> from can help with debugging to isolate where the problem came from.\n>\n> Aside from the portability concerns others have raised, I don't really\n> see why you'd need this.\n>\n> We already have the nest-level as part of the SID, so isn't it\n> sufficient (and portable) at the top-level to log what isatty says + set\n> the initial SID \"root\" in the IDE (which presumably knows about git).\n>\n> Wouldn't this log passwords in cases of e.g.:\n>\n> some-script --git-password secret # invokes \"git\"\n\nBoth valid and excellent points, I would think.\n"},{"id":"424626","messageId":"YJ70g0Nd1W1f6BIx@google.com","threadId":"55637","inReplyTo":"87im3qu4gy.fsf@evledraar.gmail.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-14T22:06:59Z","receivedAt":"2021-05-14T22:07:37Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, May 10, 2021 at 02:29:15PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> \n> \n> On Thu, May 06 2021, Emily Shaffer wrote:\n> \n> > It can be useful to tell who invoked Git - was it invoked manually by a\n> > user via CLI or script? By an IDE? Knowing where the Git invocation came\n> > from can help with debugging to isolate where the problem came from.\n> \n> Aside from the portability concerns others have raised, I don't really\n> see why you'd need this.\n> \n> We already have the nest-level as part of the SID, so isn't it\n> sufficient (and portable) at the top-level to log what isatty says + set\n> the initial SID \"root\" in the IDE (which presumably knows about git).\n\nIf you already know all the IDEs and scripts which invoke Git, sure, you\ncould set the SID root - we do this with 'repo' tool today. But actually\nwe want this because we aren't sure exactly who at Google is invoking\nGit in their tooling, how, why, etc. - this logline was supposed to help\nwith that. Chicken, egg, etc.\n\nOr else I'm misunderstanding your suggestion; to me it sounded like \"go\nfix your VSCode plugin so that it sets an additional envvar\" and I\nresponded accordingly. If you mean something else I didn't see,\nimplemented in Git land, then I'm curious to hear more.\n\n> \n> Wouldn't this log passwords in cases of e.g.:\n> \n> some-script --git-password secret # invokes \"git\"\n\nI have nothing but nasty things to say about scripts which would do\nthat, but your point is valid enough to make me think this logline\nshould be gated behind a config and posted much later (when config is\navailable - I think it's not yet, with the patch as-is).\n\n> \n> In older versions of linux reading e.g. smaps from /proc/self would\n> stall the kernel while the read was happening, I haven't checked whether\n> cmdline is such a thing (probably not), but it's a subtle thing to have\n> in mind for this / follow-ups if it's made portable (is that an issue on\n> other OS's?).\n\nThat's an interesting point and I wonder how I can validate if it\ndoes/doesn't stall in this way - I guess I can go manpage diving?\n\n - Emily\n"},{"id":"424627","messageId":"YJ70pQCuvz1chbrS@google.com","threadId":"55637","inReplyTo":"1cae0717-745b-5e3a-fadc-3966da00d80c@jeffhostetler.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-14T22:07:33Z","receivedAt":"2021-05-14T22:07:43Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, May 11, 2021 at 01:28:39PM -0400, Jeff Hostetler wrote:\n> \n> \n> \n> On 5/6/21 8:29 PM, Emily Shaffer wrote:\n> > It can be useful to tell who invoked Git - was it invoked manually by a\n> > user via CLI or script? By an IDE? Knowing where the Git invocation came\n> > from can help with debugging to isolate where the problem came from.\n> > \n> > Unfortunately, there's no cross-platform reliable way to gather the name\n> > of the parent process. If procfs is present, we can use that; otherwise\n> > we will need to discover the name another way. However, the process ID\n> > should be sufficient regardless of platform.\n> > \n> > Signed-off-by: Emily Shaffer \n> > ---\n> > We briefly discussed hiding this behind a config, internally. However, I\n> > wanted to include the parent name alongside the cmd_start event, which\n> > happens very early (maybe before config gathering?).\n> > \n> > Maybe it's better to log the parent_name as its own event, since it\n> > shouldn't change over the lifetime of the process?\n> > \n> > procfs is very non-portable, though - I think this won't even work on\n> > MacOS. So I'm curious if anybody has better suggestions for how to do\n> > this.\n> > \n> > - Emily\n> \n> \n> Look at `trace2_collect_process_info()` in `trace2.h` and\n> `compat/win32/trace2_win32_process_info.c`.\n> \n> That function is designed to let the process call out to\n> platform-specific code to do things like getting the name\n> of the invoking process.\n> \n> It is called with an enum to indicate when/why it is being\n> called.\n> \n> On Windows, I have platform code to get the process ancestry,\n> the peak VM usage, and whether the process is being debugged.\n> Two of those were easy.... And all are very platform-specific.\n> \n> They all generate events of the form:\n> \n> \ttrace2_data_*(\"process\", \"windows/\", ...)\n> \n> Some of my `t021[012]/` helper scripts exclude \"process\" category\n> messages from the output to make the t*.sh tests portable.\n> \n> \n> To implement a /proc lookup as you have suggested, I would\n> fixup the Makefile or config.mak.uname to include something\n> like a \"HAVE_...\" feature and then use that at the\n> bottom of trace2.h to declare a trace2_collect_process_info()\n> function and then have your code in compat/procinfo.c hide\n> behind my interface.\n> \n> Your code should emits events of the form:\n> \n> \ttrace2_data_*(\"process\", \"linux/\", ...)\n> \n> I notice there are several `PROCFS_*` symbols currently defined\n> in `config.mak.uname` so maybe this should be:\n> \n> \ttrace2_data_*(\"process\", \"procfs/\", ...)\n> \n> (I'm not sure how similar code for Linux and the various BSD\n> versions would be.)\n\nThanks a bunch for the pointers - this is great.\n\n - Emily\n"},{"id":"424670","messageId":"xmqqo8db72zu.fsf@gitster.g","threadId":"55637","inReplyTo":"YJ70g0Nd1W1f6BIx@google.com","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-05-16T03:48:37Z","receivedAt":"2021-05-16T03:48:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> On Mon, May 10, 2021 at 02:29:15PM +0200, Ævar Arnfjörð Bjarmason wrote:\n>\n>> We already have the nest-level as part of the SID, so isn't it\n>> sufficient (and portable) at the top-level to log what isatty says + set\n>> the initial SID \"root\" in the IDE (which presumably knows about git).\n>\n> If you already know all the IDEs and scripts which invoke Git, sure, you\n> could set the SID root - we do this with 'repo' tool today. But actually\n> we want this because we aren't sure exactly who at Google is invoking\n> Git in their tooling, how, why, etc. - this logline was supposed to help\n> with that. Chicken, egg, etc.\n\nI agreed with Æver's suggestion exectly because I failed to read the\nabove motivation from the patch. If you are trying to find out who\ncalled, then you'd need to do the \"find the parent process\" dance\nwhen your parent did not give you their SID to append your ident to.\n\nPerhaps it was obvious to the author of the patch, but it was\nunclear from the point of view of readers. Perhaps the first\nparagraph of the proposed message wants a bit of rephrasing. \"we\naren't sure exactly ... how, why, etc.\" part of the above really\nhelped.\n\nThanks.\n\n"},{"id":"424783","messageId":"YKLPYd/AYZFUzs57@google.com","threadId":"55637","inReplyTo":"xmqqo8db72zu.fsf@gitster.g","subject":"Re: [PATCH] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-17T20:17:37Z","receivedAt":"2021-05-17T20:17:43Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Sun, May 16, 2021 at 12:48:37PM +0900, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> > On Mon, May 10, 2021 at 02:29:15PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> >\n> >> We already have the nest-level as part of the SID, so isn't it\n> >> sufficient (and portable) at the top-level to log what isatty says + set\n> >> the initial SID \"root\" in the IDE (which presumably knows about git).\n> >\n> > If you already know all the IDEs and scripts which invoke Git, sure, you\n> > could set the SID root - we do this with 'repo' tool today. But actually\n> > we want this because we aren't sure exactly who at Google is invoking\n> > Git in their tooling, how, why, etc. - this logline was supposed to help\n> > with that. Chicken, egg, etc.\n> \n> I agreed with Æver's suggestion exectly because I failed to read the\n> above motivation from the patch. If you are trying to find out who\n> called, then you'd need to do the \"find the parent process\" dance\n> when your parent did not give you their SID to append your ident to.\n> \n> Perhaps it was obvious to the author of the patch, but it was\n> unclear from the point of view of readers. Perhaps the first\n> paragraph of the proposed message wants a bit of rephrasing. \"we\n> aren't sure exactly ... how, why, etc.\" part of the above really\n> helped.\n> \n> Thanks.\n\nThanks for the feedback. I was vacationing last week but should be\nsending a rework today or tomorrow, will see if I can shore up the\ncommit-msg as well as the portability stuff Jeff H raised.\n\n - Emily\n"},{"id":"425144","messageId":"20210520210546.4129620-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210507002908.1495061-1-emilyshaffer@google.com","subject":"[PATCH v2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-20T21:05:46Z","receivedAt":"2021-05-20T21:05:58Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? In some cases - like 'repo' tool -\nwe can influence the source code and set the GIT_TRACE2_PARENT_SID\nenvironment variable from the caller process. In 'repo''s case, that\nparent SID is manipulated to include the string \"repo\", which means we\ncan positively identify when Git was invoked by 'repo' tool. However,\nidentifying parents that way requires both that we know which tools\ninvoke Git and that we have the ability to modify the source code of\nthose tools. It cannot scale to keep up with the various IDEs and\nwrappers which use Git, most of which we don't know about. Learning\nwhich tools and wrappers invoke Git, and how, would give us insight to\ndecide where to improve Git's usability and performance.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient regardless of platform.\n\nGit for Windows gathers similar information and logs it as a \"data_json\"\nevent. However, since \"data_json\" has a variable format, it is difficult\nto parse effectively in some languages; instead, let's pursue a\ndedicated \"cmd_ancestry\" event to record information about the ancestry\nof the current process and a consistent, parseable way.\n\nGit for Windows also gathers information about more than one parent. In\nLinux further ancestry info can be gathered with procfs, but it's\nunwieldy to do so. In the interest of later moving Git for Windows\nancestry logging to the 'cmd_ancestry' event, and in the interest of\nlater adding more ancestry to the Linux implementation - or of adding\nthis functionality to other platforms which have an easier time walking\nthe process tree - let's make 'cmd_ancestry' accept an array of\nparentage.\n\nSigned-off-by: Emily Shaffer \n---\n\nHi folks, the comments I received in v1 were of two varieties:\n1) \"There are better ways to make this platform-safe\", and\n2) \"Your commit message doesn't convince me\".\nSince I sent v1, though, I also learned a little more about procfs, and\nabout the trace2 structure overall, so there are some pretty significant\ndifferences from v1:\n\n- I took a look at Jeff H's advice on using a \"data_json\" event to log\n this and decided it would be a little more flexible to add a new event\n instead. If we want, it'd be feasible to then shoehorn the GfW parent\n tree stuff into this new event too. Doing it this way is definitely\n easier to parse for Google's trace analysis system (which for now\n completely skips \"data_json\" as it's polymorphic), and also - I think\n - means that we can add more fields later on if we need to (thread\n info, different fields than just /proc/n/comm like exec path, argv,\n whatever).\n- Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n the \"command name\" - excluding argv, excluding path, etc. It seems\n like this is a little more safe about excluding personal information\n from the traces which take the form of \"myscript.sh\n --password=hunter2\", but would still be worrisome for something like\n \"mysupersecretproject.sh\". I'm not sure whether that means we still\n want to guard it with a config flag, though.\n- I also added a lot to the commit message; hopefully it's not too\n rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n wasn't going to cut it.\n- As for testing, I followed the lead of GfW's parentage info - \"this\n isn't portable so writing tests for it will suck, just scrub it from\n the tests\". Maybe it makes sense to do some more\n platform-specific-ness in the test suite instead? I wasn't sure.\n\nThanks, all.\n - Emily\n\n Makefile | 5 ++++\n compat/procinfo.c | 53 +++++++++++++++++++++++++++++++++++++++\n config.mak.uname | 1 +\n git-compat-util.h | 6 +++++\n t/t0210/scrub_normal.perl | 6 +++++\n t/t0211/scrub_perf.perl | 5 ++++\n t/t0212/parse_events.perl | 5 +++-\n trace2.c | 13 ++++++++++\n trace2.h | 12 ++++++++-\n trace2/tr2_tgt.h | 3 +++\n trace2/tr2_tgt_event.c | 21 ++++++++++++++++\n trace2/tr2_tgt_normal.c | 19 ++++++++++++++\n trace2/tr2_tgt_perf.c | 16 ++++++++++++\n 13 files changed, 163 insertions(+), 2 deletions(-)\n create mode 100644 compat/procinfo.c\n\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..330e4fa011 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n endif\n \n+ifdef HAVE_PROCFS_LINUX\n+\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n+\tCOMPAT_OBJS += compat/procinfo.o\n+endif\n+\n ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n endif\ndiff --git a/compat/procinfo.c b/compat/procinfo.c\nnew file mode 100644\nindex 0000000000..523600673f\n--- /dev/null\n+++ b/compat/procinfo.c\n@@ -0,0 +1,53 @@\n+#include \"cache.h\"\n+\n+#include \"strbuf.h\"\n+#include \"trace2.h\"\n+\n+char *get_process_name(int pid)\n+{\n+#ifdef HAVE_PROCFS_LINUX\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf out = STRBUF_INIT;\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n+\tif (!strbuf_read_file(&out, procfs_path.buf, 0))\n+\t{\n+\t\t/* All done with file reads, clean up early */\n+\t\tstrbuf_release(&procfs_path);\n+\t\treturn strbuf_detach(&out, NULL);\n+\t}\n+#endif\n+\n+\t/* NEEDSWORK: add non-procfs implementations here. */\n+\treturn NULL;\n+}\n+\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+\tif (!trace2_is_enabled())\n+\t\treturn;\n+\n+\t/* someday we may want to write something extra here, but not today */\n+\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\treturn;\n+\n+\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n+\t{\n+\t\t/*\n+\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n+\t\t * see compat/win32/trace2_win32_process_info.c.\n+\t\t */\n+\t\tchar *names[2];\n+\t\tnames[0] = get_process_name(getppid());\n+\t\tnames[1] = NULL;\n+\n+\t\tif (!names[0])\n+\t\t\treturn;\n+\n+\t\ttrace2_cmd_ancestry((const char**)names);\n+\n+\t\tfree(names[0]);\n+\t}\n+\n+\treturn;\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex cb443b4e02..7ad110a1d2 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n+\tHAVE_PROCFS_LINUX = YesPlease\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex a508dbe5a3..cc7d5d8a2a 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1382,4 +1382,10 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)\n \n void sleep_millisec(int millisec);\n \n+/*\n+ * Convert PID to process name (as would show in top/task manager). Returns\n+ * NULL if unimplemented - be sure to check for NULL at callsite.\n+ */\n+char *get_process_name(int pid);\n+\n #endif\ndiff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\nindex c65d1a815e..7cc4de392a 100644\n--- a/t/t0210/scrub_normal.perl\n+++ b/t/t0210/scrub_normal.perl\n@@ -42,6 +42,12 @@\n \t# so just omit it for testing purposes.\n \t# print \"cmd_path _EXE_\\n\";\n }\n+ elsif ($line =~ m/^cmd_ancestry/) {\n+\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n+\t# sake, skip it when parsing normal.\n+\t#\n+\t# print \"$line\";\n+ }\n else {\n \tprint \"$line\";\n }\ndiff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\nindex 351af7844e..d164b750ff 100644\n--- a/t/t0211/scrub_perf.perl\n+++ b/t/t0211/scrub_perf.perl\n@@ -44,6 +44,11 @@\n \t# $tokens[$col_rest] = \"_EXE_\";\n \tgoto SKIP_LINE;\n }\n+ elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n+\t# so skip it.\n+\tgoto SKIP_LINE;\n+ }\n elsif ($tokens[$col_event] =~ m/child_exit/) {\n \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n }\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..b6408560c0 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -132,7 +132,10 @@\n \t# just omit it for testing purposes.\n \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n }\n- \n+ elsif ($event eq 'cmd_ancestry') {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n+\t# just skip it for testing purposes.\n+ }\n elsif ($event eq 'cmd_name') {\n \t$processes->{$sid}->{'name'} = $line->{'name'};\n \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\ndiff --git a/trace2.c b/trace2.c\nindex 256120c7fd..b9b154ac44 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n }\n \n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tstruct tr2_tgt *tgt_j;\n+\tint j;\n+\n+\tif (!trace2_enabled)\n+\t\treturn;\n+\n+\tfor_each_wanted_builtin (j, tgt_j)\n+\t\tif (tgt_j->pfn_command_ancestry_fl)\n+\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n+}\n+\n void trace2_cmd_name_fl(const char *file, int line, const char *name)\n {\n \tstruct tr2_tgt *tgt_j;\ndiff --git a/trace2.h b/trace2.h\nindex ede18c2e06..23743ac62b 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n \n #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n \n+/*\n+ * Emit an 'ancestry' event with the process name of the current process's\n+ * parent process.\n+ * This gives post-processors a way to determine what invoked the command and\n+ * learn more about usage patterns.\n+ */\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\n@@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n \tTRACE2_PROCESS_INFO_EXIT,\n };\n \n-#if defined(GIT_WINDOWS_NATIVE)\n+#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n void trace2_collect_process_info(enum trace2_process_info_reason reason);\n #else\n #define trace2_collect_process_info(reason) \\\ndiff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\nindex 7b90469212..1f66fd6573 100644\n--- a/trace2/tr2_tgt.h\n+++ b/trace2/tr2_tgt.h\n@@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n \n typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *command_path);\n+typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n+\t\t\t\t\t\tconst char **parent_names);\n typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *name,\n \t\t\t\t\t const char *hierarchy);\n@@ -108,6 +110,7 @@ struct tr2_tgt {\n \ttr2_tgt_evt_atexit_t *pfn_atexit;\n \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n+\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..578a9a5287 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tjw_release(&jw);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tconst char *parent_name = NULL;\n+\tstruct json_writer jw = JSON_WRITER_INIT;\n+\n+\tjw_object_begin(&jw, 0);\n+\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_inline_begin_array(&jw, \"ancestry\");\n+\n+\twhile ((parent_name = *parent_names++))\n+\t\tjw_array_string(&jw, parent_name);\n+\n+\tjw_end(&jw); /* 'ancestry' array */\n+\tjw_end(&jw); /* event object */\n+\n+\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n+\tjw_release(&jw);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex 31b602c171..a5751c8864 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *parent_name = NULL;\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n+\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n+\twhile ((parent_name = *parent_names++)) {\n+\t\tstrbuf_addstr(&buf_payload, parent_name);\n+\t\t/* if we'll write another one after this, add a delimiter */\n+\t\tif (parent_names && *parent_names)\n+\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n+\t}\n+\n+\tnormal_io_write_fl(file, line, &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a8018f18cc..af4d65a0a5 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n+\t/* It's not an argv but the rules are basically the same. */\n+\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n+\tstrbuf_addch(&buf_payload, ']');\n+\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n+\t\t\t &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\n-- \n2.31.1.818.g46aad6cb9e-goog\n\n"},{"id":"425146","messageId":"021601d74dc0$326f6620$974e3260$@nexbridge.com","threadId":"55637","inReplyTo":"20210520210546.4129620-1-emilyshaffer@google.com","subject":"RE: [PATCH v2] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-05-20T21:36:25Z","receivedAt":"2021-05-20T21:36:38Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On May 20, 2021 5:06 PM, Emily Shaffer wrote:\n>To: git@vger.kernel.org\n>Cc: Emily Shaffer ; Ævar Arnfjörð Bjarmason ; Junio C Hamano ;\n>Jeff Hostetler ; Bagas Sanjaya \n>Subject: [PATCH v2] tr2: log parent process name\n>\n>It can be useful to tell who invoked Git - was it invoked manually by a user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n>we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In 'repo''s case,\n>that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by 'repo' tool.\n>However, identifying parents that way requires both that we know which tools invoke Git and that we have the ability to modify the source\n>code of those tools. It cannot scale to keep up with the various IDEs and wrappers which use Git, most of which we don't know about.\n>Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and performance.\n>\n>Unfortunately, there's no cross-platform reliable way to gather the name of the parent process. If procfs is present, we can use that;\n>otherwise we will need to discover the name another way. However, the process ID should be sufficient regardless of platform.\n\nI like this idea, but there are some platforms where this is unlikely to work. NonStop, in particular, can initiate git - and I frequently do - from a non-POSIX environment where process name is entirely different. In fact, it is something like $ABC (always beginning with a $, which makes life very difficult for shell scripts and screws up GIT_SSH_COMMAND, but I digress). I'm going to need to plug in something very platform-specific to make this work. getppid() always returns 1 in this situation, which is extraordinarily meaningless on the platform and does not represent the actual parent.\n\nI will try to put the appropriate compat hooks in once this moves into master but I can't promise it will be particularly efficient at this stage.\n\nRegards,\nRandall\n\n"},{"id":"425169","messageId":"YKbvgWpMngx76I5R@google.com","threadId":"55637","inReplyTo":"021601d74dc0$326f6620$974e3260$@nexbridge.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-20T23:23:45Z","receivedAt":"2021-05-20T23:23:54Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, May 20, 2021 at 05:36:25PM -0400, Randall S. Becker wrote:\n> \n> On May 20, 2021 5:06 PM, Emily Shaffer wrote:\n> >To: git@vger.kernel.org\n> >Cc: Emily Shaffer ; Ævar Arnfjörð Bjarmason ; Junio C Hamano ;\n> >Jeff Hostetler ; Bagas Sanjaya \n> >Subject: [PATCH v2] tr2: log parent process name\n> >\n> >It can be useful to tell who invoked Git - was it invoked manually by a user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n> >we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In 'repo''s case,\n> >that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by 'repo' tool.\n> >However, identifying parents that way requires both that we know which tools invoke Git and that we have the ability to modify the source\n> >code of those tools. It cannot scale to keep up with the various IDEs and wrappers which use Git, most of which we don't know about.\n> >Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and performance.\n> >\n> >Unfortunately, there's no cross-platform reliable way to gather the name of the parent process. If procfs is present, we can use that;\n> >otherwise we will need to discover the name another way. However, the process ID should be sufficient regardless of platform.\n> \n> I like this idea, but there are some platforms where this is unlikely to work. NonStop, in particular, can initiate git - and I frequently do - from a non-POSIX environment where process name is entirely different. In fact, it is something like $ABC (always beginning with a $, which makes life very difficult for shell scripts and screws up GIT_SSH_COMMAND, but I digress). I'm going to need to plug in something very platform-specific to make this work. getppid() always returns 1 in this situation, which is extraordinarily meaningless on the platform and does not represent the actual parent.\n\nOk. It sounds like you're saying I should be more conservative in the\ncommit message as well as in the #ifdef scope? Do you think this needs a\nreroll to made the #ifdef more aggressive, or would you rather get to it\nwhen you get to it?\n\nIt looks like the change in config.mak.uname won't affect NonStop; I\nthink also the compat/procinfo.c is probably indicative enough of \"this\nstuff is for procfs\" that it won't look like it *should* work for\nNonStop, which means that you should still get the stub for\n'trace2_collect_process_info()'. But if you think the guards aren't\nreadable enough I can try to move them around a little more.\n\n - Emily\n"},{"id":"425178","messageId":"xmqqpmxksuqa.fsf@gitster.g","threadId":"55637","inReplyTo":"20210520210546.4129620-1-emilyshaffer@google.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-05-21T02:09:49Z","receivedAt":"2021-05-21T02:09:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient regardless of platform.\n\nNot a strong objection, but I wonder if seeing random integer(s) is\nbetter than not having cmd_ancestry info at all. The latter better\nsignals that the platform does not yet have the \"parent process\nname\" feature, I would think.\n\n> Git for Windows also gathers information about more than one parent. In\n> Linux further ancestry info can be gathered with procfs, but it's\n> unwieldy to do so. In the interest of later moving Git for Windows\n> ancestry logging to the 'cmd_ancestry' event, and in the interest of\n> later adding more ancestry to the Linux implementation - or of adding\n> this functionality to other platforms which have an easier time walking\n> the process tree - let's make 'cmd_ancestry' accept an array of\n> parentage.\n\nCould we rephrase \"more than one parent\" at the beginning to\nclarify? I initially had to wonder what \"an array of parentage\"\ncontains (father and mother, or a sole parent and its sole parent,\nwhich is a sole grandparent). Since there is no \"multiple processes\nmeet and spawn a single process\", I take it is the latter. Perhaps\n\"more than one generation of\" or something?\n\n> +ifdef HAVE_PROCFS_LINUX\n> +\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n> +\tCOMPAT_OBJS += compat/procinfo.o\n> +endif\n> +\n> ifdef HAVE_NS_GET_EXECUTABLE_PATH\n> \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n> endif\n> diff --git a/compat/procinfo.c b/compat/procinfo.c\n> new file mode 100644\n> index 0000000000..523600673f\n> --- /dev/null\n> +++ b/compat/procinfo.c\n> @@ -0,0 +1,53 @@\n> +#include \"cache.h\"\n> +\n> +#include \"strbuf.h\"\n> +#include \"trace2.h\"\n> +\n> +char *get_process_name(int pid)\n> +{\n> +#ifdef HAVE_PROCFS_LINUX\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf out = STRBUF_INIT;\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n> +\tif (!strbuf_read_file(&out, procfs_path.buf, 0))\n> +\t{\n\nPlace this opening brace at the end of the previous line.\n\n> +\t\t/* All done with file reads, clean up early */\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\treturn strbuf_detach(&out, NULL);\n> +\t}\n> +#endif\n> +\n> +\t/* NEEDSWORK: add non-procfs implementations here. */\n> +\treturn NULL;\n> +}\n\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n> +\t{\n\nDitto.\n\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tchar *names[2];\n> +\t\tnames[0] = get_process_name(getppid());\n> +\t\tnames[1] = NULL;\n> +\n> +\t\tif (!names[0])\n> +\t\t\treturn;\n\nOK, so if there is no name given, we do not show pid as a\nplaceholder.\n\n> +\t\ttrace2_cmd_ancestry((const char**)names);\n> +\n> +\t\tfree(names[0]);\n> +\t}\n> +\n> +\treturn;\n> +}\n> diff --git a/config.mak.uname b/config.mak.uname\n> index cb443b4e02..7ad110a1d2 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n> \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n> \tBASIC_CFLAGS += -DHAVE_SYSINFO\n> \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n> +\tHAVE_PROCFS_LINUX = YesPlease\n\nHave all Linux instances procfs enabled and mounted? It might be\nthat we need to detect this at runtime anyway?\n\n ... goes and thinks ...\n\nAh, OK, that \"try reading from proc/%d/comm\" is the runtime\ndetection, so it is only this Makefile variable is slightly\nmisnamed (it is not \"HAVE\" but \"is worth checking for it\").\n\nMakes sense.\n\n"},{"id":"425217","messageId":"025401d74e44$25db7140$719253c0$@nexbridge.com","threadId":"55637","inReplyTo":"YKbvgWpMngx76I5R@google.com","subject":"RE: [PATCH v2] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-05-21T13:20:57Z","receivedAt":"2021-05-21T13:21:14Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On May 20, 2021 7:24 PM, Emily Shaffer wrote:\n>\n>On Thu, May 20, 2021 at 05:36:25PM -0400, Randall S. Becker wrote:\n>>\n>> On May 20, 2021 5:06 PM, Emily Shaffer wrote:\n>> >To: git@vger.kernel.org\n>> >Cc: Emily Shaffer ; Ævar Arnfjörð Bjarmason\n>> >; Junio C Hamano ; Jeff\n>> >Hostetler ; Bagas Sanjaya\n>> >\n>> >Subject: [PATCH v2] tr2: log parent process name\n>> >\n>> >It can be useful to tell who invoked Git - was it invoked manually by\n>> >a user via CLI or script? By an IDE? In some cases - like 'repo'\n>> >tool - we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In\n'repo''s\n>case, that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by\n'repo'\n>tool.\n>> >However, identifying parents that way requires both that we know\n>> >which tools invoke Git and that we have the ability to modify the source code of those tools. It cannot scale to keep up with\nthe various\n>IDEs and wrappers which use Git, most of which we don't know about.\n>> >Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and\n>performance.\n>> >\n>> >Unfortunately, there's no cross-platform reliable way to gather the\n>> >name of the parent process. If procfs is present, we can use that; otherwise we will need to discover the name another way.\nHowever,\n>the process ID should be sufficient regardless of platform.\n>>\n>> I like this idea, but there are some platforms where this is unlikely to work. NonStop, in particular, can initiate git - and I\nfrequently do -\n>from a non-POSIX environment where process name is entirely different. In fact, it is something like $ABC (always beginning with a\n$,\n>which makes life very difficult for shell scripts and screws up GIT_SSH_COMMAND, but I digress). I'm going to need to plug in\nsomething\n>very platform-specific to make this work. getppid() always returns 1 in this situation, which is extraordinarily meaningless on the\nplatform\n>and does not represent the actual parent.\n>\n>Ok. It sounds like you're saying I should be more conservative in the commit message as well as in the #ifdef scope? Do you think\nthis\n>needs a reroll to made the #ifdef more aggressive, or would you rather get to it when you get to it?\n\nI'll get to it pretty quickly once it's rolled in.\n\n>It looks like the change in config.mak.uname won't affect NonStop; I think also the compat/procinfo.c is probably indicative enough\nof \"this\n>stuff is for procfs\" that it won't look like it *should* work for NonStop, which means that you should still get the stub for\n>'trace2_collect_process_info()'. But if you think the guards aren't readable enough I can try to move them around a little more.\n\nGuards are fine. There's just a lot more work to do for me. We need to make sure that the rendering of ancestor processes are\ngeneric enough not to be just pid_t through any interfaces where this is queried. In NonStop's case, char[25], should be sufficient\nfor the short term, but I would prefer something longer, say char[128] to be safe for the future in which to stick the ancestor. To\nbe completely unique, the ancestor is going to look like \\node.$proc:sequence (where node is a 7 character name, proc is a 5\ncharacter name, and sequence is currently a long.\n\n-Randall\n\n"},{"id":"425220","messageId":"027f01d74e5d$d9866e70$8c934b50$@nexbridge.com","threadId":"55637","inReplyTo":"025401d74e44$25db7140$719253c0$@nexbridge.com","subject":"RE: [PATCH v2] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-05-21T16:24:55Z","receivedAt":"2021-05-21T16:25:11Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On May 21, 2021 9:21 AM, I wrote:\n>To: 'Emily Shaffer' \n>Cc: git@vger.kernel.org; 'Ævar Arnfjörð Bjarmason' ; 'Junio C Hamano' ; 'Jeff Hostetler'\n>; 'Bagas Sanjaya' \n>Subject: RE: [PATCH v2] tr2: log parent process name\n>\n>On May 20, 2021 7:24 PM, Emily Shaffer wrote:\n>>\n>>On Thu, May 20, 2021 at 05:36:25PM -0400, Randall S. Becker wrote:\n>>>\n>>> On May 20, 2021 5:06 PM, Emily Shaffer wrote:\n>>> >To: git@vger.kernel.org\n>>> >Cc: Emily Shaffer ; Ævar Arnfjörð Bjarmason\n>>> >; Junio C Hamano ; Jeff\n>>> >Hostetler ; Bagas Sanjaya\n>>> >\n>>> >Subject: [PATCH v2] tr2: log parent process name\n>>> >\n>>> >It can be useful to tell who invoked Git - was it invoked manually\n>>> >by a user via CLI or script? By an IDE? In some cases - like 'repo'\n>>> >tool - we can influence the source code and set the\n>>> >GIT_TRACE2_PARENT_SID environment variable from the caller process.\n>>> >In\n>'repo''s\n>>case, that parent SID is manipulated to include the string \"repo\",\n>>which means we can positively identify when Git was invoked by\n>'repo'\n>>tool.\n>>> >However, identifying parents that way requires both that we know\n>>> >which tools invoke Git and that we have the ability to modify the\n>>> >source code of those tools. It cannot scale to keep up with\n>the various\n>>IDEs and wrappers which use Git, most of which we don't know about.\n>>> >Learning which tools and wrappers invoke Git, and how, would give us\n>>> >insight to decide where to improve Git's usability and\n>>performance.\n>>> >\n>>> >Unfortunately, there's no cross-platform reliable way to gather the\n>>> >name of the parent process. If procfs is present, we can use that; otherwise we will need to discover the name another way.\n>However,\n>>the process ID should be sufficient regardless of platform.\n>>>\n>>> I like this idea, but there are some platforms where this is unlikely\n>>> to work. NonStop, in particular, can initiate git - and I\n>frequently do -\n>>from a non-POSIX environment where process name is entirely different.\n>>In fact, it is something like $ABC (always beginning with a\n>$,\n>>which makes life very difficult for shell scripts and screws up\n>>GIT_SSH_COMMAND, but I digress). I'm going to need to plug in\n>something\n>>very platform-specific to make this work. getppid() always returns 1 in\n>>this situation, which is extraordinarily meaningless on the\n>platform\n>>and does not represent the actual parent.\n>>\n>>Ok. It sounds like you're saying I should be more conservative in the\n>>commit message as well as in the #ifdef scope? Do you think\n>this\n>>needs a reroll to made the #ifdef more aggressive, or would you rather get to it when you get to it?\n>\n>I'll get to it pretty quickly once it's rolled in.\n>\n>>It looks like the change in config.mak.uname won't affect NonStop; I\n>>think also the compat/procinfo.c is probably indicative enough\n>of \"this\n>>stuff is for procfs\" that it won't look like it *should* work for\n>>NonStop, which means that you should still get the stub for 'trace2_collect_process_info()'. But if you think the guards aren't\nreadable\n>enough I can try to move them around a little more.\n>\n>Guards are fine. There's just a lot more work to do for me. We need to make sure that the rendering of ancestor processes are\ngeneric\n>enough not to be just pid_t through any interfaces where this is queried. In NonStop's case, char[25], should be sufficient for the\nshort\n>term, but I would prefer something longer, say char[128] to be safe for the future in which to stick the ancestor. To be completely\nunique,\n>the ancestor is going to look like \\node.$proc:sequence (where node is a 7 character name, proc is a 5 character name, and sequence\nis\n>currently a long.\n\nJust so we know what's coming, the code snippet to get a parent on NonStop is as follows (roughly):\n pid_t ossParent = getppid();\n if (ossParent == 1) {\n short pHandle[10];\n short pAncestor[10];\n short ancestorLength;\n short error;\n short attributes[] = { 40 }; /* MOM Process */\n short processNameLength;\n char processName[64];\n\n PROCESSHANDLE_NULLIT_(pHandle);\n PROCESS_GETINFO_(pHandle);\n error = PROCESS_GETINFOLIST_(,,,, pHandle,\n attributes, (short) sizeof(attributes)/sizeof(attributes[0]),\n pAncestor, (short) sizeof(pAncestor), &ancestorLength);\n if (error) {\n printf(\"Cannot process parent. Error %d\\n\", error);\n return;\n }\n PROCESSHANDLE_TO_FILENAME_(pAncestor, processName, (short) sizeof(processName),\n &processNameLength);\n processName[processNameLength] = '\\0';\n printf(\"GUARDIAN Parent %s\\n\", processName);\n } else {\n printf(\"OSS Parent %d\\n\", ossParent);\n }\n\nWhich in the test program generates\nGUARDIAN Parent \\HPITUG.$:3:1100:583555076\n\nRegards,\nRandall\n\n"},{"id":"425231","messageId":"YKgDxahhwK/zYznH@google.com","threadId":"55637","inReplyTo":"xmqqpmxksuqa.fsf@gitster.g","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-21T19:02:29Z","receivedAt":"2021-05-21T19:02:40Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Fri, May 21, 2021 at 11:09:49AM +0900, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> > Unfortunately, there's no cross-platform reliable way to gather the name\n> > of the parent process. If procfs is present, we can use that; otherwise\n> > we will need to discover the name another way. However, the process ID\n> > should be sufficient regardless of platform.\n> \n> Not a strong objection, but I wonder if seeing random integer(s) is\n> better than not having cmd_ancestry info at all. The latter better\n> signals that the platform does not yet have the \"parent process\n> name\" feature, I would think.\n\nHm, we could; I guess then analyzers could still correlate \"all these\nthings were called by some kind of wrapper, even if we don't know if\nthat was an IDE or a script or just the user or what, we can guess based\non other heuristics\". Ok.\n\n> \n> > Git for Windows also gathers information about more than one parent. In\n> > Linux further ancestry info can be gathered with procfs, but it's\n> > unwieldy to do so. In the interest of later moving Git for Windows\n> > ancestry logging to the 'cmd_ancestry' event, and in the interest of\n> > later adding more ancestry to the Linux implementation - or of adding\n> > this functionality to other platforms which have an easier time walking\n> > the process tree - let's make 'cmd_ancestry' accept an array of\n> > parentage.\n> \n> Could we rephrase \"more than one parent\" at the beginning to\n> clarify? I initially had to wonder what \"an array of parentage\"\n> contains (father and mother, or a sole parent and its sole parent,\n> which is a sole grandparent). Since there is no \"multiple processes\n> meet and spawn a single process\", I take it is the latter. Perhaps\n> \"more than one generation of\" or something?\n\nGood point, will refer to \"more than one generation\". Thanks.\n\n> > +\tif (!strbuf_read_file(&out, procfs_path.buf, 0))\n> > +\t{\n> \n> Place this opening brace at the end of the previous line.\n\nWill polish up this and others for v3, hopefully today.\n> > +\t\tif (!names[0])\n> > +\t\t\treturn;\n> \n> OK, so if there is no name given, we do not show pid as a\n> placeholder.\n\nBased on your suggestion above I think it will make sense to show pid as\nplaceholder after all, though. So I will change that for v3.\n\n> > \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n> > +\tHAVE_PROCFS_LINUX = YesPlease\n> \n> Have all Linux instances procfs enabled and mounted? It might be\n> that we need to detect this at runtime anyway?\n> \n> ... goes and thinks ...\n> \n> Ah, OK, that \"try reading from proc/%d/comm\" is the runtime\n> detection, so it is only this Makefile variable is slightly\n> misnamed (it is not \"HAVE\" but \"is worth checking for it\").\n\nI wonder what is better. \"MAYBE_PROCFS_LINUX\"? I don't see any other\nvars in config.mak.uname that indicate \"pretty sure but not totally\nsure\" in a quick scan. However, right above this line we seem to feel\ncertain in our guess about \"PROCFS_EXECUTABLE_PATH\"...\n\n...but when it is used in exec-cmd.c:git_get_exec_path_procfs(), invoked\nby exec-cmd.c:git_get_exec_path(), we're very tolerant to faults if it's\nnot there:\n\n static int git_get_exec_path(struct strbuf *buf, const char *argv0)\n {\n \t/*\n \t * [snip]\n \t * Each of these functions returns 0 on success, so evaluation will stop\n \t * after the first successful method.\n \t */\n \tif (\n #ifdef HAVE_BSD_KERN_PROC_SYSCTL\n \t\tgit_get_exec_path_bsd_sysctl(buf) &&\n #endif /* HAVE_BSD_KERN_PROC_SYSCTL */\n \n #ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \t\tgit_get_exec_path_darwin(buf) &&\n #endif /* HAVE_NS_GET_EXECUTABLE_PATH */\n \n #ifdef PROCFS_EXECUTABLE_PATH\n \t\tgit_get_exec_path_procfs(buf) && /*** <- OK if fails ***/\n #endif /* PROCFS_EXECUTABLE_PATH */\n \n #ifdef HAVE_WPGMPTR\n \t\tgit_get_exec_path_wpgmptr(buf) &&\n #endif /* HAVE_WPGMPTR */\n \n \t\tgit_get_exec_path_from_argv0(buf, argv0)) {\n \t\treturn -1;\n \t}\n \n [snip]\n \n #ifdef PROCFS_EXECUTABLE_PATH\n /*\n * Resolves the executable path by examining a procfs symlink.\n *\n * Returns 0 on success, -1 on failure.\n */\n static int git_get_exec_path_procfs(struct strbuf *buf)\n {\n \tif (strbuf_realpath(buf, PROCFS_EXECUTABLE_PATH, 0)) {\n \t\ttrace_printf(\n \t\t\t\"trace: resolved executable path from procfs: %s\\n\",\n \t\t\tbuf->buf);\n \t\treturn 0;\n \t}\n \treturn -1;\n }\n #endif /* PROCFS_EXECUTABLE_PATH */\n\n\nSo it seems this other procfs bit takes the \"probably but not\ndefinitely\" step and is tolerant at runtime as well. Which doesn't help\nme much to decide how to rename HAVE_PROCFS_LINUX.\n\nI'll switch it to MAYBE_PROCFS_LINUX for v3 unless someone yells, I\nguess.\n\n - Emily\n"},{"id":"425232","messageId":"1e3bb53e-895b-f571-1c03-a6ae6499746d@jeffhostetler.com","threadId":"55637","inReplyTo":"20210520210546.4129620-1-emilyshaffer@google.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-05-21T19:15:16Z","receivedAt":"2021-05-21T19:15:25Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 5/20/21 5:05 PM, Emily Shaffer wrote:\n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n> we can influence the source code and set the GIT_TRACE2_PARENT_SID\n> environment variable from the caller process. In 'repo''s case, that\n> parent SID is manipulated to include the string \"repo\", which means we\n> can positively identify when Git was invoked by 'repo' tool. However,\n> identifying parents that way requires both that we know which tools\n> invoke Git and that we have the ability to modify the source code of\n> those tools. It cannot scale to keep up with the various IDEs and\n> wrappers which use Git, most of which we don't know about. Learning\n> which tools and wrappers invoke Git, and how, would give us insight to\n> decide where to improve Git's usability and performance.\n> \n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient regardless of platform.\n> \n> Git for Windows gathers similar information and logs it as a \"data_json\"\n> event. However, since \"data_json\" has a variable format, it is difficult\n> to parse effectively in some languages; instead, let's pursue a\n> dedicated \"cmd_ancestry\" event to record information about the ancestry\n> of the current process and a consistent, parseable way.\n> \n> Git for Windows also gathers information about more than one parent. In\n> Linux further ancestry info can be gathered with procfs, but it's\n> unwieldy to do so. In the interest of later moving Git for Windows\n> ancestry logging to the 'cmd_ancestry' event, and in the interest of\n> later adding more ancestry to the Linux implementation - or of adding\n> this functionality to other platforms which have an easier time walking\n> the process tree - let's make 'cmd_ancestry' accept an array of\n> parentage.\n> \n> Signed-off-by: Emily Shaffer \n> ---\n> \n> Hi folks, the comments I received in v1 were of two varieties:\n> 1) \"There are better ways to make this platform-safe\", and\n> 2) \"Your commit message doesn't convince me\".\n> Since I sent v1, though, I also learned a little more about procfs, and\n> about the trace2 structure overall, so there are some pretty significant\n> differences from v1:\n> \n> - I took a look at Jeff H's advice on using a \"data_json\" event to log\n> this and decided it would be a little more flexible to add a new event\n> instead. If we want, it'd be feasible to then shoehorn the GfW parent\n> tree stuff into this new event too. Doing it this way is definitely\n> easier to parse for Google's trace analysis system (which for now\n> completely skips \"data_json\" as it's polymorphic), and also - I think\n> - means that we can add more fields later on if we need to (thread\n> info, different fields than just /proc/n/comm like exec path, argv,\n> whatever).\n\nI could argue both sides of this, so I guess it is fine either way.\n\nIn GFW I log a array of argv[0] strings in a generic \"data_json\" event.\nI could also log additional \"data_json\" events with more structured\ndata if needed.\n\nOn the other hand, you're proposing a \"cmd_ancestry\" event with a\nsingle array of strings. You would have to expand the call signature\nof the trace2_cmd_ancestry() API to add additional data and inside\ntr2_tgt_event.c add additional fields to the JSON being composed.\n\nSo both are about equal.\n\n(I'll avoid the temptation to make a snarky comment about fixing\nyour post processing. :-) :-) :-) )\n\nIt really doesn't matter one way or the other.\n\n> - Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n> the \"command name\" - excluding argv, excluding path, etc. It seems\n\nSo you're trying to log argv[0] of the process and not the full\ncommand line. That's what I'm doing.\n\n> like this is a little more safe about excluding personal information\n> from the traces which take the form of \"myscript.sh\n> --password=hunter2\", but would still be worrisome for something like\n> \"mysupersecretproject.sh\". I'm not sure whether that means we still\n> want to guard it with a config flag, though.\n\nYou might check whether you get the name of the script or just get\na lot of entries with just \"/usr/bin/bash\".\n\nThere's lots of PII in the data stream to worry about.\nThe name of the command is just one aspect, but I digress.\n\n> - I also added a lot to the commit message; hopefully it's not too\n> rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n> wasn't going to cut it.\n> - As for testing, I followed the lead of GfW's parentage info - \"this\n> isn't portable so writing tests for it will suck, just scrub it from\n> the tests\". Maybe it makes sense to do some more\n> platform-specific-ness in the test suite instead? I wasn't sure.\n\nyeah, that's probably best. Unless you can tokenize it properly\nso that you can predict the results in a HEREDOC in the test source.\n\nFor example, you might try to test tracing a command (where a top-level\n\"git foo\" (SPACE form) spawns a \"git-foo\" (DASHED form) and check the\noutput for the child.\n\n> \n> Thanks, all.\n> - Emily\n> \n> Makefile | 5 ++++\n> compat/procinfo.c | 53 +++++++++++++++++++++++++++++++++++++++\n> config.mak.uname | 1 +\n> git-compat-util.h | 6 +++++\n> t/t0210/scrub_normal.perl | 6 +++++\n> t/t0211/scrub_perf.perl | 5 ++++\n> t/t0212/parse_events.perl | 5 +++-\n> trace2.c | 13 ++++++++++\n> trace2.h | 12 ++++++++-\n> trace2/tr2_tgt.h | 3 +++\n> trace2/tr2_tgt_event.c | 21 ++++++++++++++++\n> trace2/tr2_tgt_normal.c | 19 ++++++++++++++\n> trace2/tr2_tgt_perf.c | 16 ++++++++++++\n> 13 files changed, 163 insertions(+), 2 deletions(-)\n> create mode 100644 compat/procinfo.c\n> \n> diff --git a/Makefile b/Makefile\n> index 93664d6714..330e4fa011 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n> \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n> endif\n> \n> +ifdef HAVE_PROCFS_LINUX\n> +\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n> +\tCOMPAT_OBJS += compat/procinfo.o\n> +endif\n> +\n> ifdef HAVE_NS_GET_EXECUTABLE_PATH\n> \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n> endif\n> diff --git a/compat/procinfo.c b/compat/procinfo.c\n> new file mode 100644\n> index 0000000000..523600673f\n> --- /dev/null\n> +++ b/compat/procinfo.c\n> @@ -0,0 +1,53 @@\n> +#include \"cache.h\"\n> +\n> +#include \"strbuf.h\"\n> +#include \"trace2.h\"\n> +\n> +char *get_process_name(int pid)\n> +{\n> +#ifdef HAVE_PROCFS_LINUX\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf out = STRBUF_INIT;\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n> +\tif (!strbuf_read_file(&out, procfs_path.buf, 0))\n> +\t{\n> +\t\t/* All done with file reads, clean up early */\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\treturn strbuf_detach(&out, NULL);\n> +\t}\n> +#endif\n> +\n> +\t/* NEEDSWORK: add non-procfs implementations here. */\n> +\treturn NULL;\n> +}\n> +\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n> +\t{\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tchar *names[2];\n> +\t\tnames[0] = get_process_name(getppid());\n> +\t\tnames[1] = NULL;\n\nYou're only logging 1 parent. That's fine to get started.\n\nI'm logging IIRC 10 parents on GFW. That might seem overkill,\nbut there are lots of intermediate parents that hide what is\nhappening. For example, a \"git push\" might spawn \"git remote-https\"\nwhich spawns \"git-remote-https\" which spawn \"git send-pack\" which\nspawns \"git pack-objects\".\n\nAnd that doesn't include who called push.\n\nAnd it's not uncommon to see 2 or 3 \"bash\" entries in the array\nbecause of the bash scripts being run.\n\n> +\n> +\t\tif (!names[0])\n> +\t\t\treturn;\n> +\n> +\t\ttrace2_cmd_ancestry((const char**)names);\n> +\n> +\t\tfree(names[0]);\n> +\t}\n> +\n> +\treturn;\n> +}\n> diff --git a/config.mak.uname b/config.mak.uname\n> index cb443b4e02..7ad110a1d2 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n> \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n> \tBASIC_CFLAGS += -DHAVE_SYSINFO\n> \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n> +\tHAVE_PROCFS_LINUX = YesPlease\n> endif\n> ifeq ($(uname_S),GNU/kFreeBSD)\n> \tHAVE_ALLOCA_H = YesPlease\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index a508dbe5a3..cc7d5d8a2a 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -1382,4 +1382,10 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)\n> \n> void sleep_millisec(int millisec);\n> \n> +/*\n> + * Convert PID to process name (as would show in top/task manager). Returns\n> + * NULL if unimplemented - be sure to check for NULL at callsite.\n> + */\n> +char *get_process_name(int pid);\n> +\n> #endif\n> diff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\n> index c65d1a815e..7cc4de392a 100644\n> --- a/t/t0210/scrub_normal.perl\n> +++ b/t/t0210/scrub_normal.perl\n> @@ -42,6 +42,12 @@\n> \t# so just omit it for testing purposes.\n> \t# print \"cmd_path _EXE_\\n\";\n> }\n> + elsif ($line =~ m/^cmd_ancestry/) {\n> +\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n> +\t# sake, skip it when parsing normal.\n> +\t#\n> +\t# print \"$line\";\n> + }\n> else {\n> \tprint \"$line\";\n> }\n> diff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\n> index 351af7844e..d164b750ff 100644\n> --- a/t/t0211/scrub_perf.perl\n> +++ b/t/t0211/scrub_perf.perl\n> @@ -44,6 +44,11 @@\n> \t# $tokens[$col_rest] = \"_EXE_\";\n> \tgoto SKIP_LINE;\n> }\n> + elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n> +\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n> +\t# so skip it.\n> +\tgoto SKIP_LINE;\n> + }\n> elsif ($tokens[$col_event] =~ m/child_exit/) {\n> \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n> }\n> diff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\n> index 6584bb5634..b6408560c0 100644\n> --- a/t/t0212/parse_events.perl\n> +++ b/t/t0212/parse_events.perl\n> @@ -132,7 +132,10 @@\n> \t# just omit it for testing purposes.\n> \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n> }\n> -\n> + elsif ($event eq 'cmd_ancestry') {\n> +\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n> +\t# just skip it for testing purposes.\n> + }\n> elsif ($event eq 'cmd_name') {\n> \t$processes->{$sid}->{'name'} = $line->{'name'};\n> \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\n> diff --git a/trace2.c b/trace2.c\n> index 256120c7fd..b9b154ac44 100644\n> --- a/trace2.c\n> +++ b/trace2.c\n> @@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n> \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n> }\n> \n> +void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tstruct tr2_tgt *tgt_j;\n> +\tint j;\n> +\n> +\tif (!trace2_enabled)\n> +\t\treturn;\n> +\n> +\tfor_each_wanted_builtin (j, tgt_j)\n> +\t\tif (tgt_j->pfn_command_ancestry_fl)\n> +\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n> +}\n> +\n> void trace2_cmd_name_fl(const char *file, int line, const char *name)\n> {\n> \tstruct tr2_tgt *tgt_j;\n> diff --git a/trace2.h b/trace2.h\n> index ede18c2e06..23743ac62b 100644\n> --- a/trace2.h\n> +++ b/trace2.h\n> @@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n> \n> #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n> \n> +/*\n> + * Emit an 'ancestry' event with the process name of the current process's\n> + * parent process.\n> + * This gives post-processors a way to determine what invoked the command and\n> + * learn more about usage patterns.\n> + */\n> +void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n> +\n> +#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n> +\n> /*\n> * Emit a 'cmd_name' event with the canonical name of the command.\n> * This gives post-processors a simple field to identify the command\n> @@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n> \tTRACE2_PROCESS_INFO_EXIT,\n> };\n> \n> -#if defined(GIT_WINDOWS_NATIVE)\n> +#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n> void trace2_collect_process_info(enum trace2_process_info_reason reason);\n> #else\n> #define trace2_collect_process_info(reason) \\\n> diff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\n> index 7b90469212..1f66fd6573 100644\n> --- a/trace2/tr2_tgt.h\n> +++ b/trace2/tr2_tgt.h\n> @@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n> \n> typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n> \t\t\t\t\t const char *command_path);\n> +typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n> +\t\t\t\t\t\tconst char **parent_names);\n> typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n> \t\t\t\t\t const char *name,\n> \t\t\t\t\t const char *hierarchy);\n> @@ -108,6 +110,7 @@ struct tr2_tgt {\n> \ttr2_tgt_evt_atexit_t *pfn_atexit;\n> \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n> \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n> +\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n> \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n> \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n> \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\n> diff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\n> index 6353e8ad91..578a9a5287 100644\n> --- a/trace2/tr2_tgt_event.c\n> +++ b/trace2/tr2_tgt_event.c\n> @@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tjw_release(&jw);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *event_name = \"cmd_ancestry\";\n> +\tconst char *parent_name = NULL;\n> +\tstruct json_writer jw = JSON_WRITER_INIT;\n> +\n> +\tjw_object_begin(&jw, 0);\n> +\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n> +\tjw_object_inline_begin_array(&jw, \"ancestry\");\n> +\n> +\twhile ((parent_name = *parent_names++))\n> +\t\tjw_array_string(&jw, parent_name);\n\nYou're building the array with the immediate parent in a[0]\nand the grandparent in a[1], and etc. This is the same as\nI did in GFW.\n\nPerhaps state this in the docs somewhere.\n\n> +\n> +\tjw_end(&jw); /* 'ancestry' array */\n> +\tjw_end(&jw); /* event object */\n> +\n> +\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n> +\tjw_release(&jw);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> diff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\n> index 31b602c171..a5751c8864 100644\n> --- a/trace2/tr2_tgt_normal.c\n> +++ b/trace2/tr2_tgt_normal.c\n> @@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tstrbuf_release(&buf_payload);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *parent_name = NULL;\n> +\tstruct strbuf buf_payload = STRBUF_INIT;\n> +\n> +\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n> +\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n> +\twhile ((parent_name = *parent_names++)) {\n> +\t\tstrbuf_addstr(&buf_payload, parent_name);\n\nDid you want to quote each parent's name?\n\n> +\t\t/* if we'll write another one after this, add a delimiter */\n> +\t\tif (parent_names && *parent_names)\n> +\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n> +\t}\n> +\n> +\tnormal_io_write_fl(file, line, &buf_payload);\n> +\tstrbuf_release(&buf_payload);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> diff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\n> index a8018f18cc..af4d65a0a5 100644\n> --- a/trace2/tr2_tgt_perf.c\n> +++ b/trace2/tr2_tgt_perf.c\n> @@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tstrbuf_release(&buf_payload);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *event_name = \"cmd_ancestry\";\n> +\tstruct strbuf buf_payload = STRBUF_INIT;\n> +\n> +\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n> +\t/* It's not an argv but the rules are basically the same. */\n> +\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n\nThis will have whitespace delimiters between the quoted strings\nrather than commas. Just checking if that's what you wanted.\n\nI'm not sure it matters, since this stream is intended for human\nparsing.\n\n> +\tstrbuf_addch(&buf_payload, ']');\n> +\n> +\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n> +\t\t\t &buf_payload);\n> +\tstrbuf_release(&buf_payload);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> \n\nWe should update Documentation/technical/api-trace2.txt too.\n\nThanks,\nJeff\n"},{"id":"425235","messageId":"YKgSc5OgVOt6HQqW@google.com","threadId":"55637","inReplyTo":"1e3bb53e-895b-f571-1c03-a6ae6499746d@jeffhostetler.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-21T20:05:07Z","receivedAt":"2021-05-21T20:05:15Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Fri, May 21, 2021 at 03:15:16PM -0400, Jeff Hostetler wrote:\n> On 5/20/21 5:05 PM, Emily Shaffer wrote:\n> > - I took a look at Jeff H's advice on using a \"data_json\" event to log\n> > this and decided it would be a little more flexible to add a new event\n> > instead. If we want, it'd be feasible to then shoehorn the GfW parent\n> > tree stuff into this new event too. Doing it this way is definitely\n> > easier to parse for Google's trace analysis system (which for now\n> > completely skips \"data_json\" as it's polymorphic), and also - I think\n> > - means that we can add more fields later on if we need to (thread\n> > info, different fields than just /proc/n/comm like exec path, argv,\n> > whatever).\n> \n> I could argue both sides of this, so I guess it is fine either way.\n> \n> In GFW I log a array of argv[0] strings in a generic \"data_json\" event.\n> I could also log additional \"data_json\" events with more structured\n> data if needed.\n> \n> On the other hand, you're proposing a \"cmd_ancestry\" event with a\n> single array of strings. You would have to expand the call signature\n> of the trace2_cmd_ancestry() API to add additional data and inside\n> tr2_tgt_event.c add additional fields to the JSON being composed.\n> \n> So both are about equal.\n> \n> (I'll avoid the temptation to make a snarky comment about fixing\n> your post processing. :-) :-) :-) )\n\n;P\n\n(I don't have much to add - this is an accurate summary of what I\nthought about, too. Thanks for writing it out.)\n\n> \n> It really doesn't matter one way or the other.\n> \n> > - Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n> > the \"command name\" - excluding argv, excluding path, etc. It seems\n> \n> So you're trying to log argv[0] of the process and not the full\n> command line. That's what I'm doing.\n\nIt's close to argv[0], yeah. POSIX docs indicate it might be truncated\nin a way that argv[0] hasn't been, but it also doesn't include the\nleading path (as far as I've seen). For example, a long-running helper\nscript I use with mutt, right now (gaffing on line length in email to\nhelp with argv clarity, sorry):\n\n $ ps aux | grep mutt\n emilysh+ 4119883 0.0 0.0 6892 3600 pts/6 S+ 12:44 0:00 /bin/bash /usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh /var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n # comm is truncated to 15ch, except apparently in the cases of some\n # kernel worker processes I saw with much longer names?\n $ cat /proc/4119883/comm\n open-vim-in-new\n # exe is a link to the executable, which means bash as this is a\n # script\n $ ls -lha /proc/4119883/exe\n lrwxrwxrwx 1 emilyshaffer primarygroup 0 May 21 12:44\n /proc/4119883/exe -> /usr/bin/bash\n # cmdline has the whole argv, separated on NUL so it runs together in\n # editor\n $ cat /proc/4119883/cmdline\n /bin/bash/usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh/var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n\nJonathan N pointed out that the process name (the thing in 'comm') can\nalso be manually manipulated by the process itself, and 'man procfs'\nalso talks about 'PR_SET_NAME' and 'PR_GET_NAME' operations in\n'prctl()', so that tracks. (It doesn't look like we can use prctl() to\nfind out the names of processes besides the current process, though, so\nthe procfs stuff is still needed. Dang.)\n\n> \n> > like this is a little more safe about excluding personal information\n> > from the traces which take the form of \"myscript.sh\n> > --password=hunter2\", but would still be worrisome for something like\n> > \"mysupersecretproject.sh\". I'm not sure whether that means we still\n> > want to guard it with a config flag, though.\n> \n> You might check whether you get the name of the script or just get\n> a lot of entries with just \"/usr/bin/bash\".\n\nSee above :)\n\n> There's lots of PII in the data stream to worry about.\n> The name of the command is just one aspect, but I digress.\n\nYes, that's what we've noticed too, so a process name isn't worrying us\nthat much more.\n\n> \n> > - I also added a lot to the commit message; hopefully it's not too\n> > rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n> > wasn't going to cut it.\n> > - As for testing, I followed the lead of GfW's parentage info - \"this\n> > isn't portable so writing tests for it will suck, just scrub it from\n> > the tests\". Maybe it makes sense to do some more\n> > platform-specific-ness in the test suite instead? I wasn't sure.\n> \n> yeah, that's probably best. Unless you can tokenize it properly\n> so that you can predict the results in a HEREDOC in the test source.\n> \n> For example, you might try to test tracing a command (where a top-level\n> \"git foo\" (SPACE form) spawns a \"git-foo\" (DASHED form) and check the\n> output for the child.\n\nYeah, I had trouble with even deciding when to attempt such a check or\nnot.\n> > +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n> > +\t{\n> > +\t\t/*\n> > +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> > +\t\t * see compat/win32/trace2_win32_process_info.c.\n> > +\t\t */\n> > +\t\tchar *names[2];\n> > +\t\tnames[0] = get_process_name(getppid());\n> > +\t\tnames[1] = NULL;\n> \n> You're only logging 1 parent. That's fine to get started.\n> \n> I'm logging IIRC 10 parents on GFW. That might seem overkill,\n> but there are lots of intermediate parents that hide what is\n> happening. For example, a \"git push\" might spawn \"git remote-https\"\n> which spawns \"git-remote-https\" which spawn \"git send-pack\" which\n> spawns \"git pack-objects\".\n> \n> And that doesn't include who called push.\n> \n> And it's not uncommon to see 2 or 3 \"bash\" entries in the array\n> because of the bash scripts being run.\n\nAgree. But it's expensive - I didn't find a handy library call to find\n\"parent ID of given process ID\", so I think we'd have to manipulate\nprocfs; and so far I only see parent ID in summary infos like\n/proc/n/status or /proc/n/stat, which contain lots of other info too and\nwould need parsing.\n\nWe could reduce the cost a little bit by grabbing the process name from\nthe status or stat as well, and therefore still only opening one file per\nprocess, but I'd want to check whether the formats are expected to be\nstable for those things.\n\n> > +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> > +{\n> > +\tconst char *event_name = \"cmd_ancestry\";\n> > +\tconst char *parent_name = NULL;\n> > +\tstruct json_writer jw = JSON_WRITER_INIT;\n> > +\n> > +\tjw_object_begin(&jw, 0);\n> > +\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n> > +\tjw_object_inline_begin_array(&jw, \"ancestry\");\n> > +\n> > +\twhile ((parent_name = *parent_names++))\n> > +\t\tjw_array_string(&jw, parent_name);\n> \n> You're building the array with the immediate parent in a[0]\n> and the grandparent in a[1], and etc. This is the same as\n> I did in GFW.\n> \n> Perhaps state this in the docs somewhere.\n\nSure, makes sense. I think I neglected any doc work whatsoever in this\npatch anyways, whoops :)\n\n> > +\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n> > +\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n> > +\twhile ((parent_name = *parent_names++)) {\n> > +\t\tstrbuf_addstr(&buf_payload, parent_name);\n> \n> Did you want to quote each parent's name?\n\nI'd rather not - since they're going into an array anyway, I'd expect\nthe array delimiters to be enough. Am I being naive? 'normal' looks to\nme like it's supposed to be mostly human readable anyways, rather than\nparseable?\n\n> > +\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n> > +\t/* It's not an argv but the rules are basically the same. */\n> > +\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n> \n> This will have whitespace delimiters between the quoted strings\n> rather than commas. Just checking if that's what you wanted.\n> \n> I'm not sure it matters, since this stream is intended for human\n> parsing.\n\nYeah, it seems fine to me as is.\n\n> \n> We should update Documentation/technical/api-trace2.txt too.\n\nYep, thanks.\n\nI appreciate the review, Jeff.\n\n - Emily\n"},{"id":"425237","messageId":"02be01d74e7f$2c282300$84786900$@nexbridge.com","threadId":"55637","inReplyTo":"YKgSc5OgVOt6HQqW@google.com","subject":"RE: [PATCH v2] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-05-21T20:23:28Z","receivedAt":"2021-05-21T20:23:46Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"\nOn May 21, 2021 4:05 PM, Emily Shaffer wrote:\n>On Fri, May 21, 2021 at 03:15:16PM -0400, Jeff Hostetler wrote:\n>> On 5/20/21 5:05 PM, Emily Shaffer wrote:\n>> > - I took a look at Jeff H's advice on using a \"data_json\" event to log\n>> > this and decided it would be a little more flexible to add a new event\n>> > instead. If we want, it'd be feasible to then shoehorn the GfW parent\n>> > tree stuff into this new event too. Doing it this way is definitely\n>> > easier to parse for Google's trace analysis system (which for now\n>> > completely skips \"data_json\" as it's polymorphic), and also - I think\n>> > - means that we can add more fields later on if we need to (thread\n>> > info, different fields than just /proc/n/comm like exec path, argv,\n>> > whatever).\n>>\n>> I could argue both sides of this, so I guess it is fine either way.\n>>\n>> In GFW I log a array of argv[0] strings in a generic \"data_json\" event.\n>> I could also log additional \"data_json\" events with more structured\n>> data if needed.\n>>\n>> On the other hand, you're proposing a \"cmd_ancestry\" event with a\n>> single array of strings. You would have to expand the call signature\n>> of the trace2_cmd_ancestry() API to add additional data and inside\n>> tr2_tgt_event.c add additional fields to the JSON being composed.\n>>\n>> So both are about equal.\n>>\n>> (I'll avoid the temptation to make a snarky comment about fixing your\n>> post processing. :-) :-) :-) )\n>\n>;P\n>\n>(I don't have much to add - this is an accurate summary of what I thought about, too. Thanks for writing it out.)\n>\n>>\n>> It really doesn't matter one way or the other.\n>>\n>> > - Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n>> > the \"command name\" - excluding argv, excluding path, etc. It\n>> > seems\n>>\n>> So you're trying to log argv[0] of the process and not the full\n>> command line. That's what I'm doing.\n>\n>It's close to argv[0], yeah. POSIX docs indicate it might be truncated in a way that argv[0] hasn't been, but it also doesn't\ninclude the\n>leading path (as far as I've seen). For example, a long-running helper script I use with mutt, right now (gaffing on line length in\nemail to\n>help with argv clarity, sorry):\n>\n> $ ps aux | grep mutt\n> emilysh+ 4119883 0.0 0.0 6892 3600 pts/6 S+ 12:44 0:00 /bin/bash\n/usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-\n>new-split.sh /var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n> # comm is truncated to 15ch, except apparently in the cases of some\n> # kernel worker processes I saw with much longer names?\n> $ cat /proc/4119883/comm\n> open-vim-in-new\n> # exe is a link to the executable, which means bash as this is a\n> # script\n> $ ls -lha /proc/4119883/exe\n> lrwxrwxrwx 1 emilyshaffer primarygroup 0 May 21 12:44\n> /proc/4119883/exe -> /usr/bin/bash\n> # cmdline has the whole argv, separated on NUL so it runs together in\n> # editor\n> $ cat /proc/4119883/cmdline\n> /bin/bash/usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh/var/tmp/mutt-podkayne-413244-1263002-\n>7433772284891386689\n>\n>Jonathan N pointed out that the process name (the thing in 'comm') can also be manually manipulated by the process itself, and 'man\n>procfs'\n>also talks about 'PR_SET_NAME' and 'PR_GET_NAME' operations in 'prctl()', so that tracks. (It doesn't look like we can use prctl()\nto find\n>out the names of processes besides the current process, though, so the procfs stuff is still needed. Dang.)\n>\n>>\n>> > like this is a little more safe about excluding personal information\n>> > from the traces which take the form of \"myscript.sh\n>> > --password=hunter2\", but would still be worrisome for something like\n>> > \"mysupersecretproject.sh\". I'm not sure whether that means we still\n>> > want to guard it with a config flag, though.\n>>\n>> You might check whether you get the name of the script or just get a\n>> lot of entries with just \"/usr/bin/bash\".\n>\n>See above :)\n>\n>> There's lots of PII in the data stream to worry about.\n>> The name of the command is just one aspect, but I digress.\n>\n>Yes, that's what we've noticed too, so a process name isn't worrying us that much more.\n>\n>>\n>> > - I also added a lot to the commit message; hopefully it's not too\n>> > rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n>> > wasn't going to cut it.\n>> > - As for testing, I followed the lead of GfW's parentage info - \"this\n>> > isn't portable so writing tests for it will suck, just scrub it from\n>> > the tests\". Maybe it makes sense to do some more\n>> > platform-specific-ness in the test suite instead? I wasn't sure.\n>>\n>> yeah, that's probably best. Unless you can tokenize it properly so\n>> that you can predict the results in a HEREDOC in the test source.\n>>\n>> For example, you might try to test tracing a command (where a\n>> top-level \"git foo\" (SPACE form) spawns a \"git-foo\" (DASHED form) and\n>> check the output for the child.\n>\n>Yeah, I had trouble with even deciding when to attempt such a check or not.\n>> > +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n>> > +\t{\n>> > +\t\t/*\n>> > +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n>> > +\t\t * see compat/win32/trace2_win32_process_info.c.\n>> > +\t\t */\n>> > +\t\tchar *names[2];\n>> > +\t\tnames[0] = get_process_name(getppid());\n>> > +\t\tnames[1] = NULL;\n>>\n>> You're only logging 1 parent. That's fine to get started.\n>>\n>> I'm logging IIRC 10 parents on GFW. That might seem overkill, but\n>> there are lots of intermediate parents that hide what is happening.\n>> For example, a \"git push\" might spawn \"git remote-https\"\n>> which spawns \"git-remote-https\" which spawn \"git send-pack\" which\n>> spawns \"git pack-objects\".\n>>\n>> And that doesn't include who called push.\n>>\n>> And it's not uncommon to see 2 or 3 \"bash\" entries in the array\n>> because of the bash scripts being run.\n>\n>Agree. But it's expensive - I didn't find a handy library call to find \"parent ID of given process ID\", so I think we'd have to\nmanipulate\n>procfs; and so far I only see parent ID in summary infos like /proc/n/status or /proc/n/stat, which contain lots of other info too\nand would\n>need parsing.\n>\n>We could reduce the cost a little bit by grabbing the process name from the status or stat as well, and therefore still only\nopening one file\n>per process, but I'd want to check whether the formats are expected to be stable for those things.\n>\n>> > +static void fn_command_ancestry_fl(const char *file, int line,\n>> > +const char **parent_names) {\n>> > +\tconst char *event_name = \"cmd_ancestry\";\n>> > +\tconst char *parent_name = NULL;\n>> > +\tstruct json_writer jw = JSON_WRITER_INIT;\n>> > +\n>> > +\tjw_object_begin(&jw, 0);\n>> > +\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n>> > +\tjw_object_inline_begin_array(&jw, \"ancestry\");\n>> > +\n>> > +\twhile ((parent_name = *parent_names++))\n>> > +\t\tjw_array_string(&jw, parent_name);\n>>\n>> You're building the array with the immediate parent in a[0] and the\n>> grandparent in a[1], and etc. This is the same as I did in GFW.\n>>\n>> Perhaps state this in the docs somewhere.\n>\n>Sure, makes sense. I think I neglected any doc work whatsoever in this patch anyways, whoops :)\n>\n>> > +\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n>> > +\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n>> > +\twhile ((parent_name = *parent_names++)) {\n>> > +\t\tstrbuf_addstr(&buf_payload, parent_name);\n>>\n>> Did you want to quote each parent's name?\n>\n>I'd rather not - since they're going into an array anyway, I'd expect the array delimiters to be enough. Am I being naive? 'normal'\nlooks to\n>me like it's supposed to be mostly human readable anyways, rather than parseable?\n>\n>> > +\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n>> > +\t/* It's not an argv but the rules are basically the same. */\n>> > +\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n>>\n>> This will have whitespace delimiters between the quoted strings rather\n>> than commas. Just checking if that's what you wanted.\n>>\n>> I'm not sure it matters, since this stream is intended for human\n>> parsing.\n>\n>Yeah, it seems fine to me as is.\n>\n>>\n>> We should update Documentation/technical/api-trace2.txt too.\n>\n>Yep, thanks.\n>\n>I appreciate the review, Jeff.\n\nI checked the performance of my NonStop parent lookup implementation. It's fast enough that no one would notice (8 microseconds on\nthe oldest slowest machine I could find).\n\nJust an FYI.\n-Randall\n\n"},{"id":"425281","messageId":"xmqqwnrrr7sp.fsf@gitster.g","threadId":"55637","inReplyTo":"YKgDxahhwK/zYznH@google.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-05-21T23:22:46Z","receivedAt":"2021-05-21T23:22:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n>> > we will need to discover the name another way. However, the process ID\n>> > should be sufficient regardless of platform.\n>> \n>> Not a strong objection, but I wonder if seeing random integer(s) is\n>> better than not having cmd_ancestry info at all. The latter better\n>> signals that the platform does not yet have the \"parent process\n>> name\" feature, I would think.\n>\n> Hm, we could...\n\nPlease don't. There is a misreading here.\n\nYou mentioned \"However, the process ID should be sufficient\" and I\nread it as \"In the worst case we can emit the process ID if we do\nnot know how to turn it into name\", and to that I said \"showing\nprocess IDs is not all that useful as they are random integers\nwithout extra info on processes that were running back when the log\nentry was taken\". Similarly, my later \"OK, we do not show pid as a\nplaceholder.\" is \"Contrary to what I thought you said earlier, you\ndo not give raw process IDs and instead honestly say we do not have\nthat information by omitting the record. I am happy to see what the\nactual patch does\".\n\nThanks.\n"},{"id":"425300","messageId":"ad593454-27bb-7807-928d-5762ccc853cb@jeffhostetler.com","threadId":"55637","inReplyTo":"YKgSc5OgVOt6HQqW@google.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-05-22T11:18:09Z","receivedAt":"2021-05-22T11:18:12Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 5/21/21 4:05 PM, Emily Shaffer wrote:\n> On Fri, May 21, 2021 at 03:15:16PM -0400, Jeff Hostetler wrote:\n>> On 5/20/21 5:05 PM, Emily Shaffer wrote:\n>>> - I took a look at Jeff H's advice on using a \"data_json\" event to log\n>>> this and decided it would be a little more flexible to add a new event\n>>> instead. If we want, it'd be feasible to then shoehorn the GfW parent\n>>> tree stuff into this new event too. Doing it this way is definitely\n>>> easier to parse for Google's trace analysis system (which for now\n>>> completely skips \"data_json\" as it's polymorphic), and also - I think\n>>> - means that we can add more fields later on if we need to (thread\n>>> info, different fields than just /proc/n/comm like exec path, argv,\n>>> whatever).\n>>\n>> I could argue both sides of this, so I guess it is fine either way.\n>>\n>> In GFW I log a array of argv[0] strings in a generic \"data_json\" event.\n>> I could also log additional \"data_json\" events with more structured\n>> data if needed.\n>>\n>> On the other hand, you're proposing a \"cmd_ancestry\" event with a\n>> single array of strings. You would have to expand the call signature\n>> of the trace2_cmd_ancestry() API to add additional data and inside\n>> tr2_tgt_event.c add additional fields to the JSON being composed.\n>>\n>> So both are about equal.\n>>\n>> (I'll avoid the temptation to make a snarky comment about fixing\n>> your post processing. :-) :-) :-) )\n> \n> ;P\n> \n> (I don't have much to add - this is an accurate summary of what I\n> thought about, too. Thanks for writing it out.)\n> \n>>\n>> It really doesn't matter one way or the other.\n>>\n>>> - Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n>>> the \"command name\" - excluding argv, excluding path, etc. It seems\n>>\n>> So you're trying to log argv[0] of the process and not the full\n>> command line. That's what I'm doing.\n> \n> It's close to argv[0], yeah. POSIX docs indicate it might be truncated\n> in a way that argv[0] hasn't been, but it also doesn't include the\n> leading path (as far as I've seen). For example, a long-running helper\n> script I use with mutt, right now (gaffing on line length in email to\n> help with argv clarity, sorry):\n> \n> $ ps aux | grep mutt\n> emilysh+ 4119883 0.0 0.0 6892 3600 pts/6 S+ 12:44 0:00 /bin/bash /usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh /var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n> # comm is truncated to 15ch, except apparently in the cases of some\n> # kernel worker processes I saw with much longer names?\n> $ cat /proc/4119883/comm\n> open-vim-in-new\n> # exe is a link to the executable, which means bash as this is a\n> # script\n> $ ls -lha /proc/4119883/exe\n> lrwxrwxrwx 1 emilyshaffer primarygroup 0 May 21 12:44\n> /proc/4119883/exe -> /usr/bin/bash\n> # cmdline has the whole argv, separated on NUL so it runs together in\n> # editor\n> $ cat /proc/4119883/cmdline\n> /bin/bash/usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh/var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n> \n> Jonathan N pointed out that the process name (the thing in 'comm') can\n> also be manually manipulated by the process itself, and 'man procfs'\n> also talks about 'PR_SET_NAME' and 'PR_GET_NAME' operations in\n> 'prctl()', so that tracks. (It doesn't look like we can use prctl() to\n> find out the names of processes besides the current process, though, so\n> the procfs stuff is still needed. Dang.)\n> \n>>\n>>> like this is a little more safe about excluding personal information\n>>> from the traces which take the form of \"myscript.sh\n>>> --password=hunter2\", but would still be worrisome for something like\n>>> \"mysupersecretproject.sh\". I'm not sure whether that means we still\n>>> want to guard it with a config flag, though.\n>>\n>> You might check whether you get the name of the script or just get\n>> a lot of entries with just \"/usr/bin/bash\".\n> \n> See above :)\n> \n>> There's lots of PII in the data stream to worry about.\n>> The name of the command is just one aspect, but I digress.\n> \n> Yes, that's what we've noticed too, so a process name isn't worrying us\n> that much more.\n> \n>>\n>>> - I also added a lot to the commit message; hopefully it's not too\n>>> rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n>>> wasn't going to cut it.\n>>> - As for testing, I followed the lead of GfW's parentage info - \"this\n>>> isn't portable so writing tests for it will suck, just scrub it from\n>>> the tests\". Maybe it makes sense to do some more\n>>> platform-specific-ness in the test suite instead? I wasn't sure.\n>>\n>> yeah, that's probably best. Unless you can tokenize it properly\n>> so that you can predict the results in a HEREDOC in the test source.\n>>\n>> For example, you might try to test tracing a command (where a top-level\n>> \"git foo\" (SPACE form) spawns a \"git-foo\" (DASHED form) and check the\n>> output for the child.\n> \n> Yeah, I had trouble with even deciding when to attempt such a check or\n> not.\n>>> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n>>> +\t{\n>>> +\t\t/*\n>>> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n>>> +\t\t * see compat/win32/trace2_win32_process_info.c.\n>>> +\t\t */\n>>> +\t\tchar *names[2];\n>>> +\t\tnames[0] = get_process_name(getppid());\n>>> +\t\tnames[1] = NULL;\n>>\n>> You're only logging 1 parent. That's fine to get started.\n>>\n>> I'm logging IIRC 10 parents on GFW. That might seem overkill,\n>> but there are lots of intermediate parents that hide what is\n>> happening. For example, a \"git push\" might spawn \"git remote-https\"\n>> which spawns \"git-remote-https\" which spawn \"git send-pack\" which\n>> spawns \"git pack-objects\".\n>>\n>> And that doesn't include who called push.\n>>\n>> And it's not uncommon to see 2 or 3 \"bash\" entries in the array\n>> because of the bash scripts being run.\n> \n> Agree. But it's expensive - I didn't find a handy library call to find\n> \"parent ID of given process ID\", so I think we'd have to manipulate\n> procfs; and so far I only see parent ID in summary infos like\n> /proc/n/status or /proc/n/stat, which contain lots of other info too and\n> would need parsing.\n> \n> We could reduce the cost a little bit by grabbing the process name from\n> the status or stat as well, and therefore still only opening one file per\n> process, but I'd want to check whether the formats are expected to be\n> stable for those things.\n\nYeah, don't force it collect a bunch of data for n parents that you\ndon't need. I guess my point was that there were lots of time when\nthere were too many \"unhelpful\" parents in the ancestry chain. So\nmaybe confirm that you can get what you need from just 1 parent.\nI needed more than 1 to tell that a particular command was interactive\nvs launched by VS or VSCode or etc.\n\n\n> \n>>> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n>>> +{\n>>> +\tconst char *event_name = \"cmd_ancestry\";\n>>> +\tconst char *parent_name = NULL;\n>>> +\tstruct json_writer jw = JSON_WRITER_INIT;\n>>> +\n>>> +\tjw_object_begin(&jw, 0);\n>>> +\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n>>> +\tjw_object_inline_begin_array(&jw, \"ancestry\");\n>>> +\n>>> +\twhile ((parent_name = *parent_names++))\n>>> +\t\tjw_array_string(&jw, parent_name);\n>>\n>> You're building the array with the immediate parent in a[0]\n>> and the grandparent in a[1], and etc. This is the same as\n>> I did in GFW.\n>>\n>> Perhaps state this in the docs somewhere.\n> \n> Sure, makes sense. I think I neglected any doc work whatsoever in this\n> patch anyways, whoops :)\n> \n>>> +\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n>>> +\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n>>> +\twhile ((parent_name = *parent_names++)) {\n>>> +\t\tstrbuf_addstr(&buf_payload, parent_name);\n>>\n>> Did you want to quote each parent's name?\n> \n> I'd rather not - since they're going into an array anyway, I'd expect\n> the array delimiters to be enough. Am I being naive? 'normal' looks to\n> me like it's supposed to be mostly human readable anyways, rather than\n> parseable?\n> \n>>> +\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n>>> +\t/* It's not an argv but the rules are basically the same. */\n>>> +\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n>>\n>> This will have whitespace delimiters between the quoted strings\n>> rather than commas. Just checking if that's what you wanted.\n>>\n>> I'm not sure it matters, since this stream is intended for human\n>> parsing.\n> \n> Yeah, it seems fine to me as is.\n> \n>>\n>> We should update Documentation/technical/api-trace2.txt too.\n> \n> Yep, thanks.\n> \n> I appreciate the review, Jeff.\n> \n> - Emily\n> \n"},{"id":"425447","messageId":"YKvyfe0arCWNR+N7@google.com","threadId":"55637","inReplyTo":"xmqqwnrrr7sp.fsf@gitster.g","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-24T18:37:49Z","receivedAt":"2021-05-24T18:37:58Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Sat, May 22, 2021 at 08:22:46AM +0900, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> >> > we will need to discover the name another way. However, the process ID\n> >> > should be sufficient regardless of platform.\n> >> \n> >> Not a strong objection, but I wonder if seeing random integer(s) is\n> >> better than not having cmd_ancestry info at all. The latter better\n> >> signals that the platform does not yet have the \"parent process\n> >> name\" feature, I would think.\n> >\n> > Hm, we could...\n> \n> Please don't. There is a misreading here.\n> \n> You mentioned \"However, the process ID should be sufficient\" and I\n> read it as \"In the worst case we can emit the process ID if we do\n> not know how to turn it into name\", and to that I said \"showing\n> process IDs is not all that useful as they are random integers\n> without extra info on processes that were running back when the log\n> entry was taken\". Similarly, my later \"OK, we do not show pid as a\n> placeholder.\" is \"Contrary to what I thought you said earlier, you\n> do not give raw process IDs and instead honestly say we do not have\n> that information by omitting the record. I am happy to see what the\n> actual patch does\".\n\nAh, thanks for clarifying. I'll see if I can make the \"PID should be\nenough\" statement less confusing, instead - what I meant was \"on all\nsystems, the result of getppid() should be sufficient to look up the\nprocess name, so this code is probably shareable\", and Randall has\npointed out elsewhere to me that that's false.\n\n - Emily\n"},{"id":"425454","messageId":"20210524201007.115124-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210520210546.4129620-1-emilyshaffer@google.com","subject":"[PATCH v3] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-24T20:10:07Z","receivedAt":"2021-05-24T20:10:11Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? In some cases - like 'repo' tool -\nwe can influence the source code and set the GIT_TRACE2_PARENT_SID\nenvironment variable from the caller process. In 'repo''s case, that\nparent SID is manipulated to include the string \"repo\", which means we\ncan positively identify when Git was invoked by 'repo' tool. However,\nidentifying parents that way requires both that we know which tools\ninvoke Git and that we have the ability to modify the source code of\nthose tools. It cannot scale to keep up with the various IDEs and\nwrappers which use Git, most of which we don't know about. Learning\nwhich tools and wrappers invoke Git, and how, would give us insight to\ndecide where to improve Git's usability and performance.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient to look up the process name on most platforms, so\nthat code may be shareable.\n\nGit for Windows gathers similar information and logs it as a \"data_json\"\nevent. However, since \"data_json\" has a variable format, it is difficult\nto parse effectively in some languages; instead, let's pursue a\ndedicated \"cmd_ancestry\" event to record information about the ancestry\nof the current process and a consistent, parseable way.\n\nGit for Windows also gathers information about more than one parent. In\nLinux further ancestry info can be gathered with procfs, but it's\nunwieldy to do so. In the interest of later moving Git for Windows\nancestry logging to the 'cmd_ancestry' event, and in the interest of\nlater adding more ancestry to the Linux implementation - or of adding\nthis functionality to other platforms which have an easier time walking\nthe process tree - let's make 'cmd_ancestry' accept an array of\nparentage.\n\nSigned-off-by: Emily Shaffer \n---\n\nSince v2, only some nits fixed - a few brace placements and some update\nto the commit message language. Thanks Jeff, Junio, and Randall for\nlooking through the patch more thoroughly.\n\nThis iteration also has passing CI:\nhttps://github.com/nasamuffin/git/actions/runs/872341549\n\nI personally think it's ready to go; Jeff H mentioned adding more\nparents but later said there's value in confirming we can get the info\nwe want from just one parent, first, which matches my intent too. I see\nroom for additional work, but it can happen later - adding more parents\nto the Linux impl if we want, and logging the Windows impl to the same\ncmd_ancestry event if we want.\n\n - Emily\n\n\n\n Makefile | 5 ++++\n compat/procinfo.c | 51 +++++++++++++++++++++++++++++++++++++++\n config.mak.uname | 1 +\n git-compat-util.h | 6 +++++\n t/t0210/scrub_normal.perl | 6 +++++\n t/t0211/scrub_perf.perl | 5 ++++\n t/t0212/parse_events.perl | 5 +++-\n trace2.c | 13 ++++++++++\n trace2.h | 12 ++++++++-\n trace2/tr2_tgt.h | 3 +++\n trace2/tr2_tgt_event.c | 21 ++++++++++++++++\n trace2/tr2_tgt_normal.c | 19 +++++++++++++++\n trace2/tr2_tgt_perf.c | 16 ++++++++++++\n 13 files changed, 161 insertions(+), 2 deletions(-)\n create mode 100644 compat/procinfo.c\n\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..330e4fa011 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n endif\n \n+ifdef HAVE_PROCFS_LINUX\n+\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n+\tCOMPAT_OBJS += compat/procinfo.o\n+endif\n+\n ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n endif\ndiff --git a/compat/procinfo.c b/compat/procinfo.c\nnew file mode 100644\nindex 0000000000..0e92fb8b7c\n--- /dev/null\n+++ b/compat/procinfo.c\n@@ -0,0 +1,51 @@\n+#include \"cache.h\"\n+\n+#include \"strbuf.h\"\n+#include \"trace2.h\"\n+\n+char *get_process_name(int pid)\n+{\n+#ifdef HAVE_PROCFS_LINUX\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf out = STRBUF_INIT;\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n+\tif (!strbuf_read_file(&out, procfs_path.buf, 0)) {\n+\t\t/* All done with file reads, clean up early */\n+\t\tstrbuf_release(&procfs_path);\n+\t\treturn strbuf_detach(&out, NULL);\n+\t}\n+#endif\n+\n+\t/* NEEDSWORK: add non-procfs implementations here. */\n+\treturn NULL;\n+}\n+\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+\tif (!trace2_is_enabled())\n+\t\treturn;\n+\n+\t/* someday we may want to write something extra here, but not today */\n+\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\treturn;\n+\n+\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\t/*\n+\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n+\t\t * see compat/win32/trace2_win32_process_info.c.\n+\t\t */\n+\t\tchar *names[2];\n+\t\tnames[0] = get_process_name(getppid());\n+\t\tnames[1] = NULL;\n+\n+\t\tif (!names[0])\n+\t\t\treturn;\n+\n+\t\ttrace2_cmd_ancestry((const char**)names);\n+\n+\t\tfree(names[0]);\n+\t}\n+\n+\treturn;\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex cb443b4e02..7ad110a1d2 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n+\tHAVE_PROCFS_LINUX = YesPlease\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex a508dbe5a3..cc7d5d8a2a 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1382,4 +1382,10 @@ static inline void *container_of_or_null_offset(void *ptr, size_t offset)\n \n void sleep_millisec(int millisec);\n \n+/*\n+ * Convert PID to process name (as would show in top/task manager). Returns\n+ * NULL if unimplemented - be sure to check for NULL at callsite.\n+ */\n+char *get_process_name(int pid);\n+\n #endif\ndiff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\nindex c65d1a815e..7cc4de392a 100644\n--- a/t/t0210/scrub_normal.perl\n+++ b/t/t0210/scrub_normal.perl\n@@ -42,6 +42,12 @@\n \t# so just omit it for testing purposes.\n \t# print \"cmd_path _EXE_\\n\";\n }\n+ elsif ($line =~ m/^cmd_ancestry/) {\n+\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n+\t# sake, skip it when parsing normal.\n+\t#\n+\t# print \"$line\";\n+ }\n else {\n \tprint \"$line\";\n }\ndiff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\nindex 351af7844e..d164b750ff 100644\n--- a/t/t0211/scrub_perf.perl\n+++ b/t/t0211/scrub_perf.perl\n@@ -44,6 +44,11 @@\n \t# $tokens[$col_rest] = \"_EXE_\";\n \tgoto SKIP_LINE;\n }\n+ elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n+\t# so skip it.\n+\tgoto SKIP_LINE;\n+ }\n elsif ($tokens[$col_event] =~ m/child_exit/) {\n \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n }\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..b6408560c0 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -132,7 +132,10 @@\n \t# just omit it for testing purposes.\n \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n }\n- \n+ elsif ($event eq 'cmd_ancestry') {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n+\t# just skip it for testing purposes.\n+ }\n elsif ($event eq 'cmd_name') {\n \t$processes->{$sid}->{'name'} = $line->{'name'};\n \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\ndiff --git a/trace2.c b/trace2.c\nindex 256120c7fd..b9b154ac44 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n }\n \n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tstruct tr2_tgt *tgt_j;\n+\tint j;\n+\n+\tif (!trace2_enabled)\n+\t\treturn;\n+\n+\tfor_each_wanted_builtin (j, tgt_j)\n+\t\tif (tgt_j->pfn_command_ancestry_fl)\n+\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n+}\n+\n void trace2_cmd_name_fl(const char *file, int line, const char *name)\n {\n \tstruct tr2_tgt *tgt_j;\ndiff --git a/trace2.h b/trace2.h\nindex ede18c2e06..23743ac62b 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n \n #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n \n+/*\n+ * Emit an 'ancestry' event with the process name of the current process's\n+ * parent process.\n+ * This gives post-processors a way to determine what invoked the command and\n+ * learn more about usage patterns.\n+ */\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\n@@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n \tTRACE2_PROCESS_INFO_EXIT,\n };\n \n-#if defined(GIT_WINDOWS_NATIVE)\n+#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n void trace2_collect_process_info(enum trace2_process_info_reason reason);\n #else\n #define trace2_collect_process_info(reason) \\\ndiff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\nindex 7b90469212..1f66fd6573 100644\n--- a/trace2/tr2_tgt.h\n+++ b/trace2/tr2_tgt.h\n@@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n \n typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *command_path);\n+typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n+\t\t\t\t\t\tconst char **parent_names);\n typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *name,\n \t\t\t\t\t const char *hierarchy);\n@@ -108,6 +110,7 @@ struct tr2_tgt {\n \ttr2_tgt_evt_atexit_t *pfn_atexit;\n \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n+\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..578a9a5287 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tjw_release(&jw);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tconst char *parent_name = NULL;\n+\tstruct json_writer jw = JSON_WRITER_INIT;\n+\n+\tjw_object_begin(&jw, 0);\n+\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_inline_begin_array(&jw, \"ancestry\");\n+\n+\twhile ((parent_name = *parent_names++))\n+\t\tjw_array_string(&jw, parent_name);\n+\n+\tjw_end(&jw); /* 'ancestry' array */\n+\tjw_end(&jw); /* event object */\n+\n+\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n+\tjw_release(&jw);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex 31b602c171..a5751c8864 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *parent_name = NULL;\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n+\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n+\twhile ((parent_name = *parent_names++)) {\n+\t\tstrbuf_addstr(&buf_payload, parent_name);\n+\t\t/* if we'll write another one after this, add a delimiter */\n+\t\tif (parent_names && *parent_names)\n+\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n+\t}\n+\n+\tnormal_io_write_fl(file, line, &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a8018f18cc..af4d65a0a5 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n+\t/* It's not an argv but the rules are basically the same. */\n+\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n+\tstrbuf_addch(&buf_payload, ']');\n+\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n+\t\t\t &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\n-- \n2.31.1.818.g46aad6cb9e-goog\n\n"},{"id":"425460","messageId":"YKwRQbPwbds1wcyW@google.com","threadId":"55637","inReplyTo":"20210524201007.115124-1-emilyshaffer@google.com","subject":"Re: [PATCH v3] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-05-24T20:49:05Z","receivedAt":"2021-05-24T20:49:16Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, May 24, 2021 at 01:10:07PM -0700, Emily Shaffer wrote:\n> \n> Git for Windows also gathers information about more than one parent. In\nOh, bah, I guess I missed changing this to \"more than one generation of\nparent\". It doesn't seem worth a reroll, but I'll make the change\nlocally now so I don't miss it with any v3->v4 nits that may come in.\n\n - Emily\n"},{"id":"425471","messageId":"877djnogbk.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"YKgSc5OgVOt6HQqW@google.com","subject":"Re: [PATCH v2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-05-24T23:33:27Z","receivedAt":"2021-05-24T23:36:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, May 21 2021, Emily Shaffer wrote:\n\n> On Fri, May 21, 2021 at 03:15:16PM -0400, Jeff Hostetler wrote:\n>> On 5/20/21 5:05 PM, Emily Shaffer wrote:\n>> > - I took a look at Jeff H's advice on using a \"data_json\" event to log\n>> > this and decided it would be a little more flexible to add a new event\n>> > instead. If we want, it'd be feasible to then shoehorn the GfW parent\n>> > tree stuff into this new event too. Doing it this way is definitely\n>> > easier to parse for Google's trace analysis system (which for now\n>> > completely skips \"data_json\" as it's polymorphic), and also - I think\n>> > - means that we can add more fields later on if we need to (thread\n>> > info, different fields than just /proc/n/comm like exec path, argv,\n>> > whatever).\n>> \n>> I could argue both sides of this, so I guess it is fine either way.\n>> \n>> In GFW I log a array of argv[0] strings in a generic \"data_json\" event.\n>> I could also log additional \"data_json\" events with more structured\n>> data if needed.\n>> \n>> On the other hand, you're proposing a \"cmd_ancestry\" event with a\n>> single array of strings. You would have to expand the call signature\n>> of the trace2_cmd_ancestry() API to add additional data and inside\n>> tr2_tgt_event.c add additional fields to the JSON being composed.\n>> \n>> So both are about equal.\n>> \n>> (I'll avoid the temptation to make a snarky comment about fixing\n>> your post processing. :-) :-) :-) )\n>\n> ;P\n>\n> (I don't have much to add - this is an accurate summary of what I\n> thought about, too. Thanks for writing it out.)\n>\n>> \n>> It really doesn't matter one way or the other.\n>> \n>> > - Jonathan N also pointed out to me that /proc/n/comm exists, and logs\n>> > the \"command name\" - excluding argv, excluding path, etc. It seems\n>> \n>> So you're trying to log argv[0] of the process and not the full\n>> command line. That's what I'm doing.\n>\n> It's close to argv[0], yeah. POSIX docs indicate it might be truncated\n> in a way that argv[0] hasn't been, but it also doesn't include the\n> leading path (as far as I've seen). For example, a long-running helper\n> script I use with mutt, right now (gaffing on line length in email to\n> help with argv clarity, sorry):\n>\n> $ ps aux | grep mutt\n> emilysh+ 4119883 0.0 0.0 6892 3600 pts/6 S+ 12:44 0:00 /bin/bash\n> /usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh\n> /var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n> # comm is truncated to 15ch, except apparently in the cases of some\n> # kernel worker processes I saw with much longer names?\n> $ cat /proc/4119883/comm\n> open-vim-in-new\n> # exe is a link to the executable, which means bash as this is a\n> # script\n> $ ls -lha /proc/4119883/exe\n> lrwxrwxrwx 1 emilyshaffer primarygroup 0 May 21 12:44\n> /proc/4119883/exe -> /usr/bin/bash\n> # cmdline has the whole argv, separated on NUL so it runs together in\n> # editor\n> $ cat /proc/4119883/cmdline\n> /bin/bash/usr/local/google/home/emilyshaffer/dotfiles/open-vim-in-new-split.sh/var/tmp/mutt-podkayne-413244-1263002-7433772284891386689\n>\n> Jonathan N pointed out that the process name (the thing in 'comm') can\n> also be manually manipulated by the process itself, and 'man procfs'\n> also talks about 'PR_SET_NAME' and 'PR_GET_NAME' operations in\n> 'prctl()', so that tracks. (It doesn't look like we can use prctl() to\n> find out the names of processes besides the current process, though, so\n> the procfs stuff is still needed. Dang.)\n>\n>> \n>> > like this is a little more safe about excluding personal information\n>> > from the traces which take the form of \"myscript.sh\n>> > --password=hunter2\", but would still be worrisome for something like\n>> > \"mysupersecretproject.sh\". I'm not sure whether that means we still\n>> > want to guard it with a config flag, though.\n>> \n>> You might check whether you get the name of the script or just get\n>> a lot of entries with just \"/usr/bin/bash\".\n>\n> See above :)\n>\n>> There's lots of PII in the data stream to worry about.\n>> The name of the command is just one aspect, but I digress.\n>\n> Yes, that's what we've noticed too, so a process name isn't worrying us\n> that much more.\n>\n>> \n>> > - I also added a lot to the commit message; hopefully it's not too\n>> > rambly, but I hoped to explain why just setting GIT_TRACE2_PARENT_SID\n>> > wasn't going to cut it.\n>> > - As for testing, I followed the lead of GfW's parentage info - \"this\n>> > isn't portable so writing tests for it will suck, just scrub it from\n>> > the tests\". Maybe it makes sense to do some more\n>> > platform-specific-ness in the test suite instead? I wasn't sure.\n>> \n>> yeah, that's probably best. Unless you can tokenize it properly\n>> so that you can predict the results in a HEREDOC in the test source.\n>> \n>> For example, you might try to test tracing a command (where a top-level\n>> \"git foo\" (SPACE form) spawns a \"git-foo\" (DASHED form) and check the\n>> output for the child.\n>\n> Yeah, I had trouble with even deciding when to attempt such a check or\n> not.\n>> > +\tif (reason == TRACE2_PROCESS_INFO_STARTUP)\n>> > +\t{\n>> > +\t\t/*\n>> > +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n>> > +\t\t * see compat/win32/trace2_win32_process_info.c.\n>> > +\t\t */\n>> > +\t\tchar *names[2];\n>> > +\t\tnames[0] = get_process_name(getppid());\n>> > +\t\tnames[1] = NULL;\n>> \n>> You're only logging 1 parent. That's fine to get started.\n>> \n>> I'm logging IIRC 10 parents on GFW. That might seem overkill,\n>> but there are lots of intermediate parents that hide what is\n>> happening. For example, a \"git push\" might spawn \"git remote-https\"\n>> which spawns \"git-remote-https\" which spawn \"git send-pack\" which\n>> spawns \"git pack-objects\".\n>> \n>> And that doesn't include who called push.\n>> \n>> And it's not uncommon to see 2 or 3 \"bash\" entries in the array\n>> because of the bash scripts being run.\n>\n> Agree. But it's expensive - I didn't find a handy library call to find\n> \"parent ID of given process ID\", so I think we'd have to manipulate\n> procfs; and so far I only see parent ID in summary infos like\n> /proc/n/status or /proc/n/stat, which contain lots of other info too and\n> would need parsing.\n\nIt sounds a bit like you're fumbling your way towards (re?)discovering:\n\n pstree -s \n\nYou can look at its implementation (or strace it) to see what it does,\nand yes, on Linux there's no handy C library for this, iterating over\nprocfs is the library.\n\nAside from the privacy, PII, usefulness of this data etc. discussions in\nthis & related threads I don't think that per-se should be an issue on a\nmodern Linux system. After all we'd just need to do it once on\nstartup. For any sub-process we spawn we'd carry it forward after the\ninitial /usr/bin/git invocation.\n"},{"id":"425482","messageId":"xmqqpmxfh3j5.fsf@gitster.g","threadId":"55637","inReplyTo":"20210524201007.115124-1-emilyshaffer@google.com","subject":"Re: [PATCH v3] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-05-25T03:54:06Z","receivedAt":"2021-05-25T03:54:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> diff --git a/compat/procinfo.c b/compat/procinfo.c\n> new file mode 100644\n> index 0000000000..0e92fb8b7c\n> --- /dev/null\n> +++ b/compat/procinfo.c\n> @@ -0,0 +1,51 @@\n> +#include \"cache.h\"\n> +\n> +#include \"strbuf.h\"\n> +#include \"trace2.h\"\n> +\n> +char *get_process_name(int pid)\n> +{\n> +#ifdef HAVE_PROCFS_LINUX\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf out = STRBUF_INIT;\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n> +\tif (!strbuf_read_file(&out, procfs_path.buf, 0)) {\n> +\t\t/* All done with file reads, clean up early */\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\treturn strbuf_detach(&out, NULL);\n> +\t}\n> +#endif\n> +\n> +\t/* NEEDSWORK: add non-procfs implementations here. */\n> +\treturn NULL;\n> +}\n\nIs the reason why this takes \"int\" and not \"pid_t\" because we may\nport to non-POSIX platforms that do not have pid_t defined?\n\n ... goes and greps ...\n\nNah, we use pid_t everywhere (including compat/mingw.c); unless\nthere is a reason not to, let's use that type.\n\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tchar *names[2];\n> +\t\tnames[0] = get_process_name(getppid());\n> +\t\tnames[1] = NULL;\n\nMakes me wonder if get_process_name() is an appropriate\nabstraction; specifically, something like\n\n\t\tconst char **names = get_ancestry_names();\n int cnt;\n\t\tif (names)\n\t\t\ttrace2_cmd_ancestry(names);\n\t\tfor (cnt = 0; names[cnt]; cnt++)\n \tfree((char *)names[cnt]);\n\t\tfree(names);\n\nwould allow platforms to decide how many levels is easy for them to\ngrab for reporting, for example (and they do not even have to have\nto assume that getting process IDs to feed get_process_name() one by\none is the easiest way to show ancestry).\n\n"},{"id":"425524","messageId":"038801d7516a$9efb1fc0$dcf15f40$@nexbridge.com","threadId":"55637","inReplyTo":"xmqqpmxfh3j5.fsf@gitster.g","subject":"RE: [PATCH v3] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-05-25T13:33:54Z","receivedAt":"2021-05-25T13:34:11Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On May 24, 2021 11:54 PM, Junio C Hamano wrote:\n>Emily Shaffer writes:\n>\n>> diff --git a/compat/procinfo.c b/compat/procinfo.c new file mode\n>> 100644 index 0000000000..0e92fb8b7c\n>> --- /dev/null\n>> +++ b/compat/procinfo.c\n>> @@ -0,0 +1,51 @@\n>> +#include \"cache.h\"\n>> +\n>> +#include \"strbuf.h\"\n>> +#include \"trace2.h\"\n>> +\n>> +char *get_process_name(int pid)\n>> +{\n>> +#ifdef HAVE_PROCFS_LINUX\n>> +\tstruct strbuf procfs_path = STRBUF_INIT;\n>> +\tstruct strbuf out = STRBUF_INIT;\n>> +\t/* try to use procfs if it's present. */\n>> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", pid);\n>> +\tif (!strbuf_read_file(&out, procfs_path.buf, 0)) {\n>> +\t\t/* All done with file reads, clean up early */\n>> +\t\tstrbuf_release(&procfs_path);\n>> +\t\treturn strbuf_detach(&out, NULL);\n>> +\t}\n>> +#endif\n>> +\n>> +\t/* NEEDSWORK: add non-procfs implementations here. */\n>> +\treturn NULL;\n>> +}\n>\n>Is the reason why this takes \"int\" and not \"pid_t\" because we may port to non-POSIX platforms that do not have pid_t defined?\n>\n> ... goes and greps ...\n>\n>Nah, we use pid_t everywhere (including compat/mingw.c); unless there is a reason not to, let's use that type.\n>\n>> +void trace2_collect_process_info(enum trace2_process_info_reason\n>> +reason) {\n>> +\tif (!trace2_is_enabled())\n>> +\t\treturn;\n>> +\n>> +\t/* someday we may want to write something extra here, but not today */\n>> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n>> +\t\treturn;\n>> +\n>> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n>> +\t\t/*\n>> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n>> +\t\t * see compat/win32/trace2_win32_process_info.c.\n>> +\t\t */\n>> +\t\tchar *names[2];\n>> +\t\tnames[0] = get_process_name(getppid());\n>> +\t\tnames[1] = NULL;\n>\n>Makes me wonder if get_process_name() is an appropriate abstraction; specifically, something like\n>\n>\t\tconst char **names = get_ancestry_names();\n> int cnt;\n>\t\tif (names)\n>\t\t\ttrace2_cmd_ancestry(names);\n>\t\tfor (cnt = 0; names[cnt]; cnt++)\n> \tfree((char *)names[cnt]);\n>\t\tfree(names);\n>\n>would allow platforms to decide how many levels is easy for them to grab for reporting, for example (and they do not even have to have\n>to assume that getting process IDs to feed get_process_name() one by one is the easiest way to show ancestry).\n\nPassing pid_t == 1 to get_process_names is non-informative. We can't tell what is really intended inside get_process_names(). Knowing that we are getting the ancestor, however, gives us an the important glue that the parent is wanted so we can interpret 1 as a non-ancestor. NonStop has pid_t defined regardless of whether it is used or relevant, so a higher-level of abstraction makes sense. The git process always has a valid pid_t, but the ancestor may not, but we do not know this at compile time. The platform code previously shared appears to be the correct technique for us. Going with get_ancestry_names() is interesting, but I think a count (how many ancestors do you want) is important (1 - just immediate, 2 - parent, grandparent, -1 - everyone). The ancestry tree can be very large (although each lookup is only about 8us). During a trace, I really would not want to care about more than 2 ancestors, typically, not the likely 8 or 10 that I can see happening in my situation (too much \n noise). The number of ancestors to trace probably needs to be in .git.config.\n\n\n"},{"id":"426806","messageId":"20210608185855.668050-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210524201007.115124-1-emilyshaffer@google.com","subject":"[PATCH v4] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-08T18:58:55Z","receivedAt":"2021-06-08T19:04:21Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? In some cases - like 'repo' tool -\nwe can influence the source code and set the GIT_TRACE2_PARENT_SID\nenvironment variable from the caller process. In 'repo''s case, that\nparent SID is manipulated to include the string \"repo\", which means we\ncan positively identify when Git was invoked by 'repo' tool. However,\nidentifying parents that way requires both that we know which tools\ninvoke Git and that we have the ability to modify the source code of\nthose tools. It cannot scale to keep up with the various IDEs and\nwrappers which use Git, most of which we don't know about. Learning\nwhich tools and wrappers invoke Git, and how, would give us insight to\ndecide where to improve Git's usability and performance.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient to look up the process name on most platforms, so\nthat code may be shareable.\n\nGit for Windows gathers similar information and logs it as a \"data_json\"\nevent. However, since \"data_json\" has a variable format, it is difficult\nto parse effectively in some languages; instead, let's pursue a\ndedicated \"cmd_ancestry\" event to record information about the ancestry\nof the current process and a consistent, parseable way.\n\nGit for Windows also gathers information about more than one generation\nof parent. In Linux further ancestry info can be gathered with procfs,\nbut it's unwieldy to do so. In the interest of later moving Git for\nWindows ancestry logging to the 'cmd_ancestry' event, and in the\ninterest of later adding more ancestry to the Linux implementation - or\nof adding this functionality to other platforms which have an easier\ntime walking the process tree - let's make 'cmd_ancestry' accept an\narray of parentage.\n\nSigned-off-by: Emily Shaffer \n---\n\nNotes:\n Since v3:\n \n Junio and Randall suggested ditching \"get_process_name(ppid)\" as the API to\n implement on various platforms, and I liked the suggestion a lot. So instead,\n I added \"get_ancestry_names(strvec *names)\".\n \n - Using a strvec instead of a char** makes cleanup easier, since strvec takes\n care of counting the number of strings in the array for us. Otherwise we'd\n need to include size as a return or out-param in get_ancestry_names(), and\n that's what the util class is for, right? :)\n - I made get_ancestry_names() static instead of putting it into\n git-compat-util.h. I think I had put get_process_name() into that header to\n facilitate non-procfs implementations, but compat/procinfo.c doesn't seem to\n me to indicate \"procfs only\", and if we do need to implement\n get_process_name() somewhere else, it'll be pretty easy to move it.\n - I added a description of \"cmd_ancestry\" to\n Documentation/technical/api-trace2.txt. I didn't see any user-facing docs to\n update (for example, \"git grep cmd_path\" produces only that one doc file).\n \n Thanks, all.\n \n - Emily\n\n Documentation/technical/api-trace2.txt | 14 ++++++\n Makefile | 5 +++\n compat/procinfo.c | 61 ++++++++++++++++++++++++++\n config.mak.uname | 1 +\n t/t0210/scrub_normal.perl | 6 +++\n t/t0211/scrub_perf.perl | 5 +++\n t/t0212/parse_events.perl | 5 ++-\n trace2.c | 13 ++++++\n trace2.h | 12 ++++-\n trace2/tr2_tgt.h | 3 ++\n trace2/tr2_tgt_event.c | 21 +++++++++\n trace2/tr2_tgt_normal.c | 19 ++++++++\n trace2/tr2_tgt_perf.c | 16 +++++++\n 13 files changed, 179 insertions(+), 2 deletions(-)\n create mode 100644 compat/procinfo.c\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex 3f52f981a2..8a0b360a0e 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -493,6 +493,20 @@ about specific error arguments.\n }\n ------------\n \n+`\"cmd_ancestry\"`::\n+\tThis event contains the text command name for the parent (and earlier\n+\tgenerations of parents) of the current process, in an array ordered from\n+\tnearest parent to furthest great-grandparent. It may not be implemented\n+\ton all platforms.\n++\n+------------\n+{\n+\t\"event\":\"cmd_ancestry\",\n+\t...\n+\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n+}\n+------------\n+\n `\"cmd_name\"`::\n \tThis event contains the command name for this git process\n \tand the hierarchy of commands from parent git processes.\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..330e4fa011 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n endif\n \n+ifdef HAVE_PROCFS_LINUX\n+\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n+\tCOMPAT_OBJS += compat/procinfo.o\n+endif\n+\n ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n endif\ndiff --git a/compat/procinfo.c b/compat/procinfo.c\nnew file mode 100644\nindex 0000000000..9417c3aa54\n--- /dev/null\n+++ b/compat/procinfo.c\n@@ -0,0 +1,61 @@\n+#include \"cache.h\"\n+\n+#include \"strbuf.h\"\n+#include \"strvec.h\"\n+#include \"trace2.h\"\n+\n+static void get_ancestry_names(struct strvec *names)\n+{\n+#ifdef HAVE_PROCFS_LINUX\n+\t/*\n+\t * NEEDSWORK: We could gather the entire pstree into an array to match\n+\t * functionality with compat/win32/trace2_win32_process_info.c.\n+\t * To do so, we may want to examine /proc//stat. For now, just\n+\t * gather the immediate parent name which is readily accessible from\n+\t * /proc/$(getppid())/comm.\n+\t */\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf name = STRBUF_INIT;\n+\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n+\t\tstrbuf_release(&procfs_path);\n+\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t}\n+\n+\treturn;\n+#endif\n+\t/* NEEDSWORK: add non-procfs-linux implementations here */\n+}\n+\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+\tif (!trace2_is_enabled())\n+\t\treturn;\n+\n+\t/* someday we may want to write something extra here, but not today */\n+\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\treturn;\n+\n+\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\t/*\n+\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n+\t\t * see compat/win32/trace2_win32_process_info.c.\n+\t\t */\n+\t\tstruct strvec names = STRVEC_INIT;\n+\n+\t\tget_ancestry_names(&names);\n+\n+\t\tif (names.nr == 0) {\n+\t\t\tstrvec_clear(&names);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\ttrace2_cmd_ancestry(names.v);\n+\n+\t\tstrvec_clear(&names);\n+\t}\n+\n+\treturn;\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex cb443b4e02..7ad110a1d2 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n+\tHAVE_PROCFS_LINUX = YesPlease\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\ndiff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\nindex c65d1a815e..7cc4de392a 100644\n--- a/t/t0210/scrub_normal.perl\n+++ b/t/t0210/scrub_normal.perl\n@@ -42,6 +42,12 @@\n \t# so just omit it for testing purposes.\n \t# print \"cmd_path _EXE_\\n\";\n }\n+ elsif ($line =~ m/^cmd_ancestry/) {\n+\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n+\t# sake, skip it when parsing normal.\n+\t#\n+\t# print \"$line\";\n+ }\n else {\n \tprint \"$line\";\n }\ndiff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\nindex 351af7844e..d164b750ff 100644\n--- a/t/t0211/scrub_perf.perl\n+++ b/t/t0211/scrub_perf.perl\n@@ -44,6 +44,11 @@\n \t# $tokens[$col_rest] = \"_EXE_\";\n \tgoto SKIP_LINE;\n }\n+ elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n+\t# so skip it.\n+\tgoto SKIP_LINE;\n+ }\n elsif ($tokens[$col_event] =~ m/child_exit/) {\n \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n }\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..b6408560c0 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -132,7 +132,10 @@\n \t# just omit it for testing purposes.\n \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n }\n- \n+ elsif ($event eq 'cmd_ancestry') {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n+\t# just skip it for testing purposes.\n+ }\n elsif ($event eq 'cmd_name') {\n \t$processes->{$sid}->{'name'} = $line->{'name'};\n \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\ndiff --git a/trace2.c b/trace2.c\nindex 256120c7fd..b9b154ac44 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n }\n \n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tstruct tr2_tgt *tgt_j;\n+\tint j;\n+\n+\tif (!trace2_enabled)\n+\t\treturn;\n+\n+\tfor_each_wanted_builtin (j, tgt_j)\n+\t\tif (tgt_j->pfn_command_ancestry_fl)\n+\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n+}\n+\n void trace2_cmd_name_fl(const char *file, int line, const char *name)\n {\n \tstruct tr2_tgt *tgt_j;\ndiff --git a/trace2.h b/trace2.h\nindex ede18c2e06..23743ac62b 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n \n #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n \n+/*\n+ * Emit an 'ancestry' event with the process name of the current process's\n+ * parent process.\n+ * This gives post-processors a way to determine what invoked the command and\n+ * learn more about usage patterns.\n+ */\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\n@@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n \tTRACE2_PROCESS_INFO_EXIT,\n };\n \n-#if defined(GIT_WINDOWS_NATIVE)\n+#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n void trace2_collect_process_info(enum trace2_process_info_reason reason);\n #else\n #define trace2_collect_process_info(reason) \\\ndiff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\nindex 7b90469212..1f66fd6573 100644\n--- a/trace2/tr2_tgt.h\n+++ b/trace2/tr2_tgt.h\n@@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n \n typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *command_path);\n+typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n+\t\t\t\t\t\tconst char **parent_names);\n typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *name,\n \t\t\t\t\t const char *hierarchy);\n@@ -108,6 +110,7 @@ struct tr2_tgt {\n \ttr2_tgt_evt_atexit_t *pfn_atexit;\n \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n+\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..578a9a5287 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tjw_release(&jw);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tconst char *parent_name = NULL;\n+\tstruct json_writer jw = JSON_WRITER_INIT;\n+\n+\tjw_object_begin(&jw, 0);\n+\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_inline_begin_array(&jw, \"ancestry\");\n+\n+\twhile ((parent_name = *parent_names++))\n+\t\tjw_array_string(&jw, parent_name);\n+\n+\tjw_end(&jw); /* 'ancestry' array */\n+\tjw_end(&jw); /* event object */\n+\n+\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n+\tjw_release(&jw);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex 31b602c171..a5751c8864 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *parent_name = NULL;\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n+\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n+\twhile ((parent_name = *parent_names++)) {\n+\t\tstrbuf_addstr(&buf_payload, parent_name);\n+\t\t/* if we'll write another one after this, add a delimiter */\n+\t\tif (parent_names && *parent_names)\n+\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n+\t}\n+\n+\tnormal_io_write_fl(file, line, &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a8018f18cc..af4d65a0a5 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n+\t/* It's not an argv but the rules are basically the same. */\n+\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n+\tstrbuf_addch(&buf_payload, ']');\n+\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n+\t\t\t &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\n-- \n2.32.0.rc1.229.g3e70b5a671-goog\n\n"},{"id":"426813","messageId":"YL/ZaEQOYkG6lRz1@google.com","threadId":"55637","inReplyTo":"20210608185855.668050-1-emilyshaffer@google.com","subject":"Re: [PATCH v4] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-08T20:56:08Z","receivedAt":"2021-06-08T20:56:29Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, Jun 08, 2021 at 11:58:55AM -0700, Emily Shaffer wrote:\n\nHmph, this is failing CI. I'll investigate it today and send a v5.\n"},{"id":"426817","messageId":"20210608221059.1935021-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210608185855.668050-1-emilyshaffer@google.com","subject":"[PATCH v5] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-08T22:10:59Z","receivedAt":"2021-06-08T22:12:20Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? In some cases - like 'repo' tool -\nwe can influence the source code and set the GIT_TRACE2_PARENT_SID\nenvironment variable from the caller process. In 'repo''s case, that\nparent SID is manipulated to include the string \"repo\", which means we\ncan positively identify when Git was invoked by 'repo' tool. However,\nidentifying parents that way requires both that we know which tools\ninvoke Git and that we have the ability to modify the source code of\nthose tools. It cannot scale to keep up with the various IDEs and\nwrappers which use Git, most of which we don't know about. Learning\nwhich tools and wrappers invoke Git, and how, would give us insight to\ndecide where to improve Git's usability and performance.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient to look up the process name on most platforms, so\nthat code may be shareable.\n\nGit for Windows gathers similar information and logs it as a \"data_json\"\nevent. However, since \"data_json\" has a variable format, it is difficult\nto parse effectively in some languages; instead, let's pursue a\ndedicated \"cmd_ancestry\" event to record information about the ancestry\nof the current process and a consistent, parseable way.\n\nGit for Windows also gathers information about more than one generation\nof parent. In Linux further ancestry info can be gathered with procfs,\nbut it's unwieldy to do so. In the interest of later moving Git for\nWindows ancestry logging to the 'cmd_ancestry' event, and in the\ninterest of later adding more ancestry to the Linux implementation - or\nof adding this functionality to other platforms which have an easier\ntime walking the process tree - let's make 'cmd_ancestry' accept an\narray of parentage.\n\nSigned-off-by: Emily Shaffer \n---\n\nSince v4:\n\nSince I'm reading from a file to discover the name of the process, the\nfile is newline-terminated. This newline caused some havoc in tests, so\nit's better to strip it if it's there. It's not part of the process name\nitself.\n\nPassing tests at\nhttps://github.com/nasamuffin/git/actions/runs/919722509\n\nRange-diff against v4:\n1: efb0a3ccb4 ! 1: 7a7e1ebbfa tr2: log parent process name\n @@ compat/procinfo.c (new)\n +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n +\t\tstrbuf_release(&procfs_path);\n ++\t\tstrbuf_trim_trailing_newline(&name);\n +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n +\t}\n +\n\n Documentation/technical/api-trace2.txt | 14 ++++++\n Makefile | 5 +++\n compat/procinfo.c | 62 ++++++++++++++++++++++++++\n config.mak.uname | 1 +\n t/t0210/scrub_normal.perl | 6 +++\n t/t0211/scrub_perf.perl | 5 +++\n t/t0212/parse_events.perl | 5 ++-\n trace2.c | 13 ++++++\n trace2.h | 12 ++++-\n trace2/tr2_tgt.h | 3 ++\n trace2/tr2_tgt_event.c | 21 +++++++++\n trace2/tr2_tgt_normal.c | 19 ++++++++\n trace2/tr2_tgt_perf.c | 16 +++++++\n 13 files changed, 180 insertions(+), 2 deletions(-)\n create mode 100644 compat/procinfo.c\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex 3f52f981a2..8a0b360a0e 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -493,6 +493,20 @@ about specific error arguments.\n }\n ------------\n \n+`\"cmd_ancestry\"`::\n+\tThis event contains the text command name for the parent (and earlier\n+\tgenerations of parents) of the current process, in an array ordered from\n+\tnearest parent to furthest great-grandparent. It may not be implemented\n+\ton all platforms.\n++\n+------------\n+{\n+\t\"event\":\"cmd_ancestry\",\n+\t...\n+\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n+}\n+------------\n+\n `\"cmd_name\"`::\n \tThis event contains the command name for this git process\n \tand the hierarchy of commands from parent git processes.\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..330e4fa011 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n endif\n \n+ifdef HAVE_PROCFS_LINUX\n+\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n+\tCOMPAT_OBJS += compat/procinfo.o\n+endif\n+\n ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n endif\ndiff --git a/compat/procinfo.c b/compat/procinfo.c\nnew file mode 100644\nindex 0000000000..f8763cacf8\n--- /dev/null\n+++ b/compat/procinfo.c\n@@ -0,0 +1,62 @@\n+#include \"cache.h\"\n+\n+#include \"strbuf.h\"\n+#include \"strvec.h\"\n+#include \"trace2.h\"\n+\n+static void get_ancestry_names(struct strvec *names)\n+{\n+#ifdef HAVE_PROCFS_LINUX\n+\t/*\n+\t * NEEDSWORK: We could gather the entire pstree into an array to match\n+\t * functionality with compat/win32/trace2_win32_process_info.c.\n+\t * To do so, we may want to examine /proc//stat. For now, just\n+\t * gather the immediate parent name which is readily accessible from\n+\t * /proc/$(getppid())/comm.\n+\t */\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf name = STRBUF_INIT;\n+\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n+\t\tstrbuf_release(&procfs_path);\n+\t\tstrbuf_trim_trailing_newline(&name);\n+\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t}\n+\n+\treturn;\n+#endif\n+\t/* NEEDSWORK: add non-procfs-linux implementations here */\n+}\n+\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+\tif (!trace2_is_enabled())\n+\t\treturn;\n+\n+\t/* someday we may want to write something extra here, but not today */\n+\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\treturn;\n+\n+\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\t/*\n+\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n+\t\t * see compat/win32/trace2_win32_process_info.c.\n+\t\t */\n+\t\tstruct strvec names = STRVEC_INIT;\n+\n+\t\tget_ancestry_names(&names);\n+\n+\t\tif (names.nr == 0) {\n+\t\t\tstrvec_clear(&names);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\ttrace2_cmd_ancestry(names.v);\n+\n+\t\tstrvec_clear(&names);\n+\t}\n+\n+\treturn;\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex cb443b4e02..7ad110a1d2 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n+\tHAVE_PROCFS_LINUX = YesPlease\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\ndiff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\nindex c65d1a815e..7cc4de392a 100644\n--- a/t/t0210/scrub_normal.perl\n+++ b/t/t0210/scrub_normal.perl\n@@ -42,6 +42,12 @@\n \t# so just omit it for testing purposes.\n \t# print \"cmd_path _EXE_\\n\";\n }\n+ elsif ($line =~ m/^cmd_ancestry/) {\n+\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n+\t# sake, skip it when parsing normal.\n+\t#\n+\t# print \"$line\";\n+ }\n else {\n \tprint \"$line\";\n }\ndiff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\nindex 351af7844e..d164b750ff 100644\n--- a/t/t0211/scrub_perf.perl\n+++ b/t/t0211/scrub_perf.perl\n@@ -44,6 +44,11 @@\n \t# $tokens[$col_rest] = \"_EXE_\";\n \tgoto SKIP_LINE;\n }\n+ elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n+\t# so skip it.\n+\tgoto SKIP_LINE;\n+ }\n elsif ($tokens[$col_event] =~ m/child_exit/) {\n \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n }\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..b6408560c0 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -132,7 +132,10 @@\n \t# just omit it for testing purposes.\n \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n }\n- \n+ elsif ($event eq 'cmd_ancestry') {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n+\t# just skip it for testing purposes.\n+ }\n elsif ($event eq 'cmd_name') {\n \t$processes->{$sid}->{'name'} = $line->{'name'};\n \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\ndiff --git a/trace2.c b/trace2.c\nindex 256120c7fd..b9b154ac44 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n }\n \n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tstruct tr2_tgt *tgt_j;\n+\tint j;\n+\n+\tif (!trace2_enabled)\n+\t\treturn;\n+\n+\tfor_each_wanted_builtin (j, tgt_j)\n+\t\tif (tgt_j->pfn_command_ancestry_fl)\n+\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n+}\n+\n void trace2_cmd_name_fl(const char *file, int line, const char *name)\n {\n \tstruct tr2_tgt *tgt_j;\ndiff --git a/trace2.h b/trace2.h\nindex ede18c2e06..23743ac62b 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n \n #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n \n+/*\n+ * Emit an 'ancestry' event with the process name of the current process's\n+ * parent process.\n+ * This gives post-processors a way to determine what invoked the command and\n+ * learn more about usage patterns.\n+ */\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\n@@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n \tTRACE2_PROCESS_INFO_EXIT,\n };\n \n-#if defined(GIT_WINDOWS_NATIVE)\n+#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n void trace2_collect_process_info(enum trace2_process_info_reason reason);\n #else\n #define trace2_collect_process_info(reason) \\\ndiff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\nindex 7b90469212..1f66fd6573 100644\n--- a/trace2/tr2_tgt.h\n+++ b/trace2/tr2_tgt.h\n@@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n \n typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *command_path);\n+typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n+\t\t\t\t\t\tconst char **parent_names);\n typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *name,\n \t\t\t\t\t const char *hierarchy);\n@@ -108,6 +110,7 @@ struct tr2_tgt {\n \ttr2_tgt_evt_atexit_t *pfn_atexit;\n \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n+\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..578a9a5287 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tjw_release(&jw);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tconst char *parent_name = NULL;\n+\tstruct json_writer jw = JSON_WRITER_INIT;\n+\n+\tjw_object_begin(&jw, 0);\n+\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_inline_begin_array(&jw, \"ancestry\");\n+\n+\twhile ((parent_name = *parent_names++))\n+\t\tjw_array_string(&jw, parent_name);\n+\n+\tjw_end(&jw); /* 'ancestry' array */\n+\tjw_end(&jw); /* event object */\n+\n+\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n+\tjw_release(&jw);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex 31b602c171..a5751c8864 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *parent_name = NULL;\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n+\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n+\twhile ((parent_name = *parent_names++)) {\n+\t\tstrbuf_addstr(&buf_payload, parent_name);\n+\t\t/* if we'll write another one after this, add a delimiter */\n+\t\tif (parent_names && *parent_names)\n+\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n+\t}\n+\n+\tnormal_io_write_fl(file, line, &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a8018f18cc..af4d65a0a5 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n+\t/* It's not an argv but the rules are basically the same. */\n+\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n+\tstrbuf_addch(&buf_payload, ']');\n+\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n+\t\t\t &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\n-- \n2.32.0.rc1.229.g3e70b5a671-goog\n\n"},{"id":"426818","messageId":"023201d75cb4$02006d10$06014730$@nexbridge.com","threadId":"55637","inReplyTo":"20210608221059.1935021-1-emilyshaffer@google.com","subject":"RE: [PATCH v5] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-06-08T22:16:57Z","receivedAt":"2021-06-08T22:17:11Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On June 8, 2021 6:11 PM, Emily Shaffer wrote:\n>It can be useful to tell who invoked Git - was it invoked manually by a user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n>we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In 'repo''s case,\n>that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by 'repo' tool.\n>However, identifying parents that way requires both that we know which tools invoke Git and that we have the ability to modify the source\n>code of those tools. It cannot scale to keep up with the various IDEs and wrappers which use Git, most of which we don't know about.\n>Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and performance.\n>\n>Unfortunately, there's no cross-platform reliable way to gather the name of the parent process. If procfs is present, we can use that;\n>otherwise we will need to discover the name another way. However, the process ID should be sufficient to look up the process name on\n>most platforms, so that code may be shareable.\n>\n>Git for Windows gathers similar information and logs it as a \"data_json\"\n>event. However, since \"data_json\" has a variable format, it is difficult to parse effectively in some languages; instead, let's pursue a\n>dedicated \"cmd_ancestry\" event to record information about the ancestry of the current process and a consistent, parseable way.\n>\n>Git for Windows also gathers information about more than one generation of parent. In Linux further ancestry info can be gathered with\n>procfs, but it's unwieldy to do so. In the interest of later moving Git for Windows ancestry logging to the 'cmd_ancestry' event, and in the\n>interest of later adding more ancestry to the Linux implementation - or of adding this functionality to other platforms which have an\n>easier time walking the process tree - let's make 'cmd_ancestry' accept an array of parentage.\n\nWe are probably going to have to discuss this one at more length. On NonStop, in some cases, I have access to the program arguments of the parent (rather like ps -ef) in POSIX-land, but not from the other personality. I do have access to the program object name, in both sides, although if someone replaces the object - which is not actually possible for a running program, but a rename is - the object may end up being somewhat meaningless or mangled. My suspicion is that I'm going to have to supply different things for the two personalities, but I'm not sure what as of yet.\n\nRegards,\nRandall\n\n"},{"id":"426819","messageId":"YL/uNYuADscBJUu2@google.com","threadId":"55637","inReplyTo":"023201d75cb4$02006d10$06014730$@nexbridge.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-08T22:24:53Z","receivedAt":"2021-06-08T22:26:02Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, Jun 08, 2021 at 06:16:57PM -0400, Randall S. Becker wrote:\n> \n> On June 8, 2021 6:11 PM, Emily Shaffer wrote:\n> >It can be useful to tell who invoked Git - was it invoked manually by a user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n> >we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In 'repo''s case,\n> >that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by 'repo' tool.\n> >However, identifying parents that way requires both that we know which tools invoke Git and that we have the ability to modify the source\n> >code of those tools. It cannot scale to keep up with the various IDEs and wrappers which use Git, most of which we don't know about.\n> >Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and performance.\n> >\n> >Unfortunately, there's no cross-platform reliable way to gather the name of the parent process. If procfs is present, we can use that;\n> >otherwise we will need to discover the name another way. However, the process ID should be sufficient to look up the process name on\n> >most platforms, so that code may be shareable.\n> >\n> >Git for Windows gathers similar information and logs it as a \"data_json\"\n> >event. However, since \"data_json\" has a variable format, it is difficult to parse effectively in some languages; instead, let's pursue a\n> >dedicated \"cmd_ancestry\" event to record information about the ancestry of the current process and a consistent, parseable way.\n> >\n> >Git for Windows also gathers information about more than one generation of parent. In Linux further ancestry info can be gathered with\n> >procfs, but it's unwieldy to do so. In the interest of later moving Git for Windows ancestry logging to the 'cmd_ancestry' event, and in the\n> >interest of later adding more ancestry to the Linux implementation - or of adding this functionality to other platforms which have an\n> >easier time walking the process tree - let's make 'cmd_ancestry' accept an array of parentage.\n> \n> We are probably going to have to discuss this one at more length. On\n> NonStop, in some cases, I have access to the program arguments of the\n> parent (rather like ps -ef) in POSIX-land, but not from the other\n> personality. I do have access to the program object name, in both\n> sides, although if someone replaces the object - which is not actually\n> possible for a running program, but a rename is - the object may end\n> up being somewhat meaningless or mangled. My suspicion is that I'm\n> going to have to supply different things for the two personalities,\n> but I'm not sure what as of yet.\n\nI guess I'm having trouble understanding - it sounds like you're\ndescribing one process with a graph of ancestry instead of a line. Does\nthat mean we shouldn't be tracing the ancestry the way we are?\n\n - Emily\n"},{"id":"426820","messageId":"023701d75cb7$1ff995a0$5fecc0e0$@nexbridge.com","threadId":"55637","inReplyTo":"YL/uNYuADscBJUu2@google.com","subject":"RE: [PATCH v5] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-06-08T22:39:15Z","receivedAt":"2021-06-08T22:39:40Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On June 8, 2021 6:25 PM, Emily Shaffer wrote\"\n>To: Randall S. Becker \n>Cc: git@vger.kernel.org; 'Ævar Arnfjörð Bjarmason' ; 'Junio C Hamano' ; 'Jeff Hostetler'\n>; 'Bagas Sanjaya' \n>Subject: Re: [PATCH v5] tr2: log parent process name\n>\n>On Tue, Jun 08, 2021 at 06:16:57PM -0400, Randall S. Becker wrote:\n>>\n>> On June 8, 2021 6:11 PM, Emily Shaffer wrote:\n>> >It can be useful to tell who invoked Git - was it invoked manually by\n>> >a user via CLI or script? By an IDE? In some cases - like 'repo'\n>> >tool - we can influence the source code and set the GIT_TRACE2_PARENT_SID environment variable from the caller process. In\n'repo''s\n>case, that parent SID is manipulated to include the string \"repo\", which means we can positively identify when Git was invoked by\n'repo'\n>tool.\n>> >However, identifying parents that way requires both that we know\n>> >which tools invoke Git and that we have the ability to modify the source code of those tools. It cannot scale to keep up with\nthe various\n>IDEs and wrappers which use Git, most of which we don't know about.\n>> >Learning which tools and wrappers invoke Git, and how, would give us insight to decide where to improve Git's usability and\n>performance.\n>> >\n>> >Unfortunately, there's no cross-platform reliable way to gather the\n>> >name of the parent process. If procfs is present, we can use that;\n>> >otherwise we will need to discover the name another way. However, the process ID should be sufficient to look up the process\nname on\n>most platforms, so that code may be shareable.\n>> >\n>> >Git for Windows gathers similar information and logs it as a \"data_json\"\n>> >event. However, since \"data_json\" has a variable format, it is\n>> >difficult to parse effectively in some languages; instead, let's pursue a dedicated \"cmd_ancestry\" event to record information\nabout the\n>ancestry of the current process and a consistent, parseable way.\n>> >\n>> >Git for Windows also gathers information about more than one\n>> >generation of parent. In Linux further ancestry info can be gathered\n>> >with procfs, but it's unwieldy to do so. In the interest of later\n>> >moving Git for Windows ancestry logging to the 'cmd_ancestry' event, and in the interest of later adding more ancestry to the\nLinux\n>implementation - or of adding this functionality to other platforms which have an easier time walking the process tree - let's make\n>'cmd_ancestry' accept an array of parentage.\n>>\n>> We are probably going to have to discuss this one at more length. On\n>> NonStop, in some cases, I have access to the program arguments of the\n>> parent (rather like ps -ef) in POSIX-land, but not from the other\n>> personality. I do have access to the program object name, in both\n>> sides, although if someone replaces the object - which is not actually\n>> possible for a running program, but a rename is - the object may end\n>> up being somewhat meaningless or mangled. My suspicion is that I'm\n>> going to have to supply different things for the two personalities,\n>> but I'm not sure what as of yet.\n>\n>I guess I'm having trouble understanding - it sounds like you're describing one process with a graph of ancestry instead of a line.\nDoes that\n>mean we shouldn't be tracing the ancestry the way we are?\n\nIt's more like this (g = Guardian, p=Posix, for illustration), for a typical interactive situation coming from the non-POSIX side):\n\ngMonitor -> gAncestor1 -> gAncestor2 -> pAncestor3 (/bin/sh) -> git\n\nAnd when started from an SSH window:\n\ngMonitor -> gAncestor1 (SSH) -> pAncestor2 (/bin/sh) -> git\n\nI can get the program object name from any of the above, and the pid from a POSIX process, or the name (or cpu and process number)\nof a Guardian process. In the case of POSIX, obtaining program arguments may be possible. An ancestor, as with Linux, can have\nmultiple children in a tree but a child can only have one parent - well, technically one at a time anyway because there are some\nfunky exceptions where a child can adopt a different parent in Guardian-land. In both cases, I can get the program object file of\nthe process (like /usr/local/bin/git), but if someone renames git because an install happened during a long-running operation, like\ngit gc --aggressive, the object may be named something else, like /usr/local/bin/ZZLDAG01, for argument sake).\n\nI'm not sure any of this is really relevant, but describes some of what is possible. It also might be useful to pull out the tty\nthat the process was running on. That is easy to get if the terminal is still connected. I think that particular bit of information\nmight be very useful, as well as user information.\n\n-Randall\n\n"},{"id":"426928","messageId":"YMEh0P3sVM8qMZji@google.com","threadId":"55637","inReplyTo":"023701d75cb7$1ff995a0$5fecc0e0$@nexbridge.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-09T20:17:20Z","receivedAt":"2021-06-09T20:18:28Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, Jun 08, 2021 at 06:39:15PM -0400, Randall S. Becker wrote:\n> On June 8, 2021 6:25 PM, Emily Shaffer wrote\"\n> >> We are probably going to have to discuss this one at more length. On\n> >> NonStop, in some cases, I have access to the program arguments of the\n> >> parent (rather like ps -ef) in POSIX-land, but not from the other\n> >> personality. I do have access to the program object name, in both\n> >> sides, although if someone replaces the object - which is not actually\n> >> possible for a running program, but a rename is - the object may end\n> >> up being somewhat meaningless or mangled. My suspicion is that I'm\n> >> going to have to supply different things for the two personalities,\n> >> but I'm not sure what as of yet.\n> >\n> >I guess I'm having trouble understanding - it sounds like you're\n> >describing one process with a graph of ancestry instead of a line.\n> >Does that mean we shouldn't be tracing the ancestry the way we are?\n> \n> It's more like this (g = Guardian, p=Posix, for illustration), for a\n> typical interactive situation coming from the non-POSIX side):\n> \n> gMonitor -> gAncestor1 -> gAncestor2 -> pAncestor3 (/bin/sh) -> git\n> \n> And when started from an SSH window:\n> \n> gMonitor -> gAncestor1 (SSH) -> pAncestor2 (/bin/sh) -> git\n> \n> I can get the program object name from any of the above, and the pid\n> from a POSIX process, or the name (or cpu and process number) of a\n> Guardian process. In the case of POSIX, obtaining program arguments\n> may be possible. An ancestor, as with Linux, can have multiple\n> children in a tree but a child can only have one parent - well,\n> technically one at a time anyway because there are some funky\n> exceptions where a child can adopt a different parent in\n> Guardian-land. In both cases, I can get the program object file of the\n> process (like /usr/local/bin/git), but if someone renames git because\n> an install happened during a long-running operation, like git gc\n> --aggressive, the object may be named something else, like\n> /usr/local/bin/ZZLDAG01, for argument sake).\n\nInteresting. One thing that might be helpful (if you don't care about\nthe later name, since it looks like it might be junk) is that\ntrace2_collect_process_info() provides some hint about when it was\ninvoked, via the 'enum trace2_process_info_reason' arg. So if you're\nworried about a long-running process being renamed, the\nTRACE2_PROCESS_INFO_STARTUP reason happens very early in common-main.c.\n(And you could capture info like \"the name changed later\" at\nTRACE2_PROCESS_INFO_EXIT if you wanted.)\n\n> I'm not sure any of this is really relevant, but describes some of\n> what is possible. It also might be useful to pull out the tty that the\n> process was running on. That is easy to get if the terminal is still\n> connected. I think that particular bit of information might be very\n> useful, as well as user information.\n> \n> -Randall\n> \n\n(I don't have much insight into what would or wouldn't be useful to log\nhere in your case, but I will say that it all sounds very cool and I\nappreciate your thorough explanation.)\n\n - Emily\n"},{"id":"427590","messageId":"xmqqo8c6p5e9.fsf@gitster.g","threadId":"55637","inReplyTo":"20210608221059.1935021-1-emilyshaffer@google.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-16T08:42:22Z","receivedAt":"2021-06-16T08:42:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\nOther than your back-and-forth with Randall on NonStop specifics\nthat didn't result in any code change, there was no comment on this\npatch. Are other people totally happy with this version, or are\nthey totally uninterested?\n\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tstruct strvec names = STRVEC_INIT;\n> +\n> +\t\tget_ancestry_names(&names);\n> +\n> +\t\tif (names.nr == 0) {\n>\n> +\t\t\tstrvec_clear(&names);\n> +\t\t\treturn;\n> +\t\t}\n>\n> +\t\ttrace2_cmd_ancestry(names.v);\n> +\n> +\t\tstrvec_clear(&names);\n\n\nMicronit. CodingGuidelines tells us not to explicitly compare with\nconstant 0, '\\0', or NULL. It may be more concise and easier to\nfollow if written like this:\n\n\t\tget_ancestry_names(&names);\n\t\tif (names.nr)\n\t\t\ttrace2_cmd_ancestry(names.v);\n\t\tstrvec_clear(&names);\n\n\nThanks.\n"},{"id":"428613","messageId":"3327f108-6cd1-1d3d-eae9-2cdff96e1375@jeffhostetler.com","threadId":"55637","inReplyTo":"20210608221059.1935021-1-emilyshaffer@google.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-06-28T16:45:24Z","receivedAt":"2021-06-28T16:45:28Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/8/21 6:10 PM, Emily Shaffer wrote:\n> It can be useful to tell who invoked Git - was it invoked manually by a\n> user via CLI or script? By an IDE? In some cases - like 'repo' tool -\n> we can influence the source code and set the GIT_TRACE2_PARENT_SID\n> environment variable from the caller process. In 'repo''s case, that\n> parent SID is manipulated to include the string \"repo\", which means we\n> can positively identify when Git was invoked by 'repo' tool. However,\n> identifying parents that way requires both that we know which tools\n> invoke Git and that we have the ability to modify the source code of\n> those tools. It cannot scale to keep up with the various IDEs and\n> wrappers which use Git, most of which we don't know about. Learning\n> which tools and wrappers invoke Git, and how, would give us insight to\n> decide where to improve Git's usability and performance.\n> \n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient to look up the process name on most platforms, so\n> that code may be shareable.\n> \n> Git for Windows gathers similar information and logs it as a \"data_json\"\n> event. However, since \"data_json\" has a variable format, it is difficult\n> to parse effectively in some languages; instead, let's pursue a\n> dedicated \"cmd_ancestry\" event to record information about the ancestry\n> of the current process and a consistent, parseable way.\n> \n> Git for Windows also gathers information about more than one generation\n> of parent. In Linux further ancestry info can be gathered with procfs,\n> but it's unwieldy to do so. In the interest of later moving Git for\n> Windows ancestry logging to the 'cmd_ancestry' event, and in the\n> interest of later adding more ancestry to the Linux implementation - or\n> of adding this functionality to other platforms which have an easier\n> time walking the process tree - let's make 'cmd_ancestry' accept an\n> array of parentage.\n> \n> Signed-off-by: Emily Shaffer \n> ---\n> \n> Since v4:\n> \n> Since I'm reading from a file to discover the name of the process, the\n> file is newline-terminated. This newline caused some havoc in tests, so\n> it's better to strip it if it's there. It's not part of the process name\n> itself.\n> \n> Passing tests at\n> https://github.com/nasamuffin/git/actions/runs/919722509\n> \n> Range-diff against v4:\n> 1: efb0a3ccb4 ! 1: 7a7e1ebbfa tr2: log parent process name\n> @@ compat/procinfo.c (new)\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> +\t\tstrbuf_release(&procfs_path);\n> ++\t\tstrbuf_trim_trailing_newline(&name);\n> +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> +\t}\n> +\n\nYou're only getting the name of the command (argv[0]) and not the\nfull command line, right? That is a good thing.\n\n> \n> Documentation/technical/api-trace2.txt | 14 ++++++\n> Makefile | 5 +++\n> compat/procinfo.c | 62 ++++++++++++++++++++++++++\n> config.mak.uname | 1 +\n> t/t0210/scrub_normal.perl | 6 +++\n> t/t0211/scrub_perf.perl | 5 +++\n> t/t0212/parse_events.perl | 5 ++-\n> trace2.c | 13 ++++++\n> trace2.h | 12 ++++-\n> trace2/tr2_tgt.h | 3 ++\n> trace2/tr2_tgt_event.c | 21 +++++++++\n> trace2/tr2_tgt_normal.c | 19 ++++++++\n> trace2/tr2_tgt_perf.c | 16 +++++++\n> 13 files changed, 180 insertions(+), 2 deletions(-)\n> create mode 100644 compat/procinfo.c\n> \n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index 3f52f981a2..8a0b360a0e 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -493,6 +493,20 @@ about specific error arguments.\n> }\n> ------------\n> \n> +`\"cmd_ancestry\"`::\n> +\tThis event contains the text command name for the parent (and earlier\n> +\tgenerations of parents) of the current process, in an array ordered from\n> +\tnearest parent to furthest great-grandparent. It may not be implemented\n> +\ton all platforms.\n> ++\n> +------------\n> +{\n> +\t\"event\":\"cmd_ancestry\",\n> +\t...\n> +\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n\nIs the second element really \"tmux: server\". Seems odd that that's what\nthe command name (argv[0]) is. Perhaps I misread something??\n\n> +}\n\nThis array is bounded and that implies that you captured all of\nthe grand parents back to \"init\" (or whatever it is called these\ndays).\n\nIs there value in having a final \"...\" or \"(truncated)\" element\nto indicate that the list incomplete? I did the latter in the\nWindows version.\n\n\n> +------------\n> +\n> `\"cmd_name\"`::\n> \tThis event contains the command name for this git process\n> \tand the hierarchy of commands from parent git processes.\n> diff --git a/Makefile b/Makefile\n> index 93664d6714..330e4fa011 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -1889,6 +1889,11 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n> \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n> endif\n> \n> +ifdef HAVE_PROCFS_LINUX\n> +\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n> +\tCOMPAT_OBJS += compat/procinfo.o\n> +endif\n> +\n> ifdef HAVE_NS_GET_EXECUTABLE_PATH\n> \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n> endif\n> diff --git a/compat/procinfo.c b/compat/procinfo.c\n> new file mode 100644\n> index 0000000000..f8763cacf8\n> --- /dev/null\n> +++ b/compat/procinfo.c\n> @@ -0,0 +1,62 @@\n> +#include \"cache.h\"\n> +\n> +#include \"strbuf.h\"\n> +#include \"strvec.h\"\n> +#include \"trace2.h\"\n> +\n> +static void get_ancestry_names(struct strvec *names)\n> +{\n> +#ifdef HAVE_PROCFS_LINUX\n> +\t/*\n> +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> +\t * functionality with compat/win32/trace2_win32_process_info.c.\n> +\t * To do so, we may want to examine /proc//stat. For now, just\n> +\t * gather the immediate parent name which is readily accessible from\n> +\t * /proc/$(getppid())/comm.\n> +\t */\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf name = STRBUF_INIT;\n> +\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\tstrbuf_trim_trailing_newline(&name);\n> +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> +\t}\n> +\n> +\treturn;\n> +#endif\n> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n> +}\n\nPerhaps this has already been discussed, but would it be better\nto have a \"compat/linux/trace2_linux_process_info.c\"\nor \"compat/procfs/trace2_procfs_process_info.c\" source file and\nonly compile it in Linux-compatible builds -- rather than #ifdef'ing\nthe source. This is a highly platform-specific feature.\n\nFor example, if I convert the Win32 version to use your new event,\nI wouldn't want to move the code.\n\nI just noticed that you have both \"BASIC_CFLAGS+=\" and a \"COMPAT_OBSJ+=\"\nlines. If you made this source file procfs-specific, you wouldn't need\nthe ifdef and you could avoid the new CFLAG.\n\n> +\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tstruct strvec names = STRVEC_INIT;\n> +\n> +\t\tget_ancestry_names(&names);\n> +\n> +\t\tif (names.nr == 0) {\n> +\t\t\tstrvec_clear(&names);\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\ttrace2_cmd_ancestry(names.v);\n> +\n> +\t\tstrvec_clear(&names);\n\nI agree with Junio here, it would be simpler to say it like this:\n\n\t\tget_ancestry_names(&names);\n\t\tif (names.nr)\n\t\t\ttrace2_cmd_ancestry(names.v);\n\t\tstrvec_clear(&names);\n\n> +\t}\n> +\n> +\treturn;\n> +}\n> diff --git a/config.mak.uname b/config.mak.uname\n> index cb443b4e02..7ad110a1d2 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -58,6 +58,7 @@ ifeq ($(uname_S),Linux)\n> \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n> \tBASIC_CFLAGS += -DHAVE_SYSINFO\n> \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n> +\tHAVE_PROCFS_LINUX = YesPlease\n> endif\n> ifeq ($(uname_S),GNU/kFreeBSD)\n> \tHAVE_ALLOCA_H = YesPlease\n> diff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\n> index c65d1a815e..7cc4de392a 100644\n> --- a/t/t0210/scrub_normal.perl\n> +++ b/t/t0210/scrub_normal.perl\n> @@ -42,6 +42,12 @@\n> \t# so just omit it for testing purposes.\n> \t# print \"cmd_path _EXE_\\n\";\n> }\n> + elsif ($line =~ m/^cmd_ancestry/) {\n> +\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n> +\t# sake, skip it when parsing normal.\n> +\t#\n> +\t# print \"$line\";\n> + }\n> else {\n> \tprint \"$line\";\n> }\n> diff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\n> index 351af7844e..d164b750ff 100644\n> --- a/t/t0211/scrub_perf.perl\n> +++ b/t/t0211/scrub_perf.perl\n> @@ -44,6 +44,11 @@\n> \t# $tokens[$col_rest] = \"_EXE_\";\n> \tgoto SKIP_LINE;\n> }\n> + elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n> +\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n> +\t# so skip it.\n> +\tgoto SKIP_LINE;\n> + }\n> elsif ($tokens[$col_event] =~ m/child_exit/) {\n> \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n> }\n> diff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\n> index 6584bb5634..b6408560c0 100644\n> --- a/t/t0212/parse_events.perl\n> +++ b/t/t0212/parse_events.perl\n> @@ -132,7 +132,10 @@\n> \t# just omit it for testing purposes.\n> \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n> }\n> -\n> + elsif ($event eq 'cmd_ancestry') {\n> +\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n> +\t# just skip it for testing purposes.\n> + }\n> elsif ($event eq 'cmd_name') {\n> \t$processes->{$sid}->{'name'} = $line->{'name'};\n> \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\n> diff --git a/trace2.c b/trace2.c\n> index 256120c7fd..b9b154ac44 100644\n> --- a/trace2.c\n> +++ b/trace2.c\n> @@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n> \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n> }\n> \n> +void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tstruct tr2_tgt *tgt_j;\n> +\tint j;\n> +\n> +\tif (!trace2_enabled)\n> +\t\treturn;\n> +\n> +\tfor_each_wanted_builtin (j, tgt_j)\n> +\t\tif (tgt_j->pfn_command_ancestry_fl)\n> +\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n> +}\n> +\n> void trace2_cmd_name_fl(const char *file, int line, const char *name)\n> {\n> \tstruct tr2_tgt *tgt_j;\n> diff --git a/trace2.h b/trace2.h\n> index ede18c2e06..23743ac62b 100644\n> --- a/trace2.h\n> +++ b/trace2.h\n> @@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n> \n> #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n> \n> +/*\n> + * Emit an 'ancestry' event with the process name of the current process's\n> + * parent process.\n> + * This gives post-processors a way to determine what invoked the command and\n> + * learn more about usage patterns.\n> + */\n> +void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n> +\n> +#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n> +\n> /*\n> * Emit a 'cmd_name' event with the canonical name of the command.\n> * This gives post-processors a simple field to identify the command\n> @@ -492,7 +502,7 @@ enum trace2_process_info_reason {\n> \tTRACE2_PROCESS_INFO_EXIT,\n> };\n> \n> -#if defined(GIT_WINDOWS_NATIVE)\n> +#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n> void trace2_collect_process_info(enum trace2_process_info_reason reason);\n> #else\n> #define trace2_collect_process_info(reason) \\\n> diff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\n> index 7b90469212..1f66fd6573 100644\n> --- a/trace2/tr2_tgt.h\n> +++ b/trace2/tr2_tgt.h\n> @@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n> \n> typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n> \t\t\t\t\t const char *command_path);\n> +typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n> +\t\t\t\t\t\tconst char **parent_names);\n> typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n> \t\t\t\t\t const char *name,\n> \t\t\t\t\t const char *hierarchy);\n> @@ -108,6 +110,7 @@ struct tr2_tgt {\n> \ttr2_tgt_evt_atexit_t *pfn_atexit;\n> \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n> \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n> +\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n> \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n> \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n> \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\n> diff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\n> index 6353e8ad91..578a9a5287 100644\n> --- a/trace2/tr2_tgt_event.c\n> +++ b/trace2/tr2_tgt_event.c\n> @@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tjw_release(&jw);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *event_name = \"cmd_ancestry\";\n> +\tconst char *parent_name = NULL;\n> +\tstruct json_writer jw = JSON_WRITER_INIT;\n> +\n> +\tjw_object_begin(&jw, 0);\n> +\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n> +\tjw_object_inline_begin_array(&jw, \"ancestry\");\n> +\n> +\twhile ((parent_name = *parent_names++))\n> +\t\tjw_array_string(&jw, parent_name);\n> +\n> +\tjw_end(&jw); /* 'ancestry' array */\n> +\tjw_end(&jw); /* event object */\n> +\n> +\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n> +\tjw_release(&jw);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> diff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\n> index 31b602c171..a5751c8864 100644\n> --- a/trace2/tr2_tgt_normal.c\n> +++ b/trace2/tr2_tgt_normal.c\n> @@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tstrbuf_release(&buf_payload);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *parent_name = NULL;\n> +\tstruct strbuf buf_payload = STRBUF_INIT;\n> +\n> +\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n> +\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n> +\twhile ((parent_name = *parent_names++)) {\n> +\t\tstrbuf_addstr(&buf_payload, parent_name);\n> +\t\t/* if we'll write another one after this, add a delimiter */\n> +\t\tif (parent_names && *parent_names)\n> +\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n> +\t}\n> +\n> +\tnormal_io_write_fl(file, line, &buf_payload);\n> +\tstrbuf_release(&buf_payload);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> diff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\n> index a8018f18cc..af4d65a0a5 100644\n> --- a/trace2/tr2_tgt_perf.c\n> +++ b/trace2/tr2_tgt_perf.c\n> @@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n> \tstrbuf_release(&buf_payload);\n> }\n> \n> +static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n> +{\n> +\tconst char *event_name = \"cmd_ancestry\";\n> +\tstruct strbuf buf_payload = STRBUF_INIT;\n> +\n> +\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n> +\t/* It's not an argv but the rules are basically the same. */\n> +\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n> +\tstrbuf_addch(&buf_payload, ']');\n> +\n> +\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n> +\t\t\t &buf_payload);\n> +\tstrbuf_release(&buf_payload);\n> +}\n> +\n> static void fn_command_name_fl(const char *file, int line, const char *name,\n> \t\t\t const char *hierarchy)\n> {\n> @@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n> \tfn_atexit,\n> \tfn_error_va_fl,\n> \tfn_command_path_fl,\n> +\tfn_command_ancestry_fl,\n> \tfn_command_name_fl,\n> \tfn_command_mode_fl,\n> \tfn_alias_fl,\n> \n\nOtherwise, this looks good to me.\nJeff\n"},{"id":"428789","messageId":"YNux62he9Mk43Y1B@google.com","threadId":"55637","inReplyTo":"3327f108-6cd1-1d3d-eae9-2cdff96e1375@jeffhostetler.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-06-29T23:51:07Z","receivedAt":"2021-06-29T23:51:20Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Jun 28, 2021 at 12:45:24PM -0400, Jeff Hostetler wrote:\n> On 6/8/21 6:10 PM, Emily Shaffer wrote:\n> > Range-diff against v4:\n> > 1: efb0a3ccb4 ! 1: 7a7e1ebbfa tr2: log parent process name\n> > @@ compat/procinfo.c (new)\n> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> > +\t\tstrbuf_release(&procfs_path);\n> > ++\t\tstrbuf_trim_trailing_newline(&name);\n> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> > +\t}\n> > +\n> \n> You're only getting the name of the command (argv[0]) and not the\n> full command line, right? That is a good thing.\n\nRoughly. The name can be reset by the process itself (that's what\nhappened, I guess, in the tmux case I pasted below) but by default it's\nargv[0]. It's also truncated to 15ch or something.\n> > ------------\n> > +`\"cmd_ancestry\"`::\n> > +\tThis event contains the text command name for the parent (and earlier\n> > +\tgenerations of parents) of the current process, in an array ordered from\n> > +\tnearest parent to furthest great-grandparent. It may not be implemented\n> > +\ton all platforms.\n> > ++\n> > +------------\n> > +{\n> > +\t\"event\":\"cmd_ancestry\",\n> > +\t...\n> > +\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n> \n> Is the second element really \"tmux: server\". Seems odd that that's what\n> the command name (argv[0]) is. Perhaps I misread something??\n\nSee above. This is what shows up in pstree, though, and by poking around in\n/proc I confirmed that this is indeed the content of /proc//comm:\n\n ├─tmux: server─┬─bash───mutt───open-vim-in-new───vim\n │ ├─bash───pstree\n │ └─mutt\n\nThis is a somewhat contrived example, though, because in Linux as of this patch,\nonly one ancestor is gathered. So maybe I had better make the doc\nreflect what's actually possible. I'm planning on sending a follow-on\nsometime soon exposing more generations of ancestry, so I guess I could\nupdate the docs back to this state around then.\n\n> \n> > +}\n> \n> This array is bounded and that implies that you captured all of\n> the grand parents back to \"init\" (or whatever it is called these\n> days).\n\nIn this case it does - pid 1 is systemd, which hasn't got a parent\nprocess.\n\n> Is there value in having a final \"...\" or \"(truncated)\" element\n> to indicate that the list incomplete? I did the latter in the\n> Windows version.\n\nHrm. I'm not the one who wants to parse these - it's someone else who's\nworking with our team internally - so I'll ask around and see what they\nthink is best.\n\n> > +#ifdef HAVE_PROCFS_LINUX\n> > +\t/*\n> > +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> > +\t * functionality with compat/win32/trace2_win32_process_info.c.\n> > +\t * To do so, we may want to examine /proc//stat. For now, just\n> > +\t * gather the immediate parent name which is readily accessible from\n> > +\t * /proc/$(getppid())/comm.\n> > +\t */\n> > +\tstruct strbuf procfs_path = STRBUF_INIT;\n> > +\tstruct strbuf name = STRBUF_INIT;\n> > +\n> > +\t/* try to use procfs if it's present. */\n> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> > +\t\tstrbuf_release(&procfs_path);\n> > +\t\tstrbuf_trim_trailing_newline(&name);\n> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> > +\t}\n> > +\n> > +\treturn;\n> > +#endif\n> > +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n> > +}\n> \n> Perhaps this has already been discussed, but would it be better\n> to have a \"compat/linux/trace2_linux_process_info.c\"\n> or \"compat/procfs/trace2_procfs_process_info.c\" source file and\n> only compile it in Linux-compatible builds -- rather than #ifdef'ing\n> the source. This is a highly platform-specific feature.\n> \n> For example, if I convert the Win32 version to use your new event,\n> I wouldn't want to move the code.\n> \n> I just noticed that you have both \"BASIC_CFLAGS+=\" and a \"COMPAT_OBSJ+=\"\n> lines. If you made this source file procfs-specific, you wouldn't need\n> the ifdef and you could avoid the new CFLAG.\n\nSure, I'll investigate it, thanks.\n\n> > +\n> > +\t\tif (names.nr == 0) {\n> > +\t\t\tstrvec_clear(&names);\n> > +\t\t\treturn;\n> > +\t\t}\n> > +\n> > +\t\ttrace2_cmd_ancestry(names.v);\n> > +\n> > +\t\tstrvec_clear(&names);\n> \n> I agree with Junio here, it would be simpler to say it like this:\n> \n> \t\tget_ancestry_names(&names);\n> \t\tif (names.nr)\n> \t\t\ttrace2_cmd_ancestry(names.v);\n> \t\tstrvec_clear(&names);\n> \n\nThanks both, done locally.\n\n> Otherwise, this looks good to me.\n\nThanks. Look for a v6 from me this week, hopefully with the build stuff\nsorted out.\n\n - Emily\n"},{"id":"428807","messageId":"87a6n7g9np.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"YNux62he9Mk43Y1B@google.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-30T06:10:59Z","receivedAt":"2021-06-30T06:16:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 29 2021, Emily Shaffer wrote:\n\n> On Mon, Jun 28, 2021 at 12:45:24PM -0400, Jeff Hostetler wrote:\n>> On 6/8/21 6:10 PM, Emily Shaffer wrote:\n>> > Range-diff against v4:\n>> > 1: efb0a3ccb4 ! 1: 7a7e1ebbfa tr2: log parent process name\n>> > @@ compat/procinfo.c (new)\n>> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n>> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n>> > +\t\tstrbuf_release(&procfs_path);\n>> > ++\t\tstrbuf_trim_trailing_newline(&name);\n>> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n>> > +\t}\n>> > +\n>> \n>> You're only getting the name of the command (argv[0]) and not the\n>> full command line, right? That is a good thing.\n>\n> Roughly. The name can be reset by the process itself (that's what\n> happened, I guess, in the tmux case I pasted below) but by default it's\n> argv[0]. It's also truncated to 15ch or something.\n\n16 including the \\0. See prctl(2). Linux has two different ways to\nset/get the name, one is the argv method, the other is\nprctl(PR_SET_NAME). They don't need to match at all. The ps(1) utility\nand some top-like utilities allow you to switch between viewing the two\nversions.\n\nAs noted in the linked manual pages you'll also potentially need to deal\nwith multithreaded programs having different names for each thread.\n\nI don't think we use this now, but FWIW one thing I've wanted to do for\na while was to have the progress.c code update this, so you see if git's\nat N% counting objects or whatever in top.\n\n>> > +#ifdef HAVE_PROCFS_LINUX\n>> > +\t/*\n>> > +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n>> > +\t * functionality with compat/win32/trace2_win32_process_info.c.\n>> > +\t * To do so, we may want to examine /proc//stat. For now, just\n>> > +\t * gather the immediate parent name which is readily accessible from\n>> > +\t * /proc/$(getppid())/comm.\n>> > +\t */\n>> > +\tstruct strbuf procfs_path = STRBUF_INIT;\n>> > +\tstruct strbuf name = STRBUF_INIT;\n>> > +\n>> > +\t/* try to use procfs if it's present. */\n>> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n>> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n>> > +\t\tstrbuf_release(&procfs_path);\n>> > +\t\tstrbuf_trim_trailing_newline(&name);\n>> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n>> > +\t}\n>> > +\n>> > +\treturn;\n>> > +#endif\n>> > +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n>> > +}\n>> \n>> Perhaps this has already been discussed, but would it be better\n>> to have a \"compat/linux/trace2_linux_process_info.c\"\n>> or \"compat/procfs/trace2_procfs_process_info.c\" source file and\n>> only compile it in Linux-compatible builds -- rather than #ifdef'ing\n>> the source. This is a highly platform-specific feature.\n>> \n>> For example, if I convert the Win32 version to use your new event,\n>> I wouldn't want to move the code.\n>> \n>> I just noticed that you have both \"BASIC_CFLAGS+=\" and a \"COMPAT_OBSJ+=\"\n>> lines. If you made this source file procfs-specific, you wouldn't need\n>> the ifdef and you could avoid the new CFLAG.\n\n\nIn general we've preferred not using ifdefs at all except for the small\nbits that absolutely need it.\n\nSo e.g. in this case the whole code should compile on non-Linux, we just\nneed a small boolean guard somewhere to check what the platform is.\n\nIt means we don't have significant pieces of code that don't compile\nexcept on platform X. It's easy to get into your code not compiling if\nyou overuse ifdefs.\n"},{"id":"430859","messageId":"YPi58/qRPQWhZkiI@google.com","threadId":"55637","inReplyTo":"87a6n7g9np.fsf@evledraar.gmail.com","subject":"Re: [PATCH v5] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-07-22T00:21:07Z","receivedAt":"2021-07-22T00:21:15Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Jun 30, 2021 at 08:10:59AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> \n> \n> On Tue, Jun 29 2021, Emily Shaffer wrote:\n> \n> > On Mon, Jun 28, 2021 at 12:45:24PM -0400, Jeff Hostetler wrote:\n> >> On 6/8/21 6:10 PM, Emily Shaffer wrote:\n> >> > Range-diff against v4:\n> >> > 1: efb0a3ccb4 ! 1: 7a7e1ebbfa tr2: log parent process name\n> >> > @@ compat/procinfo.c (new)\n> >> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> >> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> >> > +\t\tstrbuf_release(&procfs_path);\n> >> > ++\t\tstrbuf_trim_trailing_newline(&name);\n> >> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> >> > +\t}\n> >> > +\n> >> \n> >> You're only getting the name of the command (argv[0]) and not the\n> >> full command line, right? That is a good thing.\n> >\n> > Roughly. The name can be reset by the process itself (that's what\n> > happened, I guess, in the tmux case I pasted below) but by default it's\n> > argv[0]. It's also truncated to 15ch or something.\n> \n> 16 including the \\0. See prctl(2). Linux has two different ways to\n> set/get the name, one is the argv method, the other is\n> prctl(PR_SET_NAME). They don't need to match at all. The ps(1) utility\n> and some top-like utilities allow you to switch between viewing the two\n> versions.\n> \n> As noted in the linked manual pages you'll also potentially need to deal\n> with multithreaded programs having different names for each thread.\n> \n> I don't think we use this now, but FWIW one thing I've wanted to do for\n> a while was to have the progress.c code update this, so you see if git's\n> at N% counting objects or whatever in top.\n> \n> >> > +#ifdef HAVE_PROCFS_LINUX\n> >> > +\t/*\n> >> > +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> >> > +\t * functionality with compat/win32/trace2_win32_process_info.c.\n> >> > +\t * To do so, we may want to examine /proc//stat. For now, just\n> >> > +\t * gather the immediate parent name which is readily accessible from\n> >> > +\t * /proc/$(getppid())/comm.\n> >> > +\t */\n> >> > +\tstruct strbuf procfs_path = STRBUF_INIT;\n> >> > +\tstruct strbuf name = STRBUF_INIT;\n> >> > +\n> >> > +\t/* try to use procfs if it's present. */\n> >> > +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> >> > +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> >> > +\t\tstrbuf_release(&procfs_path);\n> >> > +\t\tstrbuf_trim_trailing_newline(&name);\n> >> > +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> >> > +\t}\n> >> > +\n> >> > +\treturn;\n> >> > +#endif\n> >> > +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n> >> > +}\n> >> \n> >> Perhaps this has already been discussed, but would it be better\n> >> to have a \"compat/linux/trace2_linux_process_info.c\"\n> >> or \"compat/procfs/trace2_procfs_process_info.c\" source file and\n> >> only compile it in Linux-compatible builds -- rather than #ifdef'ing\n> >> the source. This is a highly platform-specific feature.\n> >> \n> >> For example, if I convert the Win32 version to use your new event,\n> >> I wouldn't want to move the code.\n> >> \n> >> I just noticed that you have both \"BASIC_CFLAGS+=\" and a \"COMPAT_OBSJ+=\"\n> >> lines. If you made this source file procfs-specific, you wouldn't need\n> >> the ifdef and you could avoid the new CFLAG.\n> \n> \n> In general we've preferred not using ifdefs at all except for the small\n> bits that absolutely need it.\n> \n> So e.g. in this case the whole code should compile on non-Linux, we just\n> need a small boolean guard somewhere to check what the platform is.\n> \n> It means we don't have significant pieces of code that don't compile\n> except on platform X. It's easy to get into your code not compiling if\n> you overuse ifdefs.\n\nHmm. I see what you mean.\n\nHowever, since win32 is already using some conditionally compiling code,\nthat's the approach I went for. It's still running CI to make sure I\ndidn't break Windows in the process, but I'll post it (and the Actions\nruns) later today, assuming it passed.\n\nYes, the implementation I wrote would compile on any platform, but as I\nunderstand it, other platforms may need drastically different\nimplementations which may not compile as easily as this. So, I'm not\nsure if it's really appropriate to do run-time platform checks here\n(which is what I think you were describing).\n\nAnyway, I'll look forward to seeing what you folks think of the next\niteration (again, hopefully later today).\n\n - Emily\n"},{"id":"430862","messageId":"20210722012707.205776-1-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210608221059.1935021-1-emilyshaffer@google.com","subject":"[PATCH v6 0/2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-07-22T01:27:05Z","receivedAt":"2021-07-22T01:27:14Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Since v5, I reshuffled some of the platform-specific compilation recipe\naround per Jeff H's comments on v5. I think these are at odds with\nÆvar's comments, though, so I guess let's take a look and see how\nstrongly we feel? :)\n\nOtherwise I also made the logic inversion Junio suggested to make the\nlogic easier to follow in procinfo.c:trace2_collect_process_info.\n\nThanks,\n - Emily\n\nEmily Shaffer (2):\n tr2: make process info collection platform-generic\n tr2: log parent process name\n\n Documentation/technical/api-trace2.txt | 14 +++++++\n Makefile | 4 ++\n compat/linux/procinfo.c | 55 ++++++++++++++++++++++++++\n compat/stub/procinfo.c | 11 ++++++\n config.mak.uname | 3 ++\n t/t0210/scrub_normal.perl | 6 +++\n t/t0211/scrub_perf.perl | 5 +++\n t/t0212/parse_events.perl | 5 ++-\n trace2.c | 13 ++++++\n trace2.h | 16 +++++---\n trace2/tr2_tgt.h | 3 ++\n trace2/tr2_tgt_event.c | 21 ++++++++++\n trace2/tr2_tgt_normal.c | 19 +++++++++\n trace2/tr2_tgt_perf.c | 16 ++++++++\n 14 files changed, 184 insertions(+), 7 deletions(-)\n create mode 100644 compat/linux/procinfo.c\n create mode 100644 compat/stub/procinfo.c\n\nRange-diff against v5:\n-: ---------- > 1: 80084448e4 tr2: make process info collection platform-generic\n1: 7a7e1ebbfa ! 2: 485f9a24f0 tr2: log parent process name\n @@ Documentation/technical/api-trace2.txt: about specific error arguments.\n \tThis event contains the command name for this git process\n \tand the hierarchy of commands from parent git processes.\n \n - ## Makefile ##\n -@@ Makefile: ifneq ($(PROCFS_EXECUTABLE_PATH),)\n - \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n - endif\n - \n -+ifdef HAVE_PROCFS_LINUX\n -+\tBASIC_CFLAGS += -DHAVE_PROCFS_LINUX\n -+\tCOMPAT_OBJS += compat/procinfo.o\n -+endif\n -+\n - ifdef HAVE_NS_GET_EXECUTABLE_PATH\n - \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n - endif\n -\n - ## compat/procinfo.c (new) ##\n + ## compat/linux/procinfo.c (new) ##\n @@\n +#include \"cache.h\"\n +\n @@ compat/procinfo.c (new)\n +\n +static void get_ancestry_names(struct strvec *names)\n +{\n -+#ifdef HAVE_PROCFS_LINUX\n +\t/*\n +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n +\t * functionality with compat/win32/trace2_win32_process_info.c.\n @@ compat/procinfo.c (new)\n +\t}\n +\n +\treturn;\n -+#endif\n +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n +}\n +\n @@ compat/procinfo.c (new)\n +\n +\t\tget_ancestry_names(&names);\n +\n -+\t\tif (names.nr == 0) {\n -+\t\t\tstrvec_clear(&names);\n -+\t\t\treturn;\n -+\t\t}\n -+\n -+\t\ttrace2_cmd_ancestry(names.v);\n -+\n ++\t\tif (names.nr)\n ++\t\t\ttrace2_cmd_ancestry(names.v);\n +\t\tstrvec_clear(&names);\n +\t}\n +\n @@ config.mak.uname: ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n -+\tHAVE_PROCFS_LINUX = YesPlease\n ++\tHAVE_PLATFORM_PROCINFO = YesPlease\n ++\tCOMPAT_OBJS += compat/linux/procinfo.o\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\n @@ trace2.h: void trace2_cmd_path_fl(const char *file, int line, const char *pathna\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\n -@@ trace2.h: enum trace2_process_info_reason {\n - \tTRACE2_PROCESS_INFO_EXIT,\n - };\n - \n --#if defined(GIT_WINDOWS_NATIVE)\n -+#if ( defined(GIT_WINDOWS_NATIVE) || defined(HAVE_PROCFS_LINUX) )\n - void trace2_collect_process_info(enum trace2_process_info_reason reason);\n - #else\n - #define trace2_collect_process_info(reason) \\\n \n ## trace2/tr2_tgt.h ##\n @@ trace2/tr2_tgt.h: typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n-- \n2.32.0.402.g57bb445576-goog\n\n"},{"id":"430863","messageId":"20210722012707.205776-2-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210722012707.205776-1-emilyshaffer@google.com","subject":"[PATCH v6 1/2] tr2: make process info collection platform-generic","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-07-22T01:27:06Z","receivedAt":"2021-07-22T01:27:17Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"To pave the way for non-Windows platforms to define\ntrace2_collect_process_info(), reorganize the stub-or-definition schema\nto something which doesn't directly reference Windows.\n\nPlatforms which want to collect parent process information in the\nfuture should:\n\n 1. Add an implementation to compat/ (e.g. compat/somearch/procinfo.c)\n 2. Add that object to COMPAT_OBJS to config.mak.uname\n (e.g. COMPAT_OBJS += compat/somearch/procinfo.o)\n 3. Define HAVE_PLATFORM_PROCINFO in config.mak.uname\n\nIn the Windows case, this definition lives in\ncompat/win32/trace2_win32_process_info.c, which is already conditionally\nadded to COMPAT_OBJS; so let's add HAVE_PLATFORM_PROCINFO to hint to the\nbuild that compat/stub/procinfo.c should not be used.\n\nSigned-off-by: Emily Shaffer \nHelped-by: Jonathan Nieder \n---\n Makefile | 4 ++++\n compat/stub/procinfo.c | 11 +++++++++++\n config.mak.uname | 1 +\n trace2.h | 6 ------\n 4 files changed, 16 insertions(+), 6 deletions(-)\n create mode 100644 compat/stub/procinfo.c\n\ndiff --git a/Makefile b/Makefile\nindex 93664d6714..bd76689c79 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1889,6 +1889,10 @@ ifneq ($(PROCFS_EXECUTABLE_PATH),)\n \tBASIC_CFLAGS += '-DPROCFS_EXECUTABLE_PATH=\"$(procfs_executable_path_SQ)\"'\n endif\n \n+ifndef HAVE_PLATFORM_PROCINFO\n+\tCOMPAT_OBJS += compat/stub/procinfo.o\n+endif\n+\n ifdef HAVE_NS_GET_EXECUTABLE_PATH\n \tBASIC_CFLAGS += -DHAVE_NS_GET_EXECUTABLE_PATH\n endif\ndiff --git a/compat/stub/procinfo.c b/compat/stub/procinfo.c\nnew file mode 100644\nindex 0000000000..12c0a23c9e\n--- /dev/null\n+++ b/compat/stub/procinfo.c\n@@ -0,0 +1,11 @@\n+#include \"git-compat-util.h\"\n+\n+#include \"trace2.h\"\n+\n+/*\n+ * Stub. See sample implementations in compat/linux/procinfo.c and\n+ * compat/win32/trace2_win32_process_info.c.\n+ */\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex cb443b4e02..185ff79b14 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -612,6 +612,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tETAGS_TARGET = ETAGS\n \tNO_POSIX_GOODIES = UnfortunatelyYes\n \tDEFAULT_HELP_FORMAT = html\n+\tHAVE_PLATFORM_PROCINFO = YesPlease\n \tBASIC_LDFLAGS += -municode\n \tCOMPAT_CFLAGS += -DNOGDI -Icompat -Icompat/win32\n \tCOMPAT_CFLAGS += -DSTRIP_EXTENSION=\\\".exe\\\"\ndiff --git a/trace2.h b/trace2.h\nindex ede18c2e06..0d990db817 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -492,13 +492,7 @@ enum trace2_process_info_reason {\n \tTRACE2_PROCESS_INFO_EXIT,\n };\n \n-#if defined(GIT_WINDOWS_NATIVE)\n void trace2_collect_process_info(enum trace2_process_info_reason reason);\n-#else\n-#define trace2_collect_process_info(reason) \\\n-\tdo { \\\n-\t} while (0)\n-#endif\n \n const char *trace2_session_id(void);\n \n-- \n2.32.0.402.g57bb445576-goog\n\n"},{"id":"430864","messageId":"20210722012707.205776-3-emilyshaffer@google.com","threadId":"55637","inReplyTo":"20210722012707.205776-1-emilyshaffer@google.com","subject":"[PATCH v6 2/2] tr2: log parent process name","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-07-22T01:27:07Z","receivedAt":"2021-07-22T01:27:22Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"It can be useful to tell who invoked Git - was it invoked manually by a\nuser via CLI or script? By an IDE? In some cases - like 'repo' tool -\nwe can influence the source code and set the GIT_TRACE2_PARENT_SID\nenvironment variable from the caller process. In 'repo''s case, that\nparent SID is manipulated to include the string \"repo\", which means we\ncan positively identify when Git was invoked by 'repo' tool. However,\nidentifying parents that way requires both that we know which tools\ninvoke Git and that we have the ability to modify the source code of\nthose tools. It cannot scale to keep up with the various IDEs and\nwrappers which use Git, most of which we don't know about. Learning\nwhich tools and wrappers invoke Git, and how, would give us insight to\ndecide where to improve Git's usability and performance.\n\nUnfortunately, there's no cross-platform reliable way to gather the name\nof the parent process. If procfs is present, we can use that; otherwise\nwe will need to discover the name another way. However, the process ID\nshould be sufficient to look up the process name on most platforms, so\nthat code may be shareable.\n\nGit for Windows gathers similar information and logs it as a \"data_json\"\nevent. However, since \"data_json\" has a variable format, it is difficult\nto parse effectively in some languages; instead, let's pursue a\ndedicated \"cmd_ancestry\" event to record information about the ancestry\nof the current process and a consistent, parseable way.\n\nGit for Windows also gathers information about more than one generation\nof parent. In Linux further ancestry info can be gathered with procfs,\nbut it's unwieldy to do so. In the interest of later moving Git for\nWindows ancestry logging to the 'cmd_ancestry' event, and in the\ninterest of later adding more ancestry to the Linux implementation - or\nof adding this functionality to other platforms which have an easier\ntime walking the process tree - let's make 'cmd_ancestry' accept an\narray of parentage.\n\nSigned-off-by: Emily Shaffer \n---\n\nNotes:\n Since v3:\n \n Junio and Randall suggested ditching \"get_process_name(ppid)\" as the API to\n implement on various platforms, and I liked the suggestion a lot. So instead,\n I added \"get_ancestry_names(strvec *names)\".\n \n - Using a strvec instead of a char** makes cleanup easier, since strvec takes\n care of counting the number of strings in the array for us. Otherwise we'd\n need to include size as a return or out-param in get_ancestry_names(), and\n that's what the util class is for, right? :)\n - I made get_ancestry_names() static instead of putting it into\n git-compat-util.h. I think I had put get_process_name() into that header to\n facilitate non-procfs implementations, but compat/procinfo.c doesn't seem to\n me to indicate \"procfs only\", and if we do need to implement\n get_process_name() somewhere else, it'll be pretty easy to move it.\n - I added a description of \"cmd_ancestry\" to\n Documentation/technical/api-trace2.txt. I didn't see any user-facing docs to\n update (for example, \"git grep cmd_path\" produces only that one doc file).\n \n Thanks, all.\n \n - Emily\n\n Documentation/technical/api-trace2.txt | 14 +++++++\n compat/linux/procinfo.c | 55 ++++++++++++++++++++++++++\n config.mak.uname | 2 +\n t/t0210/scrub_normal.perl | 6 +++\n t/t0211/scrub_perf.perl | 5 +++\n t/t0212/parse_events.perl | 5 ++-\n trace2.c | 13 ++++++\n trace2.h | 10 +++++\n trace2/tr2_tgt.h | 3 ++\n trace2/tr2_tgt_event.c | 21 ++++++++++\n trace2/tr2_tgt_normal.c | 19 +++++++++\n trace2/tr2_tgt_perf.c | 16 ++++++++\n 12 files changed, 168 insertions(+), 1 deletion(-)\n create mode 100644 compat/linux/procinfo.c\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex 3f52f981a2..8a0b360a0e 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -493,6 +493,20 @@ about specific error arguments.\n }\n ------------\n \n+`\"cmd_ancestry\"`::\n+\tThis event contains the text command name for the parent (and earlier\n+\tgenerations of parents) of the current process, in an array ordered from\n+\tnearest parent to furthest great-grandparent. It may not be implemented\n+\ton all platforms.\n++\n+------------\n+{\n+\t\"event\":\"cmd_ancestry\",\n+\t...\n+\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n+}\n+------------\n+\n `\"cmd_name\"`::\n \tThis event contains the command name for this git process\n \tand the hierarchy of commands from parent git processes.\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nnew file mode 100644\nindex 0000000000..578fed4cd3\n--- /dev/null\n+++ b/compat/linux/procinfo.c\n@@ -0,0 +1,55 @@\n+#include \"cache.h\"\n+\n+#include \"strbuf.h\"\n+#include \"strvec.h\"\n+#include \"trace2.h\"\n+\n+static void get_ancestry_names(struct strvec *names)\n+{\n+\t/*\n+\t * NEEDSWORK: We could gather the entire pstree into an array to match\n+\t * functionality with compat/win32/trace2_win32_process_info.c.\n+\t * To do so, we may want to examine /proc//stat. For now, just\n+\t * gather the immediate parent name which is readily accessible from\n+\t * /proc/$(getppid())/comm.\n+\t */\n+\tstruct strbuf procfs_path = STRBUF_INIT;\n+\tstruct strbuf name = STRBUF_INIT;\n+\n+\t/* try to use procfs if it's present. */\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n+\t\tstrbuf_release(&procfs_path);\n+\t\tstrbuf_trim_trailing_newline(&name);\n+\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t}\n+\n+\treturn;\n+\t/* NEEDSWORK: add non-procfs-linux implementations here */\n+}\n+\n+void trace2_collect_process_info(enum trace2_process_info_reason reason)\n+{\n+\tif (!trace2_is_enabled())\n+\t\treturn;\n+\n+\t/* someday we may want to write something extra here, but not today */\n+\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\treturn;\n+\n+\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\t/*\n+\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n+\t\t * see compat/win32/trace2_win32_process_info.c.\n+\t\t */\n+\t\tstruct strvec names = STRVEC_INIT;\n+\n+\t\tget_ancestry_names(&names);\n+\n+\t\tif (names.nr)\n+\t\t\ttrace2_cmd_ancestry(names.v);\n+\t\tstrvec_clear(&names);\n+\t}\n+\n+\treturn;\n+}\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 185ff79b14..d3bd4c6843 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -58,6 +58,8 @@ ifeq ($(uname_S),Linux)\n \tFREAD_READS_DIRECTORIES = UnfortunatelyYes\n \tBASIC_CFLAGS += -DHAVE_SYSINFO\n \tPROCFS_EXECUTABLE_PATH = /proc/self/exe\n+\tHAVE_PLATFORM_PROCINFO = YesPlease\n+\tCOMPAT_OBJS += compat/linux/procinfo.o\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tHAVE_ALLOCA_H = YesPlease\ndiff --git a/t/t0210/scrub_normal.perl b/t/t0210/scrub_normal.perl\nindex c65d1a815e..7cc4de392a 100644\n--- a/t/t0210/scrub_normal.perl\n+++ b/t/t0210/scrub_normal.perl\n@@ -42,6 +42,12 @@\n \t# so just omit it for testing purposes.\n \t# print \"cmd_path _EXE_\\n\";\n }\n+ elsif ($line =~ m/^cmd_ancestry/) {\n+\t# 'cmd_ancestry' is not implemented everywhere, so for portability's\n+\t# sake, skip it when parsing normal.\n+\t#\n+\t# print \"$line\";\n+ }\n else {\n \tprint \"$line\";\n }\ndiff --git a/t/t0211/scrub_perf.perl b/t/t0211/scrub_perf.perl\nindex 351af7844e..d164b750ff 100644\n--- a/t/t0211/scrub_perf.perl\n+++ b/t/t0211/scrub_perf.perl\n@@ -44,6 +44,11 @@\n \t# $tokens[$col_rest] = \"_EXE_\";\n \tgoto SKIP_LINE;\n }\n+ elsif ($tokens[$col_event] =~ m/cmd_ancestry/) {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere,\n+\t# so skip it.\n+\tgoto SKIP_LINE;\n+ }\n elsif ($tokens[$col_event] =~ m/child_exit/) {\n \t$tokens[$col_rest] =~ s/ pid:\\d* / pid:_PID_ /;\n }\ndiff --git a/t/t0212/parse_events.perl b/t/t0212/parse_events.perl\nindex 6584bb5634..b6408560c0 100644\n--- a/t/t0212/parse_events.perl\n+++ b/t/t0212/parse_events.perl\n@@ -132,7 +132,10 @@\n \t# just omit it for testing purposes.\n \t# $processes->{$sid}->{'path'} = \"_EXE_\";\n }\n- \n+ elsif ($event eq 'cmd_ancestry') {\n+\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n+\t# just skip it for testing purposes.\n+ }\n elsif ($event eq 'cmd_name') {\n \t$processes->{$sid}->{'name'} = $line->{'name'};\n \t$processes->{$sid}->{'hierarchy'} = $line->{'hierarchy'};\ndiff --git a/trace2.c b/trace2.c\nindex 256120c7fd..b9b154ac44 100644\n--- a/trace2.c\n+++ b/trace2.c\n@@ -260,6 +260,19 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname)\n \t\t\ttgt_j->pfn_command_path_fl(file, line, pathname);\n }\n \n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tstruct tr2_tgt *tgt_j;\n+\tint j;\n+\n+\tif (!trace2_enabled)\n+\t\treturn;\n+\n+\tfor_each_wanted_builtin (j, tgt_j)\n+\t\tif (tgt_j->pfn_command_ancestry_fl)\n+\t\t\ttgt_j->pfn_command_ancestry_fl(file, line, parent_names);\n+}\n+\n void trace2_cmd_name_fl(const char *file, int line, const char *name)\n {\n \tstruct tr2_tgt *tgt_j;\ndiff --git a/trace2.h b/trace2.h\nindex 0d990db817..9b7286c572 100644\n--- a/trace2.h\n+++ b/trace2.h\n@@ -133,6 +133,16 @@ void trace2_cmd_path_fl(const char *file, int line, const char *pathname);\n \n #define trace2_cmd_path(p) trace2_cmd_path_fl(__FILE__, __LINE__, (p))\n \n+/*\n+ * Emit an 'ancestry' event with the process name of the current process's\n+ * parent process.\n+ * This gives post-processors a way to determine what invoked the command and\n+ * learn more about usage patterns.\n+ */\n+void trace2_cmd_ancestry_fl(const char *file, int line, const char **parent_names);\n+\n+#define trace2_cmd_ancestry(v) trace2_cmd_ancestry_fl(__FILE__, __LINE__, (v))\n+\n /*\n * Emit a 'cmd_name' event with the canonical name of the command.\n * This gives post-processors a simple field to identify the command\ndiff --git a/trace2/tr2_tgt.h b/trace2/tr2_tgt.h\nindex 7b90469212..1f66fd6573 100644\n--- a/trace2/tr2_tgt.h\n+++ b/trace2/tr2_tgt.h\n@@ -27,6 +27,8 @@ typedef void(tr2_tgt_evt_error_va_fl_t)(const char *file, int line,\n \n typedef void(tr2_tgt_evt_command_path_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *command_path);\n+typedef void(tr2_tgt_evt_command_ancestry_fl_t)(const char *file, int line,\n+\t\t\t\t\t\tconst char **parent_names);\n typedef void(tr2_tgt_evt_command_name_fl_t)(const char *file, int line,\n \t\t\t\t\t const char *name,\n \t\t\t\t\t const char *hierarchy);\n@@ -108,6 +110,7 @@ struct tr2_tgt {\n \ttr2_tgt_evt_atexit_t *pfn_atexit;\n \ttr2_tgt_evt_error_va_fl_t *pfn_error_va_fl;\n \ttr2_tgt_evt_command_path_fl_t *pfn_command_path_fl;\n+\ttr2_tgt_evt_command_ancestry_fl_t\t*pfn_command_ancestry_fl;\n \ttr2_tgt_evt_command_name_fl_t *pfn_command_name_fl;\n \ttr2_tgt_evt_command_mode_fl_t *pfn_command_mode_fl;\n \ttr2_tgt_evt_alias_fl_t *pfn_alias_fl;\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex 6353e8ad91..578a9a5287 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -261,6 +261,26 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tjw_release(&jw);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tconst char *parent_name = NULL;\n+\tstruct json_writer jw = JSON_WRITER_INIT;\n+\n+\tjw_object_begin(&jw, 0);\n+\tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_inline_begin_array(&jw, \"ancestry\");\n+\n+\twhile ((parent_name = *parent_names++))\n+\t\tjw_array_string(&jw, parent_name);\n+\n+\tjw_end(&jw); /* 'ancestry' array */\n+\tjw_end(&jw); /* event object */\n+\n+\ttr2_dst_write_line(&tr2dst_event, &jw.json);\n+\tjw_release(&jw);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -584,6 +604,7 @@ struct tr2_tgt tr2_tgt_event = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex 31b602c171..a5751c8864 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -160,6 +160,24 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *parent_name = NULL;\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\t/* cmd_ancestry parent <- grandparent <- great-grandparent */\n+\tstrbuf_addstr(&buf_payload, \"cmd_ancestry \");\n+\twhile ((parent_name = *parent_names++)) {\n+\t\tstrbuf_addstr(&buf_payload, parent_name);\n+\t\t/* if we'll write another one after this, add a delimiter */\n+\t\tif (parent_names && *parent_names)\n+\t\t\tstrbuf_addstr(&buf_payload, \" <- \");\n+\t}\n+\n+\tnormal_io_write_fl(file, line, &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -306,6 +324,7 @@ struct tr2_tgt tr2_tgt_normal = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a8018f18cc..af4d65a0a5 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -253,6 +253,21 @@ static void fn_command_path_fl(const char *file, int line, const char *pathname)\n \tstrbuf_release(&buf_payload);\n }\n \n+static void fn_command_ancestry_fl(const char *file, int line, const char **parent_names)\n+{\n+\tconst char *event_name = \"cmd_ancestry\";\n+\tstruct strbuf buf_payload = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&buf_payload, \"ancestry:[\");\n+\t/* It's not an argv but the rules are basically the same. */\n+\tsq_append_quote_argv_pretty(&buf_payload, parent_names);\n+\tstrbuf_addch(&buf_payload, ']');\n+\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n+\t\t\t &buf_payload);\n+\tstrbuf_release(&buf_payload);\n+}\n+\n static void fn_command_name_fl(const char *file, int line, const char *name,\n \t\t\t const char *hierarchy)\n {\n@@ -532,6 +547,7 @@ struct tr2_tgt tr2_tgt_perf = {\n \tfn_atexit,\n \tfn_error_va_fl,\n \tfn_command_path_fl,\n+\tfn_command_ancestry_fl,\n \tfn_command_name_fl,\n \tfn_command_mode_fl,\n \tfn_alias_fl,\n-- \n2.32.0.402.g57bb445576-goog\n\n"},{"id":"430938","messageId":"b69a9434-a5c0-d7f2-75f0-c4af518229bf@jeffhostetler.com","threadId":"55637","inReplyTo":"20210722012707.205776-1-emilyshaffer@google.com","subject":"Re: [PATCH v6 0/2] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-07-22T16:59:54Z","receivedAt":"2021-07-22T17:01:14Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 7/21/21 9:27 PM, Emily Shaffer wrote:\n> Since v5, I reshuffled some of the platform-specific compilation recipe\n> around per Jeff H's comments on v5. I think these are at odds with\n> Ævar's comments, though, so I guess let's take a look and see how\n> strongly we feel? :)\n> \n> Otherwise I also made the logic inversion Junio suggested to make the\n> logic easier to follow in procinfo.c:trace2_collect_process_info.\n> \n> Thanks,\n> - Emily\n...\n\nLGTM\n\nThanks,\nJeff\n\n"},{"id":"430948","messageId":"xmqqwnpi83kh.fsf@gitster.g","threadId":"55637","inReplyTo":"20210722012707.205776-3-emilyshaffer@google.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-07-22T21:02:38Z","receivedAt":"2021-07-22T21:02:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> Unfortunately, there's no cross-platform reliable way to gather the name\n> of the parent process. If procfs is present, we can use that; otherwise\n> we will need to discover the name another way. However, the process ID\n> should be sufficient to look up the process name on most platforms, so\n> that code may be shareable.\n\nIs the sentence that begin with \"However\" still relevant? The\nlatest get_ancestry_names() does not add anything when the result\nread from the procfs is unusable, but the sentence gives a false\nimpression that it may somehow fall back to show the PID.\n\n> Git for Windows also gathers information about more than one generation\n> of parent. In Linux further ancestry info can be gathered with procfs,\n> but it's unwieldy to do so. In the interest of later moving Git for\n> Windows ancestry logging to the 'cmd_ancestry' event, and in the\n> interest of later adding more ancestry to the Linux implementation - or\n> of adding this functionality to other platforms which have an easier\n> time walking the process tree - let's make 'cmd_ancestry' accept an\n> array of parentage.\n\nClearly written. Nice.\n\n> +`\"cmd_ancestry\"`::\n> +\tThis event contains the text command name for the parent (and earlier\n> +\tgenerations of parents) of the current process, in an array ordered from\n> +\tnearest parent to furthest great-grandparent. It may not be implemented\n> +\ton all platforms.\n> ++\n> +------------\n> +{\n> +\t\"event\":\"cmd_ancestry\",\n> +\t...\n> +\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n> +}\n> +------------\n\nOK.\n\n> diff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\n> new file mode 100644\n> index 0000000000..578fed4cd3\n> --- /dev/null\n> +++ b/compat/linux/procinfo.c\n> @@ -0,0 +1,55 @@\n> +#include \"cache.h\"\n> +\n> +#include \"strbuf.h\"\n> +#include \"strvec.h\"\n> +#include \"trace2.h\"\n> +\n> +static void get_ancestry_names(struct strvec *names)\n> +{\n> +\t/*\n> +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> +\t * functionality with compat/win32/trace2_win32_process_info.c.\n> +\t * To do so, we may want to examine /proc//stat. For now, just\n> +\t * gather the immediate parent name which is readily accessible from\n> +\t * /proc/$(getppid())/comm.\n> +\t */\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf name = STRBUF_INIT;\n> +\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\tstrbuf_trim_trailing_newline(&name);\n> +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n\nAt this point, we successfully read from /proc/$(ppid)/comm\nand pushed the result into names strvec. I think you would want to\nhave an explicit \"return;\" here.\n\nAlternatively, you could put the rest of the function in the\ncorresponding \"else\" clause, but I think an early return after each\nmethod successfully collects the result would make a lot more sense.\n\n> +\t}\n> +\n> +\treturn;\n> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n\nThis looks the other way around. A future non-procfs-linux\nimplementation written here will be unreachable code ;-)\n\nJust lose \"return;\" from here, have one in the if(){} body, and keep\nthe \"here is where you add more/other implementations\" comment and\nwe'd be OK, I think.\n\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n> +\t\tstruct strvec names = STRVEC_INIT;\n> +\n> +\t\tget_ancestry_names(&names);\n> +\n> +\t\tif (names.nr)\n> +\t\t\ttrace2_cmd_ancestry(names.v);\n> +\t\tstrvec_clear(&names);\n> +\t}\n> +\n> +\treturn;\n> +}\n\nGood.\n\nThanks.\n"},{"id":"431667","messageId":"87tuk8p4uh.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"20210722012707.205776-2-emilyshaffer@google.com","subject":"Re: [PATCH v6 1/2] tr2: make process info collection platform-generic","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T09:34:17Z","receivedAt":"2021-08-02T09:34:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 21 2021, Emily Shaffer wrote:\n\n> +/*\n> + * Stub. See sample implementations in compat/linux/procinfo.c and\n> + * compat/win32/trace2_win32_process_info.c.\n> + */\n\nBoth of those don't exist until 2/2, so that comment should be squashed\ninto that change shouldn't it?\n"},{"id":"431673","messageId":"87r1fcp4gy.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"20210722012707.205776-3-emilyshaffer@google.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T09:38:40Z","receivedAt":"2021-08-02T09:42:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 21 2021, Emily Shaffer wrote:\n\n> +`\"cmd_ancestry\"`::\n> +\tThis event contains the text command name for the parent (and earlier\n> +\tgenerations of parents) of the current process, in an array ordered from\n> +\tnearest parent to furthest great-grandparent. It may not be implemented\n> +\ton all platforms.\n> ++\n> +------------\n> +{\n> +\t\"event\":\"cmd_ancestry\",\n> +\t...\n> +\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n> +}\n> +------------\n> +\n\nOkey, so because of later NEEDSWORK no system that runs systemd will\ncurrrently have output like this, just Windows.\n\n> +\t/*\n> +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> +\t * functionality with compat/win32/trace2_win32_process_info.c.\n> +\t * To do so, we may want to examine /proc//stat. For now, just\n> +\t * gather the immediate parent name which is readily accessible from\n> +\t * /proc/$(getppid())/comm.\n> +\t */\n\nThis comment:\n\n> +\tstruct strbuf procfs_path = STRBUF_INIT;\n> +\tstruct strbuf name = STRBUF_INIT;\n> +\n> +\t/* try to use procfs if it's present. */\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> +\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> +\t\tstrbuf_release(&procfs_path);\n> +\t\tstrbuf_trim_trailing_newline(&name);\n> +\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> +\t}\n> +\n> +\treturn;\n> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n> +}\n> +\n> +void trace2_collect_process_info(enum trace2_process_info_reason reason)\n> +{\n> +\tif (!trace2_is_enabled())\n> +\t\treturn;\n> +\n> +\t/* someday we may want to write something extra here, but not today */\n> +\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n> +\t\treturn;\n> +\n> +\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n\nThis should be a switch/case, so we get the compiler asserting/warning\nif we don't check enum arms, in this case there's just these two, let's\nhave that be clear for readability.\n\n> +\t\t/*\n> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n> +\t\t * see compat/win32/trace2_win32_process_info.c.\n> +\t\t */\n\nSeems to remove the need for this comment, i.e. we comment on this\nlimitation of linux-specific get_ancestry_names twice, both in the\nfunction and its caller.\n\n> - \n> + elsif ($event eq 'cmd_ancestry') {\n> +\t# 'cmd_ancestry' is platform-specific and not implemented everywhere, so\n> +\t# just skip it for testing purposes.\n> + }\n\nThe rest of this code uses two \"\\n\" between elsif arms, let's be\nconsistent here.\n"},{"id":"431674","messageId":"87o8agp29o.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"20210722012707.205776-3-emilyshaffer@google.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T10:22:07Z","receivedAt":"2021-08-02T10:30:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 21 2021, Emily Shaffer wrote:\n\n> Git for Windows also gathers information about more than one generation\n> of parent. In Linux further ancestry info can be gathered with procfs,\n> but it's unwieldy to do so.\n\nHaving read the win32 get_processes() implementation and read proc(5) I\ndon't get how it's unweildy to do so on Linux? Perhaps I'm missing some\nspecial-case but this rather simple patch-on-top seems to do the job for\nme. This includes the unrelated enum/switch/case change I suggested.\n\nI can submit it as a patch-on-top with SOB etc, but maybe there's some\nsubtle reason it won't work properly. It works for me, I get e.g.:\n \n {\n \"event\": \"cmd_ancestry\",\n \"sid\": \"20210802T102731.879424Z-Hc2f5b994-P00001acc\",\n \"thread\": \"main\",\n \"time\": \"2021-08-02T10:27:31.879618Z\",\n \"file\": \"compat/linux/procinfo.c\",\n \"line\": 66,\n \"ancestry\": [\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n }\n\nIf anything the existing Windows version seems a bit unweildy, having to\napparently deal with cycles etc., I don't think that happens under\nprocfs, but maybe I'm missing some special-case.\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 578fed4cd31..671fe2395fd 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -4,51 +4,65 @@\n #include \"strvec.h\"\n #include \"trace2.h\"\n \n-static void get_ancestry_names(struct strvec *names)\n+static int stat_parent_pid(FILE *fp, char *statcomm, int *statppid)\n+{\n+\tchar statstate;\n+\tint statpid;\n+\n+\tint ret = fscanf(fp, \"%d %s %c %d\", &statpid, statcomm, &statstate,\n+\t\t\t statppid);\n+\tif (ret != 4)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+static void push_ancestry_name(struct strvec *names, pid_t pid)\n {\n-\t/*\n-\t * NEEDSWORK: We could gather the entire pstree into an array to match\n-\t * functionality with compat/win32/trace2_win32_process_info.c.\n-\t * To do so, we may want to examine /proc//stat. For now, just\n-\t * gather the immediate parent name which is readily accessible from\n-\t * /proc/$(getppid())/comm.\n-\t */\n \tstruct strbuf procfs_path = STRBUF_INIT;\n-\tstruct strbuf name = STRBUF_INIT;\n+\tchar statcomm[PATH_MAX];\n+\tFILE *fp;\n+\tint ppid;\n \n \t/* try to use procfs if it's present. */\n-\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n-\t\tstrbuf_release(&procfs_path);\n-\t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n-\t}\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n+\tfp = fopen(procfs_path.buf, \"r\");\n+\tif (!fp)\n+\t\treturn;\n \n-\treturn;\n-\t/* NEEDSWORK: add non-procfs-linux implementations here */\n+\tif (stat_parent_pid(fp, statcomm, &ppid) < 0)\n+\t\treturn;\n+\n+\t/*\n+\t * The comm field is in parenthesis, use printf + offset as a\n+\t * poor man's trimming of both ends.\n+\t */\n+\tstrvec_pushf(names, \"%.*s\", (int)strlen(statcomm) - 2, statcomm + 1);\n+\n+\t/*\n+\t * Both errors and reaching the end of the process chain are\n+\t * reported as fields of 0 by proc(5)\n+\t */\n+\tif (ppid != 0)\n+\t\tpush_ancestry_name(names, ppid);\n }\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n {\n-\tif (!trace2_is_enabled())\n-\t\treturn;\n+\tstruct strvec names = STRVEC_INIT;\n \n-\t/* someday we may want to write something extra here, but not today */\n-\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\tif (!trace2_is_enabled())\n \t\treturn;\n \n-\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n-\t\t/*\n-\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n-\t\t * see compat/win32/trace2_win32_process_info.c.\n-\t\t */\n-\t\tstruct strvec names = STRVEC_INIT;\n-\n-\t\tget_ancestry_names(&names);\n+\tswitch (reason) {\n+\tcase TRACE2_PROCESS_INFO_EXIT:\n+\t\tbreak;\n+\tcase TRACE2_PROCESS_INFO_STARTUP:\n+\t\tpush_ancestry_name(&names, getppid());\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n \t\tstrvec_clear(&names);\n+\t\tbreak;\n \t}\n \n \treturn;\n"},{"id":"431675","messageId":"87lf5kp27s.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"20210722012707.205776-3-emilyshaffer@google.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T10:30:34Z","receivedAt":"2021-08-02T10:31:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jul 21 2021, Emily Shaffer wrote:\n\n> compat/linux/procinfo.c | 55 ++++++++++++++++++++++++++\n> [...]\n> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n\nWe'd want non-Windows and non-Linux implementations of this, but that\nNEEDSWORK comment should not be in compat/linux/procinfo.c, where we're\nnever going to add non-Linux code.\n"},{"id":"431678","messageId":"877dh4ovyl.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"87r1fcp4gy.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T12:45:58Z","receivedAt":"2021-08-02T12:46:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Aug 02 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> On Wed, Jul 21 2021, Emily Shaffer wrote:\n>\n>> +`\"cmd_ancestry\"`::\n>> +\tThis event contains the text command name for the parent (and earlier\n>> +\tgenerations of parents) of the current process, in an array ordered from\n>> +\tnearest parent to furthest great-grandparent. It may not be implemented\n>> +\ton all platforms.\n>> ++\n>> +------------\n>> +{\n>> +\t\"event\":\"cmd_ancestry\",\n>> +\t...\n>> +\t\"ancestry\":[\"bash\",\"tmux: server\",\"systemd\"]\n>> +}\n>> +------------\n>> +\n>\n> Okey, so because of later NEEDSWORK no system that runs systemd will\n> currrently have output like this, just Windows.\n>\n>> +\t/*\n>> +\t * NEEDSWORK: We could gather the entire pstree into an array to match\n>> +\t * functionality with compat/win32/trace2_win32_process_info.c.\n>> +\t * To do so, we may want to examine /proc//stat. For now, just\n>> +\t * gather the immediate parent name which is readily accessible from\n>> +\t * /proc/$(getppid())/comm.\n>> +\t */\n>\n> This comment:\n\nTo clarify, this was ment to be continued with \"seems to remove the\nneed...\" below. But I later inserted the commentery about switch/case\nin-between, which made that unclear.\n\n> [...]\n>> [...]\n>> +\t\t/*\n>> +\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n>> +\t\t * see compat/win32/trace2_win32_process_info.c.\n>> +\t\t */\n>\n> Seems to remove the need for this comment, i.e. we comment on this\n> limitation of linux-specific get_ancestry_names twice, both in the\n> function and its caller.\n"},{"id":"431679","messageId":"874kc8ovw5.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"87o8agp29o.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T12:47:13Z","receivedAt":"2021-08-02T12:48:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Aug 02 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> On Wed, Jul 21 2021, Emily Shaffer wrote:\n>\n>> Git for Windows also gathers information about more than one generation\n>> of parent. In Linux further ancestry info can be gathered with procfs,\n>> but it's unwieldy to do so.\n>\n> Having read the win32 get_processes() implementation and read proc(5) I\n> don't get how it's unweildy to do so on Linux? Perhaps I'm missing some\n> special-case but this rather simple patch-on-top seems to do the job for\n> me. This includes the unrelated enum/switch/case change I suggested.\n>\n> I can submit it as a patch-on-top with SOB etc, but maybe there's some\n> subtle reason it won't work properly. It works for me, I get e.g.:\n\nJust a note[...]:\n\n\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n> +\tfp = fopen(procfs_path.buf, \"r\");\n> +\tif (!fp)\n> +\t\treturn;\n\nI think this is mostly ready, but is still clearly in POC state needing\npolishing, e.g. I didn't bother to close \"fp\" here, but a final patch to\nimplement this approach should of course do that.\n"},{"id":"431697","messageId":"48a62d5e-28e2-7103-a5bb-5db7e197a4b9@jeffhostetler.com","threadId":"55637","inReplyTo":"87o8agp29o.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-08-02T15:23:33Z","receivedAt":"2021-08-02T15:23:37Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 8/2/21 6:22 AM, Ævar Arnfjörð Bjarmason wrote:\n...\n> \n> If anything the existing Windows version seems a bit unweildy, having to\n> apparently deal with cycles etc., I don't think that happens under\n> procfs, but maybe I'm missing some special-case.\n> \n\nIt's been a while since I did the parent lookup on Windows, but...\n\nYes, on Windows we observed cases where a PID would be recycled\nand a child process could have the same PID as an ancient ancestor\nthat had long since exited.\n\nI don't think the cycle problem exists on Unix because IIRC a\nchild process' parent-PID is set to Init when the parent exits.\n\nWindows does not update the child's parent-PID when the parent\nexits, so if a recently-used PID is given to one of our child\nprocesses, we can get a loop.\n\nJeff\n"},{"id":"431702","messageId":"00a501d787b8$f8347a80$e89d6f80$@nexbridge.com","threadId":"55637","inReplyTo":"87o8agp29o.fsf@evledraar.gmail.com","subject":"RE: [PATCH v6 2/2] tr2: log parent process name","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2021-08-02T16:10:47Z","receivedAt":"2021-08-02T16:11:03Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 2, 2021 6:22 AM: Ævar Arnfjörð Bjarmason wroteL\n>On Wed, Jul 21 2021, Emily Shaffer wrote:\n>\n>> Git for Windows also gathers information about more than one\n>> generation of parent. In Linux further ancestry info can be gathered\n>> with procfs, but it's unwieldy to do so.\n>\n>Having read the win32 get_processes() implementation and read proc(5) I don't get how it's unweildy to do so on Linux? Perhaps I'm\n>missing some special-case but this rather simple patch-on-top seems to do the job for me. This includes the unrelated enum/switch/case\n>change I suggested.\n>\n>I can submit it as a patch-on-top with SOB etc, but maybe there's some subtle reason it won't work properly. It works for me, I get e.g.:\n>\n> {\n> \"event\": \"cmd_ancestry\",\n> \"sid\": \"20210802T102731.879424Z-Hc2f5b994-P00001acc\",\n> \"thread\": \"main\",\n> \"time\": \"2021-08-02T10:27:31.879618Z\",\n> \"file\": \"compat/linux/procinfo.c\",\n> \"line\": 66,\n> \"ancestry\": [\n> \"bash\",\n> \"screen\",\n> \"systemd\"\n> ]\n> }\n\nShould not the subfields of \"ancestry\" also have field names? I get that they are a list, but it seems a bit restrictive.\n\nMy preference here would be:\n\n \"ancestry\": [\n \"ancestor\": [\n\t\"program\": \"bash\",\n\t\"pid\" : 1234 ],\n \"ancestor\": [\n \"program\": \"screen\"],\n \"ancestor\": [\n \t\"program\" : \"systemd\"],\n ]\n\nWith more richness available in the ancestor.\n\n"},{"id":"431707","messageId":"xmqqwnp3x19i.fsf@gitster.g","threadId":"55637","inReplyTo":"87lf5kp27s.fsf@evledraar.gmail.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-08-02T16:24:57Z","receivedAt":"2021-08-02T16:25:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Wed, Jul 21 2021, Emily Shaffer wrote:\n>\n>> compat/linux/procinfo.c | 55 ++++++++++++++++++++++++++\n>> [...]\n>> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n>\n> We'd want non-Windows and non-Linux implementations of this, but that\n> NEEDSWORK comment should not be in compat/linux/procinfo.c, where we're\n> never going to add non-Linux code.\n\nI am puzzled. This is talking about additional implementation for\nLinux that does not use procfs, no (i.e. what to do with builds of\nLinux that compile out the procfs support or installations that do\nnot mount the /proc hierarchy)?\n\nThe comment seems to be at the right place to remind us of them,\neven though I do not know how important such an environment is.\n"},{"id":"431741","messageId":"87eebbofhq.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"00a501d787b8$f8347a80$e89d6f80$@nexbridge.com","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T18:41:36Z","receivedAt":"2021-08-02T18:42:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Aug 02 2021, Randall S. Becker wrote:\n\n> On August 2, 2021 6:22 AM: Ævar Arnfjörð Bjarmason wroteL\n>>On Wed, Jul 21 2021, Emily Shaffer wrote:\n>>\n>>> Git for Windows also gathers information about more than one\n>>> generation of parent. In Linux further ancestry info can be gathered\n>>> with procfs, but it's unwieldy to do so.\n>>\n>>Having read the win32 get_processes() implementation and read proc(5) I don't get how it's unweildy to do so on Linux? Perhaps I'm\n>>missing some special-case but this rather simple patch-on-top seems to do the job for me. This includes the unrelated enum/switch/case\n>>change I suggested.\n>>\n>>I can submit it as a patch-on-top with SOB etc, but maybe there's some subtle reason it won't work properly. It works for me, I get e.g.:\n>>\n>> {\n>> \"event\": \"cmd_ancestry\",\n>> \"sid\": \"20210802T102731.879424Z-Hc2f5b994-P00001acc\",\n>> \"thread\": \"main\",\n>> \"time\": \"2021-08-02T10:27:31.879618Z\",\n>> \"file\": \"compat/linux/procinfo.c\",\n>> \"line\": 66,\n>> \"ancestry\": [\n>> \"bash\",\n>> \"screen\",\n>> \"systemd\"\n>> ]\n>> }\n>\n> Should not the subfields of \"ancestry\" also have field names? I get that they are a list, but it seems a bit restrictive.\n>\n> My preference here would be:\n>\n> \"ancestry\": [\n> \"ancestor\": [\n> \t\"program\": \"bash\",\n> \t\"pid\" : 1234 ],\n> \"ancestor\": [\n> \"program\": \"screen\"],\n> \"ancestor\": [\n> \t\"program\" : \"systemd\"],\n> ]\n>\n> With more richness available in the ancestor.\n\nThat sounds sensible, but to be clear that's a relevant comment on\nEmily's original patch, my \"let's implement the same for Linux\" is just\nfaithfully reproducing what we're already doing in the Windows\nimplementation.\n\nBut yes, I'd think that including the PID would be a sensible thing to\ndo...\n"},{"id":"431742","messageId":"87bl6fofft.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"xmqqwnp3x19i.fsf@gitster.g","subject":"Re: [PATCH v6 2/2] tr2: log parent process name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-02T18:42:46Z","receivedAt":"2021-08-02T18:43:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Aug 02 2021, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> On Wed, Jul 21 2021, Emily Shaffer wrote:\n>>\n>>> compat/linux/procinfo.c | 55 ++++++++++++++++++++++++++\n>>> [...]\n>>> +\t/* NEEDSWORK: add non-procfs-linux implementations here */\n>>\n>> We'd want non-Windows and non-Linux implementations of this, but that\n>> NEEDSWORK comment should not be in compat/linux/procinfo.c, where we're\n>> never going to add non-Linux code.\n>\n> I am puzzled. This is talking about additional implementation for\n> Linux that does not use procfs, no (i.e. what to do with builds of\n> Linux that compile out the procfs support or installations that do\n> not mount the /proc hierarchy)?\n>\n> The comment seems to be at the right place to remind us of them,\n> even though I do not know how important such an environment is.\n\nYes, I see I misread that, nevermind the narrow suggestion then.\n\nIs there a way to do this sort of thing on Linux without procfs? Other\nthan things that use procfs themselves, e.g. parsing \"ps\" output.\n"},{"id":"433773","messageId":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"87o8agp29o.fsf@evledraar.gmail.com","subject":"[PATCH 0/6] tr2: plug memory leaks + logic errors + Win32 & Linux feature parity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:18Z","receivedAt":"2021-08-25T23:19:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The just-landed 6f64eeab605 (Merge branch\n'es/trace2-log-parent-process-name', 2021-08-24) added parent process\nname logging, but under Linux we'd only log the immediate parent, and\nthe full process chain on Windows.\n\nThis brings the Linux implementation in parity with the Windows\nimplementation. As it turns out /proc//stat is a bit of a pain to\nparse.\n\nThis is preceded by some minor memory leak fixes to\nes/trace2-log-parent-process-name, and the fixing of a bug where we'd\nlog the empty string as a parent if we didn't have procfs.\n\nÆvar Arnfjörð Bjarmason (6):\n tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n tr2: stop leaking \"thread_name\" memory\n tr2: fix memory leak & logic error in 2f732bf15e6\n tr2: do compiler enum check in trace2_collect_process_info()\n tr2: log N parent process names on Linux\n\n compat/linux/procinfo.c | 151 +++++++++++++++++++++++++++++++++-------\n trace2/tr2_tls.c | 1 +\n 2 files changed, 128 insertions(+), 24 deletions(-)\n\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433774","messageId":"patch-1.6-8c649ce3b49-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 1/6] tr2: remove NEEDSWORK comment for \"non-procfs\" implementations","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:19Z","receivedAt":"2021-08-25T23:19:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I'm fairly sure that there is no way on Linux to inspect the process\ntree without using procfs, any tool such as ps(1), top(1) etc. that\nshows this sort of information ultimately looks the information up in\nprocfs.\n\nSo let's remove this comment added in 2f732bf15e6 (tr2: log parent\nprocess name, 2021-07-21), it's setting us up for an impossible task.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 578fed4cd31..15a89676c7a 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -25,7 +25,6 @@ static void get_ancestry_names(struct strvec *names)\n \t}\n \n \treturn;\n-\t/* NEEDSWORK: add non-procfs-linux implementations here */\n }\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433775","messageId":"patch-2.6-0150e3402a7-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 2/6] tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:20Z","receivedAt":"2021-08-25T23:19:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rewrite a comment added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to describe what we might do under\nTRACE2_PROCESS_INFO_EXIT in the future, instead of vaguely referring\nto \"something extra\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 15a89676c7a..62f8aaed4cc 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -32,8 +32,12 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\t/* someday we may want to write something extra here, but not today */\n \tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\t/*\n+\t\t * The Windows version of this calls its\n+\t\t * get_peak_memory_info() here. We may want to insert\n+\t\t * similar process-end statistics here in the future.\n+\t\t */\n \t\treturn;\n \n \tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433776","messageId":"patch-3.6-1fa1bbb6743-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 3/6] tr2: stop leaking \"thread_name\" memory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:21Z","receivedAt":"2021-08-25T23:19:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix a memory leak introduced in ee4512ed481 (trace2: create new\ncombined trace facility, 2019-02-22), we were doing a free() of other\nmemory allocated in tr2tls_create_self(), but not the \"thread_name\"\n\"strbuf strbuf\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n trace2/tr2_tls.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/trace2/tr2_tls.c b/trace2/tr2_tls.c\nindex 067c23755fb..7da94aba522 100644\n--- a/trace2/tr2_tls.c\n+++ b/trace2/tr2_tls.c\n@@ -95,6 +95,7 @@ void tr2tls_unset_self(void)\n \n \tpthread_setspecific(tr2tls_key, NULL);\n \n+\tstrbuf_release(&ctx->thread_name);\n \tfree(ctx->array_us_start);\n \tfree(ctx);\n }\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433777","messageId":"patch-4.6-73e7d4eb6ac-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 4/6] tr2: fix memory leak & logic error in 2f732bf15e6","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:22Z","receivedAt":"2021-08-25T23:19:59Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In a subsequent commit I'll be replacing most of this code to log N\nparents, but let's first fix bugs introduced in the recent\n2f732bf15e6 (tr2: log parent process name, 2021-07-21).\n\nIt was using the strbuf_read_file() in the wrong way, its return value\nis either a length or a negative value on error. If we didn't have a\nprocfs, or otherwise couldn't access it we'd end up pushing an empty\nstring to the trace2 ancestry array.\n\nIt was also using the strvec_push() API the wrong way. That API always\ndoes an xstrdup(), so by detaching the strbuf here we'd leak\nmemory. Let's instead pass in our pointer for strvec_push() to\nxstrdup(), and then free our own strbuf.\n\nFurthermore we need to free that \"procfs_path\" strbuf whether or not\nstrbuf_read_file() succeeds, which was another source of memory leaks\nin 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\nsystem where we could read the file from procfs.\n\nIn combination with the preceding commit this makes all of\nt[0-9]*trace2*.sh pass under SANITIZE=leak on Linux.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 7 ++++---\n 1 file changed, 4 insertions(+), 3 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 62f8aaed4cc..c86daff2b26 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -18,12 +18,13 @@ static void get_ancestry_names(struct strvec *names)\n \n \t/* try to use procfs if it's present. */\n \tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n-\t\tstrbuf_release(&procfs_path);\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n \t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t\tstrvec_push(names, name.buf);\n+\t\tstrbuf_release(&name);\n \t}\n \n+\tstrbuf_release(&procfs_path);\n \treturn;\n }\n \n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433778","messageId":"patch-5.6-4e378da2cce-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 5/6] tr2: do compiler enum check in trace2_collect_process_info()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:23Z","receivedAt":"2021-08-25T23:20:00Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to use a switch statement without a \"default\" branch to\nhave the compiler error if this code ever drifts out of sync with the\nmembers of the \"enum trace2_process_info_reason\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 13 +++++++------\n 1 file changed, 7 insertions(+), 6 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex c86daff2b26..46a751c9a1d 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -30,29 +30,30 @@ static void get_ancestry_names(struct strvec *names)\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n {\n+\tstruct strvec names = STRVEC_INIT;\n+\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\tswitch (reason) {\n+\tcase TRACE2_PROCESS_INFO_EXIT:\n \t\t/*\n \t\t * The Windows version of this calls its\n \t\t * get_peak_memory_info() here. We may want to insert\n \t\t * similar process-end statistics here in the future.\n \t\t */\n-\t\treturn;\n-\n-\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\tbreak;\n+\tcase TRACE2_PROCESS_INFO_STARTUP:\n \t\t/*\n \t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n \t\t * see compat/win32/trace2_win32_process_info.c.\n \t\t */\n-\t\tstruct strvec names = STRVEC_INIT;\n-\n \t\tget_ancestry_names(&names);\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n \t\tstrvec_clear(&names);\n+\t\tbreak;\n \t}\n \n \treturn;\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433779","messageId":"patch-6.6-da003330800-20210825T231400Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH 6/6] tr2: log N parent process names on Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-25T23:19:24Z","receivedAt":"2021-08-25T23:20:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started\nlogging parent process names, but only logged all parents on Windows.\non Linux only the name of the immediate parent process was logged.\n\nExtend the functionality added there to also log full parent chain on\nLinux. In 2f732bf15e6 it was claimed that \"further ancestry info can\nbe gathered with procfs, but it's unwieldy to do so.\".\n\nI don't know what the author meant by that, but I think it probably\nreferred to needing to slurp this up from the FS, as opposed to having\nan API. The underlying semantics on Linux are easier to deal with than\non Windows though, at least as far as finding the parent PIDs\ngoes. See the get_processes() function used on Windows. As shown in\n353d3d77f4f (trace2: collect Windows-specific process information,\n2019-02-22) it needs to deal with cycles.\n\nWhat is more complex on Linux is getting at the process name, a\nsimpler approach is to use fscanf(), see [1] for an implementation of\nthat, but as noted in the comment being added here it would fail in\nthe face of some weird process names, so we need our own\nparse_proc_stat() to parse it out.\n\nWith this patch the \"ancestry\" chain for a trace2 event might look\nlike this:\n\n $ GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version | grep ancestry | jq -r .ancestry\n [\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\nAnd in the case of naughty process names. This uses perl's ability to\nuse prctl(PR_SET_NAME, ...). See Perl/perl5@7636ea95c5 (Set the legacy\nprocess name with prctl() on assignment to $0 on Linux, 2010-04-15)[2]:\n\n $ perl -e '$0 = \"(naughty\\nname)\"; system \"GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version\"' | grep ancestry | jq -r .ancestry\n [\n \"sh\",\n \"(naughty\\nname)\",\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\n1. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n2. https://github.com/Perl/perl5/commit/7636ea95c57762930accf4358f7c0c2dec086b5e\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 134 ++++++++++++++++++++++++++++++++++------\n 1 file changed, 116 insertions(+), 18 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 46a751c9a1d..937084126a6 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -4,27 +4,129 @@\n #include \"strvec.h\"\n #include \"trace2.h\"\n \n-static void get_ancestry_names(struct strvec *names)\n+/*\n+ * We need more complex parsing instat_parent_pid() and\n+ * parse_proc_stat() below than a dumb fscanf(). That's because while\n+ * the statcomm field is surrounded by parentheses, the process itself\n+ * is free to insert any arbitrary byte sequence its its name. That\n+ * can include newlines, spaces, closing parentheses etc. See\n+ * do_task_stat() in fs/proc/array.c in linux.git, this is in contrast\n+ * with the escaped version of the name found in /proc/%d/status.\n+ *\n+ * So instead of using fscanf() we'll read N bytes from it, look for\n+ * the first \"(\", and then the last \")\", anything in-between is our\n+ * process name.\n+ *\n+ * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by\n+ * default, but it can be raised set to values of up to 2^22. So\n+ * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n+ * we're interested in, so 2 * 7 = 14.\n+ *\n+ * We then have 4 spaces between those four values, which brings us up\n+ * to 18. Add the two parentheses and it's 20. The \"state\" is then one\n+ * character (now at 21).\n+ *\n+ * Finally the maximum length of the \"comm\" name itself is 15\n+ * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n+ * to \"123456789abcdef\". See PR_SET_NAME in prctl(2). So all in all\n+ * we'd need to read 21 + 15 = 36 bytes.\n+ *\n+ * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever\n+ * grows past 2^22 we'll be future-proof. We'll then anchor at the\n+ * last \")\" we find to locate the parent PID.\n+ */\n+#define STAT_PARENT_PID_READ_N 64\n+\n+static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,\n+\t\t\t int *statppid)\n {\n+\tconst char *lhs = strchr(sb->buf, '(');\n+\tconst char *rhs = strrchr(sb->buf, ')');\n+\tconst char *ppid_lhs, *ppid_rhs;\n+\tchar *p;\n+\tpid_t ppid;\n+\n+\tif (!lhs || !rhs)\n+\t\tgoto bad_kernel;\n+\n \t/*\n-\t * NEEDSWORK: We could gather the entire pstree into an array to match\n-\t * functionality with compat/win32/trace2_win32_process_info.c.\n-\t * To do so, we may want to examine /proc//stat. For now, just\n-\t * gather the immediate parent name which is readily accessible from\n-\t * /proc/$(getppid())/comm.\n+\t * We're at the \")\", that's followed by \" X \", where X is a\n+\t * single \"state\" character. So advance by 4 bytes.\n \t */\n+\tppid_lhs = rhs + 4;\n+\n+\tppid_rhs = strchr(ppid_lhs, ' ');\n+\tif (!ppid_rhs)\n+\t\tgoto bad_kernel;\n+\n+\tppid = strtol(ppid_lhs, &p, 10);\n+\tif (ppid_rhs == p) {\n+\t\tconst char *comm = lhs + 1;\n+\t\tint commlen = rhs - lhs - 1;\n+\n+\t\tstrbuf_addf(name, \"%.*s\", commlen, comm);\n+\t\t*statppid = ppid;\n+\n+\t\treturn 0;\n+\t}\n+\n+bad_kernel:\n+\t/*\n+\t * We were able to read our STAT_PARENT_PID_READ_N bytes from\n+\t * /proc/%d/stat, but the content is bad. Broken kernel?\n+\t * Should not happen, but handle it gracefully.\n+\t */\n+\treturn -1;\n+}\n+\n+static int stat_parent_pid(pid_t pid, struct strbuf *name, int *statppid)\n+{\n \tstruct strbuf procfs_path = STRBUF_INIT;\n-\tstruct strbuf name = STRBUF_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tsize_t n;\n+\tFILE *fp = NULL;\n+\tint ret = -1;\n \n \t/* try to use procfs if it's present. */\n-\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n-\t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, name.buf);\n-\t\tstrbuf_release(&name);\n-\t}\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n+\tfp = fopen(procfs_path.buf, \"r\");\n+\tif (!fp)\n+\t\tgoto cleanup;\n+\n+\tn = strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp);\n+\tif (n != STAT_PARENT_PID_READ_N)\n+\t\tgoto cleanup;\n+\tif (parse_proc_stat(&sb, name, statppid) < 0)\n+\t\tgoto cleanup;\n \n+\tret = 0;\n+cleanup:\n+\tif (fp)\n+\t\tfclose(fp);\n \tstrbuf_release(&procfs_path);\n+\tstrbuf_release(&sb);\n+\n+\treturn ret;\n+}\n+\n+static void push_ancestry_name(struct strvec *names, pid_t pid)\n+{\n+\tstruct strbuf name = STRBUF_INIT;\n+\tint ppid;\n+\n+\tif (stat_parent_pid(pid, &name, &ppid) < 0)\n+\t\tgoto cleanup;\n+\n+\tstrvec_push(names, name.buf);\n+\n+\t/*\n+\t * Both errors and reaching the end of the process chain are\n+\t * reported as fields of 0 by proc(5)\n+\t */\n+\tif (ppid)\n+\t\tpush_ancestry_name(names, ppid);\n+cleanup:\n+\tstrbuf_release(&name);\n \treturn;\n }\n \n@@ -44,11 +146,7 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \t\t */\n \t\tbreak;\n \tcase TRACE2_PROCESS_INFO_STARTUP:\n-\t\t/*\n-\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n-\t\t * see compat/win32/trace2_win32_process_info.c.\n-\t\t */\n-\t\tget_ancestry_names(&names);\n+\t\tpush_ancestry_name(&names, getppid());\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433790","messageId":"CAPig+cShVK1OChWP+BCx-_8wPV2BKwem8vHgTdYF2gAZX0pFUQ@mail.gmail.com","threadId":"55637","inReplyTo":"patch-6.6-da003330800-20210825T231400Z-avarab@gmail.com","subject":"Re: [PATCH 6/6] tr2: log N parent process names on Linux","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2021-08-25T23:49:21Z","receivedAt":"2021-08-25T23:49:35Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Aug 25, 2021 at 7:20 PM Ævar Arnfjörð Bjarmason\n wrote:\n> [...]\n> Extend the functionality added there to also log full parent chain on\n> Linux. In 2f732bf15e6 it was claimed that \"further ancestry info can\n> be gathered with procfs, but it's unwieldy to do so.\".\n> [...]\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> diff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\n> @@ -4,27 +4,129 @@\n> +/*\n> + * We need more complex parsing instat_parent_pid() and\n\ns/instat_parent_pid/in stat_parent_pid/\n\n> + * parse_proc_stat() below than a dumb fscanf(). That's because while\n> + * the statcomm field is surrounded by parentheses, the process itself\n> + * is free to insert any arbitrary byte sequence its its name. That\n> + * can include newlines, spaces, closing parentheses etc. See\n> + * do_task_stat() in fs/proc/array.c in linux.git, this is in contrast\n> + * with the escaped version of the name found in /proc/%d/status.\n> + *\n> + * So instead of using fscanf() we'll read N bytes from it, look for\n> + * the first \"(\", and then the last \")\", anything in-between is our\n> + * process name.\n> + *\n> + * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by\n> + * default, but it can be raised set to values of up to 2^22. So\n> + * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n> + * we're interested in, so 2 * 7 = 14.\n> + *\n> + * We then have 4 spaces between those four values, which brings us up\n> + * to 18. Add the two parentheses and it's 20. The \"state\" is then one\n> + * character (now at 21).\n> + *\n> + * Finally the maximum length of the \"comm\" name itself is 15\n> + * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n> + * to \"123456789abcdef\". See PR_SET_NAME in prctl(2). So all in all\n> + * we'd need to read 21 + 15 = 36 bytes.\n> + *\n> + * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever\n> + * grows past 2^22 we'll be future-proof. We'll then anchor at the\n> + * last \")\" we find to locate the parent PID.\n> + */\n"},{"id":"433798","messageId":"YScGBj4RJP7iISGp@nand.local","threadId":"55637","inReplyTo":"patch-3.6-1fa1bbb6743-20210825T231400Z-avarab@gmail.com","subject":"Re: [PATCH 3/6] tr2: stop leaking \"thread_name\" memory","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-26T03:09:58Z","receivedAt":"2021-08-26T03:10:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 26, 2021 at 01:19:21AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> Fix a memory leak introduced in ee4512ed481 (trace2: create new\n> combined trace facility, 2019-02-22), we were doing a free() of other\n> memory allocated in tr2tls_create_self(), but not the \"thread_name\"\n> \"strbuf strbuf\".\n\nYour patch is obviously correct, but is \"strbuf strbuf\" a typo for\n\"struct strbuf\"? It doesn't really matter, and it certainly is not worth\nre-submitting 6 patches for a single typo, but just curious.\n\nThanks,\nTaylor\n"},{"id":"433800","messageId":"YScIvuySaXPcBb/Y@nand.local","threadId":"55637","inReplyTo":"patch-4.6-73e7d4eb6ac-20210825T231400Z-avarab@gmail.com","subject":"Re: [PATCH 4/6] tr2: fix memory leak & logic error in 2f732bf15e6","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-26T03:21:34Z","receivedAt":"2021-08-26T03:21:38Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 26, 2021 at 01:19:22AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> In a subsequent commit I'll be replacing most of this code to log N\n> parents, but let's first fix bugs introduced in the recent\n> 2f732bf15e6 (tr2: log parent process name, 2021-07-21).\n>\n> It was using the strbuf_read_file() in the wrong way, its return value\n> is either a length or a negative value on error. If we didn't have a\n> procfs, or otherwise couldn't access it we'd end up pushing an empty\n> string to the trace2 ancestry array.\n\nSure, and I could believe that we didn't ever test the \"couldn't open\n/proc/pid/comm\" case, which explains why this error didn't fail any\ntests. I wondered whether it was worth writing this instead as:\n\n if (strbuf_read_file(...) < 0)\n warning(...);\n\nbefore trimming and pushing the result onto \"names\", but we should\nprobably be swallowing this case and not presenting it as an error\n(since it's completely acceptable to not be able to read from comm,\ne.g., in versions of procfs on Linux older than 2.6.33).\n\n> It was also using the strvec_push() API the wrong way. That API always\n> does an xstrdup(), so by detaching the strbuf here we'd leak\n> memory. Let's instead pass in our pointer for strvec_push() to\n> xstrdup(), and then free our own strbuf.\n\nRight, you could make strvec_push_nodup() non-static, but I don't think\nthat kind of churn is worthwhile.\n\n> Furthermore we need to free that \"procfs_path\" strbuf whether or not\n> strbuf_read_file() succeeds, which was another source of memory leaks\n> in 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\n> system where we could read the file from procfs.\n\nHmm. It's completely correct to only call strbuf_release on names inside\nof the body of the if statement, but it's extra work for the reader to\nfigure that out. Why not put both strbuf_release() calls next to each\nother (immediately before the return) to make it more obvious (since\nfreeing the STRBUF_INIT value is a noop)?\n\nThanks,\nTaylor\n"},{"id":"433801","messageId":"YScJO9SY+H4KbHFV@nand.local","threadId":"55637","inReplyTo":"patch-5.6-4e378da2cce-20210825T231400Z-avarab@gmail.com","subject":"Re: [PATCH 5/6] tr2: do compiler enum check in trace2_collect_process_info()","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-26T03:23:39Z","receivedAt":"2021-08-26T03:23:43Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 26, 2021 at 01:19:23AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> Change code added in 2f732bf15e6 (tr2: log parent process name,\n> 2021-07-21) to use a switch statement without a \"default\" branch to\n> have the compiler error if this code ever drifts out of sync with the\n> members of the \"enum trace2_process_info_reason\".\n\nOK... arguably this code should have been written with a switch\nstatement to begin with, but this looks like unnecessary churn to me.\nMaybe the next patch adds a new value of this enum? Let's see.\n\nThanks,\nTaylor\n"},{"id":"433802","messageId":"YScTaDcPTs1nrP2Y@nand.local","threadId":"55637","inReplyTo":"patch-6.6-da003330800-20210825T231400Z-avarab@gmail.com","subject":"Re: [PATCH 6/6] tr2: log N parent process names on Linux","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-26T04:07:04Z","receivedAt":"2021-08-26T04:07:10Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 26, 2021 at 01:19:24AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started\n> logging parent process names, but only logged all parents on Windows.\n> on Linux only the name of the immediate parent process was logged.\n>\n> Extend the functionality added there to also log full parent chain on\n> Linux. In 2f732bf15e6 it was claimed that \"further ancestry info can\n> be gathered with procfs, but it's unwieldy to do so.\".\n>\n> I don't know what the author meant by that, but I think it probably\n> referred to needing to slurp this up from the FS, as opposed to having\n> an API.\n\nI don't think that this (specifically, \"I don't know what the author\nmeant by that\") is necessary information to include in a patch message.\n\nIf you're looking for a replacement (and you may not be, but just my\n$.02) I would suggest:\n\n \"2f732bf15e6 does not log the full parent chain on Linux; implement\n that functionality here.\"\n\n> What is more complex on Linux is getting at the process name, a\n> simpler approach is to use fscanf(), see [1] for an implementation of\n> that, but as noted in the comment being added here it would fail in\n> the face of some weird process names, so we need our own\n> parse_proc_stat() to parse it out.\n\nThis is helpful information to have for readers that aren't familiar\nwith procfs (especially the detail about why the naive approach doesn't\nwork).\n\n> diff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\n> index 46a751c9a1d..937084126a6 100644\n> --- a/compat/linux/procinfo.c\n> +++ b/compat/linux/procinfo.c\n> @@ -4,27 +4,129 @@\n> #include \"strvec.h\"\n> #include \"trace2.h\"\n>\n> -static void get_ancestry_names(struct strvec *names)\n> +/*\n> + * We need more complex parsing instat_parent_pid() and\n> + * parse_proc_stat() below than a dumb fscanf(). That's because while\n> + * the statcomm field is surrounded by parentheses, the process itself\n> + * is free to insert any arbitrary byte sequence its its name. That\n> + * can include newlines, spaces, closing parentheses etc. See\n> + * do_task_stat() in fs/proc/array.c in linux.git, this is in contrast\n> + * with the escaped version of the name found in /proc/%d/status.\n> + *\n> + * So instead of using fscanf() we'll read N bytes from it, look for\n> + * the first \"(\", and then the last \")\", anything in-between is our\n> + * process name.\n> + *\n> + * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by\n> + * default, but it can be raised set to values of up to 2^22. So\n> + * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n> + * we're interested in, so 2 * 7 = 14.\n> + *\n> + * We then have 4 spaces between those four values, which brings us up\n> + * to 18. Add the two parentheses and it's 20. The \"state\" is then one\n> + * character (now at 21).\n\nHmm, aren't there three spaces, not four?\n\n> + * Finally the maximum length of the \"comm\" name itself is 15\n> + * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n> + * to \"123456789abcdef\". See PR_SET_NAME in prctl(2). So all in all\n> + * we'd need to read 21 + 15 = 36 bytes.\n\nAh, 36 is the right number even though you and I counted a different\nnumber of spaces, since the name is truncated when it goes over *16*\ncharacters, but that includes the NUL byte. So we both arrive at the\nsame number in the end ;).\n\nBut I agree it's safer to just read more (but not too much more) than\nwhat we need.\n\n> + * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever\n> + * grows past 2^22 we'll be future-proof. We'll then anchor at the\n> + * last \")\" we find to locate the parent PID.\n> + */\n> +#define STAT_PARENT_PID_READ_N 64\n> +\n> +static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,\n> +\t\t\t int *statppid)\n\nGoing to think aloud to make sure that this parsing looks right.\n\n> {\n> +\tconst char *lhs = strchr(sb->buf, '(');\n> +\tconst char *rhs = strrchr(sb->buf, ')');\n\nlhs and rhs are going to be on either side of the comm field (which may\nbe helpful to indicate by calling these comm_lhs and comm_rhs). And\nstrrchr makes sure to handle process names that have a ')' in them.\nLooks right.\n\n> +\tconst char *ppid_lhs, *ppid_rhs;\n> +\tchar *p;\n> +\tpid_t ppid;\n> +\n> +\tif (!lhs || !rhs)\n> +\t\tgoto bad_kernel;\n> +\n\nOK.\n\n> \t/*\n> -\t * NEEDSWORK: We could gather the entire pstree into an array to match\n> -\t * functionality with compat/win32/trace2_win32_process_info.c.\n> -\t * To do so, we may want to examine /proc//stat. For now, just\n> -\t * gather the immediate parent name which is readily accessible from\n> -\t * /proc/$(getppid())/comm.\n> +\t * We're at the \")\", that's followed by \" X \", where X is a\n> +\t * single \"state\" character. So advance by 4 bytes.\n> \t */\n> +\tppid_lhs = rhs + 4;\n> +\n> +\tppid_rhs = strchr(ppid_lhs, ' ');\n\nSkipping over the state field gives us the first character of ppid as\nyou say, good. And reading until the first space character will point us\nright after the end, good.\n\n> +\tif (!ppid_rhs)\n> +\t\tgoto bad_kernel;\n> +\n> +\tppid = strtol(ppid_lhs, &p, 10);\n> +\tif (ppid_rhs == p) {\n\nThen parse the ppid and make sure we stopped at the right-hand side\nwhere we should have. Good.\n\n> +\t\tconst char *comm = lhs + 1;\n\nSkipping past the '(', but now I feel like we really should\ns/lhs/comm_&/.\n\n> +\t\tint commlen = rhs - lhs - 1;\n\nThis is right, but you could simplify the expression to be \"rhs - comm\",\nsince you just took into account the left-hand parenthesis in the\nprevious line. Also recommend a size_t here: it's obvious we're not\ngoing to overflow int here, but it saves future readers of having to\nwonder the same thing.\n\n> +\n> +\t\tstrbuf_addf(name, \"%.*s\", commlen, comm);\n> +\t\t*statppid = ppid;\n> +\n> +\t\treturn 0;\n> +\t}\n> +\n> +bad_kernel:\n> +\t/*\n> +\t * We were able to read our STAT_PARENT_PID_READ_N bytes from\n> +\t * /proc/%d/stat, but the content is bad. Broken kernel?\n> +\t * Should not happen, but handle it gracefully.\n> +\t */\n> +\treturn -1;\n> +}\n\nPhew, all seems good. Thanks for bearing with me while I read through\nall of that ;).\n\n> +static int stat_parent_pid(pid_t pid, struct strbuf *name, int *statppid)\n> +{\n> \tstruct strbuf procfs_path = STRBUF_INIT;\n> -\tstruct strbuf name = STRBUF_INIT;\n> +\tstruct strbuf sb = STRBUF_INIT;\n> +\tsize_t n;\n> +\tFILE *fp = NULL;\n\nfopen() will return NULL, and you call it unconditionally, so no need to\ninitialize here.\n\n> +\tint ret = -1;\n>\n> \t/* try to use procfs if it's present. */\n> -\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> -\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n> -\t\tstrbuf_trim_trailing_newline(&name);\n> -\t\tstrvec_push(names, name.buf);\n> -\t\tstrbuf_release(&name);\n> -\t}\n> +\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n> +\tfp = fopen(procfs_path.buf, \"r\");\n> +\tif (!fp)\n> +\t\tgoto cleanup;\n> +\n> +\tn = strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp);\n> +\tif (n != STAT_PARENT_PID_READ_N)\n> +\t\tgoto cleanup;\n\nHmm. Wouldn't we always goto cleanup here, since STAT_PARENT_PID_READ_N\nis deliberately oversized (and not constant anyways, since process ids\ncould be anywhere from 1-7 digits long)?\n\nI think we could probably drop 'n' entirely here, and instead:\n\n if (strbuf_fread(...) < 0)\n goto cleanup;\n\n> +\tif (parse_proc_stat(&sb, name, statppid) < 0)\n> +\t\tgoto cleanup;\n>\n> +\tret = 0;\n> +cleanup:\n> +\tif (fp)\n> +\t\tfclose(fp);\n> \tstrbuf_release(&procfs_path);\n> +\tstrbuf_release(&sb);\n> +\n> +\treturn ret;\n> +}\n> +\n> +static void push_ancestry_name(struct strvec *names, pid_t pid)\n> +{\n> +\tstruct strbuf name = STRBUF_INIT;\n> +\tint ppid;\n> +\n> +\tif (stat_parent_pid(pid, &name, &ppid) < 0)\n> +\t\tgoto cleanup;\n> +\n> +\tstrvec_push(names, name.buf);\n> +\n> +\t/*\n> +\t * Both errors and reaching the end of the process chain are\n> +\t * reported as fields of 0 by proc(5)\n> +\t */\n> +\tif (ppid)\n> +\t\tpush_ancestry_name(names, ppid);\n> +cleanup:\n> +\tstrbuf_release(&name);\n> \treturn;\n> }\n\nThe rest looks good to me, but it looks like you overwrote all of the\nwork that you did in patch 4/6. I guess separating them out makes sense\nif this patch wasn't taken, but I probably would have gone right to this\npatch instead of fixing leaks that you were going to get rid of anyway.\n\nThanks,\nTaylor\n"},{"id":"433824","messageId":"patch-v2-1.6-8c649ce3b49-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 1/6] tr2: remove NEEDSWORK comment for \"non-procfs\" implementations","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:19Z","receivedAt":"2021-08-26T12:22:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I'm fairly sure that there is no way on Linux to inspect the process\ntree without using procfs, any tool such as ps(1), top(1) etc. that\nshows this sort of information ultimately looks the information up in\nprocfs.\n\nSo let's remove this comment added in 2f732bf15e6 (tr2: log parent\nprocess name, 2021-07-21), it's setting us up for an impossible task.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 578fed4cd31..15a89676c7a 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -25,7 +25,6 @@ static void get_ancestry_names(struct strvec *names)\n \t}\n \n \treturn;\n-\t/* NEEDSWORK: add non-procfs-linux implementations here */\n }\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433826","messageId":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-0.6-00000000000-20210825T231400Z-avarab@gmail.com","subject":"[PATCH v2 0/6] tr2: plug memory leaks + logic errors + Win32 & Linux feature parity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:18Z","receivedAt":"2021-08-26T12:22:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"An early re-roll due to some very good & early review by Taylor Blau,\nthis also fixes the grammar/typo pointed out by Eric Sunshine. Thanks\nboth:\n\nThis should address all the comments Taylor brought up, and hopefully\na bit more. The only thing I left outstanding was the inclusion of the\n\"while we're at it\" enum refactoring in 5/6. It's not necessary for\nthe end-state here, but I think since we're already reviewing most of\nthis file it makes sense to change it while we're at it to\nfuture-proof the code vis-as-vis getting stricted checks from the\ncompiler.\n\nÆvar Arnfjörð Bjarmason (6):\n tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n tr2: stop leaking \"thread_name\" memory\n tr2: fix memory leak & logic error in 2f732bf15e6\n tr2: do compiler enum check in trace2_collect_process_info()\n tr2: log N parent process names on Linux\n\n compat/linux/procinfo.c | 169 ++++++++++++++++++++++++++++++++++------\n trace2/tr2_tls.c | 1 +\n 2 files changed, 146 insertions(+), 24 deletions(-)\n\nRange-diff against v1:\n1: 8c649ce3b49 = 1: 8c649ce3b49 tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n2: 0150e3402a7 = 2: 0150e3402a7 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n3: 1fa1bbb6743 ! 3: 1d835d6767e tr2: stop leaking \"thread_name\" memory\n @@ Commit message\n Fix a memory leak introduced in ee4512ed481 (trace2: create new\n combined trace facility, 2019-02-22), we were doing a free() of other\n memory allocated in tr2tls_create_self(), but not the \"thread_name\"\n - \"strbuf strbuf\".\n + \"struct strbuf\".\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n4: 73e7d4eb6ac ! 4: 1aa0dbc394e tr2: fix memory leak & logic error in 2f732bf15e6\n @@ Commit message\n It was also using the strvec_push() API the wrong way. That API always\n does an xstrdup(), so by detaching the strbuf here we'd leak\n memory. Let's instead pass in our pointer for strvec_push() to\n - xstrdup(), and then free our own strbuf.\n + xstrdup(), and then free our own strbuf. I do have some WIP changes to\n + make strvec_push_nodup() non-static, which makes this and some other\n + callsites nicer, but let's just follow the prevailing pattern of using\n + strvec_push() for now.\n \n - Furthermore we need to free that \"procfs_path\" strbuf whether or not\n + We'll also need to free that \"procfs_path\" strbuf whether or not\n strbuf_read_file() succeeds, which was another source of memory leaks\n in 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\n system where we could read the file from procfs.\n \n + Let's move all the freeing of the memory to the end of the\n + function. If we're still at STRBUF_INIT with \"name\" due to not haven\n + taken the branch where the strbuf_read_file() succeeds freeing it is\n + redundant, so we could move it into the body of the \"if\", but just\n + handling freeing the same way for all branches of the function makes\n + it more readable.\n +\n In combination with the preceding commit this makes all of\n t[0-9]*trace2*.sh pass under SANITIZE=leak on Linux.\n \n @@ compat/linux/procinfo.c: static void get_ancestry_names(struct strvec *names)\n \t\tstrbuf_trim_trailing_newline(&name);\n -\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n +\t\tstrvec_push(names, name.buf);\n -+\t\tstrbuf_release(&name);\n \t}\n \n +\tstrbuf_release(&procfs_path);\n ++\tstrbuf_release(&name);\n ++\n \treturn;\n }\n \n5: 4e378da2cce = 5: 70fef093d8d tr2: do compiler enum check in trace2_collect_process_info()\n6: da003330800 ! 6: f6aac902484 tr2: log N parent process names on Linux\n @@ Commit message\n on Linux only the name of the immediate parent process was logged.\n \n Extend the functionality added there to also log full parent chain on\n - Linux. In 2f732bf15e6 it was claimed that \"further ancestry info can\n - be gathered with procfs, but it's unwieldy to do so.\".\n -\n - I don't know what the author meant by that, but I think it probably\n - referred to needing to slurp this up from the FS, as opposed to having\n - an API. The underlying semantics on Linux are easier to deal with than\n - on Windows though, at least as far as finding the parent PIDs\n - goes. See the get_processes() function used on Windows. As shown in\n - 353d3d77f4f (trace2: collect Windows-specific process information,\n - 2019-02-22) it needs to deal with cycles.\n -\n - What is more complex on Linux is getting at the process name, a\n - simpler approach is to use fscanf(), see [1] for an implementation of\n - that, but as noted in the comment being added here it would fail in\n - the face of some weird process names, so we need our own\n - parse_proc_stat() to parse it out.\n + Linux.\n +\n + This requires us to lookup \"/proc//stat\" instead of\n + \"/proc//comm\". The \"comm\" file just contains the name of the\n + process, but the \"stat\" file has both that information, and the parent\n + PID of that process, see procfs(5). We parse out the parent PID of our\n + own parent, and recursively walk the chain of \"/proc/*/stat\" files all\n + the way up the chain. A parent PID of 0 indicates the end of the\n + chain.\n +\n + It's possible given the semantics of Linux's PID files that we end up\n + getting an entirely nonsensical chain of processes. It could happen if\n + e.g. we have a chain of processes like:\n +\n + 1 (init) => 321 (bash) => 123 (git)\n +\n + Let's assume that \"bash\" was started a while ago, and that as shown\n + the OS has already cycled back to using a lower PID for us than our\n + parent process. In the time it takes us to start up and get to\n + trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP) our parent\n + process might exit, and be replaced by an entirely different process!\n +\n + We'd racily look up our own getppid(), but in the meantime our parent\n + would exit, and Linux would have cycled all the way back to starting\n + an entirely unrelated process as PID 321.\n +\n + If that happens we'll just silently log incorrect data in our ancestry\n + chain. Luckily we don't need to worry about this except in this\n + specific cycling scenario, as Linux does not have PID\n + randomization. It appears it once did through a third-party feature,\n + but that it was removed around 2006[1]. For anyone worried about this\n + edge case raising PID_MAX via \"/proc/sys/kernel/pid_max\" will mitigate\n + it, but not eliminate it.\n +\n + One thing we don't need to worry about is getting into an infinite\n + loop when walking \"/proc/*/stat\". See 353d3d77f4f (trace2: collect\n + Windows-specific process information, 2019-02-22) for the related\n + Windows code that needs to deal with that, and [2] for an explanation\n + of that edge case.\n +\n + Aside from potential race conditions it's also a bit painful to\n + correctly parse the process name out of \"/proc/*/stat\". A simpler\n + approach is to use fscanf(), see [3] for an implementation of that,\n + but as noted in the comment being added here it would fail in the face\n + of some weird process names, so we need our own parse_proc_stat() to\n + parse it out.\n \n With this patch the \"ancestry\" chain for a trace2 event might look\n like this:\n @@ Commit message\n \"systemd\"\n ]\n \n - And in the case of naughty process names. This uses perl's ability to\n - use prctl(PR_SET_NAME, ...). See Perl/perl5@7636ea95c5 (Set the legacy\n - process name with prctl() on assignment to $0 on Linux, 2010-04-15)[2]:\n + And in the case of naughty process names like the following. This uses\n + perl's ability to use prctl(PR_SET_NAME, ...). See\n + Perl/perl5@7636ea95c5 (Set the legacy process name with prctl() on\n + assignment to $0 on Linux, 2010-04-15)[4]:\n \n $ perl -e '$0 = \"(naughty\\nname)\"; system \"GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version\"' | grep ancestry | jq -r .ancestry\n [\n @@ Commit message\n \"systemd\"\n ]\n \n - 1. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n - 2. https://github.com/Perl/perl5/commit/7636ea95c57762930accf4358f7c0c2dec086b5e\n + 1. https://grsecurity.net/news#grsec2110\n + 2. https://lore.kernel.org/git/48a62d5e-28e2-7103-a5bb-5db7e197a4b9@jeffhostetler.com/\n + 3. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n + 4. https://github.com/Perl/perl5/commit/7636ea95c57762930accf4358f7c0c2dec086b5e\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ compat/linux/procinfo.c\n \n -static void get_ancestry_names(struct strvec *names)\n +/*\n -+ * We need more complex parsing instat_parent_pid() and\n ++ * We need more complex parsing in stat_parent_pid() and\n + * parse_proc_stat() below than a dumb fscanf(). That's because while\n + * the statcomm field is surrounded by parentheses, the process itself\n + * is free to insert any arbitrary byte sequence its its name. That\n -+ * can include newlines, spaces, closing parentheses etc. See\n -+ * do_task_stat() in fs/proc/array.c in linux.git, this is in contrast\n -+ * with the escaped version of the name found in /proc/%d/status.\n ++ * can include newlines, spaces, closing parentheses etc.\n ++ *\n ++ * See do_task_stat() in fs/proc/array.c in linux.git, this is in\n ++ * contrast with the escaped version of the name found in\n ++ * /proc/%d/status.\n + *\n + * So instead of using fscanf() we'll read N bytes from it, look for\n + * the first \"(\", and then the last \")\", anything in-between is our\n @@ compat/linux/procinfo.c\n + * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n + * we're interested in, so 2 * 7 = 14.\n + *\n -+ * We then have 4 spaces between those four values, which brings us up\n -+ * to 18. Add the two parentheses and it's 20. The \"state\" is then one\n -+ * character (now at 21).\n ++ * We then have 3 spaces between those four values, and we'd like to\n ++ * get to the space between the 4th and the 5th (the \"pgrp\" field) to\n ++ * make sure we read the entire \"ppid\" field. So that brings us up to\n ++ * 14 + 3 + 1 = 18. Add the two parentheses around the \"comm\" value\n ++ * and it's 20. The \"state\" value itself is then one character (now at\n ++ * 21).\n + *\n + * Finally the maximum length of the \"comm\" name itself is 15\n + * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n @@ compat/linux/procinfo.c\n +static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,\n +\t\t\t int *statppid)\n {\n -+\tconst char *lhs = strchr(sb->buf, '(');\n -+\tconst char *rhs = strrchr(sb->buf, ')');\n ++\tconst char *comm_lhs = strchr(sb->buf, '(');\n ++\tconst char *comm_rhs = strrchr(sb->buf, ')');\n +\tconst char *ppid_lhs, *ppid_rhs;\n +\tchar *p;\n +\tpid_t ppid;\n +\n -+\tif (!lhs || !rhs)\n ++\tif (!comm_lhs || !comm_rhs)\n +\t\tgoto bad_kernel;\n +\n \t/*\n @@ compat/linux/procinfo.c\n +\t * We're at the \")\", that's followed by \" X \", where X is a\n +\t * single \"state\" character. So advance by 4 bytes.\n \t */\n -+\tppid_lhs = rhs + 4;\n ++\tppid_lhs = comm_rhs + 4;\n +\n ++\t/*\n ++\t * Read until the space between the \"ppid\" and \"pgrp\" fields\n ++\t * to make sure we're anchored after the untruncated \"ppid\"\n ++\t * field..\n ++\t */\n +\tppid_rhs = strchr(ppid_lhs, ' ');\n +\tif (!ppid_rhs)\n +\t\tgoto bad_kernel;\n +\n +\tppid = strtol(ppid_lhs, &p, 10);\n +\tif (ppid_rhs == p) {\n -+\t\tconst char *comm = lhs + 1;\n -+\t\tint commlen = rhs - lhs - 1;\n ++\t\tconst char *comm = comm_lhs + 1;\n ++\t\tsize_t commlen = comm_rhs - comm;\n +\n -+\t\tstrbuf_addf(name, \"%.*s\", commlen, comm);\n ++\t\tstrbuf_add(name, comm, commlen);\n +\t\t*statppid = ppid;\n +\n +\t\treturn 0;\n @@ compat/linux/procinfo.c\n \tstruct strbuf procfs_path = STRBUF_INIT;\n -\tstruct strbuf name = STRBUF_INIT;\n +\tstruct strbuf sb = STRBUF_INIT;\n -+\tsize_t n;\n -+\tFILE *fp = NULL;\n ++\tFILE *fp;\n +\tint ret = -1;\n \n \t/* try to use procfs if it's present. */\n @@ compat/linux/procinfo.c\n -\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n -\t\tstrbuf_trim_trailing_newline(&name);\n -\t\tstrvec_push(names, name.buf);\n --\t\tstrbuf_release(&name);\n -\t}\n +\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n +\tfp = fopen(procfs_path.buf, \"r\");\n +\tif (!fp)\n +\t\tgoto cleanup;\n +\n -+\tn = strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp);\n -+\tif (n != STAT_PARENT_PID_READ_N)\n ++\t/*\n ++\t * We could be more strict here and assert that we read at\n ++\t * least STAT_PARENT_PID_READ_N. My reading of procfs(5) is\n ++\t * that on any modern kernel (at least since 2.6.0 released in\n ++\t * 2003) even if all the mandatory numeric fields were zero'd\n ++\t * out we'd get at least 100 bytes, but let's just check that\n ++\t * we got anything at all and trust the parse_proc_stat()\n ++\t * function to handle its \"Bad Kernel?\" error checking.\n ++\t */\n ++\tif (!strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp))\n +\t\tgoto cleanup;\n +\tif (parse_proc_stat(&sb, name, statppid) < 0)\n +\t\tgoto cleanup;\n @@ compat/linux/procinfo.c\n +\tif (ppid)\n +\t\tpush_ancestry_name(names, ppid);\n +cleanup:\n -+\tstrbuf_release(&name);\n - \treturn;\n - }\n + \tstrbuf_release(&name);\n \n + \treturn;\n @@ compat/linux/procinfo.c: void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \t\t */\n \t\tbreak;\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433825","messageId":"patch-v2-2.6-0150e3402a7-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 2/6] tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:20Z","receivedAt":"2021-08-26T12:22:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rewrite a comment added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to describe what we might do under\nTRACE2_PROCESS_INFO_EXIT in the future, instead of vaguely referring\nto \"something extra\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 15a89676c7a..62f8aaed4cc 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -32,8 +32,12 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\t/* someday we may want to write something extra here, but not today */\n \tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\t/*\n+\t\t * The Windows version of this calls its\n+\t\t * get_peak_memory_info() here. We may want to insert\n+\t\t * similar process-end statistics here in the future.\n+\t\t */\n \t\treturn;\n \n \tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433827","messageId":"patch-v2-3.6-1d835d6767e-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 3/6] tr2: stop leaking \"thread_name\" memory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:21Z","receivedAt":"2021-08-26T12:22:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix a memory leak introduced in ee4512ed481 (trace2: create new\ncombined trace facility, 2019-02-22), we were doing a free() of other\nmemory allocated in tr2tls_create_self(), but not the \"thread_name\"\n\"struct strbuf\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n trace2/tr2_tls.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/trace2/tr2_tls.c b/trace2/tr2_tls.c\nindex 067c23755fb..7da94aba522 100644\n--- a/trace2/tr2_tls.c\n+++ b/trace2/tr2_tls.c\n@@ -95,6 +95,7 @@ void tr2tls_unset_self(void)\n \n \tpthread_setspecific(tr2tls_key, NULL);\n \n+\tstrbuf_release(&ctx->thread_name);\n \tfree(ctx->array_us_start);\n \tfree(ctx);\n }\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433828","messageId":"patch-v2-5.6-70fef093d8d-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 5/6] tr2: do compiler enum check in trace2_collect_process_info()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:23Z","receivedAt":"2021-08-26T12:22:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to use a switch statement without a \"default\" branch to\nhave the compiler error if this code ever drifts out of sync with the\nmembers of the \"enum trace2_process_info_reason\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 13 +++++++------\n 1 file changed, 7 insertions(+), 6 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex bd01f017bc7..0b47d44990c 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -31,29 +31,30 @@ static void get_ancestry_names(struct strvec *names)\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n {\n+\tstruct strvec names = STRVEC_INIT;\n+\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\tswitch (reason) {\n+\tcase TRACE2_PROCESS_INFO_EXIT:\n \t\t/*\n \t\t * The Windows version of this calls its\n \t\t * get_peak_memory_info() here. We may want to insert\n \t\t * similar process-end statistics here in the future.\n \t\t */\n-\t\treturn;\n-\n-\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\tbreak;\n+\tcase TRACE2_PROCESS_INFO_STARTUP:\n \t\t/*\n \t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n \t\t * see compat/win32/trace2_win32_process_info.c.\n \t\t */\n-\t\tstruct strvec names = STRVEC_INIT;\n-\n \t\tget_ancestry_names(&names);\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n \t\tstrvec_clear(&names);\n+\t\tbreak;\n \t}\n \n \treturn;\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433829","messageId":"patch-v2-4.6-1aa0dbc394e-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 4/6] tr2: fix memory leak & logic error in 2f732bf15e6","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:22Z","receivedAt":"2021-08-26T12:23:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In a subsequent commit I'll be replacing most of this code to log N\nparents, but let's first fix bugs introduced in the recent\n2f732bf15e6 (tr2: log parent process name, 2021-07-21).\n\nIt was using the strbuf_read_file() in the wrong way, its return value\nis either a length or a negative value on error. If we didn't have a\nprocfs, or otherwise couldn't access it we'd end up pushing an empty\nstring to the trace2 ancestry array.\n\nIt was also using the strvec_push() API the wrong way. That API always\ndoes an xstrdup(), so by detaching the strbuf here we'd leak\nmemory. Let's instead pass in our pointer for strvec_push() to\nxstrdup(), and then free our own strbuf. I do have some WIP changes to\nmake strvec_push_nodup() non-static, which makes this and some other\ncallsites nicer, but let's just follow the prevailing pattern of using\nstrvec_push() for now.\n\nWe'll also need to free that \"procfs_path\" strbuf whether or not\nstrbuf_read_file() succeeds, which was another source of memory leaks\nin 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\nsystem where we could read the file from procfs.\n\nLet's move all the freeing of the memory to the end of the\nfunction. If we're still at STRBUF_INIT with \"name\" due to not haven\ntaken the branch where the strbuf_read_file() succeeds freeing it is\nredundant, so we could move it into the body of the \"if\", but just\nhandling freeing the same way for all branches of the function makes\nit more readable.\n\nIn combination with the preceding commit this makes all of\nt[0-9]*trace2*.sh pass under SANITIZE=leak on Linux.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 62f8aaed4cc..bd01f017bc7 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -18,12 +18,14 @@ static void get_ancestry_names(struct strvec *names)\n \n \t/* try to use procfs if it's present. */\n \tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n-\t\tstrbuf_release(&procfs_path);\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n \t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t\tstrvec_push(names, name.buf);\n \t}\n \n+\tstrbuf_release(&procfs_path);\n+\tstrbuf_release(&name);\n+\n \treturn;\n }\n \n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433830","messageId":"patch-v2-6.6-f6aac902484-20210826T121820Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v2 6/6] tr2: log N parent process names on Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:22:24Z","receivedAt":"2021-08-26T12:23:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started\nlogging parent process names, but only logged all parents on Windows.\non Linux only the name of the immediate parent process was logged.\n\nExtend the functionality added there to also log full parent chain on\nLinux.\n\nThis requires us to lookup \"/proc//stat\" instead of\n\"/proc//comm\". The \"comm\" file just contains the name of the\nprocess, but the \"stat\" file has both that information, and the parent\nPID of that process, see procfs(5). We parse out the parent PID of our\nown parent, and recursively walk the chain of \"/proc/*/stat\" files all\nthe way up the chain. A parent PID of 0 indicates the end of the\nchain.\n\nIt's possible given the semantics of Linux's PID files that we end up\ngetting an entirely nonsensical chain of processes. It could happen if\ne.g. we have a chain of processes like:\n\n 1 (init) => 321 (bash) => 123 (git)\n\nLet's assume that \"bash\" was started a while ago, and that as shown\nthe OS has already cycled back to using a lower PID for us than our\nparent process. In the time it takes us to start up and get to\ntrace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP) our parent\nprocess might exit, and be replaced by an entirely different process!\n\nWe'd racily look up our own getppid(), but in the meantime our parent\nwould exit, and Linux would have cycled all the way back to starting\nan entirely unrelated process as PID 321.\n\nIf that happens we'll just silently log incorrect data in our ancestry\nchain. Luckily we don't need to worry about this except in this\nspecific cycling scenario, as Linux does not have PID\nrandomization. It appears it once did through a third-party feature,\nbut that it was removed around 2006[1]. For anyone worried about this\nedge case raising PID_MAX via \"/proc/sys/kernel/pid_max\" will mitigate\nit, but not eliminate it.\n\nOne thing we don't need to worry about is getting into an infinite\nloop when walking \"/proc/*/stat\". See 353d3d77f4f (trace2: collect\nWindows-specific process information, 2019-02-22) for the related\nWindows code that needs to deal with that, and [2] for an explanation\nof that edge case.\n\nAside from potential race conditions it's also a bit painful to\ncorrectly parse the process name out of \"/proc/*/stat\". A simpler\napproach is to use fscanf(), see [3] for an implementation of that,\nbut as noted in the comment being added here it would fail in the face\nof some weird process names, so we need our own parse_proc_stat() to\nparse it out.\n\nWith this patch the \"ancestry\" chain for a trace2 event might look\nlike this:\n\n $ GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version | grep ancestry | jq -r .ancestry\n [\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\nAnd in the case of naughty process names like the following. This uses\nperl's ability to use prctl(PR_SET_NAME, ...). See\nPerl/perl5@7636ea95c5 (Set the legacy process name with prctl() on\nassignment to $0 on Linux, 2010-04-15)[4]:\n\n $ perl -e '$0 = \"(naughty\\nname)\"; system \"GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version\"' | grep ancestry | jq -r .ancestry\n [\n \"sh\",\n \"(naughty\\nname)\",\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\n1. https://grsecurity.net/news#grsec2110\n2. https://lore.kernel.org/git/48a62d5e-28e2-7103-a5bb-5db7e197a4b9@jeffhostetler.com/\n3. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n4. https://github.com/Perl/perl5/commit/7636ea95c57762930accf4358f7c0c2dec086b5e\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 149 +++++++++++++++++++++++++++++++++++-----\n 1 file changed, 132 insertions(+), 17 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 0b47d44990c..bc2f9382a17 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -4,26 +4,145 @@\n #include \"strvec.h\"\n #include \"trace2.h\"\n \n-static void get_ancestry_names(struct strvec *names)\n+/*\n+ * We need more complex parsing in stat_parent_pid() and\n+ * parse_proc_stat() below than a dumb fscanf(). That's because while\n+ * the statcomm field is surrounded by parentheses, the process itself\n+ * is free to insert any arbitrary byte sequence its its name. That\n+ * can include newlines, spaces, closing parentheses etc.\n+ *\n+ * See do_task_stat() in fs/proc/array.c in linux.git, this is in\n+ * contrast with the escaped version of the name found in\n+ * /proc/%d/status.\n+ *\n+ * So instead of using fscanf() we'll read N bytes from it, look for\n+ * the first \"(\", and then the last \")\", anything in-between is our\n+ * process name.\n+ *\n+ * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by\n+ * default, but it can be raised set to values of up to 2^22. So\n+ * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n+ * we're interested in, so 2 * 7 = 14.\n+ *\n+ * We then have 3 spaces between those four values, and we'd like to\n+ * get to the space between the 4th and the 5th (the \"pgrp\" field) to\n+ * make sure we read the entire \"ppid\" field. So that brings us up to\n+ * 14 + 3 + 1 = 18. Add the two parentheses around the \"comm\" value\n+ * and it's 20. The \"state\" value itself is then one character (now at\n+ * 21).\n+ *\n+ * Finally the maximum length of the \"comm\" name itself is 15\n+ * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n+ * to \"123456789abcdef\". See PR_SET_NAME in prctl(2). So all in all\n+ * we'd need to read 21 + 15 = 36 bytes.\n+ *\n+ * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever\n+ * grows past 2^22 we'll be future-proof. We'll then anchor at the\n+ * last \")\" we find to locate the parent PID.\n+ */\n+#define STAT_PARENT_PID_READ_N 64\n+\n+static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,\n+\t\t\t int *statppid)\n {\n+\tconst char *comm_lhs = strchr(sb->buf, '(');\n+\tconst char *comm_rhs = strrchr(sb->buf, ')');\n+\tconst char *ppid_lhs, *ppid_rhs;\n+\tchar *p;\n+\tpid_t ppid;\n+\n+\tif (!comm_lhs || !comm_rhs)\n+\t\tgoto bad_kernel;\n+\n \t/*\n-\t * NEEDSWORK: We could gather the entire pstree into an array to match\n-\t * functionality with compat/win32/trace2_win32_process_info.c.\n-\t * To do so, we may want to examine /proc//stat. For now, just\n-\t * gather the immediate parent name which is readily accessible from\n-\t * /proc/$(getppid())/comm.\n+\t * We're at the \")\", that's followed by \" X \", where X is a\n+\t * single \"state\" character. So advance by 4 bytes.\n \t */\n+\tppid_lhs = comm_rhs + 4;\n+\n+\t/*\n+\t * Read until the space between the \"ppid\" and \"pgrp\" fields\n+\t * to make sure we're anchored after the untruncated \"ppid\"\n+\t * field..\n+\t */\n+\tppid_rhs = strchr(ppid_lhs, ' ');\n+\tif (!ppid_rhs)\n+\t\tgoto bad_kernel;\n+\n+\tppid = strtol(ppid_lhs, &p, 10);\n+\tif (ppid_rhs == p) {\n+\t\tconst char *comm = comm_lhs + 1;\n+\t\tsize_t commlen = comm_rhs - comm;\n+\n+\t\tstrbuf_add(name, comm, commlen);\n+\t\t*statppid = ppid;\n+\n+\t\treturn 0;\n+\t}\n+\n+bad_kernel:\n+\t/*\n+\t * We were able to read our STAT_PARENT_PID_READ_N bytes from\n+\t * /proc/%d/stat, but the content is bad. Broken kernel?\n+\t * Should not happen, but handle it gracefully.\n+\t */\n+\treturn -1;\n+}\n+\n+static int stat_parent_pid(pid_t pid, struct strbuf *name, int *statppid)\n+{\n \tstruct strbuf procfs_path = STRBUF_INIT;\n-\tstruct strbuf name = STRBUF_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tFILE *fp;\n+\tint ret = -1;\n \n \t/* try to use procfs if it's present. */\n-\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n-\t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, name.buf);\n-\t}\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n+\tfp = fopen(procfs_path.buf, \"r\");\n+\tif (!fp)\n+\t\tgoto cleanup;\n+\n+\t/*\n+\t * We could be more strict here and assert that we read at\n+\t * least STAT_PARENT_PID_READ_N. My reading of procfs(5) is\n+\t * that on any modern kernel (at least since 2.6.0 released in\n+\t * 2003) even if all the mandatory numeric fields were zero'd\n+\t * out we'd get at least 100 bytes, but let's just check that\n+\t * we got anything at all and trust the parse_proc_stat()\n+\t * function to handle its \"Bad Kernel?\" error checking.\n+\t */\n+\tif (!strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp))\n+\t\tgoto cleanup;\n+\tif (parse_proc_stat(&sb, name, statppid) < 0)\n+\t\tgoto cleanup;\n \n+\tret = 0;\n+cleanup:\n+\tif (fp)\n+\t\tfclose(fp);\n \tstrbuf_release(&procfs_path);\n+\tstrbuf_release(&sb);\n+\n+\treturn ret;\n+}\n+\n+static void push_ancestry_name(struct strvec *names, pid_t pid)\n+{\n+\tstruct strbuf name = STRBUF_INIT;\n+\tint ppid;\n+\n+\tif (stat_parent_pid(pid, &name, &ppid) < 0)\n+\t\tgoto cleanup;\n+\n+\tstrvec_push(names, name.buf);\n+\n+\t/*\n+\t * Both errors and reaching the end of the process chain are\n+\t * reported as fields of 0 by proc(5)\n+\t */\n+\tif (ppid)\n+\t\tpush_ancestry_name(names, ppid);\n+cleanup:\n \tstrbuf_release(&name);\n \n \treturn;\n@@ -45,11 +164,7 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \t\t */\n \t\tbreak;\n \tcase TRACE2_PROCESS_INFO_STARTUP:\n-\t\t/*\n-\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n-\t\t * see compat/win32/trace2_win32_process_info.c.\n-\t\t */\n-\t\tget_ancestry_names(&names);\n+\t\tpush_ancestry_name(&names, getppid());\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n-- \n2.33.0.733.ga72a4f1c2e1\n\n"},{"id":"433831","messageId":"87k0k8z7er.fsf@evledraar.gmail.com","threadId":"55637","inReplyTo":"YScTaDcPTs1nrP2Y@nand.local","subject":"\"I don't know what the author meant by that...\" (was \"Re: [PATCH 6/6] tr2: log N parent process names on Linux\")","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-26T12:24:10Z","receivedAt":"2021-08-26T13:01:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Aug 26 2021, Taylor Blau wrote:\n\n> On Thu, Aug 26, 2021 at 01:19:24AM +0200, Ævar Arnfjörð Bjarmason wrote:\n>> In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started\n>> logging parent process names, but only logged all parents on Windows.\n>> on Linux only the name of the immediate parent process was logged.\n>>\n>> Extend the functionality added there to also log full parent chain on\n>> Linux. In 2f732bf15e6 it was claimed that \"further ancestry info can\n>> be gathered with procfs, but it's unwieldy to do so.\".\n>>\n>> I don't know what the author meant by that, but I think it probably\n>> referred to needing to slurp this up from the FS, as opposed to having\n>> an API.\n>\n> I don't think that this (specifically, \"I don't know what the author\n> meant by that\") is necessary information to include in a patch message.\n>\n> If you're looking for a replacement (and you may not be, but just my\n> $.02) I would suggest:\n>\n> \"2f732bf15e6 does not log the full parent chain on Linux; implement\n> that functionality here.\"\n\nI hope Taylor doesn't mind me quoting it, but he sent me this follow-up\noff-list:\n\n For what it's worth, I really struggled to write this. What I was trying\n to say was something along the lines of \"let's give Emily a little more\n credit for not doing this right off the bat\", but I didn't want to write\n that exactly, since I don't think it was your original intention.\n \n At the very least, saying something to the effect of \"I don't know what\n the original author thought was so tough, here's a patch to implement\n it\" isn't helping anybody, so I tried to focus on that.\n \n Anyway, just writing to you off-list to say that I do think you had good\n intentions, but that what you wrote may be read differently by others in\n a way that you didn't intend it to be.\n\nFirst, thanks to Taylor for pointing this out, and second I'd like to\napologize for that comment.\n\nMore than some \"sorry someone read it that way\", this really does read\neven to me, its author, shortly after having written it like something\nthat's way more on the side of sneakiness than a charitable comment.\n\nI.e. like some snipe-y paraphrasing of \"maybe they found this problem\ntoo complex, but look how easy it is!\" than something charitable that's\nmeant to barely pass under the radar.\n\nHopefully it helps that I'm honestly just being a bit of an\ninconsiderate idiot here than actively malicious.\n\nWhat I meant to accomplish here was to guide a reader of these patches\nthrough the same mental states I went through when reading the original\npatch.\n\nI.e. I took it from its description that there were some unstated\nspecial-cases in reading procfs that made it harder to deal with than\nnot. I think those comments were rather just a way to say that scraping\nprocfs was a bit of a pain, and the MVP in 2f732bf15e6 was good enough\nfor now, which is fair enough.\n\nI rephrased those comments in v2 of this patch[1]. The summary starting\nat the 4th paragraph (\"It's possible given the semantics[...]\") is an\nedge case I hadn't considered when I wrote v1. I in turn copied that\n\"unweildy\" comment from an even older version[2], where the difficulties\nof parsing the \"comm\" field out of \"/proc/*/stat\" weren't apparent to\nme.\n\nI.e. I think if I had to describe that interface now I would describe it\nas unwieldy, but wouldn't when I wrote v1[1] or that v0[2]. I.e. I think\nthere's no convenient way to get full atomic snapshot of the process\ntree, so \"give me the names of all my parents as an array\" is definitely\nsomewhere between brittle and unwieldy, in particular considering the\nhassle of parsing \"/proc/*/stat\" properly.\n\nBut I digress, which is also a problem I have with being overly verbose\nsometimes and weaving enough rope to hang myself with.\n\nThe point isn't the difficulties of the procfs(5) interface, but that\nparticularly with a medium like the text-based communication we mainly\nuse in this project it behooves the sender to think not only about what\nthey're saying, but how what they're saying is going to be interpreted\nby the recipient and bystanders alike.\n\nI think this is a case where I clearly failed at that, sorry Emily, and\nthanks Taylor for pointing this out to me.\n\n1. https://lore.kernel.org/git/cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com/\n2. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n"},{"id":"433842","messageId":"CAPig+cSXQGbOMCNNF84aMA4MDFMbRc0z5xuKVoVJUQou6c1GkA@mail.gmail.com","threadId":"55637","inReplyTo":"patch-v2-4.6-1aa0dbc394e-20210826T121820Z-avarab@gmail.com","subject":"Re: [PATCH v2 4/6] tr2: fix memory leak & logic error in 2f732bf15e6","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2021-08-26T15:58:25Z","receivedAt":"2021-08-26T15:58:39Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Aug 26, 2021 at 8:22 AM Ævar Arnfjörð Bjarmason\n wrote:\n> [...]\n> Let's move all the freeing of the memory to the end of the\n> function. If we're still at STRBUF_INIT with \"name\" due to not haven\n\ns/haven/having/\n\n> taken the branch where the strbuf_read_file() succeeds freeing it is\n> redundant, so we could move it into the body of the \"if\", but just\n> handling freeing the same way for all branches of the function makes\n> it more readable.\n> [...]\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n"},{"id":"433846","messageId":"xmqqeeagf98q.fsf@gitster.g","threadId":"55637","inReplyTo":"patch-v2-4.6-1aa0dbc394e-20210826T121820Z-avarab@gmail.com","subject":"Re: [PATCH v2 4/6] tr2: fix memory leak & logic error in 2f732bf15e6","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-08-26T16:42:29Z","receivedAt":"2021-08-26T16:42:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Subject: Re: [PATCH v2 4/6] tr2: fix memory leak & logic error in 2f732bf15e6\n\nPlease remember to write your commit titles to help readers of \"git\nshortlog\". This commit fixes get_ancestry_names() function.\n\n get_ancestry_names(): leave the parent list empty upon failure\n\nor something like that. The rest of the proposed log message looks\nperfect and so do the changes to the code.\n\n> In a subsequent commit I'll be replacing most of this code to log N\n> parents, but let's first fix bugs introduced in the recent\n> 2f732bf15e6 (tr2: log parent process name, 2021-07-21).\n>\n> It was using the strbuf_read_file() in the wrong way, its return value\n> is either a length or a negative value on error. If we didn't have a\n> procfs, or otherwise couldn't access it we'd end up pushing an empty\n> string to the trace2 ancestry array.\n>\n> It was also using the strvec_push() API the wrong way. That API always\n> does an xstrdup(), so by detaching the strbuf here we'd leak\n> memory. Let's instead pass in our pointer for strvec_push() to\n> xstrdup(), and then free our own strbuf. I do have some WIP changes to\n> make strvec_push_nodup() non-static, which makes this and some other\n> callsites nicer, but let's just follow the prevailing pattern of using\n> strvec_push() for now.\n>\n> We'll also need to free that \"procfs_path\" strbuf whether or not\n> strbuf_read_file() succeeds, which was another source of memory leaks\n> in 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\n> system where we could read the file from procfs.\n>\n> Let's move all the freeing of the memory to the end of the\n> function. If we're still at STRBUF_INIT with \"name\" due to not haven\n> taken the branch where the strbuf_read_file() succeeds freeing it is\n> redundant, so we could move it into the body of the \"if\", but just\n> handling freeing the same way for all branches of the function makes\n> it more readable.\n>\n> In combination with the preceding commit this makes all of\n> t[0-9]*trace2*.sh pass under SANITIZE=leak on Linux.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> compat/linux/procinfo.c | 8 +++++---\n> 1 file changed, 5 insertions(+), 3 deletions(-)\n>\n> diff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\n> index 62f8aaed4cc..bd01f017bc7 100644\n> --- a/compat/linux/procinfo.c\n> +++ b/compat/linux/procinfo.c\n> @@ -18,12 +18,14 @@ static void get_ancestry_names(struct strvec *names)\n> \n> \t/* try to use procfs if it's present. */\n> \tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n> -\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n> -\t\tstrbuf_release(&procfs_path);\n> +\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n> \t\tstrbuf_trim_trailing_newline(&name);\n> -\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n> +\t\tstrvec_push(names, name.buf);\n> \t}\n> \n> +\tstrbuf_release(&procfs_path);\n> +\tstrbuf_release(&name);\n> +\n> \treturn;\n> }\n"},{"id":"433889","messageId":"YSgX1ON+FvpwPjKB@nand.local","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"Re: [PATCH v2 0/6] tr2: plug memory leaks + logic errors + Win32 & Linux feature parity","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-26T22:38:12Z","receivedAt":"2021-08-26T22:38:14Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 26, 2021 at 02:22:18PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> An early re-roll due to some very good & early review by Taylor Blau,\n> this also fixes the grammar/typo pointed out by Eric Sunshine. Thanks\n> both:\n\nThanks; aside from the cosmetic issues that Junio and Eric already\nbrought up, this version looks good to me. I probably would have avoided\n4/6 and jumped straight to 6/6, but your approach is good, too.\n\nThanks for being so receptive to my comments (public and private) in the\nearlier round of this series.\n\n Reviewed-by: Taylor Blau \n\nThanks,\nTaylor\n"},{"id":"433914","messageId":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v2-0.6-00000000000-20210826T121820Z-avarab@gmail.com","subject":"[PATCH v3 0/6] tr2: plug memory leaks + logic errors + Win32 & Linux feature parity","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:12Z","receivedAt":"2021-08-27T08:03:59Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A hopefully final re-roll with minor commit message changes. Thanks\neveryone for the reviews!\n\nÆvar Arnfjörð Bjarmason (6):\n tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n tr2: stop leaking \"thread_name\" memory\n tr2: leave the parent list empty upon failure & don't leak memory\n tr2: do compiler enum check in trace2_collect_process_info()\n tr2: log N parent process names on Linux\n\n compat/linux/procinfo.c | 169 ++++++++++++++++++++++++++++++++++------\n trace2/tr2_tls.c | 1 +\n 2 files changed, 146 insertions(+), 24 deletions(-)\n\nRange-diff against v2:\n1: 8c649ce3b4 = 1: 306f14a0f7 tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n2: 0150e3402a = 2: a999e016a9 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n3: 1d835d6767 = 3: 45769da953 tr2: stop leaking \"thread_name\" memory\n4: 1aa0dbc394 ! 4: 946140691f tr2: fix memory leak & logic error in 2f732bf15e6\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - tr2: fix memory leak & logic error in 2f732bf15e6\n + tr2: leave the parent list empty upon failure & don't leak memory\n \n In a subsequent commit I'll be replacing most of this code to log N\n parents, but let's first fix bugs introduced in the recent\n @@ Commit message\n system where we could read the file from procfs.\n \n Let's move all the freeing of the memory to the end of the\n - function. If we're still at STRBUF_INIT with \"name\" due to not haven\n + function. If we're still at STRBUF_INIT with \"name\" due to not having\n taken the branch where the strbuf_read_file() succeeds freeing it is\n - redundant, so we could move it into the body of the \"if\", but just\n + redundant. So we could move it into the body of the \"if\", but just\n handling freeing the same way for all branches of the function makes\n it more readable.\n \n5: 70fef093d8 = 5: 0bea5aa9c9 tr2: do compiler enum check in trace2_collect_process_info()\n6: f6aac90248 = 6: 6eac9986c3 tr2: log N parent process names on Linux\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433915","messageId":"patch-v3-2.6-a999e016a9-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 2/6] tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:14Z","receivedAt":"2021-08-27T08:04:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rewrite a comment added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to describe what we might do under\nTRACE2_PROCESS_INFO_EXIT in the future, instead of vaguely referring\nto \"something extra\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 15a89676c7..62f8aaed4c 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -32,8 +32,12 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\t/* someday we may want to write something extra here, but not today */\n \tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\t\t/*\n+\t\t * The Windows version of this calls its\n+\t\t * get_peak_memory_info() here. We may want to insert\n+\t\t * similar process-end statistics here in the future.\n+\t\t */\n \t\treturn;\n \n \tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433916","messageId":"patch-v3-1.6-306f14a0f7-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 1/6] tr2: remove NEEDSWORK comment for \"non-procfs\" implementations","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:13Z","receivedAt":"2021-08-27T08:04:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I'm fairly sure that there is no way on Linux to inspect the process\ntree without using procfs, any tool such as ps(1), top(1) etc. that\nshows this sort of information ultimately looks the information up in\nprocfs.\n\nSo let's remove this comment added in 2f732bf15e6 (tr2: log parent\nprocess name, 2021-07-21), it's setting us up for an impossible task.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 578fed4cd3..15a89676c7 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -25,7 +25,6 @@ static void get_ancestry_names(struct strvec *names)\n \t}\n \n \treturn;\n-\t/* NEEDSWORK: add non-procfs-linux implementations here */\n }\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433917","messageId":"patch-v3-3.6-45769da953-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 3/6] tr2: stop leaking \"thread_name\" memory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:15Z","receivedAt":"2021-08-27T08:04:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix a memory leak introduced in ee4512ed481 (trace2: create new\ncombined trace facility, 2019-02-22), we were doing a free() of other\nmemory allocated in tr2tls_create_self(), but not the \"thread_name\"\n\"struct strbuf\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n trace2/tr2_tls.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/trace2/tr2_tls.c b/trace2/tr2_tls.c\nindex 067c23755f..7da94aba52 100644\n--- a/trace2/tr2_tls.c\n+++ b/trace2/tr2_tls.c\n@@ -95,6 +95,7 @@ void tr2tls_unset_self(void)\n \n \tpthread_setspecific(tr2tls_key, NULL);\n \n+\tstrbuf_release(&ctx->thread_name);\n \tfree(ctx->array_us_start);\n \tfree(ctx);\n }\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433918","messageId":"patch-v3-6.6-6eac9986c3-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 6/6] tr2: log N parent process names on Linux","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:18Z","receivedAt":"2021-08-27T08:04:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started\nlogging parent process names, but only logged all parents on Windows.\non Linux only the name of the immediate parent process was logged.\n\nExtend the functionality added there to also log full parent chain on\nLinux.\n\nThis requires us to lookup \"/proc//stat\" instead of\n\"/proc//comm\". The \"comm\" file just contains the name of the\nprocess, but the \"stat\" file has both that information, and the parent\nPID of that process, see procfs(5). We parse out the parent PID of our\nown parent, and recursively walk the chain of \"/proc/*/stat\" files all\nthe way up the chain. A parent PID of 0 indicates the end of the\nchain.\n\nIt's possible given the semantics of Linux's PID files that we end up\ngetting an entirely nonsensical chain of processes. It could happen if\ne.g. we have a chain of processes like:\n\n 1 (init) => 321 (bash) => 123 (git)\n\nLet's assume that \"bash\" was started a while ago, and that as shown\nthe OS has already cycled back to using a lower PID for us than our\nparent process. In the time it takes us to start up and get to\ntrace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP) our parent\nprocess might exit, and be replaced by an entirely different process!\n\nWe'd racily look up our own getppid(), but in the meantime our parent\nwould exit, and Linux would have cycled all the way back to starting\nan entirely unrelated process as PID 321.\n\nIf that happens we'll just silently log incorrect data in our ancestry\nchain. Luckily we don't need to worry about this except in this\nspecific cycling scenario, as Linux does not have PID\nrandomization. It appears it once did through a third-party feature,\nbut that it was removed around 2006[1]. For anyone worried about this\nedge case raising PID_MAX via \"/proc/sys/kernel/pid_max\" will mitigate\nit, but not eliminate it.\n\nOne thing we don't need to worry about is getting into an infinite\nloop when walking \"/proc/*/stat\". See 353d3d77f4f (trace2: collect\nWindows-specific process information, 2019-02-22) for the related\nWindows code that needs to deal with that, and [2] for an explanation\nof that edge case.\n\nAside from potential race conditions it's also a bit painful to\ncorrectly parse the process name out of \"/proc/*/stat\". A simpler\napproach is to use fscanf(), see [3] for an implementation of that,\nbut as noted in the comment being added here it would fail in the face\nof some weird process names, so we need our own parse_proc_stat() to\nparse it out.\n\nWith this patch the \"ancestry\" chain for a trace2 event might look\nlike this:\n\n $ GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version | grep ancestry | jq -r .ancestry\n [\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\nAnd in the case of naughty process names like the following. This uses\nperl's ability to use prctl(PR_SET_NAME, ...). See\nPerl/perl5@7636ea95c5 (Set the legacy process name with prctl() on\nassignment to $0 on Linux, 2010-04-15)[4]:\n\n $ perl -e '$0 = \"(naughty\\nname)\"; system \"GIT_TRACE2_EVENT=/dev/stdout ~/g/git/git version\"' | grep ancestry | jq -r .ancestry\n [\n \"sh\",\n \"(naughty\\nname)\",\n \"bash\",\n \"screen\",\n \"systemd\"\n ]\n\n1. https://grsecurity.net/news#grsec2110\n2. https://lore.kernel.org/git/48a62d5e-28e2-7103-a5bb-5db7e197a4b9@jeffhostetler.com/\n3. https://lore.kernel.org/git/87o8agp29o.fsf@evledraar.gmail.com/\n4. https://github.com/Perl/perl5/commit/7636ea95c57762930accf4358f7c0c2dec086b5e\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 149 +++++++++++++++++++++++++++++++++++-----\n 1 file changed, 132 insertions(+), 17 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 0b47d44990..bc2f9382a1 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -4,26 +4,145 @@\n #include \"strvec.h\"\n #include \"trace2.h\"\n \n-static void get_ancestry_names(struct strvec *names)\n+/*\n+ * We need more complex parsing in stat_parent_pid() and\n+ * parse_proc_stat() below than a dumb fscanf(). That's because while\n+ * the statcomm field is surrounded by parentheses, the process itself\n+ * is free to insert any arbitrary byte sequence its its name. That\n+ * can include newlines, spaces, closing parentheses etc.\n+ *\n+ * See do_task_stat() in fs/proc/array.c in linux.git, this is in\n+ * contrast with the escaped version of the name found in\n+ * /proc/%d/status.\n+ *\n+ * So instead of using fscanf() we'll read N bytes from it, look for\n+ * the first \"(\", and then the last \")\", anything in-between is our\n+ * process name.\n+ *\n+ * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by\n+ * default, but it can be raised set to values of up to 2^22. So\n+ * that's 7 digits for a PID. We have 2 PIDs in the first four fields\n+ * we're interested in, so 2 * 7 = 14.\n+ *\n+ * We then have 3 spaces between those four values, and we'd like to\n+ * get to the space between the 4th and the 5th (the \"pgrp\" field) to\n+ * make sure we read the entire \"ppid\" field. So that brings us up to\n+ * 14 + 3 + 1 = 18. Add the two parentheses around the \"comm\" value\n+ * and it's 20. The \"state\" value itself is then one character (now at\n+ * 21).\n+ *\n+ * Finally the maximum length of the \"comm\" name itself is 15\n+ * characters, e.g. a setting of \"123456789abcdefg\" will be truncated\n+ * to \"123456789abcdef\". See PR_SET_NAME in prctl(2). So all in all\n+ * we'd need to read 21 + 15 = 36 bytes.\n+ *\n+ * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever\n+ * grows past 2^22 we'll be future-proof. We'll then anchor at the\n+ * last \")\" we find to locate the parent PID.\n+ */\n+#define STAT_PARENT_PID_READ_N 64\n+\n+static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,\n+\t\t\t int *statppid)\n {\n+\tconst char *comm_lhs = strchr(sb->buf, '(');\n+\tconst char *comm_rhs = strrchr(sb->buf, ')');\n+\tconst char *ppid_lhs, *ppid_rhs;\n+\tchar *p;\n+\tpid_t ppid;\n+\n+\tif (!comm_lhs || !comm_rhs)\n+\t\tgoto bad_kernel;\n+\n \t/*\n-\t * NEEDSWORK: We could gather the entire pstree into an array to match\n-\t * functionality with compat/win32/trace2_win32_process_info.c.\n-\t * To do so, we may want to examine /proc//stat. For now, just\n-\t * gather the immediate parent name which is readily accessible from\n-\t * /proc/$(getppid())/comm.\n+\t * We're at the \")\", that's followed by \" X \", where X is a\n+\t * single \"state\" character. So advance by 4 bytes.\n \t */\n+\tppid_lhs = comm_rhs + 4;\n+\n+\t/*\n+\t * Read until the space between the \"ppid\" and \"pgrp\" fields\n+\t * to make sure we're anchored after the untruncated \"ppid\"\n+\t * field..\n+\t */\n+\tppid_rhs = strchr(ppid_lhs, ' ');\n+\tif (!ppid_rhs)\n+\t\tgoto bad_kernel;\n+\n+\tppid = strtol(ppid_lhs, &p, 10);\n+\tif (ppid_rhs == p) {\n+\t\tconst char *comm = comm_lhs + 1;\n+\t\tsize_t commlen = comm_rhs - comm;\n+\n+\t\tstrbuf_add(name, comm, commlen);\n+\t\t*statppid = ppid;\n+\n+\t\treturn 0;\n+\t}\n+\n+bad_kernel:\n+\t/*\n+\t * We were able to read our STAT_PARENT_PID_READ_N bytes from\n+\t * /proc/%d/stat, but the content is bad. Broken kernel?\n+\t * Should not happen, but handle it gracefully.\n+\t */\n+\treturn -1;\n+}\n+\n+static int stat_parent_pid(pid_t pid, struct strbuf *name, int *statppid)\n+{\n \tstruct strbuf procfs_path = STRBUF_INIT;\n-\tstruct strbuf name = STRBUF_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tFILE *fp;\n+\tint ret = -1;\n \n \t/* try to use procfs if it's present. */\n-\tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n-\t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, name.buf);\n-\t}\n+\tstrbuf_addf(&procfs_path, \"/proc/%d/stat\", pid);\n+\tfp = fopen(procfs_path.buf, \"r\");\n+\tif (!fp)\n+\t\tgoto cleanup;\n+\n+\t/*\n+\t * We could be more strict here and assert that we read at\n+\t * least STAT_PARENT_PID_READ_N. My reading of procfs(5) is\n+\t * that on any modern kernel (at least since 2.6.0 released in\n+\t * 2003) even if all the mandatory numeric fields were zero'd\n+\t * out we'd get at least 100 bytes, but let's just check that\n+\t * we got anything at all and trust the parse_proc_stat()\n+\t * function to handle its \"Bad Kernel?\" error checking.\n+\t */\n+\tif (!strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp))\n+\t\tgoto cleanup;\n+\tif (parse_proc_stat(&sb, name, statppid) < 0)\n+\t\tgoto cleanup;\n \n+\tret = 0;\n+cleanup:\n+\tif (fp)\n+\t\tfclose(fp);\n \tstrbuf_release(&procfs_path);\n+\tstrbuf_release(&sb);\n+\n+\treturn ret;\n+}\n+\n+static void push_ancestry_name(struct strvec *names, pid_t pid)\n+{\n+\tstruct strbuf name = STRBUF_INIT;\n+\tint ppid;\n+\n+\tif (stat_parent_pid(pid, &name, &ppid) < 0)\n+\t\tgoto cleanup;\n+\n+\tstrvec_push(names, name.buf);\n+\n+\t/*\n+\t * Both errors and reaching the end of the process chain are\n+\t * reported as fields of 0 by proc(5)\n+\t */\n+\tif (ppid)\n+\t\tpush_ancestry_name(names, ppid);\n+cleanup:\n \tstrbuf_release(&name);\n \n \treturn;\n@@ -45,11 +164,7 @@ void trace2_collect_process_info(enum trace2_process_info_reason reason)\n \t\t */\n \t\tbreak;\n \tcase TRACE2_PROCESS_INFO_STARTUP:\n-\t\t/*\n-\t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n-\t\t * see compat/win32/trace2_win32_process_info.c.\n-\t\t */\n-\t\tget_ancestry_names(&names);\n+\t\tpush_ancestry_name(&names, getppid());\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433919","messageId":"patch-v3-4.6-946140691f-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 4/6] tr2: leave the parent list empty upon failure & don't leak memory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:16Z","receivedAt":"2021-08-27T08:04:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In a subsequent commit I'll be replacing most of this code to log N\nparents, but let's first fix bugs introduced in the recent\n2f732bf15e6 (tr2: log parent process name, 2021-07-21).\n\nIt was using the strbuf_read_file() in the wrong way, its return value\nis either a length or a negative value on error. If we didn't have a\nprocfs, or otherwise couldn't access it we'd end up pushing an empty\nstring to the trace2 ancestry array.\n\nIt was also using the strvec_push() API the wrong way. That API always\ndoes an xstrdup(), so by detaching the strbuf here we'd leak\nmemory. Let's instead pass in our pointer for strvec_push() to\nxstrdup(), and then free our own strbuf. I do have some WIP changes to\nmake strvec_push_nodup() non-static, which makes this and some other\ncallsites nicer, but let's just follow the prevailing pattern of using\nstrvec_push() for now.\n\nWe'll also need to free that \"procfs_path\" strbuf whether or not\nstrbuf_read_file() succeeds, which was another source of memory leaks\nin 2f732bf15e6, i.e. we'd leak that memory as well if we weren't on a\nsystem where we could read the file from procfs.\n\nLet's move all the freeing of the memory to the end of the\nfunction. If we're still at STRBUF_INIT with \"name\" due to not having\ntaken the branch where the strbuf_read_file() succeeds freeing it is\nredundant. So we could move it into the body of the \"if\", but just\nhandling freeing the same way for all branches of the function makes\nit more readable.\n\nIn combination with the preceding commit this makes all of\nt[0-9]*trace2*.sh pass under SANITIZE=leak on Linux.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex 62f8aaed4c..bd01f017bc 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -18,12 +18,14 @@ static void get_ancestry_names(struct strvec *names)\n \n \t/* try to use procfs if it's present. */\n \tstrbuf_addf(&procfs_path, \"/proc/%d/comm\", getppid());\n-\tif (strbuf_read_file(&name, procfs_path.buf, 0)) {\n-\t\tstrbuf_release(&procfs_path);\n+\tif (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {\n \t\tstrbuf_trim_trailing_newline(&name);\n-\t\tstrvec_push(names, strbuf_detach(&name, NULL));\n+\t\tstrvec_push(names, name.buf);\n \t}\n \n+\tstrbuf_release(&procfs_path);\n+\tstrbuf_release(&name);\n+\n \treturn;\n }\n \n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"433920","messageId":"patch-v3-5.6-0bea5aa9c9-20210827T080054Z-avarab@gmail.com","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"[PATCH v3 5/6] tr2: do compiler enum check in trace2_collect_process_info()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-08-27T08:02:17Z","receivedAt":"2021-08-27T08:04:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 2f732bf15e6 (tr2: log parent process name,\n2021-07-21) to use a switch statement without a \"default\" branch to\nhave the compiler error if this code ever drifts out of sync with the\nmembers of the \"enum trace2_process_info_reason\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/linux/procinfo.c | 13 +++++++------\n 1 file changed, 7 insertions(+), 6 deletions(-)\n\ndiff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c\nindex bd01f017bc..0b47d44990 100644\n--- a/compat/linux/procinfo.c\n+++ b/compat/linux/procinfo.c\n@@ -31,29 +31,30 @@ static void get_ancestry_names(struct strvec *names)\n \n void trace2_collect_process_info(enum trace2_process_info_reason reason)\n {\n+\tstruct strvec names = STRVEC_INIT;\n+\n \tif (!trace2_is_enabled())\n \t\treturn;\n \n-\tif (reason == TRACE2_PROCESS_INFO_EXIT)\n+\tswitch (reason) {\n+\tcase TRACE2_PROCESS_INFO_EXIT:\n \t\t/*\n \t\t * The Windows version of this calls its\n \t\t * get_peak_memory_info() here. We may want to insert\n \t\t * similar process-end statistics here in the future.\n \t\t */\n-\t\treturn;\n-\n-\tif (reason == TRACE2_PROCESS_INFO_STARTUP) {\n+\t\tbreak;\n+\tcase TRACE2_PROCESS_INFO_STARTUP:\n \t\t/*\n \t\t * NEEDSWORK: we could do the entire ptree in an array instead,\n \t\t * see compat/win32/trace2_win32_process_info.c.\n \t\t */\n-\t\tstruct strvec names = STRVEC_INIT;\n-\n \t\tget_ancestry_names(&names);\n \n \t\tif (names.nr)\n \t\t\ttrace2_cmd_ancestry(names.v);\n \t\tstrvec_clear(&names);\n+\t\tbreak;\n \t}\n \n \treturn;\n-- \n2.33.0.736.g68690aaec9a\n\n"},{"id":"434208","messageId":"YS11MuO6j9N1mzvm@nand.local","threadId":"55637","inReplyTo":"cover-v3-0.6-0000000000-20210827T080054Z-avarab@gmail.com","subject":"Re: [PATCH v3 0/6] tr2: plug memory leaks + logic errors + Win32 & Linux feature parity","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-08-31T00:17:54Z","receivedAt":"2021-08-31T00:17:58Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Aug 27, 2021 at 10:02:12AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> Range-diff against v2:\n> 1: 8c649ce3b4 = 1: 306f14a0f7 tr2: remove NEEDSWORK comment for \"non-procfs\" implementations\n> 2: 0150e3402a = 2: a999e016a9 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under Linux\n> 3: 1d835d6767 = 3: 45769da953 tr2: stop leaking \"thread_name\" memory\n> 4: 1aa0dbc394 ! 4: 946140691f tr2: fix memory leak & logic error in 2f732bf15e6\n> @@ Metadata\n> Author: Ævar Arnfjörð Bjarmason \n>\n> ## Commit message ##\n> - tr2: fix memory leak & logic error in 2f732bf15e6\n> + tr2: leave the parent list empty upon failure & don't leak memory\n\nI agree with Junio that the earlier subject line could be improved, but\nthe replacement is a little verbose for my taste. On the other hand, I'm\nnot sure how to simplify it without it becoming:\n\n tr2: memory leaks and bug fixes\n\n;-).\n\nSo I think that what you wrote here is perfectly good. This version\nlooks good to me, too.\n\nThanks,\nTaylor\n"}]}