git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/6] tr2: log N parent process names on Linux

From
Taylor Blau <me@ttaylorr.com>
Date
Aug 26, 2021, 04:07 UTC
Message-ID
<YScTaDcPTs1nrP2Y@nand.local>
In-Reply-To
<patch-6.6-da003330800-20210825T231400Z-avarab@gmail.com>
On Thu, Aug 26, 2021 at 01:19:24AM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 11 quoted lines
> In 2f732bf15e6 (tr2: log parent process name, 2021-07-21) we started
> logging parent process names, but only logged all parents on Windows.
> on Linux only the name of the immediate parent process was logged.
>
> Extend the functionality added there to also log full parent chain on
> Linux. In 2f732bf15e6 it was claimed that "further ancestry info can
> be gathered with procfs, but it's unwieldy to do so.".
>
> I don't know what the author meant by that, but I think it probably
> referred to needing to slurp this up from the FS, as opposed to having
> an API.

I don't think that this (specifically, "I don't know what the author meant by that") is necessary information to include in a patch message.

If you're looking for a replacement (and you may not be, but just my $.02) I would suggest:

    "2f732bf15e6 does not log the full parent chain on Linux; implement
    that functionality here."
Show 5 quoted lines
> What is more complex on Linux is getting at the process name, a
> simpler approach is to use fscanf(), see [1] for an implementation of
> that, but as noted in the comment being added here it would fail in
> the face of some weird process names, so we need our own
> parse_proc_stat() to parse it out.

This is helpful information to have for readers that aren't familiar with procfs (especially the detail about why the naive approach doesn't work).

Show 30 quoted lines
> diff --git a/compat/linux/procinfo.c b/compat/linux/procinfo.c
> index 46a751c9a1d..937084126a6 100644
> --- a/compat/linux/procinfo.c
> +++ b/compat/linux/procinfo.c
> @@ -4,27 +4,129 @@
>  #include "strvec.h"
>  #include "trace2.h"
>
> -static void get_ancestry_names(struct strvec *names)
> +/*
> + * We need more complex parsing instat_parent_pid() and
> + * parse_proc_stat() below than a dumb fscanf(). That's because while
> + * the statcomm field is surrounded by parentheses, the process itself
> + * is free to insert any arbitrary byte sequence its its name. That
> + * can include newlines, spaces, closing parentheses etc. See
> + * do_task_stat() in fs/proc/array.c in linux.git, this is in contrast
> + * with the escaped version of the name found in /proc/%d/status.
> + *
> + * So instead of using fscanf() we'll read N bytes from it, look for
> + * the first "(", and then the last ")", anything in-between is our
> + * process name.
> + *
> + * How much N do we need? On Linux /proc/sys/kernel/pid_max is 2^15 by
> + * default, but it can be raised set to values of up to 2^22. So
> + * that's 7 digits for a PID. We have 2 PIDs in the first four fields
> + * we're interested in, so 2 * 7 = 14.
> + *
> + * We then have 4 spaces between those four values, which brings us up
> + * to 18. Add the two parentheses and it's 20. The "state" is then one
> + * character (now at 21).
Hmm, aren't there three spaces, not four?
> + * Finally the maximum length of the "comm" name itself is 15
> + * characters, e.g. a setting of "123456789abcdefg" will be truncated
> + * to "123456789abcdef". See PR_SET_NAME in prctl(2). So all in all
> + * we'd need to read 21 + 15 = 36 bytes.

Ah, 36 is the right number even though you and I counted a different number of spaces, since the name is truncated when it goes over *16* characters, but that includes the NUL byte. So we both arrive at the same number in the end ;).

But I agree it's safer to just read more (but not too much more) than what we need.

Show 8 quoted lines
> + * Let's just read 2^6 (64) instead for good measure. If PID_MAX ever
> + * grows past 2^22 we'll be future-proof. We'll then anchor at the
> + * last ")" we find to locate the parent PID.
> + */
> +#define STAT_PARENT_PID_READ_N 64
> +
> +static int parse_proc_stat(struct strbuf *sb, struct strbuf *name,
> +			    int *statppid)
Going to think aloud to make sure that this parsing looks right.
>  {
> +	const char *lhs = strchr(sb->buf, '(');
> +	const char *rhs = strrchr(sb->buf, ')');

lhs and rhs are going to be on either side of the comm field (which may be helpful to indicate by calling these comm_lhs and comm_rhs). And strrchr makes sure to handle process names that have a ')' in them. Looks right.

Show 7 quoted lines
> +	const char *ppid_lhs, *ppid_rhs;
> +	char *p;
> +	pid_t ppid;
> +
> +	if (!lhs || !rhs)
> +		goto bad_kernel;
> +
OK.
Show 12 quoted lines
>  	/*
> -	 * NEEDSWORK: We could gather the entire pstree into an array to match
> -	 * functionality with compat/win32/trace2_win32_process_info.c.
> -	 * To do so, we may want to examine /proc/<pid>/stat. For now, just
> -	 * gather the immediate parent name which is readily accessible from
> -	 * /proc/$(getppid())/comm.
> +	 * We're at the ")", that's followed by " X ", where X is a
> +	 * single "state" character. So advance by 4 bytes.
>  	 */
> +	ppid_lhs = rhs + 4;
> +
> +	ppid_rhs = strchr(ppid_lhs, ' ');

Skipping over the state field gives us the first character of ppid as you say, good. And reading until the first space character will point us right after the end, good.

Show 5 quoted lines
> +	if (!ppid_rhs)
> +		goto bad_kernel;
> +
> +	ppid = strtol(ppid_lhs, &p, 10);
> +	if (ppid_rhs == p) {

Then parse the ppid and make sure we stopped at the right-hand side where we should have. Good.

> +		const char *comm = lhs + 1;

Skipping past the '(', but now I feel like we really should s/lhs/comm_&/.

> +		int commlen = rhs - lhs - 1;

This is right, but you could simplify the expression to be "rhs - comm", since you just took into account the left-hand parenthesis in the previous line. Also recommend a size_t here: it's obvious we're not going to overflow int here, but it saves future readers of having to wonder the same thing.

Show 15 quoted lines
> +
> +		strbuf_addf(name, "%.*s", commlen, comm);
> +		*statppid = ppid;
> +
> +		return 0;
> +	}
> +
> +bad_kernel:
> +	/*
> +	 * We were able to read our STAT_PARENT_PID_READ_N bytes from
> +	 * /proc/%d/stat, but the content is bad. Broken kernel?
> +	 * Should not happen, but handle it gracefully.
> +	 */
> +	return -1;
> +}

Phew, all seems good. Thanks for bearing with me while I read through all of that ;).

Show 7 quoted lines
> +static int stat_parent_pid(pid_t pid, struct strbuf *name, int *statppid)
> +{
>  	struct strbuf procfs_path = STRBUF_INIT;
> -	struct strbuf name = STRBUF_INIT;
> +	struct strbuf sb = STRBUF_INIT;
> +	size_t n;
> +	FILE *fp = NULL;

fopen() will return NULL, and you call it unconditionally, so no need to initialize here.

Show 17 quoted lines
> +	int ret = -1;
>
>  	/* try to use procfs if it's present. */
> -	strbuf_addf(&procfs_path, "/proc/%d/comm", getppid());
> -	if (strbuf_read_file(&name, procfs_path.buf, 0) > 0) {
> -		strbuf_trim_trailing_newline(&name);
> -		strvec_push(names, name.buf);
> -		strbuf_release(&name);
> -	}
> +	strbuf_addf(&procfs_path, "/proc/%d/stat", pid);
> +	fp = fopen(procfs_path.buf, "r");
> +	if (!fp)
> +		goto cleanup;
> +
> +	n = strbuf_fread(&sb, STAT_PARENT_PID_READ_N, fp);
> +	if (n != STAT_PARENT_PID_READ_N)
> +		goto cleanup;

Hmm. Wouldn't we always goto cleanup here, since STAT_PARENT_PID_READ_N is deliberately oversized (and not constant anyways, since process ids could be anywhere from 1-7 digits long)?

I think we could probably drop 'n' entirely here, and instead:
    if (strbuf_fread(...) < 0)
      goto cleanup;
Show 33 quoted lines
> +	if (parse_proc_stat(&sb, name, statppid) < 0)
> +		goto cleanup;
>
> +	ret = 0;
> +cleanup:
> +	if (fp)
> +		fclose(fp);
>  	strbuf_release(&procfs_path);
> +	strbuf_release(&sb);
> +
> +	return ret;
> +}
> +
> +static void push_ancestry_name(struct strvec *names, pid_t pid)
> +{
> +	struct strbuf name = STRBUF_INIT;
> +	int ppid;
> +
> +	if (stat_parent_pid(pid, &name, &ppid) < 0)
> +		goto cleanup;
> +
> +	strvec_push(names, name.buf);
> +
> +	/*
> +	 * Both errors and reaching the end of the process chain are
> +	 * reported as fields of 0 by proc(5)
> +	 */
> +	if (ppid)
> +		push_ancestry_name(names, ppid);
> +cleanup:
> +	strbuf_release(&name);
>  	return;
>  }

The rest looks good to me, but it looks like you overwrote all of the work that you did in patch 4/6. I guess separating them out makes sense if this patch wasn't taken, but I probably would have gone right to this patch instead of fixing leaks that you were going to get rid of anyway.

Thanks, Taylor

Previous: Eric SunshineNext: Ævar Arnfjörð Bjarmason
Message 64 of 87 in “tr2: log parent process name”
  1. tr2: log parent process nameEmily Shaffer, May 7, 2021
  2. Bagas SanjayaMay 7, 2021
  3. Emily ShafferMay 7, 2021
  4. Ævar Arnfjörð BjarmasonMay 10, 2021
  5. Junio C HamanoMay 11, 2021
  6. Emily ShafferMay 14, 2021
  7. Junio C HamanoMay 16, 2021
  8. Emily ShafferMay 17, 2021
  9. Jeff HostetlerMay 11, 2021
  10. Emily ShafferMay 14, 2021
  11. tr2: log parent process nameEmily Shaffer, May 20, 2021
  12. Randall S. BeckerMay 20, 2021
  13. Emily ShafferMay 20, 2021
  14. Randall S. BeckerMay 21, 2021
  15. Randall S. BeckerMay 21, 2021
  16. Junio C HamanoMay 21, 2021
  17. Emily ShafferMay 21, 2021
  18. Junio C HamanoMay 21, 2021
  19. Emily ShafferMay 24, 2021
  20. Jeff HostetlerMay 21, 2021
  21. Emily ShafferMay 21, 2021
  22. Randall S. BeckerMay 21, 2021
  23. Jeff HostetlerMay 22, 2021
  24. Ævar Arnfjörð BjarmasonMay 24, 2021
  25. tr2: log parent process nameEmily Shaffer, May 24, 2021
  26. Emily ShafferMay 24, 2021
  27. Junio C HamanoMay 25, 2021
  28. Randall S. BeckerMay 25, 2021
  29. tr2: log parent process nameEmily Shaffer, Jun 8, 2021
  30. Emily ShafferJun 8, 2021
  31. tr2: log parent process nameEmily Shaffer, Jun 8, 2021
  32. Randall S. BeckerJun 8, 2021
  33. Emily ShafferJun 8, 2021
  34. Randall S. BeckerJun 8, 2021
  35. Emily ShafferJun 9, 2021
  36. Junio C HamanoJun 16, 2021
  37. Jeff HostetlerJun 28, 2021
  38. Emily ShafferJun 29, 2021
  39. Ævar Arnfjörð BjarmasonJun 30, 2021
  40. Emily ShafferJul 22, 2021
  41. 0/2 tr2: log parent process nameEmily Shaffer, Jul 22, 2021
  42. 1/2 tr2: make process info collection platform-genericEmily Shaffer, Jul 22, 2021
  43. Ævar Arnfjörð BjarmasonAug 2, 2021
  44. 2/2 tr2: log parent process nameEmily Shaffer, Jul 22, 2021
  45. Junio C HamanoJul 22, 2021
  46. Ævar Arnfjörð BjarmasonAug 2, 2021
  47. Ævar Arnfjörð BjarmasonAug 2, 2021
  48. Ævar Arnfjörð BjarmasonAug 2, 2021
  49. Ævar Arnfjörð BjarmasonAug 2, 2021
  50. Jeff HostetlerAug 2, 2021
  51. Randall S. BeckerAug 2, 2021
  52. Ævar Arnfjörð BjarmasonAug 2, 2021
  53. 0/6 tr2: plug memory leaks + logic errors + Win32 & Linux feature parityÆvar Arnfjörð Bjarmason, Aug 25, 2021
  54. 1/6 tr2: remove NEEDSWORK comment for "non-procfs" implementationsÆvar Arnfjörð Bjarmason, Aug 25, 2021
  55. 2/6 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under LinuxÆvar Arnfjörð Bjarmason, Aug 25, 2021
  56. 3/6 tr2: stop leaking "thread_name" memoryÆvar Arnfjörð Bjarmason, Aug 25, 2021
  57. Taylor BlauAug 26, 2021
  58. 4/6 tr2: fix memory leak & logic error in 2f732bf15e6Ævar Arnfjörð Bjarmason, Aug 25, 2021
  59. Taylor BlauAug 26, 2021
  60. 5/6 tr2: do compiler enum check in trace2_collect_process_info()Ævar Arnfjörð Bjarmason, Aug 25, 2021
  61. Taylor BlauAug 26, 2021
  62. 6/6 tr2: log N parent process names on LinuxÆvar Arnfjörð Bjarmason, Aug 25, 2021
  63. Eric SunshineAug 25, 2021
  64. Taylor BlauAug 26, 2021
  65. "I don't know what the author meant by that..." (was "Re: [PATCH 6/6] tr2: log N parent process names on Linux")Ævar Arnfjörð Bjarmason, Aug 26, 2021
  66. 0/6 tr2: plug memory leaks + logic errors + Win32 & Linux feature parityÆvar Arnfjörð Bjarmason, Aug 26, 2021
  67. 1/6 tr2: remove NEEDSWORK comment for "non-procfs" implementationsÆvar Arnfjörð Bjarmason, Aug 26, 2021
  68. 2/6 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under LinuxÆvar Arnfjörð Bjarmason, Aug 26, 2021
  69. 3/6 tr2: stop leaking "thread_name" memoryÆvar Arnfjörð Bjarmason, Aug 26, 2021
  70. 5/6 tr2: do compiler enum check in trace2_collect_process_info()Ævar Arnfjörð Bjarmason, Aug 26, 2021
  71. 4/6 tr2: fix memory leak & logic error in 2f732bf15e6Ævar Arnfjörð Bjarmason, Aug 26, 2021
  72. Eric SunshineAug 26, 2021
  73. Junio C HamanoAug 26, 2021
  74. 6/6 tr2: log N parent process names on LinuxÆvar Arnfjörð Bjarmason, Aug 26, 2021
  75. Taylor BlauAug 26, 2021
  76. 0/6 tr2: plug memory leaks + logic errors + Win32 & Linux feature parityÆvar Arnfjörð Bjarmason, Aug 27, 2021
  77. 2/6 tr2: clarify TRACE2_PROCESS_INFO_EXIT comment under LinuxÆvar Arnfjörð Bjarmason, Aug 27, 2021
  78. 1/6 tr2: remove NEEDSWORK comment for "non-procfs" implementationsÆvar Arnfjörð Bjarmason, Aug 27, 2021
  79. 3/6 tr2: stop leaking "thread_name" memoryÆvar Arnfjörð Bjarmason, Aug 27, 2021
  80. 6/6 tr2: log N parent process names on LinuxÆvar Arnfjörð Bjarmason, Aug 27, 2021
  81. 4/6 tr2: leave the parent list empty upon failure & don't leak memoryÆvar Arnfjörð Bjarmason, Aug 27, 2021
  82. 5/6 tr2: do compiler enum check in trace2_collect_process_info()Ævar Arnfjörð Bjarmason, Aug 27, 2021
  83. Taylor BlauAug 31, 2021
  84. Ævar Arnfjörð BjarmasonAug 2, 2021
  85. Junio C HamanoAug 2, 2021
  86. Ævar Arnfjörð BjarmasonAug 2, 2021
  87. Jeff HostetlerJul 22, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.