perf lock contention: Add -x option for CSV style output

perf lock contention: Add -x option for CSV style output

Sometimes we want to process the output by external programs.  Let's add
the -x option to specify the field separator like perf stat.

  $ sudo ./perf lock con -ab -x, sleep 1
  # output: contended, total wait, max wait, avg wait, type, caller
  19, 194232, 21415, 10222, spinlock, process_one_work+0x1f0
  15, 162748, 23843, 10849, rwsem:R, do_user_addr_fault+0x40e
  4, 86740, 23415, 21685, rwlock:R, ep_poll_callback+0x2d
  1, 84281, 84281, 84281, mutex, iwl_mvm_async_handlers_wk+0x135
  8, 67608, 27404, 8451, spinlock, __queue_work+0x174
  3, 58616, 31125, 19538, rwsem:W, do_mprotect_pkey+0xff
  3, 52953, 21172, 17651, rwlock:W, do_epoll_wait+0x248
  2, 30324, 19704, 15162, rwsem:R, do_madvise+0x3ad
  1, 24619, 24619, 24619, spinlock, rcu_core+0xd4

The first line is a comment that shows the output format.  Each line is
separated by the given string ("," in this case).  The time is printed
in nsec without the unit so that it can be parsed easily.

The characters can be used in the output like (":", "+" and ".") are not
allowed for the -x option.

  $ ./perf lock con -x:
  Cannot use the separator that is already used

   Usage: perf lock contention [<options>]

      -x, --field-separator <separator>
                            print result in CSV format with custom separator

The stacktraces are printed in the same line separated by ":".  The
header is updated to show the stacktrace.  Also the debug output is
added at the end as a comment.

  $ sudo ./perf lock con -abv -x, -F wait_total sleep 1
  Looking at the vmlinux_path (8 entries long)
  symsrc__init: cannot get elf header.
  Using /proc/kcore for kernel data
  Using /proc/kallsyms for symbols
  # output: total wait, type, caller, stacktrace
  37134, spinlock, rcu_core+0xd4, 0xffffffff9d0401e4 _raw_spin_lock_irqsave+0x44: 0xffffffff9c738114 rcu_core+0xd4: ...
  21213, spinlock, raw_spin_rq_lock_nested+0x1b, 0xffffffff9d0407c0 _raw_spin_lock+0x30: 0xffffffff9c6d9cfb raw_spin_rq_lock_nested+0x1b: ...
  20506, rwlock:W, ep_done_scan+0x2d, 0xffffffff9c9bc4dd ep_done_scan+0x2d: 0xffffffff9c9bd5f1 do_epoll_wait+0x6d1: ...
  18044, rwlock:R, ep_poll_callback+0x2d, 0xffffffff9d040555 _raw_read_lock_irqsave+0x45: 0xffffffff9c9bc81d ep_poll_callback+0x2d: ...
  17890, rwlock:W, do_epoll_wait+0x47b, 0xffffffff9c9bd39b do_epoll_wait+0x47b: 0xffffffff9c9be9ef __x64_sys_epoll_wait+0x6d1: ...
  12114, spinlock, futex_wait_queue+0x60, 0xffffffff9d0407c0 _raw_spin_lock+0x30: 0xffffffff9d037cae __schedule+0xbe: ...
  # debug: total=7, bad=0, bad_task=0, bad_stack=0, bad_time=0, bad_data=0

Also note that some field (like lock symbols) can be empty.

  $ sudo ./perf lock con -abl -x, -E 10 sleep 1
  # output: contended, total wait, max wait, avg wait, address, symbol, type
  6, 275025, 61764, 45837, ffff9dcc9f7d60d0, , spinlock
  18, 87716, 11196, 4873, ffff9dc540059000, , spinlock
  2, 6472, 5499, 3236, ffff9dcc7f730e00, rq_lock, spinlock
  3, 4429, 2341, 1476, ffff9dcc7f7b0e00, rq_lock, spinlock
  3, 3974, 1635, 1324, ffff9dcc7f7f0e00, rq_lock, spinlock
  4, 3290, 1326, 822, ffff9dc5f4e2cde0, , rwlock
  3, 2894, 1023, 964, ffffffff9e0d7700, rcu_state, spinlock
  1, 2567, 2567, 2567, ffff9dcc7f6b0e00, rq_lock, spinlock
  4, 1259, 596, 314, ffff9dc69c2adde0, , rwlock
  1, 934, 934, 934, ffff9dcc7f670e00, rq_lock, spinlock

Acked-by: Ian Rogers <irogers@google.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/r/20230628200141.2739587-3-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>