Processes with brackets in `ps` output
Have you ever wondered why some processes have brackets around their names?
PID TTY STAT TIME COMMAND
1 ? Ss 0:03 /usr/lib/systemd/systemd --switched-root --system --deserialize
2 ? S 0:00 [kthreadd]
4 ? I< 0:00 [kworker/0:0H]
6 ? I< 0:00 [mm_percpu_wq]
...
The COMMAND
column shows the command that was used to start that process and all its arguments in a string. By reading the ps(1) man page, you will find out that the brackets mean the arguments weren’t available. But where do they come from exactly?
First, let’s find out where ps itself comes from:
$ rpm -qf /usr/bin/ps
procps-ng-3.3.10-15.fc27.x86_64
OK, the proc filesystem.
This is an file-based interface to internal data structures in the kernel. Each process gets a directory under /proc with a bunch of files that make it easy to retrieve that information. We are interested here in the cmdlin
file:
Table 1-1: Process specific entries in /proc
..............................................................................
File Content
clear_refs Clears page referenced bits shown in smaps output
cmdline Command line arguments
cpu Current and last cpu in which it was executed (2.4)(smp)
cwd Link to the current working directory
environ Values of environment variables
exe Link to the executable of this process
fd Directory, which contains all file descriptors
maps Memory maps to executables and library files (2.4)
mem Memory held by this process
root Link to the root directory of this process
stat Process status
statm Process memory status information
status Process status in human readable form
wchan Present with CONFIG_KALLSYMS=y: it shows the kernel <span style="color:#fff;font-weight:boldfunction</span>
symbol the task is blocked in - or <span style="color:#0ff;font-weight:bold"0"</span> <span style="color:#fff;font-weight:boldif</span> not blocked.
pagemap Page table
stack Report full stack trace, <span style="color:#fff;font-weight:boldenable</span> via CONFIG_STACKTRACE
smaps an extension based on maps, showing the memory consumption of
each mapping and flags associated with it
numa_maps an extension based on maps, showing the memory locality and
binding policy as well as mem usage (in pages) of each mapping.
Now that we know where this information is coming from, let’s dive into the procps-ng source code.
By using your favorite code editor (or simply grep), you will find out that the read_unvectored()
function is called to read the contents of /proc/%u/cmdline
and will return zero when there’s nothing in it. The fill_cmdline_cvt()
function then calls escape_command()
with the ESC_BRACKETS
flag, which adds the brackets we see in ps’ output.
You can check for yourself that such processes really don’t have arguments (zero file size):
]$ stat /proc/2/cmdline
File: /proc/2/cmdline
Size: 0 Blocks: 0 IO Block: 1024 regular empty file
Device: 14h/20d Inode: 28849 Links: 1
Access: (0444/-r--r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:system_r:kernel_t:s0
Access: 2017-08-06 23:31:28.110131480 +0000
Modify: 2017-08-06 23:31:28.110131480 +0000
Change: 2017-08-06 23:31:28.110131480 +0000
Birth: -
But why not? Most commonly, these will be kernel threads implementing helper functions, specific subsystems, work queues, etc. These tasks will register as system time in top.
They can also be processes which were execve’ed with an empty list of arguments; or even a process that overwrote its argv[] with empty data. These occur less often.
If you’re curious to understand what these processes do in more detail, see below. This list includes links to useful information about each process.
- acpi_thermal_pm:: ACPI Thermal Zone driver
- ata_sff: ATA driver
- cfg80211: 802.11 device configuration API
- comp:
- cpuhp: CPU hotplug, one process for each core
- crypto: Access to the crypto API
- devfreq_wq: Voltage and Frequency scaling for Non-CPU devices
- dm_bufio_cache: Device Mapper Buffered I/O cache layer
- edac-poller: Error Detection And Correction (EDAC) Devices
- ext4-rsv-conver: ext4 reserved space conversion
- gfx:
- i915/signal: Intel i915 gpu driver signaler thread
- ipv6_addrconf: IPv6 address autoconfiguration
- irq: Interrupt threads
- jbd2: Journaling Block Device
- kauditd: Worker thread to send audit records to userspace
- kblocked: Operations on block devices
- kcompactd: Memory compaction
- kdevtmpfs: Maintains /dev tmpfs
- khugepaged: Transparent Hugepage Support
- kintegrityd: Block device data integrity extensions
- kmemstick: Sony MemStick work queue
- krfcommd: Bluetooth RFCOMM implementation
- ksmd: Memory deduplication
- ksoftirqd: Software interrupt handler
- kstrp: Stream Parser work queue
- kswapd: Kernel Swap Daemon
- kthreadd: Interface for starting new kthreads
- kthrotld: Block device I/O Throttling work queue
- kworker: Executes work queue requests
- md: Device Mapper
- migration: Migrates threads amongst processors to achieve balance
- mm_percpu_wq: Runs per-CPU memory management tasks
- nets: Network namespace work queue
- oom_reaper: Out-Of-Memory (OOM) Killer
- rcu_bh: Read-copy update (RCU) bh mechanism
- rcu_sched: RCU scheduler
- rpciod: Sun RPC work queue
- rtsx_usb_ms: Realtek USB card reader driver
- scsi_eh: SCSI Error Handling
- scsi_tmf: SCSI Management
- sdma: Smart Direct Access Memory (Infiniband, GPU, etc)
- ttm_swap: GPU memory management
- watchdog: Kernel watchdog API
- writeback: Writes pages to disk
- xprtiod: Sun RPC work queue
The: Documentation directory in the Linux kernel sources is an invaluable resource. And there’s always the source code itself if you’re in doubt about something.