Tag: linux debugging

bcc: BPF based kernel analysis utilities

tux_compbcc is a collection of tracing and monitoring tools on Linux written around eBPF (Extended Berkeley Packet Filter), an in-kernel VM. BPF is one of the latest mechanisms of its kind built into the kernel (at the time of writing) and finds its use in networking, tracing, in-kernel optimizations and hardware optimizations. bcc provides both kernel-level and user-level tracing options. Continue reading bcc: BPF based kernel analysis utilities

vmtouch: portable file cache analyzer

search_compvmtouch is a useful utility to analyze and control the virtual memory used by the filesystem. It works on Unix-ish systems and BSD. It is a portable utility with minimal dependecies in the source code. In addition to understanding cache usage by files, you can also use vmtouch to preload files and speed up subsequent operations. Continue reading vmtouch: portable file cache analyzer

ftrace: trace kernel function calls

hacker_compFor those who have used strace for userspace development ftrace would be a familiar concept. ftrace is an internal tracing facility for the Linux kernel developed by Red Hat. It internally uses the kernel’s tracepoint mechanism and can track several characteristics of the running kernel. However, in this article we will concentrate on enabling it to trace the commonest one – function calls in the Linux kernel. Continue reading ftrace: trace kernel function calls

dstat: live system information

$ dstat -a
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw 
 11   3  85   1   0   0|  51k   58k|   0     0 |   0     0 | 172   864 
  1   0  99   0   0   0|   0  4096B|   0     0 |   0     0 |  81   224 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  57   174 
  1   0  99   0   0   0|   0     0 |  14k  121B|   0     0 | 104   306 
  1   0  99   0   0   0|   0     0 |  97B  121B|   0     0 | 124   596 
  1   0  99   0   0   0|   0     0 | 218B  172B|   0     0 | 106   407 
  0   0 100   0   0   0|   0     0 | 229B  121B|   0     0 |  84   296 
  1   1  99   0   0   0|   0     0 | 785B  723B|   0     0 | 120   344 
  0   0  99   0   0   0|   0     0 | 163B  121B|   0     0 |  67   208 
  0   0 100   0   0   0|   0     0 |  54B   86B|   0     0 |  67   217 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  52   161 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  49   163 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  56   206 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  88   304 
  1   0  99   0   0   0|   0     0 |   0     0 |   0     0 | 107   461 
  1   0  99   0   0   0|   0     0 |   0     0 |   0     0 |  60   243 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  50   148 
  0   0 100   0   0   0|   0     0 |   0     0 |   0     0 |  47   128 
  1   0  99   0   0   0|   0     0 |   0     0 |   0     0 |  90   431 
  1   0  99   0   0   0|   0     0 |   0     0 |   0     0 | 100   408

What if you could have the functionality of vmstat, iostat, mpstat, netstat and ifstat in a single tool? dstat is a top like utility to fetch all the information from a running system and show it in the terminal with regular updates. Continue reading dstat: live system information

csysdig: trace your system

The sysdig utility is a open source tool to trace, explore, capture system state and activity from a running Linux instance, then save, filter and analyze. It is a combination of tools like strace, tcpdump, htop, iftop and lsof. sysdig allows you to dig into system metrics including CPU, memory, disk I/O, network I/O, application activity and more. Continue reading csysdig: trace your system

Dump, debug, resume process with criu

hacker_compcriu (previously crtools) is a userspace tool to dump the complete state of a process as a set of image files and resume it later from the same checkpoint. Another advantage of CRIU is you can convert it to a core dump to analyse in GDB. This is particularly useful to debug hung processes. Continue reading Dump, debug, resume process with criu