Collectl – A Powerful Tool to Collect Various Linux Performance Data

Limit sample count & change interval

By default collectl takes sample 1 second interval and unlimited sample count. If you want to limit sample counts with different interval ? just add c followed by No of sample count and i followed by Interval Second with collectl command. For better understanding, we have added timestamps too.

# collectl -c5 -i2 -oT
waiting for 2 second sample...
#         <--------CPU--------><----------Disks-----------><----------Network---------->
#Time     cpu sys inter  ctxsw KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut
15:41:37    8   0   799    759      1      1      0      1    441    426     78     270
15:41:39   14   1  1454   1698      1      1     36      4    667    727    218     541
15:41:41    2   0   285    409      1      1      0      1     66    126     60     117
15:41:43   16   1  1493   1477      3      1      0      1    850    815    170     507
15:41:45   11   1  1154   1218      1      1     24      4    565    593    146     421

Command Equivalence Matrix

Refer the following Matrix before going to deeper collectl with other utility maps.

Use collectl like top

Initially we told you, collectl capable to generate the output of few utility. Here collectl generating top command output, see below.

[ac-button size=”large” color=”orange” style=”flat” icon=”fa-link” url=”https://www.2daygeek.com/top-command-examples-to-monitor-server-performance/” target=”_blank”]Top Command Examples[/ac-button]

# collectl --top
waiting for 1 second sample...
# TOP PROCESSES sorted by time (counters are /sec) 18:05:47
# PID  User     PR  PPID THRD S   VSZ   RSS CP  SysT  UsrT Pct  AccuTime  RKB  WKB MajF MinF Command
 2862  webmaste 20  2981    0 R  383M   44M  1  0.03  0.63  66   0:22.79    0    0    0 3954 /usr/sbin/apache2
 3013  webmaste 20  2981    0 S  370M   32M  1  0.00  0.16  16   0:08.83    0    0    0  427 /usr/sbin/apache2
 3244  root     20  5899    0 R   60M   20M  2  0.00  0.07   7   0:00.80    0    0    0    0 /usr/bin/perl
 5698  root     20  5849    0 S  105M    5M  1  0.00  0.01   1   0:13.05    0    0    0    0 sshd:
    1  root     20     0    0 S   39M    1M  3  0.00  0.00   0   0:27.57    0    0    0    0 /sbin/init
    2  root     20     0    0 S     0     0  2  0.00  0.00   0   0:01.49    0    0    0    0 kthreadd
    3  root     20     2    0 S     0     0  0  0.00  0.00   0 143:32.45    0    0    0    0 ksoftirqd/0
    6  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:03.55    0    0    0    0 migration/0
    7  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:52.25    0    0    0    0 watchdog/0
    8  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:12.50    0    0    0    0 migration/1
   10  root     20     2    0 S     0     0  1  0.00  0.00   0 505:47.44    0    0    0    0 ksoftirqd/1
   11  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:55.66    0    0    0    0 watchdog/1
   12  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:08.04    0    0    0    0 migration/2
   14  root     20     2    0 S     0     0  2  0.00  0.00   0  47:20.11    0    0    0    0 ksoftirqd/2
   15  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:50.22    0    0    0    0 watchdog/2
   16  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:29.06    0    0    0    0 migration/3
   18  root     20     2    0 S     0     0  3  0.00  0.00   0  30:12.18    0    0    0    0 ksoftirqd/3
   19  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:48.62    0    0    0    0 watchdog/3
   20  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:08.21    0    0    0    0 migration/4
   22  root     20     2    0 S     0     0  4  0.00  0.00   0  20:01.33    0    0    0    0 ksoftirqd/4
   23  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:49.75    0    0    0    0 watchdog/4
   24  root     RT     2    0 S     0     0  5  0.00  0.00   0   0:29.75    0    0    0    0 migration/5
   25  root     20     2    0 S     0     0  5  0.00  0.00   0   0:00.00    0    0    0    0 kworker/5:0
   26  root     20     2    0 S     0     0  5  0.00  0.00   0  15:59.53    0    0    0    0 ksoftirqd/5
   27  root     RT     2    0 S     0     0  5  0.00  0.00   0   0:51.22    0    0    0    0 watchdog/5
   28  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 cpuset
   29  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 khelper
   30  root     20     2    0 S     0     0  2  0.00  0.00   0   0:00.00    0    0    0    0 kdevtmpfs
   31  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 netns
   33  root     20     2    0 S     0     0  1  0.00  0.00   0   0:55.05    0    0    0    0 sync_supers
   34  root     20     2    0 S     0     0  3  0.00  0.00   0   0:01.50    0    0    0    0 bdi-default
   35  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 kintegrityd
   36  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 kblockd

Use collectl like iotop

Colletcl generating the output same as iotop.

[ac-button size=”large” color=”green” style=”flat” icon=”fa-link” url=”https://www.2daygeek.com/monitor-disk-io-activity-using-iotop-iostat-commands-on-linux/” target=”_blank”]iotop Command Examples[/ac-button]

# collectl --top iokb
waiting for 1 second sample...
# TOP PROCESSES sorted by iokb (counters are /sec) 15:58:26
# PID  User     PR  PPID THRD S   VSZ   RSS CP  SysT  UsrT Pct  AccuTime  RKB  WKB MajF MinF Command
28520  webmaste 20  2981    0 S  368M   30M  1  0.01  0.05   6   0:15.32    0    4    0  265 /usr/sbin/apache2
 1675  root      0     2    0 S     0     0  3  0.00  0.00   0  11:43.20    1    0    0    0 o2hb-AACFA88021
    1  root     20     0    0 S   39M    1M  4  0.00  0.00   0   0:27.56    0    0    0    0 /sbin/init
    2  root     20     0    0 S     0     0  3  0.00  0.00   0   0:01.49    0    0    0    0 kthreadd
    3  root     20     2    0 S     0     0  0  0.00  0.00   0 143:27.25    0    0    0    0 ksoftirqd/0
    6  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:03.55    0    0    0    0 migration/0
    7  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:52.22    0    0    0    0 watchdog/0
    8  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:12.49    0    0    0    0 migration/1
   10  root     20     2    0 S     0     0  1  0.01  0.00   1 505:28.64    0    0    0    0 ksoftirqd/1
   11  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:55.63    0    0    0    0 watchdog/1
   12  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:08.03    0    0    0    0 migration/2
   14  root     20     2    0 S     0     0  2  0.00  0.00   0  47:18.68    0    0    0    0 ksoftirqd/2
   15  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:50.19    0    0    0    0 watchdog/2
   16  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:29.04    0    0    0    0 migration/3
   18  root     20     2    0 S     0     0  3  0.00  0.00   0  30:11.30    0    0    0    0 ksoftirqd/3
   19  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:48.59    0    0    0    0 watchdog/3
   20  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:08.21    0    0    0    0 migration/4
   22  root     20     2    0 S     0     0  4  0.00  0.00   0  20:00.87    0    0    0    0 ksoftirqd/4
   23  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:49.71    0    0    0    0 watchdog/4
   24  root     RT     2    0 S     0     0  5  0.00  0.00   0   0:29.73    0    0    0    0 migration/5
   25  root     20     2    0 S     0     0  5  0.00  0.00   0   0:00.00    0    0    0    0 kworker/5:0
   26  root     20     2    0 S     0     0  5  0.00  0.00   0  15:59.26    0    0    0    0 ksoftirqd/5
   27  root     RT     2    0 S     0     0  5  0.00  0.00   0   0:51.18    0    0    0    0 watchdog/5
   28  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 cpuset
   29  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 khelper
   30  root     20     2    0 S     0     0  2  0.00  0.00   0   0:00.00    0    0    0    0 kdevtmpfs
   31  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 netns
   33  root     20     2    0 S     0     0  1  0.00  0.00   0   0:55.04    0    0    0    0 sync_supers
   34  root     20     2    0 S     0     0  2  0.00  0.00   0   0:01.50    0    0    0    0 bdi-default
   35  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 kintegrityd
   36  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 kblockd
   37  root      0     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 ata_sff
   38  root     20     2    0 S     0     0  3  0.00  0.00   0   0:00.00    0    0    0    0 khubd

To know more list of –top’s sort types, fire the below command on terminal.

# collectl --showtopopts

Use collectl like vmstat

Colletcl generating the output same as vmstat.

# collectl --vmstat
waiting for 1 second sample...
#procs ---------------memory (KB)--------------- --swaps-- -----io---- --system-- ----cpu-----
# r  b   swpd   free   buff  cache  inact active   si   so    bi    bo   in    cs us sy  id wa
  2  0   103M   614M   325M  1867M  1568M   623M    0    0     0     0 3385  3415 34  3  62  0
  1  0   103M   599M   325M  1867M  1568M   623M    0    0     1    68 1722  1655 14  1  84  0
  2  0   103M   613M   325M  1867M  1568M   623M    0    0     0     0  478   772  5  0  93  0
  0  0   103M   614M   325M  1867M  1568M   623M    0    0     1     1  435   608  5  0  94  0
  1  0   103M   603M   325M  1867M  1568M   623M    0    0     0     0 1079   721 10  0  88  0
  1  0   103M   615M   325M  1867M  1568M   623M    0    0     1     0 2100  2325 20  1  77  0

Use collectl like ps

Colletcl generating the output same as ps.

# collectl -sZ
waiting for 60 second sample...

### RECORD    1 >>> 2daygeek <<< (1483658400.001) (Thu Jan  5 16:20:00 2017) ###

# PROCESS SUMMARY (counters are /sec)
# PID  User     PR  PPID THRD S   VSZ   RSS CP  SysT  UsrT Pct  AccuTime  RKB  WKB MajF MinF Command
    1  root     20     0    0 S   39M    1M  4  0.00  0.00   0   0:27.56    0    0    0    0 /sbin/init
    2  root     20     0    0 S     0     0  2  0.00  0.00   0   0:01.49    0    0    0    0 kthreadd
    3  root     20     2    0 S     0     0  0  0.05  0.00   0 143:28.17    0    0    0    0 ksoftirqd/0
    6  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:03.55    0    0    0    0 migration/0
    7  root     RT     2    0 S     0     0  0  0.00  0.00   0   0:52.22    0    0    0    0 watchdog/0
    8  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:12.49    0    0    0    0 migration/1
   10  root     20     2    0 S     0     0  1  0.16  0.00   0 505:31.83    0    0    0    0 ksoftirqd/1
   11  root     RT     2    0 S     0     0  1  0.00  0.00   0   0:55.63    0    0    0    0 watchdog/1
   12  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:08.04    0    0    0    0 migration/2
   14  root     20     2    0 S     0     0  2  0.01  0.00   0  47:18.96    0    0    0    0 ksoftirqd/2
   15  root     RT     2    0 S     0     0  2  0.00  0.00   0   0:50.20    0    0    0    0 watchdog/2
   16  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:29.04    0    0    0    0 migration/3
   18  root     20     2    0 S     0     0  3  0.01  0.00   0  30:11.46    0    0    0    0 ksoftirqd/3
   19  root     RT     2    0 S     0     0  3  0.00  0.00   0   0:48.60    0    0    0    0 watchdog/3
   20  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:08.21    0    0    0    0 migration/4
   22  root     20     2    0 S     0     0  4  0.01  0.00   0  20:00.96    0    0    0    0 ksoftirqd/4
   23  root     RT     2    0 S     0     0  4  0.00  0.00   0   0:49.72    0    0    0    0 watchdog/4

Record Mode

If you want to collects some data for limited duration like couple of minutes to several hours or more, you can do by issuing following command. Still we are not using collectl as a daemon, the default sampling rate is 1 second. Raw data is recorded in a single compressed file with the extension raw.gz.

# collectl -c10 -f/opt

Details :

  • -c10 : Sample counts
  • -f/opt : Store output file @ /opt

Playback Mode

If you want to playback the Raw data file, just add lowercase -p followed by raw file stored location.

# collectl -m -p /var/log/collectl/2daygeek-20170105-150152.raw.gz -oT

Playing back /var/log/collectl/2daygeek-20170105-150152.raw.gz
#         <----CPU------------><----------Disks-----------><----------Network---------->
#Time     cpu sys inter  ctxsw KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut
15:01:54    0   0    47     32      0      0      0      0      0      0      0       0
15:01:55    0   0   180    215      0      0     20      5      0      0      0       0
15:01:56    0   0   206    333      0      0      0      0      2     14      2      15
15:01:57    0   0  1195    805      0      0      0      0      3     26     13      37
15:01:58    0   0    87     52      0      0     52      6      0      0      0       0
15:01:59    0   0    72    111      0      0      0      0      0      0      0       0
15:02:00    0   0    50     29      0      0      0      0      0      0      0       0
15:02:01    0   0    78    110      0      0      0      0      0      0      0       0
15:02:02    0   0    40     24      0      0      0      0      0      0      0       0
15:02:03    0   0    82    115      0      0      0      0      0      0      0       0

If you want to print any particular duration of data, use --from time & --thru time.

# collectl -m -p /var/log/collectl/2daygeek-20170105-150152.raw.gz -oT --from 15:02:00 --thru 15:02:05

Playing back /var/log/collectl/2daygeek-20170105-150152.raw.gz
#         <----CPU[HYPER]-----><----------Disks-----------><----------Network---------->
#Time     cpu sys inter  ctxsw KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut
15:02:00    0   0    50     29      0      0      0      0      0      0      0       0
15:02:01    0   0    78    110      0      0      0      0      0      0      0       0
15:02:02    0   0    40     24      0      0      0      0      0      0      0       0
15:02:03    0   0    82    115      0      0      0      0      0      0      0       0

How to run collectl as a daemon

Start the collectl daemon, which will create a raw file @ /var/log/collect with raw.gz extension and collects all possible data. You can use the raw file for later analysis.

[For SysVinit System]
# service collectl start

[For systemd System]
# systemctl start collectl

Read more about collect

If you want to know more option which is available for collectl, simply navigate to man page.

[Collectl man page]
# collectl --help
or
# man collectl

[show all the options]
# collectl --showoptions

[show all the subsystems]
# collectl --showsubsys

[show all subsystem specific options]
# collectl --showsubopts

[show --top options]
# collectl --showtopopts

About Magesh Maruthamuthu

Love to play with all Linux distribution

View all posts by Magesh Maruthamuthu

Leave a Reply

Your email address will not be published. Required fields are marked *