awk: Parsing 'free -m' output to get memory usage/consumption
Although I know this problem is already solved by collectd and New Relic I wanted to write a little shell script that showed me the memory usage on a bunch of VMs by parsing the output of http://linux.about.com/library/cmd/blcmdl1_free.htm.
The output I was playing with looks like this:
$ free -m
total used free shared buffers cached
Mem: 365 360 5 0 59 97
-/+ buffers/cache: 203 161
Swap: 767 13 754
I wanted to find out what % of the memory on the machine was being used and as I understand it the numbers that we would use to calculate this are the 'total' value on the 'Mem' line and the 'used' value on the 'buffers/cache' line.
I initially thought that the 'used' value I was interested in should be the one on the 'Mem' line but this number includes memory that Linux has borrowed for disk caching so it isn’t the true number.
There’s another quite interesting article showing some experiments you can do to prove this.
So what I wanted to do was get the result of the calculation '203/365' which I wasn’t sure how to do until I realised you can match multiple regular expressions with awk like so:
$ free -m | awk '/Mem:/ { print $2 } /buffers\/cache/ { print $3 }'
365
203
We’ve now filtered the output down to just our two numbers but another neat thing you can do with awk is change what it uses as its field and record separator.
In this case we want to change the field separator to be the new line character and we’ll set the record separator to be nothing because otherwise it defaults to the new line character which will mess with our field separator.
Those two values are set by using the 'RS' and 'FS' variables:
$ free -m |
awk '/Mem:/ { print $2 } /buffers\/cache/ { print $3 }' |
awk 'BEGIN { RS = "" ; FS = "\n" } { print $2 / $1 }'
0.556164
This is still sub optimal because we’re using two awk commands rather than one! We can get around that by storing the two memory values in variables and printing them out in an END block:
$ free -m |
awk '/Mem:/ { total=$2 } /buffers\/cache/ { used=$3 } END { print used/total}'
0.556164
About the author
I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.