Analyzing Memory Usage of an Application

Introduction

Memory usage of an application is a key factor to monitor. Specially in production systems you need to set alarms to make sure that system is stable. Therefore memory usage can be considered as a probe to measure the health of the system. Usually production systems are installed on Linux servers. OS itself help in many ways to provide a clear view of application’s memory usage.

featured_image

In this post, I am going to discuss different commands and tools which can be used to measure memory usage of applications, specially Java applications. This post will guide you from higher-level to lower-level under following topics.

  1. Monitoring Overall System Memory Usage
  2. Monitoring Application’s Memory Usage
  3. Analyzing Java Application’s Memory Usage

Further information on commands and tools can be gained by going through the external links provided.

Monitoring Overall System Memory Usage

For this purpose I am going to discuss using “free” command in Linux. Free command gives an overview of complete system memory usage. Therefore please note that this is not an efficient way of measuring an applications performance. Because the system can host many applications and each application has its own boundaries. However, let’s look in to some usages of free command. (hint: use free -h to get a human readable output)

[root@operation_node ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          7873       7360        512          0         70        920
-/+ buffers/cache:       6369       1503
Swap:        11143        431      10712

In the above output, in the first line, you can see total physical memory is 7873 MB and 7360 MB is used. Because of that only 512 MB is remaining. However this does not imply that memory is completely being used. Linux OS is good at using memory effectively. Therefore it uses caching for make the memory access efficient. And that cached memory is showing as used in the first line.
What you should look at is the second line which removed the cache & buffer usage of the physical memory. In used column of second line shows actual use of memory without cache & buffer. In the free column of second row you can see 1503 MB of free memory, which is generated by accumulating free + cache & buffer. So actually you have 1503 MB of free physical memory for use. In addition, according to the third line, you have around 10 GB of swap memory ready for use. Please refer [1] for further information.

In modern versions of Linux kernels, the output of free command has changed. It would be look like something below.

me@my-pc ~ $ free -m

                  total        used        free      shared  buff/cache   available

Mem:           7882        3483         299         122        4100        3922

Swap:          9535           0        9535

In above case, the formula for calculation is as below [2] [3]:

total = used + free + buffers + cache

available : is the amount of memory which is available for allocation to a new process or to existing processes.

free : is the amount of memory which is currently not used for anything. This number should be small, because memory which is not used is simply wasted.

Monitoring Application's Memory Usage

In many cases we want to target monitoring of a single application than overall system. Overall memory usage reflects the use of memory including OS-level operations. Therefore we can use top command in Linux for this purpose. Following is a sample output of top command.

top-cmd-result
A result of a top command

What you should actually focus is the RES value and %MEM value of the application (for this, first you need to identify the process-id of an application using ps -aux | grep "application_name" command). You can use simple "e" to toggle the unit of displaying memory.

RES -- Resident Memory Size : The non-swapped physical memory a task has used.

%MEM -- Memory Usage (RES) : A task's currently used share of available physical memory (RAM).

According to above discussion, top command directly reveals the memory consumption of an application. For further information on top command, you may refer [4] [5].

Analyzing Java Application's Memory Usage

If your application is a Java application, then you might need to look at what objects consumes the high amount of memory. For that one option is taking a heap-dump of that Java application. You may use following command to take a heap dump of the application. Prior to that you should know the process-id of that running Java application.

jmap -dump:format=b,file=heap_dump.hprof <process_id>

Once you execute the command, you will get the file heap_dump.hprof  containing heap usage of Java program. Since the file is in binary format, you need to use a special tool to analyze it. Commonly using tool to inspect heap-dump is Eclipse Memory Analyzer Tool (MAT) [6], which is built on top of Eclipse platform. You just need to download the pack and extract it. Executing MemoryAnalyzer will open up a GUI application to analyze the heap-dump. When you open the heap-dump using MAT, tool will prompt you to generate reports based on heap-dump. You may interest about Leak Suspects Report which shows the large object which takes large portion of the memory.

MAT-leak_suspects
Memory Analyzer Tool with Leak Suspects Report

Another interesting view of this tool is the Dominator Tree, which shows large objects along with ones who keep them. According to the definition [7];

An object x dominates an object y if every path in the object graph from the start (or the root) node to y must go through x.

In the dominator tree view, you will see list of objects and the amount of memory they took when you take the heap-dump.

dominator_tree
Dominator Tree view of Memory Analyzer

In dominator tree view, you can go expanding each entry and see how they have composed. Two columns showing in this view are Shallow Heap and Retained Heap. By default the list is sorted by Retained Heap value, descending order. Following definition [8] clearly explain the meaning of those two values.

Shallow heap is the memory consumed by one object. An object needs 32 or 64 bits (depending on the OS architecture) per reference, 4 bytes per Integer, 8 bytes per Long, etc. Depending on the heap dump format the size may be adjusted (e.g. aligned to 8, etc...) to model better the real consumption of the VM.

Retained set of X is the set of objects which would be removed by GC when X is garbage collected.

Retained heap of X is the sum of shallow sizes of all objects in the retained set of X, i.e. memory kept alive by X.

Generally speaking, shallow heap of an object is its size in the heap and retained size of the same object is the amount of heap memory that will be freed when the object is garbage collected.

Therefore in case of out-of-memory or high memory indications, you should definitely focus on this Retained Heap values of the dominator tree view.

Conclusion

In this post I want to give an clear idea on using several tools to analyze the memory usage of an application running on Linux OS. I have used commands which comes with Linux itself. However you may find tools which can be installed to analyze memory. In the last section I spent on discussing how Eclipse Memory Analyzer can be used to examine heap usage of a Java program. Hope those will help you as well.

References

[1] Understanding Linux free memory : https://thecodecave.com/understanding-free-memory-in-linux/

[2] Usage of free memory : https://stackoverflow.com/questions/30772369/linux-free-m-total-used-and-free-memory-values-dont-add-up

[3] Ask Ubuntu clarification on free command : https://askubuntu.com/questions/867068/what-is-available-memory-while-using-free-command

[4] Super-user forum top command explanation : https://superuser.com/questions/575202/understanding-top-command-in-unix

[5] Linuxarea blog top command explanation : https://linuxaria.com/howto/understanding-the-top-command-on-linux

[6] Eclipse Memory Analyzer : https://www.eclipse.org/mat/

[7] MAT Dominator Tree : https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Fconcepts%2Fdominatortree.html

[8] MAT Shallow Heap and Retained Heap explanation : https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Fconcepts%2Fdominatortree.html

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s