Saturday, 5 November 2011

Android - track down memory leaks

My current Android application project is starting to make sense. Unfortunately it crasches after a few levels of playing due to java.lang.OutOfMemoryError. Up to that point I hadn't put much thinking into the memory model of Android applications and simply consumed memory without hesitations. I've now been forced to rewrite some critical parts of the application and i thought I'll write a few words to remember the most useful tools I came across.

First of all, Android apps have small heaps. And with different sizes, it's up to the vendor of the device to decide. Here's a few numbers I came across:

  • G1 = 16 Mb
  • Droid = 24 Mb
  • Nexus One = 32 Mb
  • Xoom = 48 Mb
  • GalaxyTab = 64 Mb

So you see that allocated heaps are far from using the entire RAM of the devices since no application should be able to clog the system. The natural approach to solving a memory problem would be to increase the heap but that is not so easy. If you have a rooted phone you may edit

/system/build.props
and set the heap size via
dalvik.vm.heapsize=24m


Or, if you're running on a tablet (3.x) Android version there is a manifest setting to ask for a large heap

<application android:label="@string/app_name" android:hardwareAccelerated="true" android:largeHeap="true" android:debuggable="true">
but that is no guarantee and you will instead be punished with longer GC cycle times.

On the other hand, changing the VM heap size in your emulator is easy, and could be a good thing in order to verify that your app works on devices with smaller heaps. To do that, fire up your Android SDK and AVD Manager and click edit on your virtual device. Under hardware, there is a setting Max VM application heap size.

So the conclusion is that you have to live with small heaps and limited memory. How to get an estimate of  your consumed memory and how much there is available then?
Run your application in the emulator or connect your real device via USB and use the Android Debug Bridge (adb). It's located in your Android SDK tools folder.

To dump memory info for all your running applications use
$>adb shell dumpsys meminfo

or for your specific application


$>adb shell dumpsys meminfo se.noren.android.othello


Applications Memory Usage (kB):
Uptime: 8979886 Realtime: 8979886


** MEMINFO in pid 1073 [se.noren.android.othello] **
                    native   dalvik    other    total
            size:    24648    10119      N/A    34767
       allocated:    10869     7335      N/A    18204
            free:        2     2784      N/A     2786
           (Pss):     2857     8568     9385    20810
  (shared dirty):     1508     4092     2556     8156
    (priv dirty):     2656     6020     7732    16408


 Objects
           Views:        0        ViewRoots:        0
     AppContexts:        0       Activities:        0
          Assets:        2    AssetManagers:        2
   Local Binders:        6    Proxy Binders:       10
Death Recipients:        0
 OpenSSL Sockets:        0


 SQL
            heap:        0       memoryUsed:        0
pageCacheOverflo:        0  largestMemAlloc:        0


To understand this table we must know that you have a managed heap, dalvik, and a native heap. For example some graphics are stored in native heap. But important, it is the sum of these heaps that can not exceed the VM heap size. so you can't fool the runtime by putting more stuff in either native or managed heap. So to me, the most important numbers are the number under dalvik and total above. The dalvik heap is the managed VM heap and the native numbers are memory allocated by native libraries (malloc).
You'll probably see these numbers fluctating each time you run the command, that is because objects are allocated by the runtime all the time but GCs are not run particularly often. So, in order to know that you really have garbage collected all unused objects you must either wait for the Android debug log in logcat to say something like
GC_FOR_MALLOC or GC_EXTERNAL_MALLOC or similar to that which indicates that the GC has been invoked. Still, this does not mean that all unused memory has been released since the inner workings of the GC might not have done a complete sweep.

You can of course ask for a GC programmatically by System.gc();
But that is never a good option. You should have trust in the VM to garbage collect for you. If you for example want to allocate a large memory chunk the gc will be invoked if necessary.

You can force a gc using the Dalvik Debug Monitor (DDMS). Either start it from Eclipse or from the ddms tool in the Android SDK installation folders.


If you can't see your process right away, go to menu Actions and Reset adb. After that you can turn on heap updates via the green icon Show heap updates. To force a GC, click on Cause GC.

If you wish to monitor the memory usage programmatically there are a few APIs you can use.

ActivityManager.getMemoryInfo() can be used to get an idea of how the memory situation is for the whole Android system. If running low on the gauges you can expect background processes to be killed off soon.

To start inspecting your process in particular use the Debug APIs http://developer.android.com/intl/de/reference/android/os/Debug.html#getMemoryInfo(android.os.Debug.MemoryInfo. There's an excellent explanation of the data you can retrieve from this here http://stackoverflow.com/questions/2298208/how-to-discover-memory-usage-of-my-application-in-android

For example, to see how much memory is allocated in the native heap, use:
Debug.getNativeHeapAllocatedSize()

So back to DDMS. This tool can also create heap dumps which are particulary useful when tracing down memory leaks. To dump the heap, click on the icon Dump HPROF file. There are many tools for analyzing heap dumps but the one I'm most familiar is the Eclipse Memory Analyzer (MAT). Download it from http://www.eclipse.org/mat/.

MAT can't handle the DDMS heap dumps right away, but there is a converter tool in the Android SDK.
So simply run this command.

C:\Temp\hprof>hprof-conv rawdump.hprof converteddump.hprof

Then you can open the converted heap dump in MAT. An important concept here is retained size. The retained size of an object found in the heap is how much memory could be freed if this object could be garbage collected. That includes the object itself, but also child objects which no other objects outside of the retained set has references to.

MAT gives you an overview of where your memory is allocated and has some good tooling on finding suspicious allocations that could be memory leaks.


So to find my memory leak, I used the dominator tree tab which sorts the allocated objects by retained heap
and I soon discovered that the GLRendered object held far too many references to a large 512x512 texture.

The tool becomes even more valuable when the leaking objects are small but many. The dominator tree tell you right away that you have a single object holding a much larger retained heap than you would expect it to.

If you want to learn more, check out this speech by Patrick Dubroy on Android memory management from Google IO 2011 where he explains the Android memory model in more detail.






10 comments:

  1. I still dont understand how you found out about the texture reference.

    Another question i got is that MAT tells me about 4 possible memory leaks but i dont understand anything about them.

    ReplyDelete
  2. Good point, so what I did was make a heap dump after say 1 minute of running the app. This for reference. Then I ran the app for some further amount of time and made a new heap dump. Opening the two dumps beside eachother it is clear that the later had allocated 3 mb more memory as texture objects. After that you must of course know your program well enough to recognise this as a bug.

    ReplyDelete
  3. Giulio Piancastelli20 July 2012 16:02

    So I assume that if the dominators are system classes (e.g. Resources or Bitmap in an application using huge images) that's the sign there is no leak, and images needs to be scaled down, resampled, or whatever it takes to reduce their size, since that's the real cuprit.

    ReplyDelete
  4. Yep very helpful i'm having an issue where i run out of memory after at least 20minutes of a "chat" channel which has images refreshing every 10seconds. I'll try as you did and take a dump early...let the program run (refreshing away...) for 10+ minutes and see what comparisons i can make between the two in MAT.

    BTW i've tried inspecting using DDMS Dump heap > Threads, Heap and allocation tracker and couldn't find anything. I assume MAT is easier to find problems??

    Thanks

    ReplyDelete
  5. It's marvelous I didn't know about all this until the moment I saw this entry of yours. In addition to that I want to ask you a question which is extremely intriguing for me. Do you own any helpful data about how to protect your personal intellectual property from being used without your awareness about it?

    ReplyDelete
  6. Thanks, everything worked out setting it up. Found link to this on StackOverflow.

    ReplyDelete
  7. That's interesting! Can you please share more about it? Thank you.

    -----------------------------------------------------------------------------
    Android Application Developer India & Android Application Development Company

    ReplyDelete
  8. Android is a versatile working framework that has been remarkably great due to its staggering characteristics. The majority of the Pdas in the business sector run on Android OS. Since the accomplishment of Android, parcel of Android Improvement Companies has prodded up all over. We are additionally putting great rationale to create all the more in this field.
    Develop Android Apps // Mobile
    Application Development
    // Android Application Development

    ReplyDelete