Finding memory leaks in Flutter applications

Aug 22, 2023

Sérgio Martins

In the realm of C/C++ programming, it's considered essential to run projects under memory leak detectors throughout the development process. This practice is frequently automated by configuring your Continuous Integration (CI) pipeline to execute tests using LSAN, promptly terminating the process if even a single memory leak is detected.

Similar principles hold true for Flutter. Despite Dart benefiting from garbage collection, sizeable applications often incorporate a significant volume of C code through FFI. Even if you're not using C directly, chances are you're using some pub.dev package that calls into a 3rd party native library.

The importance of fixing memory leaks is particularly relevant in the world of embedded devices. Given their prolonged uptime, addressing memory leaks becomes a critical concern to maintain the device's performance and stability over time. Conversely, in the mobile landscape, where apps are often restarted daily, the impact of leaks is usually barely noticed.

Requirements

For the purpose of this article, we'll be using Leak Sanitizer on Linux and Flutter 3.10.

Be sure to have /usr/lib/libasan.so or similar installed. Usually it's included with gcc. Clang with its static ASAN runtime should be fine as well.

You'll need dart and flutter in $PATH.

Clone this repo, which contains the samples we'll work on.

Overview of Common Challenges

If you're working with different platforms or using alternative memory tools, I've distilled the essence of our research into this section. Uncovering these hurdles took quite some time, so I'll get straight to the point and present them here. This compilation should hopefully accelerate your investigation, if you're employing different tools.

The problems were mostly related to symbolization of stack frames. While C/C++ function names and line numbers should be printed out of the box, stack frames containing dart code would just show hexadecimal addresses instead of proper names.

Make sure you're building in AOT mode, as JIT traces will be unreadable
Make sure your AOT snapshot has debug symbols. While dart compile includes debug symbols by default, this is not the case for flutter run --release, you'll need to pass --no-strip all the way to dart's gen_snapshot binary.
When using pure dart, be sure to use dart compile aot-snapshot and not dart compile exe. While the latter is technically still AOT, it's using non-standard ELF hacks, basically embedding an ELF inside an ELF instead of linking.
LSAN sometimes doesn't symbolize the stack traces, in that case try passing it to asan_symbolize.py.
Sometimes Dart will unmap the AOT snapshot early, before LSAN gets a chance to print it. Resulting in a trace that while accurate does not show function names. Try ending your program with an exception, like throw Null; which seems to prevent this. This is not needed with Flutter, only pure Dart.
When using GDB, to get nice backtraces, you'll need to load the AOT snapshot, usually named libapp.so, manually. Otherwise GDB doesn't know about it. Maybe because it's dlopened at runtime.

Creating a leak

Just as event organizers might discreetly place garbage on a beach before participants clean it up during company team-building activities, in a similar vein, we'll deliberately introduce memory leaks before illustrating how to identify them. Keep in mind that neither of these actions should be done in a production environment :).

A simple malloc.call() from dart:fii is enough for demonstration purposes, but a more common leak is when passing strings from Dart to C, as toNativeUtf8() actually allocates memory and requires a manual free() eventually, which you might forget.

Flutter

Here we'll search for leaks in a flutter application, as opposed to pure dart, which involves a slightly different process.

Our sample is based on the default app template from flutter create, but with a memory leak.

cd blog-flutter-leak-santizer/flutter_leak/

Now, we'll build our application in AOT mode (hence the --release flag). Don't forget to tell flutter to not strip out debug symbols.

export EXTRA_GEN_SNAPSHOT_OPTIONS=--no-strip
flutter build linux --release
file ./build/linux/x64/release/bundle/lib/libapp.so # Confirm it's not stripped

Now let's run:

# Usually unneeded as it's the default on Linux, but not on all platforms, so please do it just in case:
export ASAN_OPTIONS=detect_leaks=1 
./build/linux/x64/release/bundle/flutter_leak

Press the + button to generate a leak and close the application.

If all goes well you'll have the leak printed for you. It's in flutter_leak/main.dart:69.

Direct leak of 23 byte(s) in 1 object(s) allocated from:
    #0 0x55a6dab7eec9 in malloc (build/linux/x64/release/bundle/flutter_leak+0xd9ec9) (BuildId: 05ef2556bb59b33549f8c962343474459a21e821)
    #1 0x7f3490072a3c in FfiTrampoline_posixMalloc dart:ffi
    #2 0x7f34900729df in FfiTrampoline_posixMalloc dart:ffi
    #3 0x7f3490072688 in MallocAllocator.allocate package:ffi/src/allocation.dart:65
    #4 0x7f34900723cc in StringUtf8Pointer.toNativeUtf8 package:ffi/src/utf8.dart
    #5 0x7f349007234a in _MyHomePageState._incrementCounter.<anonymous closure> package:flutter_leak/main.dart:69
    #6 0x7f34900265d2 in State.setState package:flutter/src/widgets/framework.dart:1139
    #7 0x7f349007230c in _MyHomePageState._incrementCounter package:flutter_leak/main.dart:62
    #8 0x7f34900722b2 in _MyHomePageState._incrementCounter package:flutter_leak/main.dart:61
    #9 0x7f3490065298 in _InkResponseState.handleTap package:flutter/src/material/ink_well.dart:1154

Note: Try not to use flutter run -d linux and instead run the executable directly, as suggested above.

To replicate in your own project add this early to CMakeLists.txt:

add_compile_options(-fsanitize=address -fno-omit-frame-pointer)
add_link_options(-fsanitize=address -fno-omit-frame-pointer)

Note: If you're targetting a custom embedder you can still just build for linux and copy the resulting build/linux/x64/release/bundle/lib/libapp.so, which is embedder-agnostic. Alternatively, follow these instructions, which should be more correct, but a lot more involved. You can also try flutterpi_tool to generate the AOT snapshot for you, if you're targeting ARM.

Pure Dart

Here our debugee is Dart program with no Flutter dependency.

Dart was previously coupled with tcmalloc, which posed challenges for utilizing LSAN. Fortunately, tcmalloc has been removed from the main branch. To operate with a tcmalloc-free Dart, you can either compile the Dart SDK on your own or await an upcoming version release.

If you want to go down the route of compiling Dart SDK, the command should look something like ./tools/build.py --no-goma --mode debug --arch x64 --sanitizer lsan create_sdk. The pivotal switch being --sanitizer lsan.

Once you have a Dart version free from tcmalloc, you can conveniently set the LD_PRELOAD=/usr/lib/libasan.so environment variable before executing dartaotruntime. Be sure dartaotruntime is in PATH.

Let's try it in practice:

cd blog-flutter-leak-santizer/dart_leak/

While we can leak directly from Dart, it's more of a real case scenario if we have some FFI in the mix:

gcc mylib.c -shared -g -fPIC -o mylib.so -Wno-unused-result

As explained before, we'll want to run it under dartaotruntime. Do not use dart compile exe.

dart pub get
dart compile aot-snapshot main.dart

If you built your own Dart SDK with the --sanitizer lsan option, then simply:

dartaotruntime main.aot

If instead, you have a normal SDK (as long as tcmalloc-free), then:

LD_PRELOAD=/usr/lib/libasan.so dartaotruntime main.aot

If everything went well, you'll see a properly symbolized trace indicating where the leak happened.

In our case, dart_leak/main.dart:21 is calling into C, which leaks inside dart_leak/mylib.c and there's a 2nd leak at dart_leak/main.dart:9.

==133278==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 100 byte(s) in 1 object(s) allocated from:
    #0 0x55de7714203e in __interceptor_malloc (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0xad203e)
    #1 0x7fa2bcb06116 in createCLeak /blog-lsan/dart_leak/mylib.c:3:22
    #2 0x7fa2ba7f7c2d in FfiTrampoline_createLeak2 dart:ffi
    #3 0x7fa2ba7f7bd0 in FfiTrampoline_createLeak2 dart:ffi
    #4 0x7fa2ba7f7b71 in createLeak2 file:///blog-lsan/dart_leak/main.dart:21
    #5 0x7fa2ba7f7abf in createLeaks file:///blog-lsan/dart_leak/main.dart:26
    #6 0x7fa2ba7f7a8e in main file:///blog-lsan/dart_leak/main.dart:30
    #7 0x7fa2ba7f7a8e in main file:///blog-lsan/dart_leak/main.dart
    #8 0x7fa2ba7f8d2f in _Closure.call dart:core-patch/function.dart
    #9 0x7fa2ba75dbc5 in _delayEntrypointInvocation.<anonymous closure> dart:isolate-patch/isolate_patch.dart:289
    #10 0x7fa2ba7f8d2f in _Closure.call dart:core-patch/function.dart
    #11 0x7fa2ba79828c in _RawReceivePort._handleMessage dart:isolate-patch/isolate_patch.dart:184
    #12 0x7fa2ba728a4a in stub InvokeDartCode (/blog-lsan/dart_leak/main.aot+0xe0a4a) (BuildId: b44fdaafe233826999d93fb8af8b13d3)
    #13 0x55de77799553 in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x1129553)
    #14 0x55de777990b3 in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11290b3)
    #15 0x55de777a3106 in dart::DartLibraryCalls::HandleMessage(long, dart::Instance const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x1133106)
    #16 0x55de7782ac42 in dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message>>) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11bac42)
    #17 0x55de7786baee in dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11fbaee)
    #18 0x55de7786ed6f in dart::MessageHandler::TaskCallback() (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11fed6f)
    #19 0x55de7787399d  (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x120399d)
    #20 0x55de77e9fbdb in dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x182fbdb)
    #21 0x55de77ea20a5 in dart::ThreadPool::Worker::Main(unsigned long) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x18320a5)
    #22 0x55de77c633d4  (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x15f33d4)
    #23 0x7fa2bd5a744a  (/usr/lib/libc.so.6+0x8744a) (BuildId: 2f005a79cd1a8e385972f5a102f16adba414d75e)

Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0x55de7714203e in __interceptor_malloc (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0xad203e)
    #1 0x7fa2ba7c4130 in FfiTrampoline_posixMalloc dart:ffi
    #2 0x7fa2ba7c40d3 in FfiTrampoline_posixMalloc dart:ffi
    #3 0x7fa2ba7c3e3a in _MallocAllocator.allocate package:ffi/src/allocation.dart:63
    #4 0x7fa2ba7c2b53 in Utf8Codec.encode dart:convert/utf.dart
    #5 0x7fa2ba7c2b53 in StringUtf8Pointer.toNativeUtf8 package:ffi/src/utf8.dart:82
    #6 0x7fa2ba7f7aba in createLeak1 file:///blog-lsan/dart_leak/main.dart:9
    #7 0x7fa2ba7f7aba in createLeaks file:///blog-lsan/dart_leak/main.dart:25
    #8 0x7fa2ba7f7a8e in main file:///blog-lsan/dart_leak/main.dart:30
    #9 0x7fa2ba7f7a8e in main file:///blog-lsan/dart_leak/main.dart
    #10 0x7fa2ba7f8d2f in _Closure.call dart:core-patch/function.dart
    #11 0x7fa2ba75dbc5 in _delayEntrypointInvocation.<anonymous closure> dart:isolate-patch/isolate_patch.dart:289
    #12 0x7fa2ba7f8d2f in _Closure.call dart:core-patch/function.dart
    #13 0x7fa2ba79828c in _RawReceivePort._handleMessage dart:isolate-patch/isolate_patch.dart:184
    #14 0x7fa2ba728a4a in stub InvokeDartCode (/blog-lsan/dart_leak/main.aot+0xe0a4a) (BuildId: b44fdaafe233826999d93fb8af8b13d3)
    #15 0x55de77799553 in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x1129553)
    #16 0x55de777990b3 in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11290b3)
    #17 0x55de777a3106 in dart::DartLibraryCalls::HandleMessage(long, dart::Instance const&) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x1133106)
    #18 0x55de7782ac42 in dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message>>) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11bac42)
    #19 0x55de7786baee in dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11fbaee)
    #20 0x55de7786ed6f in dart::MessageHandler::TaskCallback() (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x11fed6f)
    #21 0x55de7787399d  (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x120399d)
    #22 0x55de77e9fbdb in dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x182fbdb)
    #23 0x55de77ea20a5 in dart::ThreadPool::Worker::Main(unsigned long) (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x18320a5)
    #24 0x55de77c633d4  (/dart/dart-sdk/sdk/out/DebugASANX64/dart-sdk/bin/dartaotruntime+0x15f33d4)
    #25 0x7fa2bd5a744a  (/usr/lib/libc.so.6+0x8744a) (BuildId: 2f005a79cd1a8e385972f5a102f16adba414d75e)

SUMMARY: AddressSanitizer: 112 byte(s) leaked in 2 allocation(s).

Conclusion

Drawing from over two decades of experience in C++ at KDAB, it's very fun to see our accumulated knowledge and tools can seamlessly be transposed to Flutter. The shared principles and practices between these two domains have brought about a synergy that facilitates a smoother development process specially for embedded devices where memory leaks are more pressing.

While this article primarily revolved around the LSAN leak sanitizer, it's hopefully useful for your own developer tools that rely on stack trace symbolization when using Flutter.

In an upcoming blog post, we might consider delving into other common C/C++ performance tools within the context of Flutter.