Over the past few days, I’ve been busy making MediaPipe work with cross-platform app development frameworks like React Native and Ionic. While it’s well-known that MediaPipe seamlessly integrates with native app builders such as Android Studio, extending its functionality to cross-platform frameworks proved to be a more difficult task.
My attempt was guided by the recommendations provided in this MediaPipe GitHub issue: link to issue.
What Is AAR?
Looking at the definition from the Android Studio documentation pages, “an Android library is structurally the same as an Android app module. It includes everything needed to build an app, including source code, resource files, and an Android manifest.“
According to the comments on the official MediaPipe issue, this was required in order to be able to access the solutions provided by the framework.
The comments from the MediaPipe repository redirected developers to a guide to Getting started with MediaPipe Android Library and those were the steps I attempted to follow.
Setting Up The Groundwork
To actually build an AAR, it was necessary to prepare first by installing all the tools necessary to generate a MediaPipe AAR by myself.
The first thing to do was to Install MediaPipe on my system. To do that, it was necessary to install Bazel through Homebrew, following the guidelines on the official website: Bazel Installation. Following this, I cloned the MediaPipe repository, installed OpenCV and FFmpeg according to the instructions, and successfully ran the Hello World example from the documentation—a promising start. However, when attempting to build the example from MediaPipe, I encountered some errors, necessitating further installations.
This, of course, I discovered meant that I needed Android SDK and NDK installed on my system. Fortunately, I found a file provided by the official MediaPipe repository that took care of that – and made sure to set the environment paths too. One more thing to do was to install JRE (Java Runtime Environment) for MacOS for the two tools to work.
Roadblocks
Despite these efforts, attempting to build the AAR resulted in another error:
Regrettably, this roadblock has temporarily stopped my efforts in generating my own MediaPipe AAR but I will be revisiting it in the future.
However, it’s not a big problem – this is more of a sidestep because MediaPipe Pose Landmark detection is not supported on iOS so it would have been necessary to look into another pose detection framework eventually.