SureshJoshi.com ▼

Android NDK in Android Studio with SWIG


2014-12-31

Happy soon-to-be New Year everyone! There’s one subject that has given me no end of grief this year, so it seems like a fitting topic to make my last post of 2014. The Android NDK…

Update: 2017, almost 2018! 3 years after the original post! This page still contains a lot of good information and is worth a read - but now we’re using Android Studio 3.0, Gradle 4.1, and CMake. My Github repo (https://github.com/sureshjoshi/android-ndk-swig-example) contains AS1, AS2, and AS3 solutions - but only AS3 is being maintained.

There are no more command line calls to SWIG, as that’s handled by CMake, and I just recently solved an issue where Android Studio craps out when the SWIG files aren’t present (because AS runs the CMake portion after it tries to precompile some code) by using an afterExecute task hook.

So, as of today, the native + generated + Java/Kotlin code builds are seamless. Happy days…

Update: Happy 2016 everyone! At the bottom of this page, you’ll find my latest updates to this post, which include using the experimental Gradle plugin for improved NDK integration in Android Studio.

I’ve needed to write a decent amount of NDK code for a few projects, and I have to say that there is a stark contrast in how easy it is to setup now, than it was just 12 months ago. Android Studio 1.0 (and above), alongside Gradle 2.2.1 (or Gradle Tools 1.0 I suppose), have streamlined this process immensely. Makefiles are automatically generated, and the whole process can run inside Gradle (I used to have custom Android and Application makefiles, alongside some shell scripts for everything to work semi-smoothly).

There are a lot of great recent tutorials on starting up with the Android NDK in Android Studio 1.0+ (such as this one and this one) and I used them as a reference to transition myself away from external scripting.

One last tidbit I’ve not yet seen mentioned though, is easily setting up C++ code to be used by Android without hand-rolling wrapper functions. The traditional setup for NDK coding is that you have your C++ code built by the ndk-build command, but you first need to write custom export functions so that Java can communicate with C++. For anyone who has done this on non-trivial examples, you’ll agree with me when I say this is miserable.

So, why do that ourselves, when we can use a tool that’s been in use for years (decades even) for this exact purpose? SWIG to the rescue!

NDK directory structure

There are two things you need to know here. By default, if you have external libs that you want loaded into the Android application, they are looked for in the (module)/src/main/jniLibs. You can change this by using setting sourceSets.main.jniLibs.srcDirs in your module’s build.gradle. You’ll need a subdirectory with libraries for each architecture you’re targeting (e.g. x86, arm, mips, arm64-v8a, etc…)

The code you want to be compiled by default by the NDK toolchain will be located in (module)/src/main/jni and similarly to above, you can change it by setting sourceSets.main.jni.srcDirs in your module’s build.gradle

Code in that directory will automatically be pulled into Android Studio, much like the Java code is.

Oh, and additionally, much as you set your sdk.dir in local.properties, you’ll need to set an ndk.dir… Otherwise, Android Studio won’t know where to look for your ndk installation.

Some sample code

I’ve written a minimalistic Android app with C++ functions in my Android/NDKExample code located in BitBucket. located in Github: https://github.com/sureshjoshi/android-ndk-swig-example

A lot of tutorials I’ve seen stop at simply having a function that can be called from Java (usually a void return, or maybe an int return, that takes in a few primitives). I decided that if I’m going to do this at all, at the very least, I’ll need to show code including STL and how to pass vectors and strings between Java and C++.

So, for sample C++ functions, I have the primitives in/out functionality (multiplying ints and doubles). For something a bit more complicated, I created a random number generator (using \<random> functionality, instead of just using ‘std::rand’) which returns a vector of uniformly distributed integers between 1 and 100).

std::vector< int > SeePlusPlus::RandomNumbers( const int size )
{
  // Seed a generator with a real random value, if available
  std::random_device rd;
  std::mt19937 generator( rd() );

  // Create a uniform distribution between 1 and 100
  std::uniform_int_distribution<> uniformDistribution(1, 100);

  // Fill the vector with random data
  std::vector< int > randomNumbers( size );
  for (auto& datapoint : randomNumbers)
  {
    datapoint = uniformDistribution( generator );
  }

  return randomNumbers;
}

To show a list being passed in, I decided to make a sorting function which takes in a vector of strings, sorts them, and returns a vector of sorted strings.

std::vector< std::string > SeePlusPlus::Sort( const std::vector< std::string >& inStringList )
{
  std::vector< std::string > sortedList( inStringList );
  std::sort( std::begin( sortedList ), std::end( sortedList ) );
  return sortedList;
}

As well, I created an interface class and a concrete class, to illustrate how to do it in SWIG.

Compiling C++

Once you have your sources and headers in the correct directory (I have some testing to do for code in subdirectories), making your project will automatically generate a makefile (located in (module)/build/intermediates/ndk/debug) and compile it. Any compilation errors will pop up in the Android Studio logging.

Note: There is a REALLY annoying bug in the NDK toolchain, where compilation will fail if there is only one source file to compile… This makes no sense, but you’ll get an error complaining about “no rule to make target”. I researched a bit, and this has been an issue for at least a year, with no signs of getting fixed. That’s why in the above image, you can see a file called ‘NDKBug.cpp’. This file’s sole purpose is to workaround the ndk toolchain bug (you just need an empty file, so I put in comments linking to the stackoverflow post about this issue).

Now, anything other than a trivial bunch of C++ code will require some Makefile manipulation. Android Studio gives you some limited access to the Makefile, via your .gradle file. This will be enough for our sample app (C++11 code with STL). The cFlags for -std=c++11 is required for the sample code in my NDKExample in BitBucket. The -fexceptions is to deal with the generated exception handling code generated by SWIG’s wrapper functions. For no particular reason, I used gnustl_shared instead of stlport_shared.

ndk {
  moduleName "SeePlusPlus" // Name of C++ module (i.e. libSeePlusPlus)
  cFlags "-std=c++11 -fexceptions" // Add provisions to allow C++11 functionality
  stl "gnustl_shared" // Which STL library to use: gnustl or stlport
}

Re-build and everything should be compiling nicely. Your libraries will be located at (module)/build/intermediates/ndk/debug/lib

Accessing C++ from Java

So, now you have compiled shared libraries containing C++ code, but what good is it if you can’t access it?

There are two steps to this part. First, create wrapper functionality using JNI so that Android can see your C++ code, and secondly, import your library into the Android app.

This blog post provides example of a manually written wrapper, so I won’t re-hash. My preferred method to generating wrappers is by using SWIG. I use it all the time, and it’s great (occasionally a pain, usually good though). With SWIG, you can setup a build script so that your wrappers are automatically re-generated every build as well, meaning that your changes are automatically imported into Android (huge time saver).

  1. Install SWIG

  2. Setup SWIG in your system path

  3. Create your SWIG interface file (this file is necessary to let SWIG know what files you want wrapped and for any ‘unique’ wrapping - such as with STL)

  4. Create a task in your module’s build.gradle file which calls SWIG to compile the interface file

The first two steps are pretty self-explanatory, so here is a sample interface file (also in my BitBucket repo). When compiled, it will create a SeePlusPluswrap.cxx which contains the C++/Java bridge code (which contains really well-formed method names, such as _SWIGEXPORT jdouble JNICALL Java_com_sureshjoshi_core_SeePlusPlus_1WrapperJNI_SeePlusPlus_1Multiply_1_1SWIG_11(JNIEnv jenv, jclass jcls, jlong jarg1, jobject jarg1_, jdouble jarg2, jdouble jarg3)*):

/* File : SeePlusPlus.i */
%module SeePlusPlus_Wrapper

/* Anything in the following section is added verbatim to the .cxx wrapper file */
%{
#include "SeePlusPlus.h"
%}

/* This will allow us to iterate through arrays defined by STL containers */
%include "std_string.i"
%include "std_vector.i"

/* Create a template for std::vector< std::string > and std::vector< int > in Java */
namespace std {
%template(IntVector) vector<int>;
%template(StringVector) vector<string>;
}

/* This is the list of headers to be wrapped */
/* For Java, it seems we need the file of interest and all files up the inheritance tree */
%include "ISeePlusPlus.h"
%include "SeePlusPlus.h"

The last piece of the puzzle is the Gradle task to run SWIG. As of today, this task runs perfectly when called explicitly from the command line, but silently fails when called from Android Studio. I’ll need to debug this a bit further.

def coreWrapperDir = new File("${projectDir}/src/main/java/com/sureshjoshi/core")

task createCoreWrapperDir {
  coreWrapperDir.mkdirs()
}

// For this to work, it's assumed SWIG is installed
// TODO: This only works when called from Command Line (gradlew runSwig)
task runSwig(type:Exec, dependsOn: ['createCoreWrapperDir']) {

    String osName = System.getProperty("os.name").toLowerCase();
    if (osName.contains("windows")) {
        workingDir '/src/main/jni'   // This implicitly starts from $(projectDir) evidently
        commandLine 'cmd', '/c', 'swig'
        args '-c++', '-java', '-package', 'com.sureshjoshi.core', '-outdir', coreWrapperDir.absolutePath, 'SeePlusPlus.i'
    }
    else {
        commandLine 'swig'
        args '-c++', '-java', '-package', 'com.sureshjoshi.core', '-outdir', coreWrapperDir.absolutePath, "${projectDir}/src/main/jni/SeePlusPlus.i"
    }

}

Putting it all together

After setting up SWIG and the new Gradle tasks, re-compile your program and you’ll now have a bunch of .java files representing your wrapped C++ classes. They’re used just like normal Java classes. To access STL vectors, you’ll need to use the templated method (StringVector in my example).

 void runSortString() {
    // Create an unsorted set of strings by splitting on commas
    String unsortedString = "lorem,ipsum,dolor,sit,amet,consectetur,adipiscing,elit";
    String[] unsortedStrings = unsortedString.split(",");

    StringVector unsortedStringVector = new StringVector();
    for (String value : unsortedStrings) {
        unsortedStringVector.add(value);
    }

    StringVector sortedStringVector = mCpp.Sort(unsortedStringVector);

    String message = getString(R.string.unsorted_strings) + " " + unsortedString;
    mTextviewUnsortedStrings.setText(message);

    StringBuilder sortedMessage = new StringBuilder();
    sortedMessage.append(getString(R.string.sorted_strings))
            .append(" ");
    int size = (int) sortedStringVector.size();
    for (int i = 0; i < size; ++i) {
        sortedMessage.append(sortedStringVector.get(i))
                .append(",");
    }
    mTextviewSortedStrings.setText(sortedMessage.toString());
}

And there you have it! That’s all you need to do to use Android with C++ code through the JNI and NDK. As mentioned earlier, I have a full sample app located in my repo. I hope to shortly have an update which fixes that SWIG task failing to run correctly from Android Studio.

Update - August 6, 2015

I’ve put this in a comment, but I’ll also put it here for clarity. Unfortunately, the runSwig problem, I haven’t had time to try to play around with it - but maybe Android Studio 1.3 fixes it (just haven’t tested it in depth - if anyone can and can comment, I would really appreciate it!).

Here are the from-scratch build steps (if you were going to build in your home directory on Linux/Mac):

cd ~/
git clone https://github.com/sureshjoshi/android-ndk-swig-example.git
[Open in Android Studio and let Gradle run and the project get indexed - this creates local.properties and the sdk.dir]
cd android-ndk-swig-example/NDKExample
./gradlew runSwig
(In Android Studio) Make Project

That should build error-free (assuming you’ve set up your ndk.dir in local.properties). I’m going to test Android Studio 1.3 and later versions to see when the JetBrains CLion plugin can work error-free, so we can remove the NDK dependency.

Update - January 5, 2016

If you’re up for using the experimental Gradle plugin for Android Studio, I’ve updated my Github repo (on the develop branch https://github.com/sureshjoshi/android-ndk-swig-example/tree/develop) to use Android Studio 1.5, and the latest experimental Gradle plugin (0.6.0-alpha3).

With this new plugin, there are a few changes required to the build.gradle files… First, in your main Gradle file, use the new plugin:

classpath "com.android.tools.build:gradle-experimental:0.6.0-alpha3"

And, in your app’s build.gradle, you need to update to the new ‘model’ syntax (I don’t know if I like it):

apply plugin: 'com.android.model.application'

// Manifest version information
def versionMajor = 1
def versionMinor = 1
def versionPatch = 0

model {
    android {
        compileSdkVersion = 22
        buildToolsVersion = "22.0.1"

        defaultConfig.with {
            applicationId = "com.sureshjoshi.android.ndkexample"
            minSdkVersion.apiLevel = 15
            targetSdkVersion.apiLevel = 22
            versionCode = versionMajor * 10000 + versionMinor * 100 + versionPatch
            versionName = "${versionMajor}.${versionMinor}.${versionPatch}"
        }
    }

    android.buildTypes {
        release {
            minifyEnabled = false
        }
    }

    android.ndk {
        moduleName = "SeePlusPlus" // Name of C++ module (i.e. libSeePlusPlus)
        cppFlags.add("-std=c++11") // Add provisions to allow C++11 functionality
        cppFlags.add("-fexceptions")
        stl = "gnustl_shared" // Which STL library to use: gnustl or stlport
    }
}

From here, you can follow the build steps from the previous update and you should be good to go.

However, if you want to spice things up a little - you could also use Android Studio’s ‘native’ keyword to auto-generate your C++ to Java wrappers:

NDK-AutoGen

Please note, though, if you try to use SWIG AND the native auto-generation, the auto-generated code will be placed in the incorrect location in your SeePlusPlus_wrap.cxx file (it manually needs to be moved inside the “extern C” block.

NDK-AutoGen-Moved

Feature Photo credit: JD Hancock / Foter / CC BY