My relationship with splash screens is… complicated. I love them, they’re often quite pretty to look at, they also settle the mood for the app you’re gonna use, they’re often easy to implement for the value they give: they’re great both for the users and for the devs.

But sometimes, I hate them. Especially when I have to stare at them for a minimum of 10 seconds while nothing happens. Malus points for those with a loading animation that accentuates this frustration of “nothing’s happening but a lot is happening and you never know what”. It used to be a major issue for large React Native apps, but fortunately, these are old times now, since Hermes has been the default JavaScript engine for any React Native app since v0.70. But did it really improve performance that much…? Can we measure the difference? That’s where Flashlight will come in handy!

‍

But first, what’s Hermes? Now we do Greek mythology?

Hermes is an engine you can use on Android and iOS React Native apps. As explained in this article, it affects 3 parameters to improve the app usage :

• Time To Interact (TTI), the startup time of the app, which was exactly the issue with splash screens I was describing
• Memory usage (that can have repercussions on other aspects, like battery usage)
• APK bundle size

To improve these 3 aspects, Hermes basically does the JavaScript parsing and compiling in the build phase, instead of letting the app do that when it launches on your device! No more JavaScript in the final app though (Hermes compiled it in byte code) but the startup time of the app should be shorter. Yay!

‍

Hermes team claims they improved performance regarding TTI, but by how much? How come it did even have an impact on startup time? We’re gonna have to shed some light on this, thanks to Flashlight!

‍

Flashlight? Why do you need light now?

Flashlight is a free and open source tool we’re developing at BAM, started by Alexandre Moureaux with the goal of measuring performance in mobile applications easily. It’s like Google’s Lighthouse except it’s for apps on mobiles, hence the name: a portable Lighthouse! Flashlight lets you measure in real time on a connected device how your app performs : FPS, CPU usage, etc. You can also generate an analysis report for your app on a user journey you define, thanks to Maestro’s Flows. Flashlight also has an open beta for a cloud version of the tool, using an AWS device farm to generate your performance report. Don’t hesitate to stop by our Slack to give your feedback or ask questions about anything Flashlight!

‍

What and how to test performance

Okay now we’re all set, let’s measure!

For this example, I set up a new React Native project from scratch. To better reflect the reality of a mobile app, I bloated my freshly new project with a lot of useless lines in several files. Doing this was necessary to trick Babel into keeping these lines. You can use the snippet below then run it with ++code>npx ts-node ./snippet.ts++/code> to do the same (but do you really want to add this purposeless code…? Oh well, here’s the full version of it then.)

Then of course in the App.tsx file, we’re gonna have to let it check a non existent native module or else it would be discarded from the APK bundle:

Final touch would be to enable Hermes in ++code>gradle.properties++/code> when needed for my purposes:

‍

After all that, I built 4 different APK bundles :

• One without Hermes enabled, and without the bloated code, that I’ll name “WithoutHermesWithoutCode”
• One without Hermes enabled but with the bloated code, that will be “WithoutHermesWithCode”
• One with Hermes enabled and without the bloated code, that I’ll name “WithHermesWithoutCode”
• And finally one with Hermes enabled and with the bloated code, “WithHermesWithCode”

‍

JS Bundle sizes

‍

Before even opening them, we can inspect the JavaScript bundle size in the APKs. As a reminder, you can find them if you unzip your APK, under ++code>assets/index.android.bundle++/code>. As announced by Hermes, the bundle size is smaller whether there’s bloated code or not, up to a 25% difference.

‍

Performance through Flashlight

And now the most interesting part: how do our 4 APKs compare performance wise? We’re gonna use Flashlight to tell! We used the cloud version of Flashlight to measure everything and there wasn’t even much to do to have our results: went onto the website, maybe read the docs to know what to do, uploaded one APK at a time, indicated a text I would see right after startup (”See Your Changes”) and voilà we’re all set. To add a bit more details, the device type we use on our AWS device farm is a Samsung A10S, an entry level device meaning lower performance but also a higher market share, reflecting more the reality of mobile usage.

You can find the Flashlight reports combined into one here!

‍

TTI

Since our test protocol finishes measuring when we see the first screen of the app, we can use the average test runtime measure to check. It’s basically a TTI then!

Let’s get the elephant out of the room first. We notice the WithoutHermesWithCode app to have a startup time nearly 10 times longer than the other ones. While it implies Hermes seems not affected by the code (we’ll see that in details later), we may wonder why the bloated app can take that much more time to start… Maybe something is saturated and that’s causing us more waiting times than needed?

‍

CPU Usage

And we notice CPU usage to be constantly saturated in the WithoutHermesWithCode app. A whole core of the CPU is even taken by the JS thread (which is the main source of CPU usage in our report, hence I won’t show the other ones)!

What we don’t see though is the startup time is so long that the graph, being 10 seconds long, doesn’t capture the whole sequence… At least it won’t be saturated forever, but that’s still a bad thing to even be saturated for so long. That explains the terrible score of 2 out of 100.

Additionally, you may have noticed the total CPU usage graph going over 100% for the WithoutHermesWithCode app and may be wondering why our device for testing is not melting already. The Samsung A10S has 8 cores in its CPU and Flashlight indicates 100% per core used: since the JS thread is using the whole core to the max, another one is used for the other threads. Theoretically, we could reach a maximum of 800% CPU usage (but our device may be really melting at this point…)

Another thing we can notice on this graph is that even the WithoutHermesWithoutCode app uses more CPU at the beginning than the Hermes versions… How? This may be explained by the fact that the device still has to parse and compile that JS code before doing anything else!

Hermes team is also claiming it reduces memory usage, but by how much? Luckily, Flashlight can also measure it!

‍

RAM usage

Similarly to the CPU usage, we notice the WithoutHermesWithCode app is not like the others and just hoards RAM continuously. The nice surprise here is the WithHermesWithCode app taking just a bit more RAM than the other apps without the bloated code, and it doesn’t increase over time during the startup!

‍

Does the bundle size correlates to performance as well?

Thanks to our reports and using the APK JS Bundle Size I wrote earlier, we can try to draw a link between bundle size and performance and how Hermes scales better

‍

If we doubted of it before, now we’re certain: in a real situation (and not a simple example like this one), since Hermes avoids the parse and compile phase at the startup of the app, even if our app is big, we would not see much of a difference during startup. We can also notice for bigger bundles Hermes optimized RAM usage.

You can look at the reports’ videos below!

‍

‍

So did Hermes really improve performance?

Seeing the graphs from above, we can see not only Hermes reduced the startup time of a React Native app, but also the memory usage and the bundle size, as they claimed. Another great effect of better performances that we tend to forget is lower energy consumption: if we don’t overload the CPU and RAM, we use less battery and all of this is better for our environment! At last, I’m less frustrated on my splash screen (though there was no splash screen in my example app…)! You need to know however that Hermes won’t solve all your performance issues obviously (for example, a heavy image in your app will still cause performance issues, with or without Hermes). Sorry if this ruined your day. Finally, Hermes may break some very specific features used by some packages. I haven’t encountered such issues before but it’s worth mentioning!

Now it’s your turn! Use Flashlight to analyse where in your app there may be performance issues and don’t hesitate to share the results on our Slack or our socials! We’ll also gladly listen to feedback or issues you encountered!