Flutter and embedded UIs

Hi there. It has been quite a while since my last post.
In the meantime, a bunch of things happened. Corona is over 🥳😉, Apple released a monitor for the masses (webcam cough) and some crazy guy from the east decided to question the European borders 🥴.
But that shall not be the topic of this blog post. Today’s topic is: Is Flutter a good candidate for embedded UIs?
The answer: Yes!
See you next time …

but wait. Let’s explain a bit.

What is this Flutter you are talking about?

Flutter is a cross-platform UI framework created and driven by Google. The first target group was mobile developers, so it provided Android and iOS as potential target platforms. After that, Google extended Flutter to target the Web and (currently in the process of going into the stable channel) desktops (Windows, Mac, Linux).
Flutter uses Dart as a programming language. It is a very modern but reduced (this is a good thing) programming language that is easy to learn while being fast due to native compilation.
The way Flutter is architected allows it to be used in all those different environments without touching the very core of it.

Flutter architecture - https://docs.flutter.dev/resources/architectural-overview

The fundamental part is that Flutter does all the drawing itself - not relying on any platform-specific UI framework underneath. Other cross-platform frameworks (like Xamarin or React native) do that while basically "remote controlling" the platform native UI. By using a direct rendering approach this dependency is completely gone. This makes porting Flutter much more lightweight compared to other cross-platform technologies. The platform-specific adaption happens in so-called embedders. Those embedders provide the bridge between the Flutter engine and the platform itself.

And embedded?

The fact that Flutter is built for mobile (it has to be very fast and resource-efficient) and that its architecture allows a rather easy (at least compared to other UI frameworks 😉) porting made it possible for other companies to start working on an embedder for embedded Linux.
Most of the action in that space is happening by Toyota and Sony.
Toyota’s approach is to include Flutter in the in-vehicle infotainment by providing Yocto layers that include Flutter capabilities. Sony’s approach is to provide an embedder that can be cross-compiled for the embedded system it is used in. Sony provides different types of embedders to integrate with the different window / graphic backends that are used in embedded Linux systems (Wayland, DRM).

Flutter embedded Linux overview - https://github.com/sony/flutter-embedded-linux

Is this the future?

It might sound harsh but in my opinion C++ is not the right programming language to develop user interfaces. I already did an extensive post about that topic here. Qt - what we currently use for embedded UIs - tries to cover that C++ madness but there is no way to hide that completely. All that countless hours that are simply wasted because of the unnecessary complexity C++ brings to the table sigh.

Additionally by using a technology like Qt we have to build smartphone like UI elements and behavior ourselves (like hero animations) or simply can’t use technologies that are already well established in the mobile development community (like Lottie) because they are simply not available or do not support the expected featureset.
Our designers incorporate more and more UX elements from the smartphone world because that is what the users already know and expect.

The sugar on the top that Flutter brings to the table are its developer tools. It is insane what you get without any additional tooling or setup by just launching a Flutter application. You can have a very deep look at what the performance characteristics of your application are, why that one frame took so long and where the engine had to compile shaders. You can even start CPU and memory profiling sessions right from the developer tools. I have never seen such an integrated and feature rich developer tools that just works out of the box.

Dart DevTools - https://docs.flutter.dev/development/tools/devtools/overview

And let’s not forget about the licensing cost. Our business is heavily driven by per unit costs and having a license that has royalties bound to the number of units you create is a big disadvantage.

Of course “nothing is free”. It’s like that saying:

Linux is only free if your time has no value

Using an open source solution also has its downsides. There is no one that has a contractual liability to provide fixes for bugs or is even willing to fix / extend something if you throw money at the problem.

Flutter additionally has the “problem” that it is very young. Things are changing fast, and you might run into bugs more likely as with older technologies like Qt.

But given all of this. The distribution of Flutter, the fast-growing community, the mobile driven feature set, the resource optimized engine, the developer tools and the crazy rich world of available packages while not having a per unit based royalty (none at all) and using a high level programming language to use all of this makes this a really nice package.

I did some initial testing on our hardware porting the Flutter engine and Sony’s embedder over to our system to run a small Flutter app that shows a list of recipes while using some performance relevant features (like page transitions and hero animations) as well as including some features that are simply not available using Qt (at least not to that extent, Lottie) I got a fantastic running UI.
The developer tools have some problems in our IT managed environment, but I tried this at home and there it worked out of the box. All the Flutter developer tools right in front of me.

So is this the future? I don’t know for sure but from what I can see and from the experiments I already did I can say that this technology has the potential to disrupt the embedded UI world.

I will continue experimenting with it and push for an (experimental) adaption inside BSH because I think that this has a true potential to bring UI development to the next level.