Developer Commentary: Philosophy, Evolution of TrueOS/Lumina, and Other Thoughts.
by Tim | Apr 19, 2017 | History | 1 comment
TrueOS and Lumina Developer Ken Moore recently took to Discourse to answer a few questions about the development philosophy for TrueOS/Lumina and provide some insight why these projects have evolved the way they did. I’ve edited and reproduced his answers here, but please feel free to browse, contribute to the original threads, or make new threads for questions on Discourse:
Thread 1: https://discourse.trueos.org/t/developer-blog-entries/1221/16
Thread 2: https://discourse.trueos.org/t/upstream-packages-will-start-breaking-the-future-of-trueos/1252/6
Philosophy of Development
No project is an island.
Every single project needs or uses some other external utility, library, communications format, standards compliance, and more in order to be useful. This is especially true with graphical applications, since they need to utilize the current: display system (X11/Wayland), audio system (OSS, ALSA, PulseAudio, SNDIO, etc.), and more.
A static project is typically a dead project.
A project needs regular upkeep and maintenance to ensure it continues to build and run with the current ecosystem of libraries and utilities, even if the project has no considerable changes to the code base or feature set. The ecosystem is a moving target. If a project stops upkeep, the passage of time increases the odds the project ceases to build and run.
“Upstream” decisions can have drastic consequences on your project.
Through no fault of yours, your project can be rendered obsolete or broken by changing standards in the global ecosystem that affect your project’s dependencies. For example, look at the the audio subsystem “wars”. ALSA vs PulseAudio is the fight that people are more familiar with (ALSA lost), but do you remember the “JACK” audio system? JACK was soundly beaten by ALSA, even though Jack was/is better at low-latency functionality. However, JACK losing to ALSA meant nearly all development/support for Jack has disappeared. Furthermore, all those audio recording/mixing utilities which used Jack are rendered nearly unusable by this upstream decision.
Operating system focus is key
What OS is the project originally designed for? This determines how the “upstream” dependencies list appears and which “heartbeat” to monitor. For Linux-based utilities like Okular or Firefox, this means the only supported audio subsystem is PulseAudio (even if we have OSS on FreeBSD which works 10,000x better), because PulseAudio is the only audio system officially supported on Linux, the app’s primary development platform. For FreeBSD and TrueOS, this presents a massive challenge to continue to support Okular and Firefox, as those applications are now essentially incompatible with the native subsystems used on our OS. While not impossible to support, it takes a lot of time and effort to “port” those apps over to work with FreeBSD, often introducing bugs and other usability issues along the way.
Evolution of PC-BSD, Lumina, and TrueOS
With these principals in mind – lets look at PC-BSD, Lumina, and TrueOS.
PC-BSD was largely designed around KDE on FreeBSD. KDE/Plasma5 has been available for Linux OS’s for well over a year, but is still not generally available on FreeBSD. It is still tucked away in the experimental “area51” repository where people are trying to get it working first.
What about KDE4 on FreeBSD?
This runs into the problem with static projects discussed above. It is basically a dead project, as it is replaced by version 5. This means it is continuing to accumulate security vulnerabilities like hald, OS-incompatibilities, and is relying on other dead or dying projects for critical functionality (policykit/consolekit – both replaced by systemd on Linux). For PC-BSD 10.x, we tried to mitigate a lot of the changes in KDE by making our OS configuration utilities desktop environment (DE) agnostic, but that only gave us a short reprieve from the underlying decay. It also introduced a a “split-brain” framework where our functional and supported tools were available simultaneously with the DE’s dysfunctional tools.
As a developer with PC-BSD for a long time, and a tester from nearly the beginning of the project, I was keenly aware the “winds of change” were blowing in the open-source ecosystem. This was especially true in the desktop environment subsystem, long before systemd was announced. Because of this, I started working on a FreeBSD based desktop environment that did not rely on all these Linux based subsystems and frameworks. This was so we could ensure a FreeBSD based desktop OS could outlast other Linux based desktop implementations which now regularly have their fundamental frameworks ripped out and replaced.
All of these ecosystem changes finally came to a head for us near the beginning of 2016. KDE4 was starting to deteriorate underneath us, and the FreeBSD “Release” branch would never allow us to compete with the rate of graphics driver or standards changes coming out of the Linux camp.
In response, we started rolling monthly “snapshots” of PC-BSD based on the FreeBSD “Current” branch (also to prepare for FreeBSD 11.0). Surprisingly, this “rolling” release model for monthly snapshots was solving a lot of the longstanding issues with PC-BSD. There are newer drivers. Bugfixes are available in a timely manner. Critically, the project now has a single “supported version”, which makes better use of our limited developer hours.
At the same time, we started switching to Lumina instead of using KDE by default. During this switch, we discovered much of what we were working on started “clicking” together. The interface was stable. There is only a single set of OS configuration utilities. The number of “upstream” bugs from Linux effecting us dropped considerably. Is Lumina as “finished” as KDE? Definitely not! However, critical issues are now resolved simply and permanently.
The Rename and Next Steps
With all of these changes and the lack of a clear “upgrade” path from PC-BSD to the new systems, we decided it was necessary to change the project itself (name and all). To us, this was the only way to ensure people were aware of the differences, and that TrueOS really is a different kind of project from PC-BSD. Note this was not a “hostile takeover” of the PC-BSD project by rabid FreeBSD fanatics. This was more a refocusing of the PC-BSD project into something that could ensure longevity and reliability for the foreseeable future.
Does TrueOS have bugs and issues? Of course! That is the nature of “rolling” with upstream changes all the time. Not only do you always get the latest version of something (a good thing), you also find yourself on the “front line” for finding and reporting bugs in those same applications (a bad thing if you like consistency or stability). What you are also seeing is just how much “churn” happens in the open-source ecosystem at any given time. Although, I personally think the “churn” is a bit more aggressive now. Because the changeover to a rolling model is still fairly fresh, the developers and contributors are still adjusting to some of these new realities and working on implementing contingency frameworks and/or automated QA tests to help mitigate some of the churn. However, all this takes time.
We are devoted to providing our users (and ourselves – don’t forget we use TrueOS every day too!) a stable, reliable, and secure experience. Please be patient as we continue striving toward this goal in the best way possible, not just doing what works for the moment, but the project’s future too.
Lumina Desktop Utilities: Why so many?
There are a few reasons to develop these small utilities:
Missing functionality in TrueOS
Sometimes we cannot find an open-source application to add to the TrueOS base install which fits our criteria. However, filling the functionality is critical. The Insight file manager (lumina-fm) and lumina-archiver are two good examples of tools developed to “fill a hole” in TrueOS.
Sometimes a utility we’ve been using for a long time starts losing functionality. This usually occurs when the upstream project starts expecting more dependencies either unavailable on FreeBSD or add numerous undesired. When this happens, we look for a replacement or write one ourselves (see first point).
Note this process sometimes starts quite a while in advance of the current app completely breaking. We try to proactively anticipate development trends for any utility TrueOS relies on, so if the time comes we have no glaring holes in our project.
lumina-pdf is a utility that falls into this camp. okular may currently work, but we expect that to change soon. While qpdfviewer is currently a valid replacement for okular, moving to Wayland will result in that utility becoming nonfunctional. Because Wayland integration is “down the road”, lumina-pdf is on the back burner.
Developer “Fun Time”
As a developer, it is very useful to take those small bits of extra time to do something different. This provides a “playground” to re-think an application, experiment with new libraries or features, and grow as a developer. If one of these projects becomes something useful, then more “official” time is used to polish it up and release it. However, not everything makes the cut.
lumina-mediaplayer is a good example. It began as a hobby project to replace pithos as my Pandora radio application. Pithos is fragile with LibreSSL versions and GNOME3 integrations, which frustrates me. Through tinkering, lumina-mediaplayer has gradually become useful to more people than myself. The local-file playback functionality is largely to provide baseline functionality for the user who doesn’t want Pandora. It also fills a niche as a “fallback” utility in case somebody doesn’t need the depth that VLC or another “full-featured” media player provides.
The TrueOS stance on external applications:
Where TrueOS is concerned, it is not our responsibility to ensure the availability or functionality of Linux-based apps. This is more the responsibility of the FreeBSD community and ports committers. What we focus on is the reliability, stability, and dependencies of the “base” applications that we pre-install on systems. This ensures the most general use-cases are reliable.
Here are a few types of application functionality we generally review:
– General Applications –
File Manager – Can you browse/open files graphically?
Terminal – Can you get a virtual CLI?
Web Browser – Can you get to the internet?
Email Client – As old fashioned developers, we like working clients.
– File Support Applications –
PDF Viewer – Can you download, view, and print official documents and/or give presentations?
Text editor – Both CLI and graphical.
Note these applications don’t need to be massive or cover every possible file type or use-case. Rather, they need to cover the basics (like an office workstation or web-user) and be functional all the time.
Concurrently, the “desktop” functionality needs consideration:
Is it possible to login with an arbitrary user account (AD/LDAP/Local)?
Can the user launch or manage applications easily?
Are system notifications and status easily visible? (battery status, network connectivity, etc)
Do TrueOS + Lumina run on low-end systems or systems without GPU acceleration?
These points are what we consider the “core” functionality of TrueOS – all other use cases (audio/video production, graphic design, radio broadcasting, [your-usage-here]) are not seen as the responsibility of the TrueOS team itself, but more the larger FreeBSD community to ensure “Application X” is available and functional.
Raul Rodriguez on April 20, 2017 at 1:55 am
Sorry you may have felt you had to write this. But I am glad you did. I knew sooner or later linux apps would become a problem for BSD. I sometimes felt I should switch to linux because of that. I would have hated that. This blog will make me stick it out. I always felt more apps should be written as native BSD. I know it ain’t easy. I wish I could contribute, but don’t BSD nor linux. Grew up on windows, I knew enough just to survive that OS. I think your doing a great job. Thanks for all you do.