TTS, Genode, Haiku

[About us] : Who we are, why the musings about Genode from us Haiku enthusiasts over at TTS
[Questions] : If they get answers, that page might evolve into a (small) special-purpose FAQ :-)
[Roadmap / To-Dos] : A dashboard of sorts, about my current and future milestones

Migrating 9jan19

My next posts should, in theory, be found on genodians.org

However it looks like I can't find my way out of a wet paper bag with git; no big surprise with me..

It won't let me modify my author.txt file:

# git diff
diff --git a/author.txt b/author.txt
index 51ef602..b4be640 100644
--- a/author.txt
+++ b/author.txt
@@ -1 +1,3 @@
 I write radio automation software for a living. Very interested in Genode, and now trying to port CommandCenter (and companion apps) to Genode.
+
+*Company* [https://www.tunetrackersystems.com - TuneTracker Systems LLC]

# git commit
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
        modified:   author.txt

no changes added to commit

Also tried git pull, git push ..etc to no effect. Hopefully that will be resolved soon.

Anyway if all goes well, Fossil will get a "git" compatibility back-end in a few months, and I'll be able to pilot git repos from the comfort of Fossil. Fingers crossed.

Audio, commo 6jan19

Started shooting photos of the screen on my test rig today. Those will make for nice memories once I (hopefully) complete this haiku-on-genode project. I'm now realizing what I embarked upon though: not just porting a graphical toolkit, but (part of) an operating system as well -- most classes rely on BMessage, which relies on write_port() / read_port() system calls ..etc; so quite a few bridges to build and gaps to fill.

Anyway, this past week's emails fostered more reflexion:

Audio

For our (TTS) needs, a simple stereo output with medium -- or even high -- latency is good enough at this point. Our focus will be not so much on the audio "sink" as on the "source", i.e. flexibility of file formats: our stations use mp3, mp2, ogg/vorbis, WAV files and so on. On Haiku we relied on ffmpeg/lib-av-something; thanks to abstraction through the BMediaFile class, the application didn't have to care about where the decoded data came from.

Later on we'll focus more on the 'chain' which leads to the sink: we'll need the ability to insert some audio "nodes" that intercept audio traffic and modify it (for various audio effects) before it reaches the mixer/sink. But that's farther down the road.

If some reflexion about audio latency is in order though, maybe the Be newsletter archive (see below) could help... I'm not overly familiar with the depths of the Media Kit (kind of a paradox for someone writing audio broadcasting apps); but I remember e.g. that each BMediaNode has accessors to report its internal Latency(), one's downstream latency, and so on. That way, when building an audio processing chain you can make informed decisions.

It was commonly agreed that the Be (and Haiku) Media Kit was the least mature of the kits though. Hard to get into, hard to code for and get right even when experienced. Genode would do well to tread carefully there.

Just now I see that libav and avplay are being moved from libports to "world"; will look into doing a git clone and follow the readme soon.

A blast from the past : The Be newsletters of old

Back when dinosaurs roamed the Earth, Be Inc published an email newsletter about BeOS, more or less every week. There might be some ideas in there for Genode:

If memory serves, it had a recurrent structure:

an article for advanced developers, with a bit of sample code
an article for beginner developers (new to BeOS, not new to C++), with some ready-to-compile and execute sample code.
some strategic commentary from Jean Louis Gassée.

See for instance Issue 5-1 or this longish one: Issue 3-31

Given how time consuming writing a newsletter is though, it might make sense to have those written not by the 'core' Genode team but by a third party who is very familiar with Genode.

(bonus link: http://bedope.com/)

Grep'ing the source, indexing the source

Learned this trick a couple years ago; probably common knowledge, but in case it helps others too:

A few years ago I found out that grep -R SomeSymbolNameEtcHere . is blazingly fast compared to other solutions, because there is no fork()ing involved: there is exactly one instance of /bin/grep running, at full speed, from the beginning to the end of the search.

In fact, when running it for the second time around and the disk data is cached in the OS "file cache", it returns results very very fast, even on a quite big tree like Genode.

There is something complementary, that the Genode team should look into though: talk to someone who hosts a cross-reference indexer, so that they index the Genode source, and keep the indexing current.

For example, Haiku is indexed (together with FreeBSD, Fuschia..) over here: http://xref.plausible.coop/source

The ability to click any symbol and go to its definition (or go to references to it) is invaluable, for experienced devs doing hardcore work and for newbie explorers alike. More convenient than even a recursive grep.

Maybe the indexer can even be hosted on the genode.org server itself, no idea.

HCL

Should file these 4 entries to the Hardware Compatibility List later:

On all four of these, the laptop's touchpad and keyboard work. All four of those work with the "make run/demo KERNEL=nova" scenario; two also support the newer Sculpt scenario, including wired networking. Not much success with acpica.

Thinkpad R61 (audio ok)
Thinkpad T410
Thinkpad T530 (also runs Sculpt/VC) (audio ok)
Asus F2A55M with AMD CPU (also runs Sculpt/VC) (no audio ??)

Various

USB keyboard troubles are mostly gone: was just a matter of "don't touch what you don't understand" :-).. Re-instated a part of the .run script I had commented out, and now I can use F1/F2/NumLock again, and Nitpicker is no longer locked into xray mode. Interestingly, the LEDs are still blinking. I see in the .run file they are directed from a ROM. Might look at that later.
During the week, also noted Genode would 'freeze' if left idle for a few hours. I'm not reading too much into that incident, at this point -- maybe it was just the USB mouse driver that stopped working and the OS itself was fine.
Will watch for it to happen again now that the keyboard is working (see above), should file a ticket if it persists of course. Need to further inspect my .run script first.
Culture shock: no backtrace in case of segfault! :-) -- experienced my first seg-fault during my workflow this week, and... the tracing in case of crash is austere :-) No backtrace:
```
[init -> hello_client] BWindow Ctor for: Zwei
	no RM attachment (READ pf_addr=0x0 pf_ip=0x1000ab7 from
	 pager_object: pd='init -> hello_client' thread='ep')
```
It just says there was a NULL indirection, at instruction-pointer such and such. Wonder if one can guess which function/method is involved, from that info or some other hints? I have so much left to read about Genode, maybe there is a way to get more info and I just didn't run into it yet.
UPDATE: d'oh. Now I remember, when building the GCC 6.3 toolchain last month, I commented out building of GDB. When I get around to re-building the tool chain (doing a "clean" build) I'll make sure to include GDB this time -- that must be what is invoked for userland crashes, and reporting to the genode log..
Anyway I'm adding massive Genode::log() tracing to my code, helps a lot.

Topics 1jan19

Part of me really wants to focus on the code and remain quiet until accomplishing someting tangible to talk about. But looking at access stats I see there's a few accesses per day. No IP or anything is provided so no idea if those are random/unrelated hits or not. Anyway, might warrant posting a short item every now and then. So I'm thinking of those topics for future posts:

USB-HID: been using the dde_linux one, in my latest scenarios: mouse works like a champ but keyboard does not (what with the funny "blinking" led's). Not a problem for now, all my tests are mouse driven.
ACPI: this is the one thing I completely failed to make work on any machine here -- no power-off/reset is possible, have to hard cycle when testing on bare metal; libports/src/apps/acpica/README has some detailed info which might help me; will look into why I get warnings/errors loading acpica in Sculpt. Maybe adding acpica to my own scenarios and tweaking the configuration will do the trick.
I keep coming back to this line of thought, like a broken record, about the importance of Genode: when some userland is broken/missing, most programmers can add the missing bits; or they can look for a suitable replacement among a cornucopia of "off the shelf" userland components and carry on with their life. However, when your trouble is with the kernelland and you look for a new kernel (or as is the case with Genode, a "break with the original sin and move most of kernel stuff to userland" framework), your're out of luck, as there is such a dearth of usable, well designed kernels out there. Probably because it's orders of magnitude more difficult to produce than userland is. Hence my feeling lucky that Genode exists :-)

Status-wise, I have HaikuControlLook.cpp compiling and (almost) linnking; hoping to display an actual BCheckBox in a Nitpicker virt. framebuffer soon. Then it'll be the turn of the BView arborescence, then an input-server like class, and then the (real) fun begins.

Some roadmap clues 27dec18

File system with attributes

Stumbled upon this, doing some early legwork looking for a future FS to use for our software:

https://www.haiku-os.org/guides/daily-tasks/access_bfs_with_fuse

That page makes it sound like it is not rocket science to compile and use FUSE/BFS on another OS (including Genode's noux ?).

Also noticed this post from @ehmry:

As for my personal roadmap, in the past few weeks I've begun work on a distributed non-hierarchal storage layer. The intention is to support a subset of the de facto file-system semantics with quick composition using the formalism of set logic. This is a pivot of my roadmap from a year ago and reuses code and ideas from another storage project of mine, so progress is swift. The use-case is distributed and redundant storage for things like my permanent music collection as well as the package depot.

Even if I'm somewhat successful with FUSE ..etc, I'll still follow the above project for a 'native' Genode FS: it has potential to be most robust, better integrated. If in addition to that it also sports attributes (or allows extending with plugins ..etc to support them) I'll be a most enthusiastic supporter!

The 'non-hierarchal' bit somewhat reminds me of Be Inc's quest for a file system, that initially led them to a database-like (instead of conventional) file system. I was not yet in the Be community at that time but followed it on the web, I think it was around BeOS AAPR or DR1/DR2 era. Fun times.

NitPicker and BWindow

Looks like os/src/test/nitpicker/main.cc is all I needed to get started. The coding style and paradigm are unfamiliar of course, but I'm having fun discovering, creating boot images with my experiments and so on. Still can't get over how the image.elf file is a couple MB big, and boots almost in the blink of an eye.

Curiously, I get this warning in qemu; sounds a bit ominous and won't go away even if I increase platform_drv caps from 800 to 1800:

[init -> platform_drv] Warning: Out_of_caps during Device_component construction

Seems to be without any (visible) consequence though. Anyway, next thing I know, I'm going to ask for a "screenshot" feature so that I can capture interim states of my progress, for posterity *g*.

RFA/Request for articles 27dec18

There's talks of the Genode team doing a bit of micro-blogging every now and then, not just one huge quarterly update with release-notes. Here's my "thumbs up", eager as I am for Genode info. So I'll submit a couple ideas if they (or anyone) need inspiration:

Showing off: Interesting use-cases, with screenshots.
Reasoning: it seems quite a few are using Sculpt as a day to day OS; and that's thanks both to its ability to do the same things others can do, and to its being able to do unique things; I'm interested in both the 'mundane' and the out-of-the-ordinary examples.. Might also pick up ideas about which 'simple' apps still need to be coded (and yes, I enjoy coding even 'boring' apps, so long as they have a GUI; I'm such a misfit).
Native apps vs. ported apps: which way is best (and why) for creating ergonomic, high performance Genode applications.
Reasoning: I'm still getting to grips with what Genode is and where it wants to be in the future; maybe the above is a good way to reframe that question? It is usually accepted that 'native' apps have an inherent advantage in terms of look and feel, performance; whereas cross-platform apps benefit from a larger user base; still, I lean toward the former, provided the OS has an outstanding API/SDK; any thoughts ?
GUI/UX: here's what we at Genode have learned to like and dislike
Reasoning: reading "I'm probably too opinionated about user interfaces to be very helpful here." makes me want to learn more :-) I'm no stranger to GUI controversies, lots of them over the years between this coder and his TTS "head honcho"! I've come to consider a set of ergonomy "best practices" over the years so as to make my apps consistent and to improve 'discoverability'. They might be too conservative (dull?) though. Curious to hear from others.

None of these need be of "magnum opus" length: keep it fun and quick to write so as to sustain, say, at least a weekly pace.

Showing how to use Sculpt and Genode to raise interest 26dec18

Before diving into techie stuff (coming soon), I'll give a bit of feedback on a particular aspect discussed by @nfeske on the mailing-list:

Should we prioritize documenting the use of Sculpt or the use of Genode? The former would look only at the building blocks (but not inside them). Such documentation can be in the form of light-hearted and practical use-case anecdotes, which are fun to read and immediately applicable by Sculpt users without any programming.

Speaking as a newcomer, and wanting to attract more newcomers, I'd recommend spending (slightly) more time on the former, documenting what the building blocks can do. From the simple to the elaborate.

As to documenting how those use cases are implemented: obviously we want the Genode developer community to thrive and help them do their programming job; but 1) developers must know something exists, before wishing to use it or build upon it -- Sculpt scenarios documentation is a good way to do that. And 2) even when it comes to getting up to speed coding things, I find myself grep'ping dozens of times per day... And learning a lot that way. Developers can in many cases find their way even if the documentation lags behind.

Your mileage may vary, but that's it for me as of today.

The year of Genode for TTS ? 22dec18

Throughout 2018 our hope was to continue deploying TTS software on Haiku as-is, as we've been doing for years. I was aware of Genode, had looked at several pages of genode.org, but thought it was still overkill for our needs. Let's just keep it on our radar screen as a remote 'plan B', I thought.

That state of mind evolved going into December though. Already in Q32018, kernel-land problems were overwhelming. Then they became unbearable, and finally we went into full brainstorm mode. It now looks like we might be able to benefit from micro-kernel stability, while not burning our bridges: if we can pull off making a solid Haiku compatibility layer on Genode, then Haiku applications can be written targetting either Haiku proper or Genode, with little adaptation work needed to support both. In theory. Anyway that's what we are hoping for.

If all goes well this year will see the realization of that, and 2019 will see Command Center and hopefully more TTS apps (and maybe even other, non-TTS apps?) running like a champ on both target platforms.

Early work has started: on one end, I'm stubbing out some classes in the app, support, storage, interface kits and so on. On the other end, I'm examining and tweaking Genode 'gem' GUI code. Will slowly get both ends closer to each other until I obtain a nascent haiku.lib.so.

William Blum 20dec18

William Blum passed away a few days ago... R.I.P., you'll be missed.