Last updated: 19jun22
(the official TTS status page is over at http://tunetrackersystems.com/status -- that's the "go to" page for customers, this page below is mostly for internal use and task tracking)
Apps: [+] AutoCast [ ] LiveEncoder [+] ArmyKnife [+] MakeMyDay [+] Lightning InterfaceKit: [x] Mouse interaction [+] Keyboard interaction [x] Window manager, window decorations (bare-bones->yellow tabs->full-fledged) StorageKit: [X] stat/mkdir/creat/open/read/write/close [x] BFile/BDirectory/BPath/BVolume [X] BNode.ReadAttrString() (attributes support) [x] BQuery, indexing (on names and attributes) Storage Drivers: [x] NTFS (FUSe) r/W, including reading/writing attributes [.] BFS (FUSe) r/o, including reading attrs ? (to Rx logs, mp3 files, etc) [x] FUSe in vfs_server (once downstream-FUSe is free of libc dependancies) [.] AHCI: detect and mount /boot (first NTFS partition ?), and maybe BFS [ ] USB mass storage (for Rx'ing mp3 files) Other Drivers: [+] HDAudio (on M73 ? on FM2A55 ?) [ ] Serial (UART) (for switchers) MediaKit: [+] clean-up/commit media classes [+] MP3 etc playback NetworkKit: [ ] BNetEndpoint (for LiveEncoder et al) Net services: [ ] ftpd (to Rx mp3 files) [ ] VNC server IPC and registrar: [x] Broker component for Inter-app BMessage'ing (e.g. "Open mp3 in AK") [x] Registrar, in support of Deskbar [ ] Launching apps (AK, MMD...) [ ] DnD: implement "cooperative" DnD ? Desktop components: [+] Deskbar [x] Tracker, incl. BFilePanel from libtracker [.] icon resources [.] Sys-log style terminal Genode core: [+] keep up with new Genode releases (viz. toolchain) [ ] integration with Genode apps (Falkon...)
My projects seem increasingly futile these days, as the (western) world slides further
down the right-hand part of the Hubbert Peak curve (not to mention lemmings-like
mass psychosis not seen in Europe since the thirties -- that was to be expected given the Big Picture
but still feels gloomy)... But I feel compelled to continue anyway.
So I worked on Deskbar, Tracker, throughout feb/mar/apr, with a rush this mid-june after I was done with factory work. Things are falling into place nicely, if slowly, see the screenshot, right.
Starting from the beginning:
19feb22: libtracker.so almost compiles and links. This is going to be a big SCM commit...
12mar22: Tracker works, shows desktop "boot" volume. Deskbar shows up, shows time, clicking time somewhat opens up the calendar, but the (more interesting) main menu is empty. I'll withhold most of the commits until things get in decent shape.
27apr22: Made several commits those past weeks, re. the Broker (a new component I came up with which emulates Haiku's write_port() in support of BMessenger IPC, bridging the Genode/Haiku divide). Building upon that, I got Registrar to run ; that's a Haiku server which relays BMessages, providing "roster" services to e.g. Deskbar.
Deskbar's presence is still symbolic at this stage, in that it doesn't work well enough to be useful yet. I'm not pinning much on it at this stage : in the near future Deskbar ought to provide easy access to shutdown/restart, and some limited app launching capability, but that's about it -- I strongly suspect most/all of the window handling features won't work initially, until I get VERY intimate with NitPicker (next year ?).
27may22: Tracker's desktop gets laid out more or less correctly (sans icons though). I can open /boot, its window opens up, its menu items are there etc ; the window is empty of (child) items though, it does not show the /boot/home child for some reason (GetNextDirent() issue ?), but still, took a screenshot as a day landmark.
13jun22: Windows are decorated, can be dragged around by their yellow-ish title tab and so on. Tracker windows list their sub-folders/files. So things are somewhat functional now, at last. Since Tracker is on the "very good to have" list, that's quite a relief.
19jun22: Had lingering kernel-emul/locking problems, which I solved yesterday, allowing me to post this today (made little sense to post a screenshot about Deskbar/Tracker "working" if some crippling bugs lurked behind the seemingly pretty image). Anyhow, fixed wait_for_thread() in haiku-emul code (note to self: kernel Giant Lock must be held for as short a time as possible ; my lack of systems programming experience shows). Tested on bare metal on a ThinkCenter tower and several laptops, good to leave Qemu aside for a bit. Will have to fix a USB-HID regression, maybe it's just running out of caps or ram ? Probably won't be long to fix.
I'll leave it at that for today, with the one screenshot above for you guys to chew on. For completeness' sake, the screen grab is laid out so as to shows some bugs: off-by-ones, layout issues, missing bits and pieces... But nothing that's a deal breaker for running AutoCast, and that's the whole point isn't it. Thus overall, I can mark Deskbar and Tracker as "+" in the dashboard and press on with what's urgently needed to run an AutoCast-based radio station, and revisit desktop stuff later. I'm mostly done with Phase 1 (creating the pieces of the whole puzzle), now entering Phase 2 (assembling the pieces into a coherent image) with my code and repo in a good shape.
So that somewhat concludes the 'trail blazing' part of the project. No doubt I'll stumble on more truckloads of bugs on the way to production, it won't be a walk in the park, but I can now see the light at the end of the tunnel, which hopefully is not a freight train's headlamp.
aug21: Started by augmenting/fixing fs.cpp, so that class BFile and friends would work. Testing was originally done against ram-disk (the so called "ram" node in Genode vfs config).
aug21: Storage Kit code was getting confused by errno being positive in FreeBSD libc (e.g. "file not found" has value 2 instead of -2) so I had to battle with "errno" a bit; libroot-build uses tricks to deal with the FreeBSD/Haiku discrepancy in errno signed-ness, including rerouting to "real_errnop()"; but I'd rather not use haiku/src/libs/posix_error_mapper/misc.cpp and friends due to hacks (asm ".hidden" statement, would that work in Genode ? and more importantly, the wrapper it creates for each function use a _static_ return var, so not thread safe). Instead, I patched fs.cpp itself (which is the main user of errno)... Let's hope it's good enough. Seems ok so far.
11aug21: MMD (MakeMyDay) can create/edit/delete log "clock" files (just had to patch MMD to account for missing node monitoring).
13aug21: MMD can create/edit/delete Master logs, and tries to generate a Program Log, after adding the errno fixes and work-arounds for missing node monitoring. However, in order to be interesting, the generated ProgLog should contain "Random X Y" lines, and I need keyboard support to type the X and Y values (attribute name and attribute value). Then I'll be able to type e.g. "Genre Rock". When that's done I'll focus on the NTFS-FUSe-VFS-attributes-ioctl-BQuery conundrum.
18aug21: Keyboard works well enough to navigate with Tab, type "Genre Rock" in the MMD text field and so on. I'm now hitting the BVolume/BVolumeRoster stubs (shortcomings), need to implement that stuff.
19aug21: BVolume et al done, now we're getting the "BQuery - no file found on /boot that matches Genre Rock" message, at last.
18jan 2022 (!): yeah I got busy, then got carried away coding, then busy again... All in all it was a lot of work, but as of this week it's done: I tackled Genode's ticket 193 (turning the NTFS driver and other FUSe drivers into a VFS plug-in), and gained enough knowledge about the NTFS-3g code to enable "xattr" support. Then, still advised by Norman, I set up an ioctl-like system for userland code to communicate about xattrs to the FS drivers. So we can not only BFile.Read/Write(), but also ReadAttr/WriteAttr() now. Then I got working on an Indexer component, that will sit between the Storage Kit and the NTFS driver, to maintain an attribute index and respond to queries. As of today, all tests pass. Next up, let's recompile MMD and see if it generates a Program Log now.
25jan2022: There! MMD can generate a ProgramLog from a MasterLog that goes "Random Genre Rock", meaning it's going to lift the mp3 file from the index on the NTFS partition, calculate its path and insert it in the ProgramLog, as it does on Haiku-on-Haiku :-). Took some fighting with libroot-build again, for handling SymLinks (they are important in TTS code, to access the "symlink to station folder" link). Also had to finish cleaning up my build system. I can now build all TTS software for either Genode or Haiku, from the same set of Jamfiles, yay. Next I'll look into doing a proper vfs_server, instead of hosting the NTFS+indexer FS inside MMD, then I'll probably embark on the libtracker biggie, so that ArmyKnife can open a BFilePanel to actually work on files (!).
25jul21: MakeMyDay compiles and launches. Was painless, just needed to enable a DrawBitmapAsync() variant in class BView and clean up a hack -- MMD was using BApp.CountWindows() for finding its worker thread, yuck.
So what now ? I'll have to recompute my 'roadmap', but it's already clear I should turn my attention especially towards the Storage Kit, connect nuts and bolts to enable BFile handling, the works.
And yes that was quite a few weeks elapsed with not all that much to show for it. Been working away from home with a long commute time, quite tired when getting back home and not anxious to get on the keyboard... I'll be making slow progress for a while more. In August I'll get a break though. See you then.
|
30jun21: started working on AK.
Needed to work on the build system first, so that it would start compiling at all (need to provide for clean "out of tree" app building, without using the hacks I made for AutoCast), Then had to add missing symbols in haiku lib etc. Got AK compiling-and-almost-linking in a short time. To be clear, compiling it can be done "out of the box" with virtually zero changes (I just commented out the code for customizing keyboard shortcuts, is all, as there is not support for that in HoG). 2jul21: AK links into an executable, but LD fails to load it, looking into it. 3jul21: Was caused by build confusion between "foo.abi.so" and "foo.lib.so", AK runs now. The screenshot (right) looks almost like it was taken on Haiku, kinda proud of that. Of course I need to implement all the back-end code (e.g. Storage) that is stubbed out. But let's finish the set of TTS applications first. We're pulling down the tunetrackersystems.com website in the near future as it misleads people into thinking we're still offering the old CC5 product line... We'll setup something again once AutoCast-on-Genode is ready. |
|
But as of today he reports it works, yay!
As of this afternoon, HoG builds against Genode 21.05, including our flagship app AutoCast 5.99.1.6 -- didn't hit any significant snag, despite that version being coded for GCC 10.3/binutils 2.36, it all seems fine. Took me no more than ca. three weeks, or about one week per year of "lateness" :-).
In the coming days I'll clean up and commit the updated haiku-kinda-kernel (it's a big "tweak" that makes it use pthreads as required by the new Genode revs), and then I can get in touch with Norman again.
Then I'll return to porting our stuff, back to the salt mines.
> (will the new system work on existing station hardware?)Thinking about this also prodded me into publishing this updated timeline to neocities, instead of it gathering dust in the dark corners of my hard-drive. So there.Well could be. (..) I've prioritized ugprading my work copy of Genode. It was 3 years old, so I'm migrating to 2021 code, which has a newer audio driver, which might work on the AMD based systems we've been producing for years.
Lots of those out there, so big incentive to make audio work on existing station hardware, even if it means pausing my work on higher-level AutoCast code. Though worst case, if I can't get things to work on AMD systems, we'll find cheap Intel-style micro-computers for stations that don't want to spend much on hardware. Intel makes "NUC" micro units that are industrial strength (reliable) and very cheap, among other options.
Now that I'm reading/writing this exchange, I'm thinking again of publishing my "blog" by the way. It's very technical in nature, much more than I ever thought it would be, as every time I open it I feel the need to document my findings of the day, down to the compiler options I've used to solve such and such problem.
But technical mumbo-jumbo or not, in matters of public relations, something is better than nothing :-) Would be frustrating if people thought I was idle at the switch, and not working my butt off to get things done ^^
It would also give an idea of the scale of work involved, the number of tasks I still need to get done etc, so that people pray for me :grin: , plus the big perks we'll get with this new platform. Oh, and screenshots.
Build Genode 19.05 (git checkout etc, as documented):
PS - also "fixed" AutoCast transparency (the usual BBox hack, strange that it is still needed in this public branch). Added (uncommited for now) libroot stuff too, so that background.jpeg can be found with an entry_ref, without hacks, and so that beos.mp3 can be found ditto.
When rebooting to my USB stick tonight (that's when I generally test ISO images, right before ending my programming day), AC should produce audio. Turns out there was a missing "acpi..." statement in report_rom. That was fixed in my private branch, but I hadn't duplicated the fix in the chiselapp public branch that was forked from it, oops.
Until now I was relying on a year-old ISO image made from my private branch, to launch AC with audio. Now I'll be able to run fresh builds.
Got in touch with Norman this past week ; we agreed it would be interesting to integrate haiku.lib.so as a package for SculptOS, but there's a stumbling block : my code is built against the Genode 18.11 API, and GCC6 ; I'd need to upgrade to newer Genode code, and it's doubtful whether I can compile the newest Genode code with the old compiler. We'll see.
So, newest roadmap:
When that's done, I can look into extending the set of supported features : Network Kit, keyboard support, BFS attributes, yellow tabs, compile AK/Lightning/MMD, and so on.
Will also have to look into this Genode-only bug some day (maybe it was just lurking, when in Haiku, and gets revealed only now ?) :
A.C./minimalist-lib/etc: Thread exception/crash/whatever in AC, on HoG, Settings window, clicking on "button pad" items: Error: attempt to reserve zero sized block from heap The main window/thread remains responsive, but the Settings window thread is dead.
2) thought of this marketing pitch I could "sell" to Dane about Q.A.:
Just occured to me that with Genode, ISO images are so small that we can keep a copy of each ISO sent to each customer, for optimal tech support ; when a customer gets in touch with us, we can just dig up the ISO we sent them and replicate the issue (more) reliably.
3) ToDo: get in touch with Norman as well ?
4) ToDo: fix audio output in AC
It was easy getting the AutoCast code back in shape and compiling in HoG, took just a few hours to catch up...
Now as expected, I'm hitting tons of "missing symbols" errors, since AC has wider requirements than what I implemented for hog-demos (Media Kit, BSlider, BUrl, BMailAccount, BQuery, BSerialPort...) Just have to get in the saddle tomorrow and get on it.
UPD: 26 april: AutoCast shows up in qemu ! Had to tweak my "run" file and fix broken swap_float() for jpeg translator so that background pic shows up Flashes A LOT though, now that Pulse-driven clock invalidates are done.. Let's see if bare metal flashes less than qemu..
UPD: yep, wasn't all that much hard to optimize Invalidate/painting.
Do-list:
Making progress with my "demo" scenario/ISO. Ran the demo scenario bare metal for the first time (for my public branch I mean, the old "private branch" work was often done on bare metal too).
Things mostly work as expected, except for some menus :
A couple radio stations who use our (TuneTracker Systems) software are asking what's going on these days. That's "popular" enough demand (on our scale :-) to start micro-blogging about it, say, once a week. Plus it helps me sort my thoughts as well. Since this is targeted at our stations, I'll try to keep technical mumbo-jumbo to a minimum, or explain tech stuff in simple terms.
So what's up exactly ? If you're in touch with Dane or me you might have heard that I'm working on "merging" Haiku and Genode. This is long, tedious work, but it feels great to mix a super-smooth user experience (Haiku) and a fantastic operating system framework (Genode). Let's start by describing my planning, broken down into sub-parts.
When that's done, I'll make sure AutoCast compiles and runs as well, on a simple scenario (makeshift BFS partition, not a real file system).
Eventually, I'll reach a stage where I need to port the low-level part of Haiku that is actually responsible for file access (called "libroot") and add (extend ?) attribute support to Genode, for using in BFS or NTFS. I suspect NTFS might be good to us, with its extended attributes support.
In parallel with the above, I should upgrade Genode, since currently I'm still working with a version from 2018. Updating to 2019 and 2020 revs will require adapting to some Genode API changes, and more frighteningly :-) adapting to compiler (tool chain) change. This part of the project might not be top prioritary, but good-to-have, because newer Genode versions have better hardware support, which helps with TuneTracker software.
If all goes as planned, we'll need to "wrap up", i.e. not only support AutoCast but we'll also want as much as possible of its software suite to work in real-world (not makeshift) scenarios. Not to mention the Haiku "desktop". It's unlikely I will be able to port the whole Haiku desktop (called "Deskbar" and "Tracker"), so we'll need to make our apps flexible enough to deal with their absence, e.g. use a more austere "launcher".
In parallel with the above: it might be possible to compile something of use to fellow Genodians, to build community interest. Maybe a simple Mail client ? Or more likely, since it's much less dependant on BFS file system features, an IRC client (Vision's source code) ? Genode has a great port of the "Qt" library; it seems to me this allows Genode to natively run a huge library of software out there, but maybe there's interest in apps with a smaller footprint, rather than the comparatively bigger Qt apps ? Let's hear from fellow Genodians.
Currently I'm getting close to the end of the "Pulse" phase. And I know from last year's experience that once I'm there, it does not take much more to run AutoCast "bare bones". The rest is another can of worms though. Not quite terra incognita, but still hard to evaluate yet.
Back to the current "Pulse" phase: that phase is itself sub-divided into the "baseline commit" and "patch" tasks. I'm mostly done with baseline commits, but need to fix a couple bugs (ETA unknown) before I commit my big "patch" on Haiku's baseline code to make it Genode compatible. The patch itself is divided into the "90%" part, and the "plat de résistance" (BView, BWindow, and posix headers). It should take at least a couple days to do each of BView/BWindow/posix. Then I'll create an ISO deliverable. Then say, about a week for the bare-bones AutoCast part. So there you go, I gave you the traditional "just a few more weeks !", but in sophisticated format *g*.
I have a rough idea of the "deliverables" I'm planning on in the future: 1) prep an ISO with Pulse/Mandelbrot, email fellow Genodians and propose to try it out 2) AutoCast: put something in Dane's hands for him to test 3) AutoCast again: get closer to production and/or base it on a newer Genode with better hardware support.
My next micro-blog entry will hopefully conclude on the current phase and introduce the AutoCast phase.
