[delta-chat] Re: The state of building deltachat

Hi Floris, all, thanks for the writeup ... I'd like to look at the "state of building deltachat" from the outcome perspective. We have at least 5 different user-visible downloads that we want to provide: - Android (currently works with the caveats also discussed below) installable via f-droid, gplay and as direct apk from github releases etc. - iOS (currently works but build is done in a ios/mac specific way) installable via testflight - Linux (currently Desktop works with flathub, which requires several manual steps at least on ubuntu -- what's keeping appimage's from working btw?) - OSX (works ASFAIK but is not signed so leads to warnings) - Windows (does not work, some experimental deltachat-core building works) It seems the biggest challenge remains a Windows download and having a mostly-automated, improved way to produce the Android/iOS/Linux/OSX outcomes that are already working. One advantage of having dependencies inlined is that we know which deps/versions are contained in eg "Deltachat Desktop Linux 0.200", even if they are outdated. Inlined deps also simplify it for devs to get started with doing PRs. However, coming from a debian or system packager perspective, i can totally see how inlined dependencies are considered evil ;) But I am afraid that if we prioritize doing things "right" from the sys-packaging perspective, the build process and steps get more complex, and it might further delay getting the outcomes we want for users. Also deps are indeed changing as we are starting to replace C with Rust dependencies so in a couple of months the dependency tree might look quite different -- is it really worth then to modularize all the current C dependencies and make the build of deltachat-core more flexible? holger On Mon, Mar 18, 2019 at 20:54 +0100, Floris Bruynooghe wrote: > Hi all, > > On IRC I was told the current build situation is unclear (no surprise!) > and an attempt to sketch out the current and desired next state would be > useful. Here's my attempt, it's only partial obviously and I probably > missed many things, got many wrong and many will have changed within a > few days (e.g. we're about to add a Rust dependency so who knows how > that'll look). > > > Anyway, currently deltachat-core ships with a bunch of (outdated and > limited) dependencies bundled in the libs directory. Other than netpgp > you should try to avoid to need these when building. However they are > used for the android build which is why they can't be simply removed. > I'll come back to fixing the android situation later. > > If you continue on unix it's probably because you want a > deltachat-desktop install. The recommended way to install on linux is > to use the flatpak: https://flathub.org/apps/details/chat.delta.desktop > it works correctly on all linuxes right now. I'll continue the journey > into how we produce said flatpak, which will bear many similarities with > how to build on other unixes. > > Next you need to build the deltachat-node bindings. This currently > builds libdeltachat.a itself rather than using the system installed > version. Likewise it uses the bundled libetpan. The next improvement > to make here is to change the build to use a system-installed > libdeltachat.so and thus giving control to packagers to decide which > dependencies are used. It means that downstream users no longer depend > on the git subproject of deltachat-core which happens to be in > deltachat-node, thus taking deltachat-node out of release work (unless > it needs code changes itself). This is also a desired step to support > packaging on linux distros who will otherwise never package > deltachat-node. As a result of this we also want to stop posting > prebuilt binaries of deltachat-node to npm. This will break some CIs in > deltachat-desktop which will have to start building their own > dependencies. > > Further possible scope creep here is whether to also support several > variations of building, e.g. against a static libdeltachat.a? Against a > non-installed version? But it seems this is not needed initially due to > the slow rate of changes in deltachat-node. > > Next up is deltachat-desktop, like deltachat-core this does see a lot of > changes for each release. This simply depends on deltachat-node, so > once dc-node is updated to depend on the system libdeltachat.so everyone > building this will need to build and install libdeltachat.so first. > > Finally the actual packaging work happens, e.g. in the flatpak repo. > These need to build all dependencies, dc-core and then npm install > dc-dekstop (which will pull in dc-node via npm and build it against > system dc-core). This currently does not install dc-core or all of it's > dependencies because dc-node currently uses a weird mix. > > > Now to get back to Android. This currently uses all the bundled > dependencies in the dc-core libs directory, it does however not use the > meson build definitions instead lists all the .c and .h files in it's > own build system. These bundled dependencies still are too outdated and > hard to upgrade though. > > There are two choices here (well, maybe more - tell me!): > > - Do the same as on other unixes and build all the dependencies > externally, pointing the android build to the ready .a files. We'd > probably want to do this on a CI system. We can create a docker image > which builds all these dependencies and can build dc-core & dc-android > using them. > > - Still bundle the dependencies but make it easier to upgrade them by > splitting them into their own repos, tying them in via meson > subprojects. This would then also allow to cross-build each project > using meson and producing the necessary .a files which then again get > used in the android build. The advantage of this is that these > subprojects would also give us identical builds on all unixes if we > wanted. > > > Hope this is somewhat helpful. Feel free to comment. > > Cheers, > Floris > _______________________________________________ > delta mailing list -- delta(a)codespeak.net > To unsubscribe send an email to delta-leave(a)codespeak.net