RejectedSoftware Forums

Sign up

Pages: 1 2

(re-)package dub packages in Debian

Hi all,

I know dub was designed to overcome distro packaging, but in order to get some packages into Debian (specifically I'm interested in getting vibe.d), I have to get its dependencies in Debian first.
So basically I have to replicate dub's workflow while creating Debian packages.

So I'd like to understand how it works, I see that it depends on openssl and libev (amongst others).
In most of the packages I see two directories, C and deimos.

Could someone explain to me, for example, what should be the preferred way to package those? Should I package dub and make it build and install the files in some prefixed directory and just package that and copy it to a system wide directory (like /usr/include/d/ or something else)?

I would be prepared to package and maintain many of those packages in Debian (I am a Debian developer myself) but I'd like to understand what would be the best way to do it. I also currently took over maintainance of ldc in Debian, so I have an interest in providing better support for the D language in the distribution eventually.

Thanks

Konstantinos

Re: (re-)package dub packages in Debian

Thanks for you continuous effort on improving D Debian presence.

Could someone explain to me, for example, what should be the preferred way to package those? Should I package dub and make it build and install the files in some prefixed directory and just package that and copy it to a system wide directory (like /usr/include/d/ or something else)?

I think it is most simple an generic thing to do. Just run dub build on a source and then copy resulting binary and any extra files (man pages, modules, etc) to package. Exact placement in resulting file system does not have any common standard. For example, Jordi (maintainer of http://d-apt.sourceforge.net/) uses /usr/include/dmd as base for all imports and I have chosen common /usr/include/dlang/{dmd,ldc,gdc,whatever} for Arch Linux. I think you should actually have best judgement being Debian developer :)

dub does not have a make install alternative so it is completely up to package to chose exact file system placement.

Lot of dub packages are source libraries though (including vibe.d) - it is enough to just copy necessary module and define dependencies to get a usable package. Example of dub-free packaging I do for vibe.d in Arch : https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/PKGBUILD (a bit outdated, oops!)

I'd love to read your opinion about desired scheme by the way ;)

Re: (re-)package dub packages in Debian

On Wed, 05 Mar 2014 18:13:57 GMT, Dicebot wrote:

Thanks for you continuous effort on improving D Debian presence.

You're welcome :)

I think it is most simple an generic thing to do. Just run dub build on a source and then copy resulting binary and any extra files (man pages, modules, etc) to package. Exact placement in resulting file system does not have any common standard. For example, Jordi (maintainer of http://d-apt.sourceforge.net/) uses /usr/include/dmd as base for all imports and I have chosen common /usr/include/dlang/{dmd,ldc,gdc,whatever} for Arch Linux. I think you should actually have best judgement being Debian developer :)

The previous maintainer chose /usr/include/d/ (and /usr/include/d/ldc for ldc-specific stuff), and this is what gdc also uses (it's put in /usr/include/d/4.8), so for now I'll also continue using this, but I'm not really in favour or against it, if D community says otherwise, I'd be happy to change it.

dub does not have a make install alternative so it is completely up to package to chose exact file system placement.

Lot of dub packages are source libraries though (including vibe.d) - it is enough to just copy necessary module and define dependencies to get a usable package. Example of dub-free packaging I do for vibe.d in Arch : https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/PKGBUILD (a bit outdated, oops!)

Ok, I think I understand it better now, but Debian uses a more fine-grained approach when it comes to language bindings for existing libraries (eg. libevent/openssl). I'll have to provide openssl-dlang or openssl-d (not sure which is the best name, I'll probably go for the former as it's more easy to search for), same for libevent-dlang, etc.

So will it be enough to just copy ~/.dub/packages/openssl-master/deimos/openssl/* to /usr/include/d/openssl and it would be detected? And what about the stuff in the C directory, is it just duplication of the original C headers (from first glance it looks like it, but better make sure).

I'd love to read your opinion about desired scheme by the way ;)

Thanks for all the help, the D community so far has been one of the friendlist I've encountered :)
I'll start with openssl-dlang and libevent-dlang and see if it works and makes building of dub and vibe.d possible and I'll post progress here.

Konstantinos

Re: (re-)package dub packages in Debian

On Wed, 05 Mar 2014 20:58:25 GMT, Konstantinos Margaritis wrote:

I think it is most simple an generic thing to do. Just run dub build on a source and then copy resulting binary and any extra files (man pages, modules, etc) to package. Exact placement in resulting file system does not have any common standard. For example, Jordi (maintainer of http://d-apt.sourceforge.net/) uses /usr/include/dmd as base for all imports and I have chosen common /usr/include/dlang/{dmd,ldc,gdc,whatever} for Arch Linux. I think you should actually have best judgement being Debian developer :)

The previous maintainer chose /usr/include/d/ (and /usr/include/d/ldc for ldc-specific stuff), and this is what gdc also uses (it's put in /usr/include/d/4.8), so for now I'll also continue using this, but I'm not really in favour or against it, if D community says otherwise, I'd be happy to change it.

I don't think there is any sort of community standard for this sort of things. If Debian/Ubuntu users will have issues with your choice they will surely let you know ;) I have chosen "dlang" over "d" simply because it is more grep'able.

dub does not have a make install alternative so it is completely up to package to chose exact file system placement.

Lot of dub packages are source libraries though (including vibe.d) - it is enough to just copy necessary module and define dependencies to get a usable package. Example of dub-free packaging I do for vibe.d in Arch : https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/PKGBUILD (a bit outdated, oops!)

Ok, I think I understand it better now, but Debian uses a more fine-grained approach when it comes to language bindings for existing libraries (eg. libevent/openssl). I'll have to provide openssl-dlang or openssl-d (not sure which is the best name, I'll probably go for the former as it's more easy to search for), same for libevent-dlang, etc.

Another option can be to use "deimos-openssl" naming as such bindings are usually part of Deimos project : https://github.com/D-Programming-Deimos

So will it be enough to just copy ~/.dub/packages/openssl-master/deimos/openssl/* to /usr/include/d/openssl and it would be detected? And what about the stuff in the C directory, is it just duplication of the original C headers (from first glance it looks like it, but better make sure).

You will also need to make sure that ldc2.conf / dmd.conf you package with compiler have those import paths included (https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/ldc/ldc2.conf) and that relative module paths match fully qualified names of modules. If module has fully qualified name "deimos.aaa.bbb" then it has to be available via relative path ./deimos/aaa/bbb.d from one of listed base import paths (-I).

One scheme I personally find most convenient for system-wide installation is to have file system layout like this:

/usr/include/dlang/dmd
/usr/include/dlang/ldc
/usr/include/dlang/common

..so that one can put "-I/usr/include/dlang/dmd -I/usr/include/dlang/common" for dmd.conf and "-I/usr/include/dlang/ldc -I/usr/include/dlang/common" for ldc2.conf

Then source libraries that work for all compilers can be installed to "common" subdir and compiler-tweaked ones (like standard library) - to compiler-specific subdir, without any additional modification of .conf file.

gdc is a bit special though, as it may be required to conform gcc path/naming scheme once included in gcc upstream. Right now I package it in same way as ldc/dmd but probably will be forced to change it at that point.

I'd love to read your opinion about desired scheme by the way ;)

Thanks for all the help, the D community so far has been one of the friendlist I've encountered :)
I'll start with openssl-dlang and libevent-dlang and see if it works and makes building of dub and vibe.d possible and I'll post progress here.

Looking forward to it ;)

Re: (re-)package dub packages in Debian

P.S.

And what about the stuff in the C directory, is it just duplication of the original C headers (from first glance it looks like it, but better make sure).

As such bindings are usually result of direct 1-to-1 translation (sometimes even automatic), C folder contains original headers used as base for generation in case reference is needed. It does not need to be deployed.

Re: (re-)package dub packages in Debian

On Wed, 05 Mar 2014 18:13:57 GMT, Dicebot wrote:

Lot of dub packages are source libraries though (including vibe.d) - it is enough to just copy necessary module and define dependencies to get a usable package. Example of dub-free packaging I do for vibe.d in Arch : https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/PKGBUILD (a bit outdated, oops!)

Ok, after some messing around with that, I managed to produce working packages of openssl, libevent2 library and to a point, vibe.d. The first 2 install fine (in /usr/include/d/common as you suggested) and are found fine in the include path. vibe.d is also installed there and also found without problems, but I seem to be getting link errors (it's expecting a library I guess), is there supposed to be a library built with vibe.d? It's not obvious from your Arch recipe if a library is being built.

If I manage to solve this one, without resolving to dub, I'll be one step closer to providing a multitude for D packages in Debian :)

Re: (re-)package dub packages in Debian

On Sat, 08 Mar 2014 21:24:19 GMT, Konstantinos Margaritis wrote:

On Wed, 05 Mar 2014 18:13:57 GMT, Dicebot wrote:

Lot of dub packages are source libraries though (including vibe.d) - it is enough to just copy necessary module and define dependencies to get a usable package. Example of dub-free packaging I do for vibe.d in Arch : https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/PKGBUILD (a bit outdated, oops!)

Ok, after some messing around with that, I managed to produce working packages of openssl, libevent2 library and to a point, vibe.d. The first 2 install fine (in /usr/include/d/common as you suggested) and are found fine in the include path. vibe.d is also installed there and also found without problems, but I seem to be getting link errors (it's expecting a library I guess), is there supposed to be a library built with vibe.d? It's not obvious from your Arch recipe if a library is being built.

What kind of linking errors? Application written using vibe.d need to be linked to libevent and openssl. dub takes care of it automatically but for stand-alone package I have written this simple runner script: https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/viberun

vibe.d itself can be pre-built as a static library to speed up compilation of user code but it is not necessary

Re: (re-)package dub packages in Debian

On Wed, 05 Mar 2014 17:07:53 GMT, Konstantinos Margaritis wrote:

Hi all,

I know dub was designed to overcome distro packaging, but in order to get some packages into Debian (specifically I'm interested in getting vibe.d), I have to get its dependencies in Debian first.
So basically I have to replicate dub's workflow while creating Debian packages.

So I'd like to understand how it works, I see that it depends on openssl and libev (amongst others).
In most of the packages I see two directories, C and deimos.

Could someone explain to me, for example, what should be the preferred way to package those? Should I package dub and make it build and install the files in some prefixed directory and just package that and copy it to a system wide directory (like /usr/include/d/ or something else)?

I would be prepared to package and maintain many of those packages in Debian (I am a Debian developer myself) but I'd like to understand what would be the best way to do it. I also currently took over maintainance of ldc in Debian, so I have an interest in providing better support for the D language in the distribution eventually.

Thanks

Konstantinos

Just to add a bit to the things already said, our current direction on how to integrate DUB with system libraries is roughly described in this github comment. The basic idea is to not try to turn DUB packages into DEB (or similar) packages, but rather just augment all bindings packages, such as the "openssl" one with system specific dependencies.

However, at some point I'd love to have all information necessary to build system packages available in the DUB package description, so that a "dub create-deb-package" command becomes possible. But this has so many implications if done properly for all platforms, that it is more a long-term idea than a concrete plan.

Re: (re-)package dub packages in Debian

On Mon, 10 Mar 2014 17:10:17 GMT, Sönke Ludwig wrote:

Just to add a bit to the things already said, our current direction on how to integrate DUB with system libraries is roughly described in this github comment. The basic idea is to not try to turn DUB packages into DEB (or similar) packages, but rather just augment all bindings packages, such as the "openssl" one with system specific dependencies.

However, at some point I'd love to have all information necessary to build system packages available in the DUB package description, so that a "dub create-deb-package" command becomes possible. But this has so many implications if done properly for all platforms, that it is more a long-term idea than a concrete plan.

Hi Ludwig,

I already commented on the github comment, but would like to add some extra comments here as well. IMHO, that's the wrong approach, to directly create deb packages from DUB itself. Apart from the fact, that in order to be included in Debian itself, the package has to be provided in source form -and not a port-specific .deb package- which will then be sent to the autobuilders, which have a strict policy not to build packages that perform downloading extra content in the build phase -yes, technically the builders run in isolated mode with only access to an APT repository- so something like dub which gets stuff online, is out of the question anyway.
Plus, you also have to take extra care to follow the distro policy -which changes, not very often, but it does. Instead of trying to replicate what a number of dedicated tools are written specifically for, you could instead try to help them. The best method I could think of is by exporting not the package itself, but a makefile.

That way, a packager can just do a simple 'make' and just handle the packaging as usual. In any case, D binding packages are not really that different than other language bindings, in fact they're supposed to be simpler than, say, java packages in Debian. It can be really easy to integrate D packages in Debian and have them readily available to any distro user interested.

Re: (re-)package dub packages in Debian

On Sun, 09 Mar 2014 13:10:02 GMT, Dicebot wrote:

What kind of linking errors? Application written using vibe.d need to be linked to libevent and openssl. dub takes care of it automatically but for stand-alone package I have written this simple runner script: https://github.com/Dicebot/Arch-PKGBUILDs/blob/master/vibed/viberun

Linking errors like this:

http://dpaste.com/hold/1715523/

I tried with viberun but I still get the link errors. Btw, I'm using ldc2, not dmd, I guess that's the reason, though I modified the script to call ldc2/ldmd instead.

vibe.d itself can be pre-built as a static library to speed up compilation of user code but it is not necessary

It may be the case that I'll have to provide a static library then if ldc needs it. As I understand it, dmd backend is using a proprietary license and I can't include it in Debian, I hope I'm wrong there :-/

Pages: 1 2