RejectedSoftware Forums

Sign up

Problems building with Dub

I tried to start to use Dub for one of my project and noticed a problem. Dub builds all files in one go. This leads to a couple of problems:

  1. Compiler flags are applied to all files when they in fact should only be applied to a limited set of files (those of a project)

  2. Require linking of functions not actually used. An example of this is Tango which depends on zlib. But when Tango is built as a static library
    those functions are only pulled in to the executable if they're used in the executable. Since Dub builds all files in one go they will always be
    pulled in to the executable, regardless if they're used in the code or not

It would be better if Dub compiled each package separately, then link everything together.

I guess the first issue not as big issue but the second issue is a blocker for me.
I will not link with zlib when my application doesn't require it. Is there a workaround?

Re: Problems building with Dub

Am 11.12.2013 09:05, schrieb Jacob Carlborg:> I tried to start to use Dub for one of my project and noticed a problem. Dub builds all files in one go. This leads to a couple of problems:

  1. Compiler flags are applied to all files when they in fact should only be applied to a limited set of files (those of a project)

  2. Require linking of functions not actually used. An example of this is Tango which depends on zlib. But when Tango is built as a static library

those functions are only pulled in to the executable if they're used in the executable. Since Dub builds all files in one go they will always be
pulled in to the executable, regardless if they're used in the code or not

Sounds like the compiler should be smarter here. Wasn't there some issue that DMD always emits all symbols in Posix systems, which is supposed to be fixed by a proper "export" definition?

It would be better if Dub compiled each package separately, then link everything together.

It's planned to make that the default behavior and the first step was already taken in 0.9.20 (caching the build result of each package). It will be the next addition to the build system, but I don't know when exactly I can get around to do it. See also https://github.com/rejectedsoftware/dub/wiki/Separate-compilation-and-caching-of-dependencies.

I guess the first issue not as big issue but the second issue is a blocker for me.
I will not link with zlib when my application doesn't require it. Is there a workaround?

A workaround is to define two configurations in each dependency, the first "library" one just outputs a "library" target as normal, and a second "import" configuration defines the necessary flags for importing that library (consisting of only import folders + the library import flags).

Then each dependency needs to be built once using "dub build (dependency)" and the main project should put a "subConfigurations": {"(dependency)": "import"} entry for each such dependency to use the pre-compiled version.

Well, hopefully this workaround won't be necessary for long.

Re: Problems building with Dub

On Wed, 11 Dec 2013 08:36:29 GMT, Sönke Ludwig wrote:

Sounds like the compiler should be smarter here. Wasn't there some issue that DMD always emits all symbols in Posix systems, which is supposed to be fixed by a proper "export" definition?

I don't think so. I'm pretty sure it's only for shared libraries.

It's planned to make that the default behavior and the first step was already taken in 0.9.20 (caching the build result of each package). It will be the next addition to the build system, but I don't know when exactly I can get around to do it. See also https://github.com/rejectedsoftware/dub/wiki/Separate-compilation-and-caching-of-dependencies.

Ok, I see.

A workaround is to define two configurations in each dependency, the first "library" one just outputs a "library" target as normal, and a second "import" configuration defines the necessary flags for importing that library (consisting of only import folders + the library import flags).

Then each dependency needs to be built once using "dub build (dependency)" and the main project should put a "subConfigurations": {"(dependency)": "import"} entry for each such dependency to use the pre-compiled version.

Well, hopefully this workaround won't be necessary for long.

I'm not sure I understand how this workaround i supposed to work.

Re: Problems building with Dub

Am 11.12.2013 16:32, schrieb Jacob Carlborg:

On Wed, 11 Dec 2013 08:36:29 GMT, Sönke Ludwig wrote:

Sounds like the compiler should be smarter here. Wasn't there some issue that DMD always emits all symbols in Posix systems, which is supposed to be fixed by a proper "export" definition?

I don't think so. I'm pretty sure it's only for shared libraries.

It's planned to make that the default behavior and the first step was already taken in 0.9.20 (caching the build result of each package). It will be the next addition to the build system, but I don't know when exactly I can get around to do it. See also https://github.com/rejectedsoftware/dub/wiki/Separate-compilation-and-caching-of-dependencies.

Ok, I see.

A workaround is to define two configurations in each dependency, the first "library" one just outputs a "library" target as normal, and a second "import" configuration defines the necessary flags for importing that library (consisting of only import folders + the library import flags).

Then each dependency needs to be built once using "dub build (dependency)" and the main project should put a "subConfigurations": {"(dependency)": "import"} entry for each such dependency to use the pre-compiled version.

Well, hopefully this workaround won't be necessary for long.

I'm not sure I understand how this workaround i supposed to work.

I've prepared a minimal example:
https://gist.github.com/s-ludwig/7916080

(replace "-" by "/" for the file names)

Assuming that both packages are in DUBs search path, "dub build mylib"
followed by "dub build myapp" should do the trick.

Re: Problems building with Dub

On 2013-12-11 19:52, Sönke Ludwig wrote:

I've prepared a minimal example:
https://gist.github.com/s-ludwig/7916080

(replace "-" by "/" for the file names)

Assuming that both packages are in DUBs search path, "dub build mylib"
followed by "dub build myapp" should do the trick.

Hmm, ok. I think I understand. I'll if I can get it to work.

/Jacob Carlborg

Re: Problems building with Dub

On 2013-12-11 19:52, Sönke Ludwig wrote:

I've prepared a minimal example:
https://gist.github.com/s-ludwig/7916080

(replace "-" by "/" for the file names)

Assuming that both packages are in DUBs search path, "dub build mylib"
followed by "dub build myapp" should do the trick.

If I understand that correctly I need to modify a third package, in this
case Tango? How will this work when my package is used. Will the
standard installation procedure work?

/Jacob Carlborg

Re: Problems building with Dub

Am 16.12.2013 20:33, schrieb Jacob Carlborg:

On 2013-12-11 19:52, Sönke Ludwig wrote:

I've prepared a minimal example:
https://gist.github.com/s-ludwig/7916080

(replace "-" by "/" for the file names)

Assuming that both packages are in DUBs search path, "dub build mylib"
followed by "dub build myapp" should do the trick.

If I understand that correctly I need to modify a third package, in this
case Tango? How will this work when my package is used. Will the
standard installation procedure work?

People would need to explicitly "dub build tango" once. But I'd say that
separate compilation should be pushed as the next feature for DUB. It
seems to be the one with the highest priority anyway. I'll try for 0.9.21.

Re: Problems building with Dub

On 2013-12-16 21:01, Sönke Ludwig wrote:

People would need to explicitly "dub build tango" once. But I'd say that
separate compilation should be pushed as the next feature for DUB. It
seems to be the one with the highest priority anyway. I'll try for 0.9.21.

That would be very much appreciated, thanks.

/Jacob Carlborg