On Thu, 12 Dec 2013 17:32:53 +0100, Sönke Ludwig wrote:
What exactly were the issues where it doesn't scale?
There were some issues related to file layout - it didn't like our default which implies several different "main" files in same subfolder with rest of source mostly shared, tried to compile all despite different configuration with different target files used. Most likely a bug I will have a look at if try this again at some point.
But actual scalability issues pretty much come from simplicity of format used. Being unable to define meta-variables resulted in lot of added duplication over original makefiles (there were several lists used in different contexts). Typical makefile dependency chain (use dstep to generate module from c file which is generated by external tool from spec file) simply did not map at all - only option was to use a custom pre-build script but than generated files did not get into dub describe and were not seen in Mono-D project.
I also don't like added amount of "includeXXX"/"excludeYYY"-style directives which makes it rather hard to follow project description with many configurations.
Remember that the build system is still in its infancy and hasn't
received much work. But everything else should work fine in projects of
any size.
Well, its infancy is exactly the chance to figure out what is missing here and transition to more powerful system :) My initial proposal to separate package.json was because package management looks mostly fine right now and are unlikely to undergo any major changes (well, apart from dub CLI itself, I'd love to rewrite it completely some years later :P) - build part does not look that smooth.
The problem is that the more generalized you make the system the more
specialized will the "build descriptions" become - like Heisenberg's
uncertainty principle ;). At some point you lose the "just works" and
have to adjust everything for every dependency and every
platform/whatever. Also you'll frequently discover packages which don't
support this or that kind of build (debug build, or building as shared
library, or 64-bit). My goal is to find a sweet spot that fits and has
advantages for >90% of the projects, but still allows to handle the rest
of the projects one way or another.
I don't think that fitting everyone into overly simple base is a good way to address it. Much better way is to actually abuse dub registry with some CI on top to get the information about platforms and configurations supported, probably try to motivate authors by assigning some kind of "rating" to submitted projects that conform desired look&feel well. There will always be some project that won't fit well into constraints and still be potentially useful to someone (and of course, I don't think I am the only one who had an idea of using local dub registry to handle internal company environment).
It is a very difficult task to define system that is both flexible for actual usage and constrained enough to allow dub to infer necessary metadata from it, sure. But unless it is proven otherwise, I dont' think it is impossible. Now there are quite lot of projects submitted to gather the information about and use at as input to brainstorm better system. If requirements can be somewhat formally defined, I am pretty sure thread asking for ideas in that domain will have a huge success in D newsgroup :)
If a project has a structure that doesn't really fit, on one hand you
could argue that it likely could be simplified by splitting it up into
smaller units. On the other hand you can also always do things like
making a wrapper DUB package that just defines import paths and import
libraries, but invokes "make" or another build system from the pre-build
or pre-generate commands to actually build the package.
Yep, but I consider dub describe and its nice integration with IDE's a very important feature to lose.