RejectedSoftware Forums

Sign up

Migrating to dub

I'm thinking about migrating my libraries/tools to dub. I have a couple
of questions.

  • Can I limit a package to a specific set of platforms?

  • The structure of DWT currently looks like this:

  • dwt
    | base (git submodule, linux and windows)
    |
    org.eclipse.swt.gtk.linux.x86 (git submodule, linux)
    | org.eclipse.swt.win32.win32.x86 (git submodule, windows)
    |
    org.eclipse.swt.snippets (submodule, optional)

Ideally I would like a package, "dwt", that automatically picks the
correct submodule/package for the given platform. Is that possible?

  • How do I best deal with external dependencies? Libraries that should
    be installed using the system package manager.
  • How are libraries compiled by default? It just compiles everything in
    the source directory?
  • How do I best deal with packages I don't feel yet are ready to be
    added to the registry but other packages depend on? Currently I use git
    submodules to handle this.

/Jacob Carlborg

Re: Migrating to dub

Am 24.10.2013 11:17, schrieb Jacob Carlborg:> I'm thinking about migrating my libraries/tools to dub. I have a couple

of questions.

  • Can I limit a package to a specific set of platforms?

Yes, it requires manually specifying the configurations:

{
    ...
    "configurations": [
        {
            "name": "library",
            "platforms": ["linux", "osx-x86_64"]
        }
    ]
}

All platforms with no matching configurations will abort with an error then. This should probably also be displayed for each package on the package registry in the future...

  • The structure of DWT currently looks like this:

  • dwt

| base (git submodule, linux and windows)
|
org.eclipse.swt.gtk.linux.x86 (git submodule, linux)
| org.eclipse.swt.win32.win32.x86 (git submodule, windows)
|
org.eclipse.swt.snippets (submodule, optional)

Ideally I would like a package, "dwt", that automatically picks the
correct submodule/package for the given platform. Is that possible?

Yes, you can specify additional configuration specific dependencies inside of each configuration:

{
    ...
    "configurations": [
        {
            "name": "gtk",
            "platforms": ["linux"]
            "dependencies": {
                "swt-gtk": {"version": ">=1.0.0"}
            }
        }
    ]
}

The only problem is that sub modules (AFAIK) are not included when the main package is downloaded from GitHub. I see three options how to solve this:

  • Somehow manually prepare a zip file on the registry with all sub modules included (with unknown server resource usage consequences)
  • Support GIT or HTTP URLs for dependencies and use those instead of (or in addition to) sub modules
  • Do include the dependencies in the registry, but mark them as not ready for general use (e.g. in the description)
  • How do I best deal with external dependencies? Libraries that should

be installed using the system package manager.

For now that's probably best left to the README or similar. The recent talk about this topic hasn't brought any news and the impression still is that handling this automatically will be a difficult/elaborate task. Of course it is possible to use a custom script based solution and run that script for example as a "preGenerateCommands" entry.

  • How are libraries compiled by default? It just compiles everything in

the source directory?

Yes, by default, if there is a "source" or "src" directory, all .d files contained in it will be compiled in. "excludedSourceFiles" Can be used to throw-out single files, or "sourcePaths" can be specified manually to only include certain sub-directories.

  • How do I best deal with packages I don't feel yet are ready to be

added to the registry but other packages depend on? Currently I use git
submodules to handle this.

See above. I think the sub-module approach should just work, but the question is how to best go about it considering the lack of GitHub support. The obvious alternative is to implement dependencies with a source URL #50. But the exact semantics need to be determined. In particular, I think that such a URL should only be a fallback when the dependency isn't found locally or on the registry, so that it's for example still possible to manually check out using GIT and use the checked out version.

Re: Migrating to dub

The only problem is that sub modules (AFAIK) are not included when the main package is downloaded from GitHub

Rationale? I'd expect it to do `git clone --recursive" :(

Re: Migrating to dub

On 2013-10-24 12:35, Sönke Ludwig wrote:

Yes, it requires manually specifying the configurations:

{
     ...
     "configurations": [
         {
             "name": "library",
             "platforms": ["linux", "osx-x86_64"]
         }
     ]
}

All platforms with no matching configurations will abort with an error then. This should probably also be displayed for each package on the package registry in the future...

Just specifying the OS would mean just for that OS but for any architecture?

Yes, you can specify additional configuration specific dependencies inside of each configuration:

{
     ...
     "configurations": [
         {
             "name": "gtk",
             "platforms": ["linux"]
             "dependencies": {
                 "swt-gtk": {"version": ">=1.0.0"}
             }
         }
     ]
}

The only problem is that sub modules (AFAIK) are not included when the main package is downloaded from GitHub. I see three options how to solve this:

  • Somehow manually prepare a zip file on the registry with all sub modules included (with unknown server resource usage consequences)
  • Support GIT or HTTP URLs for dependencies and use those instead of (or in addition to) sub modules

If dub supports the git protocol, either through the git executable or
libgit2, then git submodules would work. Just run git clone <address> <br>--recursive or the equivalent using libgit2.

  • Do include the dependencies in the registry, but mark them as not ready for general use (e.g. in the description)

Ok. It sounds like the easiest would be to just include them in the
registry. I might restructure the whole DWT package anyway. There's some
duplicated code.

For now that's probably best left to the README or similar. The recent talk about this topic hasn't brought any news and the impression still is that handling this automatically will be a difficult/elaborate task. Of course it is possible to use a custom script based solution and run that script for example as a "preGenerateCommands" entry.

Ok, I see. What about having a dedicated field for this? I would not
install anything, just be a list. This would allow to have an official
place to state the external dependencies.

/Jacob Carlborg

Re: Migrating to dub

On 2013-10-24 14:47, Dicebot wrote:

Rationale? I'd expect it to do `git clone --recursive" :(

I think it downloads the zip from GitHub and that doesn't include the
submodules. I guess the advantage is that git isn't required to be
installed. I assume libgit2 could be used to avoid the dependency on
git, but that would be more work to implement.

/Jacob Carlborg

Re: Migrating to dub

Am 24.10.2013 15:04, schrieb Jacob Carlborg:> On 2013-10-24 12:35, Sönke Ludwig wrote:

Yes, it requires manually specifying the configurations:

{
     ...
     "configurations": [
         {
             "name": "library",
             "platforms": ["linux", "osx-x86_64"]
         }
     ]
}

All platforms with no matching configurations will abort with an error
then. This should probably also be displayed for each package on the
package registry in the future...

Just specifying the OS would mean just for that OS but for any
architecture?

It accepts the same combination of identifiers as the build setting suffixes, the example above matches Linux (any architecture, any compiler) or OS X on X86-64 (any compiler).

  • Support GIT or HTTP URLs for dependencies and use those instead of
    (or in addition to) sub modules

If dub supports the git protocol, either through the git executable or
libgit2, then git submodules would work. Just run git clone <address> <br>--recursive or the equivalent using libgit2.

My idea in this case was to specify the git URL each dependency instead of relying on submodules. But this is of course an alternative. The only drawback is that it would tie everything more to GIT, whereas now with the .zip system it is completely VCS agnostic. Having said that, I don't find it to be particularly bad to have a standard VCS (like GIT is for Go).

  • Do include the dependencies in the registry, but mark them as not
    ready for general use (e.g. in the description)

Ok. It sounds like the easiest would be to just include them in the
registry. I might restructure the whole DWT package anyway. There's some
duplicated code.

For now that's probably best left to the README or similar. The recent
talk about this topic hasn't brought any news and the impression still
is that handling this automatically will be a difficult/elaborate
task. Of course it is possible to use a custom script based solution
and run that script for example as a "preGenerateCommands" entry.

Ok, I see. What about having a dedicated field for this? I would not
install anything, just be a list. This would allow to have an official
place to state the external dependencies.

Hm maybe a simple string like "description"? That way we have a clear cut between now and the point when we support machine readable external dependency specifications (if ever). Thinking of something like this:

"systemDependencies": "OpenSSL 0.9.x or 1.0.x, libevent 2.0.x and GTK+ 3.8"


Re: Migrating to dub

On 2013-10-24 15:31, Sönke Ludwig wrote:

It accepts the same combination of identifiers as the build setting suffixes, the example above matches Linux (any architecture, any compiler) or OS X on X86-64 (any compiler).

Ok, I see.

My idea in this case was to specify the git URL each dependency instead of relying on submodules. But this is of course an alternative. The only drawback is that it would tie everything more to GIT, whereas now with the .zip system it is completely VCS agnostic. Having said that, I don't find it to be particularly bad to have a standard VCS (like GIT is for Go).

Aha, I think I understand what you're meaning.

Hm maybe a simple string like "description"? That way we have a clear cut between now and the point when we support machine readable external dependency specifications (if ever). Thinking of something like this:

 "systemDependencies": "OpenSSL 0.9.x or 1.0.x, libevent 2.0.x and GTK+ 3.8"

Yeah, that could work.

/Jacob Carlborg