RejectedSoftware Forums

Sign up

Pages: 1 2 3 4

Re: Questions about how DUB works

Am 11.09.2013 09:10, schrieb Jacob Carlborg:

Compiler:

The compiler needs to be exactly the same since it doesn't have any binary or source compatibility. A field for specifying the supported compiler would probably be a good idea. An idea would be to hook dub up with DVM, if you're interested. Then dub could automatically install a switch compilers for different packages.

Somehow I managed to skip this... DVM is something that I've actually
thought about how it could be meaningfully integrated. I already wanted
to ask you, but wasn't sure enough how exactly to go about it (questions
like support for GDC/LDC to not make them more second class than they
should be came up).

Only a manually maintained compiler compatibility field is something
that I fear will not work out (people being lazy or too slow with
updates and possibly using different compilers). But if this would be
combined with a CI that automates compatibility checks this would get
really interesting.

Re: Questions about how DUB works

On Wed, 11 Sep 2013 07:12:53 GMT, Jacob Carlborg wrote:

On Wed, 04 Sep 2013 08:48:16 GMT, Sönke Ludwig wrote:

Installing binary files was/is on the agenda, but mostly just for application packages

I would say installing binary executables for tool/application packages is a must.

No way. You just can't do it properly without lot of efforts (infrastructure efforts, not development) and introducing yet another broken package manager is not good idea.

Leave this up to people who know and care about packaging.

Re: Questions about how DUB works

Am 11.09.2013 19:51, schrieb Dicebot:

On Wed, 11 Sep 2013 07:12:53 GMT, Jacob Carlborg wrote:

On Wed, 04 Sep 2013 08:48:16 GMT, Sönke Ludwig wrote:

Installing binary files was/is on the agenda, but mostly just for application packages

I would say installing binary executables for tool/application packages is a must.

No way. You just can't do it properly without lot of efforts (infrastructure efforts, not development) and introducing yet another broken package manager is not good idea.

Leave this up to people who know and care about packaging.

There is one thing that should definitely be supported, though: Calling
compiled binaries of dependent packages while building another packages.
But of course this is not installation.

What I would like to support at some point, though, would be to
support (semi-)automatic generation of system packages to reduce the
amount of work needed to support different distributions. But I fear
that this will require quite a complex configuration system (and an
accompanying runtime API) due to all the OS specialties (mostly the
differences in the standard file system hierarchy).

Re: Questions about how DUB works

On Wed, 11 Sep 2013 21:01:11 +0200, Sönke Ludwig wrote:

What I would like to support at some point, though, would be to
support (semi-)automatic generation of system packages to reduce the
amount of work needed to support different distributions. But I fear
that this will require quite a complex configuration system (and an
accompanying runtime API) due to all the OS specialties (mostly the
differences in the standard file system hierarchy).

It may be an interesting approach to define an API for OS reflection which can be implemented by experienced maintainers of given systems / distributions as a separate module / plugin. Both for package generation and, for example, querying existing installed software. If it is simple enough, dub maintainers on that systems can use own specific implementation when providing dub package. That should be both convenient and hygienic but is damn complicated. For example what shall one do on system with no even slightest idea about packages? (hello, Windows)

Re: Questions about how DUB works

On Wed, 11 Sep 2013 10:37:13 GMT, Sönke Ludwig wrote:

Use the NNTP interface! (Just saying that because I started adding email notifications almost a year ago, but never got the guts to finish them ;))

I tried using the NNTP interface but it says I need to log in to post to the server. I can't see how to specify a password in Thunderbird :(

That and possibly other compiler flags, depending on the desired use case (e.g. debug vs. release version or the variant of an external link dependency that is used). But already all combinations of compiler (version), architecture and version flags can be far too many to install them all by default -- but if we accept that, we are back to caching compiled binaries lazily (and maybe compiling some kind of default configuration eagerly if desired), which is exactly what is planned.

Obviously not all combinations cannot be installed by default. I was thinking a release build should be installed, possibly a version with debug symbols enabled. I don't know how useful installing a debug build would be. I think one have to assume that the packages used are working perfectly. If some package doesn't, then one can start installing debug builds or similar.

This is more or less what I have in mind, except that binaries would be put into a directory that is unique for all aspects that affect binary generation (i.e. the above) instead of only the architecture. Or rather into multiple directories, each for one combination. The unique part could be a hash value of all related factors or we could try to get something human readable, such as "(compiler)-(compiler version)-(os)-(arch)-(configuration)-..."

Ok, I see.

DUB has "configurations", but only one configuration can be active at a time for each package. But the latter variant using a sub directory is what I'm also imagining (just using a scheme that includes all aspects as mentioned above).

I'm not 100% sure how MacPorts work when you need to install an additional variant. But several variants are possibly to combine. Say you have a library for manipulating images. By default it will build support JPEG and PNG. Variants exist for GIF and PSD as well. Of course all these variants will need to be able to be used together.

I don't know how "configurations" work in DUB but if variants corresponds to -version flags there is no problem with multiple variants. Just append all version flags, "-version=GIF -version=PSD -version=JPEG" and so on.

If you have only installed two variants out of four. Then need to install additional variants I would imagine it would just rebuild the library with the old variants and the newly added ones.

I MacPorts you can have multiple versions of a package installed but only have one active. What it does is that it will install the package in a path which is not accessed by default from PATH or the paths looked in for libraries. When setting a package to active it will create symbolic or hard links.

I agree that "~issue/2" should at least never be the default - this needs to be changed so that "~master" comes first. However, it's difficult to say what should be available. For example, there may be a "~2.x-devel" or a "~stable" branch that may very well make sense to use in some situations.

In general I agree of course that versioned releases are by far preferable, but during early development, or when having tightly coupled packages, it can be necessary to work on branches. I just don't want to make that impossible or force more structure/process on developers than necessary.

I would make it possible to in the registry specify which versions and branches are available. There could also be a flag in the registry to turn off default tracking of version tags and branches. Then one need to manually add them. I would also change so that by default you need to push a version tag before dub starts tracking that package.

Note that I'm not talking about uploading zip archives or similar. I'm talking about just be able to specify for the registry that only these three tags and this one branch is available.

The semantic versioning scheme says what you can and cannot break when making a new release. But it also basically says that you can break how much you want in between releases.

I don't like if some starts using a non-released version of any of my packages, then says: "hey you broke the library in the last commit". If dub uses the master branch by default then I can't say "don't use HEAD, stick with releases".

Using Bundler the standard way is to specify versions, which are downloaded from RubyGems. It's also possibly to specify a git repository, pointing to a branch, tag or any commit. But doing the latter you're kind of on own your own and have to accept possibly code breakage in any new commit.

Short summary: use in between releases on your own risk. I don't think dub should encourage this.

/Jacob Carlborg

Re: Questions about how DUB works

On Wed, 11 Sep 2013 13:39:10 +0200, Sönke Ludwig wrote:

Somehow I managed to skip this... DVM is something that I've actually
thought about how it could be meaningfully integrated. I already wanted
to ask you, but wasn't sure enough how exactly to go about it (questions
like support for GDC/LDC to not make them more second class than they
should be came up).

Currently DMD is only supported. At least when it comes to automatically download and install. As far as I know neither LDC or GDC has a site/FTP listing all available releases. Also I'm not sure if even GDC provides pre-compiled binaries. Compiling GDC or LDC will complicate things, a lot.

Only a manually maintained compiler compatibility field is something
that I fear will not work out (people being lazy or too slow with
updates and possibly using different compilers). But if this would be
combined with a CI that automates compatibility checks this would get
really interesting.

That's a good idea. Specifying a minimum supported version might be a good idea (this might not change that often) and if not all compilers are supported. The the latest working compiler version could be figured out as you've described above.

Re: Questions about how DUB works

On Wed, 11 Sep 2013 13:39:10 +0200, Sönke Ludwig wrote:

Somehow I managed to skip this... DVM is something that I've actually
thought about how it could be meaningfully integrated. I already wanted
to ask you, but wasn't sure enough how exactly to go about it (questions
like support for GDC/LDC to not make them more second class than they
should be came up).

I forgot this:

Currently DVM isn't really usable as a library, which I really would like it to be. But it's always possible to call the binary using shell, or what the function is called.

Re: Questions about how DUB works

On Wed, 11 Sep 2013 17:51:56 GMT, Dicebot wrote:

No way. You just can't do it properly without lot of efforts (infrastructure efforts, not development) and introducing yet another broken package manager is not good idea.

It's not that hard, just put all the binaries in a directory and add it to PATH. What's the point in having a package manager that only clones a repository? What does it give me that git already doesn't?

Re: Questions about how DUB works

On Sat, 14 Sep 2013 12:12:02 GMT, Jacob Carlborg wrote:

On Wed, 11 Sep 2013 17:51:56 GMT, Dicebot wrote:

No way. You just can't do it properly without lot of efforts (infrastructure efforts, not development) and introducing yet another broken package manager is not good idea.

It's not that hard, just put all the binaries in a directory and add it to PATH. What's the point in having a package manager that only clones a repository? What does it give me that git already doesn't?

Note that I'm talking about developer tools here.

Re: Questions about how DUB works

On Sat, 14 Sep 2013 11:59:57 GMT, Jacob Carlborg wrote:

On Wed, 11 Sep 2013 10:37:13 GMT, Sönke Ludwig wrote:

Use the NNTP interface! (Just saying that because I started adding email notifications almost a year ago, but never got the guts to finish them ;))

I tried using the NNTP interface but it says I need to log in to post to the server. I can't see how to specify a password in Thunderbird :(

That and possibly other compiler flags, depending on the desired use case (e.g. debug vs. release version or the variant of an external link dependency that is used). But already all combinations of compiler (version), architecture and version flags can be far too many to install them all by default -- but if we accept that, we are back to caching compiled binaries lazily (and maybe compiling some kind of default configuration eagerly if desired), which is exactly what is planned.

Obviously not all combinations cannot be installed by default. I was thinking a release build should be installed, possibly a version with debug symbols enabled. I don't know how useful installing a debug build would be. I think one have to assume that the packages used are working perfectly. If some package doesn't, then one can start installing debug builds or similar.

Since the default build type is "debug" that should definitely be included. Executing the equivalent of "dub build" and "dub build --build=release" for each newly installed package would basically do this. Once a better build system is in place that would be an option.

This is more or less what I have in mind, except that binaries would be put into a directory that is unique for all aspects that affect binary generation (i.e. the above) instead of only the architecture. Or rather into multiple directories, each for one combination. The unique part could be a hash value of all related factors or we could try to get something human readable, such as "(compiler)-(compiler version)-(os)-(arch)-(configuration)-..."

Ok, I see.

DUB has "configurations", but only one configuration can be active at a time for each package. But the latter variant using a sub directory is what I'm also imagining (just using a scheme that includes all aspects as mentioned above).

I'm not 100% sure how MacPorts work when you need to install an additional variant. But several variants are possibly to combine. Say you have a library for manipulating images. By default it will build support JPEG and PNG. Variants exist for GIF and PSD as well. Of course all these variants will need to be able to be used together.

I don't know how "configurations" work in DUB but if variants corresponds to -version flags there is no problem with multiple variants. Just append all version flags, "-version=GIF -version=PSD -version=JPEG" and so on.

If you have only installed two variants out of four. Then need to install additional variants I would imagine it would just rebuild the library with the old variants and the newly added ones.

I MacPorts you can have multiple versions of a package installed but only have one active. What it does is that it will install the package in a path which is not accessed by default from PATH or the paths looked in for libraries. When setting a package to active it will create symbolic or hard links.

I think translated to D we can then just store a build per combination of version flags (and probably debug version flags). Different "configurations" could then end up in the same folder, but if all parameters are the same there is no reason to put the result into a different folder.

I agree that "~issue/2" should at least never be the default - this needs to be changed so that "~master" comes first. However, it's difficult to say what should be available. For example, there may be a "~2.x-devel" or a "~stable" branch that may very well make sense to use in some situations.

In general I agree of course that versioned releases are by far preferable, but during early development, or when having tightly coupled packages, it can be necessary to work on branches. I just don't want to make that impossible or force more structure/process on developers than necessary.

I would make it possible to in the registry specify which versions and branches are available. There could also be a flag in the registry to turn off default tracking of version tags and branches. Then one need to manually add them. I would also change so that by default you need to push a version tag before dub starts tracking that package.

Difficult call with the branches and starting tracking just after the first version tag. It would be a lot cleaner, but on the other hand there are a lot of packages now where the author doesn't feel comfortable yet to start tagging versions (even if that is an unjustified feeling). IF we end up with half the number of packages by forcing version tags, that would arguably be worse overall for the system.

Hmm.. maybe a little psychological trick can help here: non-tagged packages could always occur after tagged packages in the list (assuming that they are in a pre-release state) or get some kind of visual "beta" tag, so that the author is kindly forced to start making tagged versions to get more people to use their packages.

Note that I'm not talking about uploading zip archives or similar. I'm talking about just be able to specify for the registry that only these three tags and this one branch is available.

The semantic versioning scheme says what you can and cannot break when making a new release. But it also basically says that you can break how much you want in between releases.

I don't like if some starts using a non-released version of any of my packages, then says: "hey you broke the library in the last commit". If dub uses the master branch by default then I can't say "don't use HEAD, stick with releases".

True, but I also often have the situation with vibe.d that a quick fix is needed for someone and making a release would be too much effort. But maybe in that situation I should start thinking more about pre-release versions (e.g. "0.7.18-pre.1"). I'm not even sure if and how these are currently matched by DUB when using ">=0.7.17".

Using Bundler the standard way is to specify versions, which are downloaded from RubyGems. It's also possibly to specify a git repository, pointing to a branch, tag or any commit. But doing the latter you're kind of on own your own and have to accept possibly code breakage in any new commit.

Short summary: use in between releases on your own risk. I don't think dub should encourage this.

I agree. Currently it encourages version based usage not strong enough.

Pages: 1 2 3 4