RejectedSoftware Forums

Sign up

sourcePaths vs. importPaths

I was a bit confused at first about what sourcePaths did (I though it was the same as a source folder in Eclipse, in a sense), but now I get it. However, I can't say I like the design of it.
When someone defines a sourcePath in a DUB package, in the vast majority of times, they will want that to be in the import path as well (so it has to be defined in importPaths as well). You see, if you pass a D source file to DMD, but that source file is not part of the import path (or it is, but the (full) module name doesn't match the directory structure up to the import root), then that source file is not importable anywhere in D code.
This has to be a very, very niche use case then (I'd like to see examples otherwise). And partially covered by "sourceFiles" already. I would estimate that 99% of sourcePaths will be importPaths as well. And yet for that use case (a sourcePath folder is also an importPath folder) there is no way to do that with just one attribute in the .json. It would be better if we had such a mechanism.

I guess not that many people have been annoyed by this yet, only due to the implicitly defined source/import folders (that is, if you are using the "source" or "src" folders). In this way both sourcePath and importPaths will be the same automatically, because they match to the same folders, and nothing was explicitly defined.

Re: sourcePaths vs. importPaths

On Thu, 05 Dec 2013 15:42:55 GMT, Bruno Medeiros wrote:

I was a bit confused at first about what sourcePaths did (I though it was the same as a source folder in Eclipse, in a sense), but now I get it. However, I can't say I like the design of it.
When someone defines a sourcePath in a DUB package, in the vast majority of times, they will want that to be in the import path as well (so it has to be defined in importPaths as well).

Actually, I have to disagree here, and the current reality (in terms of existing packages) supports this. There seem to be 2 cases that cover >90% of the cases:

  1. Sources are all in "source/" or "src/" and the import path is auto-inferred by DUB
  2. Sources are in "<packagename>/" and the import path is set to "./"

You see, if you pass a D source file to DMD, but that source file is not part of the import path (or it is, but the (full) module name doesn't match the directory structure up to the import root), then that source file is not importable anywhere in D code.

I agree, but for completeness sake, it does work if the source file has a module declaration and is compiled together on the same command line as the module that imports it. But o course this cannot be assumed in general.

This has to be a very, very niche use case then (I'd like to see examples otherwise). And partially covered by "sourceFiles" already. I would estimate that 99% of sourcePaths will be importPaths as well. And yet for that use case (a sourcePath folder is also an importPath folder) there is no way to do that with just one attribute in the .json. It would be better if we had such a mechanism.

The big issue I see with this approach is that it will have much worse results in the above case 2, if used accidentally (quite likely, actually). Currently there will be a warning emitted that there is no import path defined if you just set "sourcePaths". But when "sourcePaths" also implies "importPaths", that will mean that there is no warning anymore, but the wrong import root is used. It would be necessary to parse the source files to automatically detect this.

I guess not that many people have been annoyed by this yet, only due to the implicitly defined source/import folders (that is, if you are using the "source" or "src" folders). In this way both sourcePath and importPaths will be the same automatically, because they match to the same folders, and nothing was explicitly defined.

Automatic inference and a common source root is an integral part of DUB's design, meant to facilitate this kind of source structure (without forcing it). I haven't yet seen the case that someone has a source/import root folder named differently - only as in case 2, and case 2 is, I'd argue, a very bad organization style for D projects, because it makes files importable (and possible conflict with other packages) that are not meant to be imported:

examples/
         x.d
test/
     testa.d
somelib/
        a.d
        b.d

Now test.testa or examples.x would be a valid import, although that would be senseless. And then this would possibly conflict with a separate test package.

Re: sourcePaths vs. importPaths

On 05/12/2013 16:43, Sönke Ludwig wrote:

On Thu, 05 Dec 2013 15:42:55 GMT, Bruno Medeiros wrote:

I was a bit confused at first about what sourcePaths did (I though it was the same as a source folder in Eclipse, in a sense), but now I get it. However, I can't say I like the design of it.
When someone defines a sourcePath in a DUB package, in the vast majority of times, they will want that to be in the import path as well (so it has to be defined in importPaths as well).

Actually, I have to disagree here, and the current reality (in terms of existing packages) supports this. There seem to be 2 cases that cover >90% of the cases:

  1. Sources are all in "source/" or "src/" and the import path is auto-inferred by DUB

I spoke incorrectly: I originally meant to say the majority of source
folders will be import paths as well (not just defined sourcePaths, but
including the implicitly defined ones as well). So that includes case 1.

  1. Sources are in "<packagename>/" and the import path is set to "./"

I didn't foresee this use case.

You see, if you pass a D source file to DMD, but that source file is not part of the import path (or it is, but the (full) module name doesn't match the directory structure up to the import root), then that source file is not importable anywhere in D code.

I agree, but for completeness sake, it does work if the source file has a module declaration and is compiled together on the same command line as the module that imports it. But o course this cannot be assumed in general.

You're right, it does work if passed on the same invocation of DMD as
the module that imports it. I'd forgotten about that.
However this doesn't work if the files are compiled separately, so in
order to support that, the compilation/import model shouldn't rely on
that fact.

This has to be a very, very niche use case then (I'd like to see examples otherwise). And partially covered by "sourceFiles" already. I would estimate that 99% of sourcePaths will be importPaths as well. And yet for that use case (a sourcePath folder is also an importPath folder) there is no way to do that with just one attribute in the .json. It would be better if we had such a mechanism.

The big issue I see with this approach is that it will have much worse results in the above case 2, if used accidentally (quite likely, actually). Currently there will be a warning emitted that there is no import path defined if you just set "sourcePaths". But when "sourcePaths" also implies "importPaths", that will mean that there is no warning anymore, but the wrong import root is used. It would be necessary to parse the source files to automatically detect this.

I don't think DUB would have to bother parsing source files to emit a
warning for this. DMD import errors should be enough to realize that an
error in structure has been made.
Or perhaps I misunderstood your example.
I any case, like you said, case 2 is a arguably a bad organization style
for the project source (I agree).

I guess not that many people have been annoyed by this yet, only due to the implicitly defined source/import folders (that is, if you are using the "source" or "src" folders). In this way both sourcePath and importPaths will be the same automatically, because they match to the same folders, and nothing was explicitly defined.

Automatic inference and a common source root is an integral part of DUB's design, meant to facilitate this kind of source structure (without forcing it). I haven't yet seen the case that someone has a source/import root folder named differently - only as in case 2, and case 2 is, I'd argue, a very bad organization style for D projects, because it makes files importable (and possible conflict with other packages) that are not meant to be imported:

examples/
          x.d
test/
      testa.d
somelib/
         a.d
         b.d

Now test.testa or examples.x would be a valid import, although that would be senseless. And then this would possibly conflict with a separate test package.

I don't understand what you're trying to say here..

Re: sourcePaths vs. importPaths

Bruno Medeiros wrote:

I was a bit confused at first about what sourcePaths did (I though it was the
same as a source folder in Eclipse, in a sense), but now I get it. However, I
can't say I like the design of it. When someone defines a sourcePath in a DUB
package, in the vast majority of times, they will want that to be in the
import path as well (so it has to be defined in importPaths as well). You see,
if you pass a D source file to DMD, but that source file is not part of the
import path (or it is, but the (full) module name doesn't match the directory
structure up to the import root), then that source file is not importable
anywhere in D code. This has to be a very, very niche use case then (I'd
like to see examples otherwise). And partially covered by "sourceFiles"
already. I would estimate that 99% of sourcePaths will be importPaths as well.
And yet for that use case (a sourcePath folder is also an importPath folder)
there is no way to do that with just one attribute in the .json. It would be
better if we had such a mechanism.

I guess not that many people have been annoyed by this yet, only due to the
implicitly defined source/import folders (that is, if you are using the
"source" or "src" folders). In this way both sourcePath and importPaths will
be the same automatically, because they match to the same folders, and nothing
was explicitly defined.

Are you kidding? :)

How about a project that has the same source tree, but produces, say three
executables, and two libraries? :D Furthermore, these two libraries may be in
completely independent packages, one is com.vendorX.productX, and second is
so.ddn.sql.driver.vendorX ... In this case you have one sourcePath, and at least
two import paths. Not to mention your project's dependencies...

Unfortunately, D still has no project (that I know of) that is of such
complexity. Wait until D gets more adopted by big companies, and gets used in
large projects...

Dejan Lekic
dejan.lekic (a) gmail.com
http://dejan.lekic.org

Re: sourcePaths vs. importPaths

On 2013-12-06 21:51, Dejan Lekic wrote:

Are you kidding? :)

How about a project that has the same source tree, but produces, say three
executables, and two libraries? :D Furthermore, these two libraries may be in
completely independent packages, one is com.vendorX.productX, and second is
so.ddn.sql.driver.vendorX ... In this case you have one sourcePath, and at least
two import paths. Not to mention your project's dependencies...

Unfortunately, D still has no project (that I know of) that is of such
complexity. Wait until D gets more adopted by big companies, and gets used in
large projects...

DWT has quite a complex structure if you count the snippets. Although
they are currently in a git submoulde. DWT builds a static library and
can build a bunch of small executables for the snippets.

/Jacob Carlborg