RejectedSoftware Forums

Sign up

Embedded dub for scripting?

Hi Sönke,

Recently I sneaked D into our team by using it as a scripting language instead of using Python/bash.

Although I've not actually coded in D very much, the scripting experience was smooth. Soon I found that writing standalone little scripts with only phobos to assist was not enough, I came to be needing scriptlike, bzip and other facilities. So I came to realize that what I really need is dub support for scripting. I'd really like to just 'import' those third party dependencies with my script, like this:

#!/usr/bin/dub

@dependencies(`{
    "scriptlike" : ">=0.6.0",
    "bzip2" : "~master"
}`)

import scriptlike;

void main()
{
   // ... actual D scripting code
}

Then when invoked like ./my_script.d -c -i {args}, dub will do some preprocessing and treat this single file just like a dub project with dub.json read from the header comments and the D file as src/app.d

My philosophy is that to compete with python and other scripting languanges, a directory tree with config files is too heavy. I'd like to go even further to support something like this:

#!/usr/bin/dub

@require("scriptlike", ">=0.6.0");
@requrre("bzip2"); // version is optional

// don't even need to write import and (even) void main(), dub will generate them
run("cat hello >> output.txt");
bzip2("output.txt");

with this ability and carelly written scripting libraries like scriptlike, writing scripts in D would really shine IMHO.

What do you think?

Re: Embedded dub for scripting?

On Wed, 21 May 2014 10:19:36 GMT, zhaopuming wrote:

Hi Sönke,

Recently I sneaked D into our team by using it as a scripting language instead of using Python/bash.

Although I've not actually coded in D very much, the scripting experience was smooth. Soon I found that writing standalone little scripts with only phobos to assist was not enough, I came to be needing scriptlike, bzip and other facilities. So I came to realize that what I really need is dub support for scripting. I'd really like to just 'import' those third party dependencies with my script, like this:

#!/usr/bin/dub

@dependencies(`{
    "scriptlike" : ">=0.6.0",
    "bzip2" : "~master"
}`)

import scriptlike;

void main()
{
   // ... actual D scripting code
}

Then when invoked like ./my_script.d -c -i {args}, dub will do some preprocessing and treat this single file just like a dub project with dub.json read from the header comments and the D file as src/app.d

My philosophy is that to compete with python and other scripting languanges, a directory tree with config files is too heavy. I'd like to go even further to support something like this:

#!/usr/bin/dub

@require("scriptlike", ">=0.6.0");
@requrre("bzip2"); // version is optional

// don't even need to write import and (even) void main(), dub will generate them
run("cat hello >> output.txt");
bzip2("output.txt");

with this ability and carelly written scripting libraries like scriptlike, writing scripts in D would really shine IMHO.

What do you think?

Hi Puming,

I think this is generally a very worthwhile direction to go. Regarding the @require("scriptlike"); approach, I think the only way to make this work would be to pre-process the source code and actually replace those lines with actual import statements before passing it to the compiler. But that would be not so pretty (IMO) for the reason that it would tie it to DUB as a build tool.

The current idea was to use a specially formatted comment instead of UDAs, because that may be easier to parse and doesn't require additional work to keep the source code a valid D program. See #103 and this thread for some basic ideas.

But inverse way was also one of the ideas - DUB would search for import statements and query the registry for matching packages. Additional comments or pragmas could be used for fine tuning (selecting specific version ranges).

Basically what is missing is just a complete, well thought out proposal, before it can be implemented.

Re: Embedded dub for scripting?

Hi Sönke,

I think this is generally a very worthwhile direction to go. Regarding the @require("scriptlike"); approach, I think the only way to make this work would be to pre-process the source code and actually replace those lines with actual import statements before passing it to the compiler. But that would be not so pretty (IMO) for the reason that it would tie it to DUB as a build tool.

The current idea was to use a specially formatted comment instead of UDAs, because that may be easier to parse and doesn't require additional work to keep the source code a valid D program. See #103 and this thread for some basic ideas.

Yes, I like the Comment idea :-) I chose UDA only because it makes this dependency mechanism more like a language feature (see node.js), and that would be good to newbies.

So this has been proposed earlier in that thread in a different use case. Happy to know that people needs this.

But inverse way was also one of the ideas - DUB would search for import statements and query the registry for matching packages. Additional comments or pragmas could be used for fine tuning (selecting specific version ranges).

A minor issue would be that import module name might not be the same with registered dub package name, adding another indirection.

But overall i think this approach is better.

Basically what is missing is just a complete, well thought out proposal, before it can be implemented.

Where should we put proposals about DUB, create a D(ub)IP ?

Re: Embedded dub for scripting?

On Thu, 22 May 2014 02:08:14 GMT, zhaopuming wrote:

Hi Sönke,

I think this is generally a very worthwhile direction to go. Regarding the @require("scriptlike"); approach, I think the only way to make this work would be to pre-process the source code and actually replace those lines with actual import statements before passing it to the compiler. But that would be not so pretty (IMO) for the reason that it would tie it to DUB as a build tool.

The current idea was to use a specially formatted comment instead of UDAs, because that may be easier to parse and doesn't require additional work to keep the source code a valid D program. See #103 and this thread for some basic ideas.

Yes, I like the Comment idea :-) I chose UDA only because it makes this dependency mechanism more like a language feature (see node.js), and that would be good to newbies.

So this has been proposed earlier in that thread in a different use case. Happy to know that people needs this.

UDAs definitely look more friendly. We could also use custom pragma()s, such as:

pragma(dub_dependency, "scriptlike: ~>1.0.0");
import scriptlike.something;

.. maybe nicer semantics that using comments, but with considerable syntax overhead.

But inverse way was also one of the ideas - DUB would search for import statements and query the registry for matching packages. Additional comments or pragmas could be used for fine tuning (selecting specific version ranges).

A minor issue would be that import module name might not be the same with registered dub package name, adding another indirection.

But overall i think this approach is better.

Yes, there are also definitely potential ambiguities here, so there still needs to be a way to explicitly state which package is required.

Basically what is missing is just a complete, well thought out proposal, before it can be implemented.

Where should we put proposals about DUB, create a D(ub)IP ?

The DIP-like idea sounds nice. I've created a little wiki page for this (Using DUB Enhancement Proposals for the lack of a better name).

Re: Embedded dub for scripting?

But inverse way was also one of the ideas - DUB would search for import statements and query the registry for matching packages. Additional comments or pragmas could be used for fine tuning (selecting specific version ranges).

A minor issue would be that import module name might not be the same with registered dub package name, adding another indirection.

But overall i think this approach is better.

  1. Is it sane to include a D parser into DUB?
  2. I don't really want all my source files to be parsed each time I build. With which parser?

Re: Embedded dub for scripting?

On Thu, 29 May 2014 22:08:37 GMT, ponce wrote:

But inverse way was also one of the ideas - DUB would search for import statements and query the registry for matching packages. Additional comments or pragmas could be used for fine tuning (selecting specific version ranges).

A minor issue would be that import module name might not be the same with registered dub package name, adding another indirection.

But overall i think this approach is better.

  1. Is it sane to include a D parser into DUB?
  2. I don't really want all my source files to be parsed each time I build. With which parser?

No, I wouldn't want to include a full parser and it also wouldn't scan all files on each build, just a single file in case of script-like single file projects. Depending on the approach, it would simply search for a pattern like "/**DUB\n" and the following "*/", or it would invoke DMD with the -ignore -v switches to let it output any pragmas.