Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Posted Fri, 31 Oct 2014 13:59:54 GMT

Reply

When using dub to init projects, it can sometimes be annoying to manually look up each project on code.dlang.org to find the newest version for each required dependency, just so you can copy-paste that into the dub.json file.

Would there be much appetite for extending the init command to take in a list of dependency projects, and have it go and look up the DB to find the most recent version string and create the required dub file for you?

So:
dub init myProject openssl logger

would automatically add: (correct at time of writing)

"openssl": ">=1.1.3+1.0.1g"
"logger": ">=0.3.3"

to your dependencies section.

It already supports this somewhat with passing vibe.d to the init command, though I dont know if this goes to the server to find the most recent version number.

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Matt Soucy

Posted Sun, 02 Nov 2014 17:09:04 -0500 in reply to Colin Grogan

Reply

On 10/31/2014 09:59 AM, Colin Grogan wrote:

When using dub to init projects, it can sometimes be annoying to manually look up each project on code.dlang.org to find the newest version for each required dependency, just so you can copy-paste that into the dub.json file.

Would there be much appetite for extending the init command to take in a list of dependency projects, and have it go and look up the DB to find the most recent version string and create the required dub file for you?

So:
dub init myProject openssl logger

would automatically add: (correct at time of writing)

"openssl": ">=1.1.3+1.0.1g"
"logger": ">=0.3.3"

to your dependencies section.

It already supports this somewhat with passing vibe.d to the init command, though I dont know if this goes to the server to find the most recent version number.

Assuming it can fail gracefully, so that I can start a project without having an internet connection, I think it'd be a fantastic feature.

Matt Soucy

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Colin Grogan

Posted Tue, 04 Nov 2014 20:13:02 GMT in reply to Matt Soucy

Reply

On Sun, 02 Nov 2014 17:09:04 -0500, Matt Soucy wrote:

On 10/31/2014 09:59 AM, Colin Grogan wrote:

When using dub to init projects, it can sometimes be annoying to manually look up each project on code.dlang.org to find the newest version for each required dependency, just so you can copy-paste that into the dub.json file.

Would there be much appetite for extending the init command to take in a list of dependency projects, and have it go and look up the DB to find the most recent version string and create the required dub file for you?

So:
dub init myProject openssl logger

would automatically add: (correct at time of writing)

"openssl": ">=1.1.3+1.0.1g"
"logger": ">=0.3.3"

to your dependencies section.

It already supports this somewhat with passing vibe.d to the init command, though I dont know if this goes to the server to find the most recent version number.

Assuming it can fail gracefully, so that I can start a project without having an internet connection, I think it'd be a fantastic feature.

Matt Soucy

Yeah, thats an important aspect of it alright. It should be seamless to what is currently there I think.

I've been tinkering with dub and there's a few ways of doing it.

The way I'm favoring would be to download the package list (Using dub update-local-list or something like that) and save it to a file in ~/.dub.
When you run

dub init dep1 dep2

it checks that file for dep1, dep2 and sticks it into your dub.json.

That would probably be the easiest and most non-intrusive way of doing it.

If dub had to search the server for every init it would probably make it notably slower (network latency be slow and all), plus you run into situations where there's no network connection as you say, complicating logic.

I'm tinkering with it anyway, will post something here soon!

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

notna

Posted Thu, 06 Nov 2014 22:45:52 GMT in reply to Colin Grogan

Reply

Given that code.dlang.org has only GitHub packages, if I'm not wrong, I'd wonder why you'd like to make things so complicated?

I'd simply "encourage", or maybe even enforce :O, the projects, which want to have DUB support, to provide a special tag, something like "latest", "dub", "dubreleased" or "4dubinit" in the GitHub repo.

In parallel DUB should provide a command line flag to write this "tag info" into the dub.json file for all the given packages with the init option, like:

"dependencies": {
"openssl": "latest",
"logger": "latest",
}

Think you got the idea ;)

Regards
Anton Oks

P.S.: I know about "~master", which could also be the "special tag" and was used before. But Sönke posted some time ago why this "flag" is deprecated, so...

On Tue, 04 Nov 2014 20:13:02 GMT, Colin Grogan wrote:

On Sun, 02 Nov 2014 17:09:04 -0500, Matt Soucy wrote:

On 10/31/2014 09:59 AM, Colin Grogan wrote:

When using dub to init projects, it can sometimes be annoying to manually look up each project on code.dlang.org to find the newest version for each required dependency, just so you can copy-paste that into the dub.json file.

Would there be much appetite for extending the init command to take in a list of dependency projects, and have it go and look up the DB to find the most recent version string and create the required dub file for you?

So:
dub init myProject openssl logger

would automatically add: (correct at time of writing)

"openssl": ">=1.1.3+1.0.1g"
"logger": ">=0.3.3"

to your dependencies section.

It already supports this somewhat with passing vibe.d to the init command, though I dont know if this goes to the server to find the most recent version number.

Assuming it can fail gracefully, so that I can start a project without having an internet connection, I think it'd be a fantastic feature.

Matt Soucy

Yeah, thats an important aspect of it alright. It should be seamless to what is currently there I think.

I've been tinkering with dub and there's a few ways of doing it.

The way I'm favoring would be to download the package list (Using dub update-local-list or something like that) and save it to a file in ~/.dub.
When you run
dub init dep1 dep2
it checks that file for dep1, dep2 and sticks it into your dub.json.

That would probably be the easiest and most non-intrusive way of doing it.

If dub had to search the server for every init it would probably make it notably slower (network latency be slow and all), plus you run into situations where there's no network connection as you say, complicating logic.

I'm tinkering with it anyway, will post something here soon!

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Colin Grogan

Posted Fri, 07 Nov 2014 00:35:52 GMT in reply to notna

Reply

So, I spent more hours than I'd like to admit understanding the dub code and where I need to put stuff etc.

Anyway, it turned out to be MUCH simpler than I thought it would be to do this.

I simply have a new function in init.d that simply gets the versions for each of the strings in deps, if it doesnt exist on code.dlang.org it ignores it.

It then calls writePackageJson as dub always has.

void initMultiDepPackage(Path root_path, string[] deps){
	//import std.stdio;
	import dub.packagesupplier : RegistryPackageSupplier;

	string[string] depVers;
	auto ps = new RegistryPackageSupplier(URL.parse("http://code.dlang.org/")); // hacky, need to get Dub's Package Supplier service, will work better
	logDebug("Searching server for latest versions for %s", deps);
	foreach(dep; deps){
		try{
			auto versionStrings = ps.getVersions(dep);
			depVers[dep] = versionStrings[$-1].toString;
		} catch(Exception e){
			logError("Error, no package %s found", dep);
			logError(e.msg);
		}
	}
	logDebug("Dependancy Versions: %s", depVers);
	writePackageJson(root_path, "A D application", depVers); // same as usual
	createDirectory(root_path ~ "source");
	write((root_path ~ "source/app.d").toNativeString(),
q{import std.stdio;

	void main()
	{
		writeln("Edit source/app.d to start your project.");
	}
});
}

Two other changes are required in commandline.d and dub.d, both are minor (calling the new function basically)

Have done some minor testing, seems to work pretty well. Will spend more time at it tomorrow to test properly and iron out some things.

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Colin Grogan

Posted Fri, 07 Nov 2014 17:26:16 GMT in reply to Colin Grogan

Reply

Ok, so re-worked it and am pretty happy with it now.

The searching of new packages is done by the Dub class now, so no hacky

auto ps = new RegistryPackageSupplier(URL.parse("http://code.dlang.org/"));

anymore.

I've tested it a bit (with a list of good package names, bad package names, good and bad, no package names etc.)

The current "type" functionality where you pass "minimal", "vibe.d" or "deimos" to dub init has been moved into the --type option.

For example, the command:

dub init myProject vibe-d openssl logger --type=vibe.d

will produce the following dub.json

{
	"name": "myproject",
	"description": "A simple vibe.d server application.",
	"copyright": "Copyright © 2014, Colin",
	"authors": ["Colin"],
	"dependencies": {
		"logger": "0.3.3",
		"vibe-d": "0.7.21-rc.4",
		"openssl": "1.1.3+1.0.1g"
	},
	"versions": ["VibeDefaultMain"]
}

I've checked what running it with no network connection, and while it doesn't fall over, it does wait for the server request to time-out (so 2 or 3 seconds for each package name in the list), which is a little annoying, but not overly debilitating.

It's up here:
https://github.com/grogancolin/dub/tree/init-with-dependencies

Would appreciate if people give it a quick test to see if it works for them?

I've used Windows 7 and a linux box (Ubuntu 12.04 I think...)

Thanks!

Re: Possible improvement for dub's "init" command

Permalink: HTTP NNTP

Colin Grogan

Posted Fri, 07 Nov 2014 17:44:01 GMT in reply to Colin Grogan

Reply

Also, pull request:
https://github.com/D-Programming-Language/dub/pull/453