dub randomly fails to write a static library file

Permalink: HTTP NNTP

Posted Sun, 11 May 2014 20:15:10 GMT

Reply

Every once in a while I get this error:

Error: Error writing file '..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\DerelictASSIMP3.lib'

It seems to be completely random. It works 99% of the time, and then fails. My disk has plenty of free space. Could it be some kind of locking issue?

Here's the stack trace:

Full exception: object.Exception@source\dub\compilers\dmd.d(173): DMD compile run failed with exit code 1
----------------
0x004B3926 in pure @safe void std.exception.bailOut(immutable(char)[], uint, const(char[]))
0x0042CE34 in void dub.compilers.dmd.DmdCompiler.invoke(const(dub.compilers.compiler.BuildSettings), const(dub.compilers.compiler.BuildPlatform)) at C:\Users\build\AppData\Roaming\dub-ci-workspace\current\dub\source\dub\compilers\dmd.d(173)
0x00435192 in void dub.generators.build.BuildGenerator.buildWithCompiler(dub.generators.generator.GeneratorSettings, dub.compilers.compiler.BuildSettings) at C:\Users\build\AppData\Roaming\dub-ci-workspace\current\dub\source\dub\generators\build.d(351)
0x00433D4D in void dub.generators.build.BuildGenerator.performCachedBuild(dub.generators.generator.GeneratorSettings, dub.compilers.compiler.BuildSettings, const(dub.package_.Package), immutable(char)[], immutable(char)[]) at C:\Users\build\AppData\Roaming\dub-ci-workspace\current\dub\source\dub\generators\build.d(139)
0x004339D1 in void dub.generators.build.BuildGenerator.buildTarget(dub.generators.generator.GeneratorSettings, dub.compilers.compiler.BuildSettings, const(dub.package_.Package), immutable(char)[]) at C:\Users\build\AppData\Roaming\dub-ci-workspace\current\dub\source\dub\generators\build.d(97)
0x004337C8 in void dub.generators.build.BuildGenerator.generateTargets(dub.generators.generator.GeneratorSettings, const(dub.generators.generator.ProjectGenerator.TargetInfo[immutable(char)[]])).void buildTargetRec(immutable(char)[]) at C:\Users\build\AppData\Roaming\dub-ci-workspace\current\dub\source\dub\generators\build.d(65)

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Andrej Mitrovic

Posted Sun, 11 May 2014 20:25:36 GMT in reply to Andrej Mitrovic

Reply

On Sun, 11 May 2014 20:15:10 GMT, Andrej Mitrovic wrote:

Every once in a while I get this error:

Looks like it's not dub, but DMD:

dmd -lib -of..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\DerelictASSIMP3.lib -debug -g -w -version=gl3n -version=glad -version=SDLImage2 -version=DisplayFrameRate -version=Have_derelict_assimp3 -version=Have_derelict_util -I..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\source -I..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-util-1.0.1\source ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\source\derelict\assimp3\assimp.d ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\source\derelict\assimp3\types.d

Error: Error writing file '..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\DerelictASSIMP3.lib'

I'll investigate further.

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Andrej Mitrovic

Posted Sun, 11 May 2014 20:32:10 GMT in reply to Andrej Mitrovic

Reply

On Sun, 11 May 2014 20:25:36 GMT, Andrej Mitrovic wrote:

Looks like it's not dub, but DMD:

Further reduced, this seems to be some kind of line-length limit issue:

test.bat:

@echo off

rem This works, path has 182 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfa.lib

rem This fails, path has 183 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfaX.lib

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Andrej Mitrovic

Posted Sun, 11 May 2014 20:42:02 GMT in reply to Andrej Mitrovic

Reply

On Sun, 11 May 2014 20:32:10 GMT, Andrej Mitrovic wrote:

test.bat:

Note that that example isn't good without having the existing path, so here's a full batch file that another person has been able to reproduce as well:

@echo off
setlocal enableextensions
md C:\dev\projects\opengl-tutorials\ports\opengl-tutorial.org\tutorials\11_text
endlocal

cd C:\dev\projects\opengl-tutorials\ports\opengl-tutorial.org\tutorials\11_text

rem This works, path has 182 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfa.lib

rem This fails, path has 183 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfaX.lib

Anyway the first command is exactly 260 characters (MAX_PATH), the second command exceeds it. Is there any way for dub to work around this?

Vladimir Panteleev gave some insights:

[22:41] <CyberShadow> From D code you can use the \\?\ prefix (assuming you use the Unicode APIs) to bypass that limitation

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Sönke Ludwig

Posted Mon, 12 May 2014 07:48:23 +0200 in reply to Andrej Mitrovic

Reply

Am 11.05.2014 22:42, schrieb Andrej Mitrovic:

On Sun, 11 May 2014 20:32:10 GMT, Andrej Mitrovic wrote:

test.bat:

Note that that example isn't good without having the existing path, so here's a full batch file that another person has been able to reproduce as well:
@echo off
setlocal enableextensions
md C:\dev\projects\opengl-tutorials\ports\opengl-tutorial.org\tutorials\11_text
endlocal

cd C:\dev\projects\opengl-tutorials\ports\opengl-tutorial.org\tutorials\11_text

rem This works, path has 182 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfa.lib

rem This fails, path has 183 chars
echo. 2>  ..\..\..\..\..\..\..\Users\Administrator\AppData\Roaming\dub\packages\derelict-assimp3-master\.dub\build\library-debug-windows-x86-dmd-7F811CB10FFAE5D52D1E21B73C0EC98D\aasdfasdfaX.lib
Anyway the first command is exactly 260 characters (MAX_PATH), the second command exceeds it. Is there any way for dub to work around this?

Vladimir Panteleev gave some insights:

[22:41] <CyberShadow> From D code you can use the \\?\ prefix (assuming you use the Unicode APIs) to bypass that limitation

Sounds like that could simply be added to Path.toNativeString(). Hmm..
but I guess the \\?\ syntax will always make up an absolute path, so
there would also need to be some logic to avoid the Path.relativeTo()
call in various places, so it will probably require an additional
shortenPath(path, base) function that conditionally does the right
thing. I can look into that.

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Andrej Mitrovic

Posted Mon, 12 May 2014 09:00:32 GMT in reply to Sönke Ludwig

Reply

On Mon, 12 May 2014 07:48:23 +0200, Sönke Ludwig wrote:

so it will probably require an additional
shortenPath(path, base) function that conditionally does the right
thing. I can look into that.

Yeah I was just thinking that in some cases use absolute paths could create shorter paths, so this could be a good idea.

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Andrej Mitrovic

Posted Tue, 13 May 2014 18:29:03 GMT in reply to Andrej Mitrovic

Reply

On Mon, 12 May 2014 09:00:32 GMT, Andrej Mitrovic wrote:

On Mon, 12 May 2014 07:48:23 +0200, Sönke Ludwig wrote:

so it will probably require an additional
shortenPath(path, base) function that conditionally does the right
thing. I can look into that.

Yeah I was just thinking that in some cases use absolute paths could create shorter paths, so this could be a good idea.

Is there any workaround I can use right now other than to run set APPDATA=C:\TEMP before running dub to shorten the paths? I'm now constantly running into this issue because I have enough dependencies now which trigger path length issues.

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Sönke Ludwig

Posted Tue, 13 May 2014 21:27:44 GMT in reply to Andrej Mitrovic

Reply

On Tue, 13 May 2014 18:29:03 GMT, Andrej Mitrovic wrote:

On Mon, 12 May 2014 09:00:32 GMT, Andrej Mitrovic wrote:

On Mon, 12 May 2014 07:48:23 +0200, Sönke Ludwig wrote:

so it will probably require an additional
shortenPath(path, base) function that conditionally does the right
thing. I can look into that.

Yeah I was just thinking that in some cases use absolute paths could create shorter paths, so this could be a good idea.

Is there any workaround I can use right now other than to run set APPDATA=C:\TEMP before running dub to shorten the paths? I'm now constantly running into this issue because I have enough dependencies now which trigger path length issues.

As a temporary workaround, you could move all packages in %APPDATA%\dub\packages to C:\dubpacks or similar and then dub add-path C:\dubpacks.

Re: dub randomly fails to write a static library file

Permalink: HTTP NNTP

Byron Heads

Posted Wed, 16 Dec 2015 15:40:55 GMT in reply to Sönke Ludwig

Reply

On Tue, 13 May 2014 21:27:44 GMT, Sönke Ludwig wrote:

On Tue, 13 May 2014 18:29:03 GMT, Andrej Mitrovic wrote:

On Mon, 12 May 2014 09:00:32 GMT, Andrej Mitrovic wrote:

On Mon, 12 May 2014 07:48:23 +0200, Sönke Ludwig wrote:

so it will probably require an additional
shortenPath(path, base) function that conditionally does the right
thing. I can look into that.

Yeah I was just thinking that in some cases use absolute paths could create shorter paths, so this could be a good idea.

Is there any workaround I can use right now other than to run set APPDATA=C:\TEMP before running dub to shorten the paths? I'm now constantly running into this issue because I have enough dependencies now which trigger path length issues.

As a temporary workaround, you could move all packages in %APPDATA%\dub\packages to C:\dubpacks or similar and then dub add-path C:\dubpacks.

Has any solution been found for this issue? I am trying to get dub in our CI builds :) Sadly I am stuck using Maven to drive dub at the moment. (work in a Java shop, getting a small part of the project in D was tricky enough)