RejectedSoftware Forums

Sign up

Pages: 1 2

Are you sure about SDL?

I am currently working on DUB to support the new SDL package format. Initially I was very happy to see a new format, especially one that seemed very simple and intuitive. However, after getting intimate with SDL I found that even though it seemed simple and stripped down, it actually has some odd "eccentricities". I wanted to see if there was a language that was as easy to read/write as SDL but didn't have so many odd rules. I spent some time looking at other formats but ended up attempting to write my own. I started with a clean slate and when I was finished I realized that I just made another variation of JSON...so I called it ASON( Application Specific Object Notation). I know that this discussion has already been had here, but I thought this language had enough advantages over SDL that it was worth it to see what people think before we take the final plunge and implement the SDL package format.

The first thing I'll say is that ASON is a superset of JSON, so every JSON file is also a valid ASON file. This means that if DUB were to start using ASON, we could swap out the parser and every DUB package file would still be valid. Furthermore, even though ASON supports different ways to write the same data, it represents the exact same data structures as JSON, so there would be no need to change the build specification. This is not the case with SDL. SDL is basically a variant of XML. Every object in Dub's build specification will require a specification of which fields should be represented as values, attributes or child tags, i.e.

dependency "vibe-d" ">=1.0.0"
// OR
dependency "vibe-d" version=">=1.0.0"
// OR
dependency name="vibe-d" version=">=1.0.0"
// OR
dependency "vibe-d" {
    version ">=1.0.0"
}
// There are more variations but you get the idea.

The next critique I have of SDL is it doesn't have a solid specification. It DOES NOT include a grammar and when I attempted to write one it was clear that writing it was not a trivial task. I was in correspondence with the creator of SDL and I brought up many odd corner cases that were not in the specification. He recommended I use the Java/C# implementations to make sure I was following the specification. ASON on the other hand includes a full grammar, complete with visual railroad version (like the one for JSON).

I've included a list of s-ludwig's initial critique of JSON and how each language solves them:

  • No Comments

    • SDL: Supports comments
    • ASON: Supports comments
  • Key names must be quoted

    • SDL: Uses "tags" that cannot have quotes
    • ASON: Supports non-quoted strings
  • Commas between fields

    • SDL: No Commas
    • ASON: Commas are optional, and supports a trailing comma
  • Easy to learn but verbose

    • SDL: Concise and easy to learn as long as you don't do anything odd
    • ASON: Concise and easy to learn, especially those familair with JSON
  • Deep nesting

    • SDL: Solves by ammending the specification to specify that list objects can be written a different way
    • ASON: Solves using the "SingularName" feature of the language

If you liked SDL you will also like ASON since they look so similar. In fact if you don't use attributes, many SDL files are also valid ASON files. You could say that ASON is what SDL would be if it was based from JSON instead of XML. Here's what the dub package format would look like using ASON:

name "my-package"
description "A package for demonstration purposes"

dependency "vibe-d"       {version ">=0.7.13"}
dependency "sub-package"  {version "~master", path "./sub-package"}

// command line version
configuration "console" {
    targetType "executable"
    versions ["ConsoleApp"]
    lib-windows ["gdi32" "user32"]
}

// Win32 based GUI version
configuration "gui" {
    targetType "executable"
    versions ["UseWinMain"]
    lib-windows ["gdi32" "user32"]    
}

Re: Are you sure about SDL?

I kind of get the feeling you're trying to push dub to use ASON.

My first reaction is, please don't invent yet another markup language for developers to learn. Especially as this new language is only relevant to dub. Honestly it would seriously make me not want to use dub. Just pick one of the existing ones, Yaml, Json, SDL (i've never heard of this one.) and stick with it. All other package managers seem to do just fine. Don't fix what isn't broken.

http://xkcd.com/927/

Seriously though are you also willing to break all existing dub.json files? That's 317 (http://code.dlang.org/ as of today) packages this change is going to mess with. Are you really going to do that? Just as things are starting to take off?

I'm in favour of just sticking to plain, normal old Json. Everyone knows it, everyone understands it and everything else does just fine using it.

Re: Are you sure about SDL?

Please read the entire post when you reply.

I kind of get the feeling you're trying to push dub to use ASON.

Yes, in fact I explicitly stated that was my intention.

My first reaction is, please don't invent yet another markup language for developers to learn. Especially as this new language is only relevant to dub.

That would have also been my first reaction. However, instead of immediately dismissing the idea, it would be more productive to state why you think the language is not a good fit, or why you think SDL will do just fine. I explained why I think ASON would be a better fit than SDL, do you have a counter-argument?

Honestly it would seriously make me not want to use dub. Just pick one of the existing ones, Yaml, Json, SDL (i've never heard of this one.) and stick with it. All other package managers seem to do just fine. Don't fix what isn't broken.

It would make you not want to use DUB? Could you please explain why? If we used ASON you wouldn't have to change anything, you could just continue to use regular JSON since it is already valid ASON.

Seriously though are you also willing to break all existing dub.json files? That's 317 (http://code.dlang.org/ as of today) packages this change is going to mess with. Are you really going to do that? Just as things are starting to take off?

No, I wouldn't be willing to break anything. That's why I like ASON, it doesn't break anything. Please read the post before you reply next time.

I'm in favour of just sticking to plain, normal old Json. Everyone knows it, everyone understands it and everything else does just fine using it.

That's fine, if we used ASON you could just continue to use plain old JSON.

Re: Are you sure about SDL?

On 8/25/2014 2:27 PM, Gary Willoughby wrote:

Seriously though are you also willing to break all existing dub.json files?

OMG, every time a JSON alternative for dub ever gets brought up,
somebody claims that it's an all-or-nothing deal. NOT TRUE. THERE IS NO
BREAKING INVOLVED HERE.

There has NEVER been ANY intent by ANYONE for dub to replace JSON.
ONLY to supplement it with an additional option. Adding support for
one language does not remove support for another. There is no
"switching" languages going on, only adding.

Pardon my reaction, but this has already been reiterated over, and over,
and over, and over.

Another thing I should address before it too gets brought up yet again:
Adding support for SDL would NOT complicate tools that wish to work with
dub or force them to support two languages: The standard way to read a
dub package description is to use the output of "dub describe", not to
parse dub.json directly.

Re: Are you sure about SDL?

On 8/25/2014 12:25 PM, Jonathan Marler wrote:

The next critique I have of SDL is it doesn't have a solid specification. It DOES NOT include a grammar and when I attempted to write one it was clear that writing it was not a trivial task.

The LL parsing section of my SDL lib is annotated with a grammar
(including lookahead information) for SDL's nonterminals:

https://github.com/Abscissa/SDLang-D/blob/master/src/sdlang_/parser.d

Reproduced here (Ident and Value are terminals handles by the lexer):

<Root> ::= <Tags> (Lookaheads: Anything)

<Tags>

 ::= <Tag> <Tags>  (Lookaheads: Ident Value)
 |   EOL   <Tags>  (Lookaheads: EOL)
 |   {empty}       (Lookaheads: Anything else)

<Tag>

 ::= <IDFull> <Values> <Attributes> <OptChild> <TagTerminator> 

(Lookaheads: Ident)

 |   <Value>  <Values> <Attributes> <OptChild> <TagTerminator> 

(Lookaheads: Value)

<IDFull> ::= Ident <IDSuffix> (Lookaheads: Ident)

<IDSuffix>

 ::= ':' Ident  (Lookaheads: ':')
 ::= {empty}    (Lookaheads: Anything else)

<Values>

 ::= Value <Values>  (Lookaheads: Value)
 |   {empty}         (Lookaheads: Anything else)

<Attributes>

 ::= <Attribute> <Attributes>  (Lookaheads: Ident)
 |   {empty}                   (Lookaheads: Anything else)

<Attribute> ::= <IDFull> '=' Value (Lookaheads: Ident)

<OptChild>

  ::= '{' EOL <Tags> '}'  (Lookaheads: '{')
  |   {empty}             (Lookaheads: Anything else)

<TagTerminator>

 ::= EOL  (Lookahead: EOL)
 |   EOF  (Lookahead: EOF)

Note: That's (sort of) the GOLD Parser Builder syntax (nonterminals are
in angle brackets):

<NonTerminal Token> ::= set of tokens | alternate set | etc...

I'll go through my SDL lexer and post a grammar for its behavior, and
then ask the main SDL guy to check it over.

Re: Are you sure about SDL?

On 8/25/2014 12:25 PM, Jonathan Marler wrote:

I am currently working on DUB to support the new SDL package format. Initially I was very happy to see a new format, especially one that seemed very simple and intuitive. However, after getting intimate with SDL I found that even though it seemed simple and stripped down, it actually has some odd "eccentricities". I wanted to see if there was a language that was as easy to read/write as SDL but didn't have so many odd rules. I spent some time looking at other formats but ended up attempting to write my own. I started with a clean slate and when I was finished I realized that I just made another variation of JSON...so I called it ASON( Application Specific Object Notation). I know that this discussion has already been had here, but I thought this language had enough advantages over SDL that it was worth it to see wh

at people think before we take the final plunge and implement the SDL package format.

I'm finding that I'm having a hard time wrapping my head around the
rules of ASON. For example, one of your examples has this:

// The following are all the same
"Jim" 42 "Banker";
"Jim" {age 42 occupation "Banker"}
{name "Jim" age 42 occupation "Banker"}

I find the fact that the second and third are equivalent to be very
confusing.

Also, at times, it seems like ASON's semantics are dependent on the D
struct being filled. Or am I mistaken about that?

How are these two lines interpreted by ASON?:

a b c

a b c d

What is the equivalent JSON for those? Is it:

"a": {"b": "c"}
"a": {"b": ["c", "d"]}

or:

"a": "b", "c": null
"a": "b", "c": "d"

or:

"a": "b", "c": {

 "a": "b", "c": "d"

}

or:

"a": "b", "c": "a", "b": "c", "d": null

Or something else?

Re: Are you sure about SDL?

I'm finding that I'm having a hard time wrapping my head around the
rules of ASON. For example, one of your examples has this:

// The following are all the same
"Jim" 42 "Banker";
"Jim" {age 42 occupation "Banker"}
{name "Jim" age 42 occupation "Banker"}

I find the fact that the second and third are equivalent to be very
confusing.

BTW, thanks for mentioning your SDL grammar, I'm looking over it.

I agree that nameless fields be confusing. This example was merely a demonstration of how nameless object fields work, however, it is up to the application to specify what fields can become nameless. In most cases nameless fields should be avoided to prevent this confusion. In the case of a dub package file, I would probably allow any "name" field such as "dependency name", or "subPackage name", or "configuration name" to become nameless, but that's about it.

dependency {name "vibe-d" version ">=1.0.0"}
// OR
dependency "vibe-d" {version ">=1.0.0"}

Also, at times, it seems like ASON's semantics are dependent on the D
struct being filled. Or am I mistaken about that?

How are these two lines interpreted by ASON?:

a b c

a b c d

What is the equivalent JSON for those? Is it:

"a": {"b": "c"}
"a": {"b": ["c", "d"]}

or:

"a": "b", "c": null
"a": "b", "c": "d"

or:

"a": "b", "c": {

 "a": "b", "c": "d"

}

or:

"a": "b", "c": "a", "b": "c", "d": null

Or something else?

Yes you are correct! ASON's semantics are dependent on the application. That's where the name "Application Specific" came from. The following features make the text "Application Specific"

  1. optional colons/commas
  2. omitting root level braces or brackets
  3. Nameless object fields.

I've noticed that in most applications it's unnecessary to specify the structure of the data you are representing because the application already knows what kind of data it is expecting. This concept is what makes ASON unique from other JSON variants. This is the main concept I wanted to get people's opinions on. There are advantages and disadvantages to it. Thanks for taking the time to read it and understand what makes it unique.

Re: Are you sure about SDL?

Here is my opinion

The fact that ASON does is a superset of JSON is pretty cool of course.

ASON

  • ASON is a JSON superset, avoid to have several parsers in IDEs
  • ASON does not account for whitespace (good)
  • optional colons and commas, I dislike them
  • I don't understand not want ASON nameless fields, even more after the example with "equivalent" literals
  • for some reason I don't really like unquoted string for values, sounds like easy to forget a word and offset all the parsing silently

SDL

  • SDL has a readily available parser package sdlang-d, in a stand-alone library
  • SDL account for whitespace (EOL) if I'm not mistaken, ASON does not
  • SDL has a "date" data-type which seems useless for DUB purpose

So I was sceptic at first but after writing down pluses and minuses I would favour ASON indeed, and more so if the format becomes stricter.

Re: Are you sure about SDL?

On Mon, 25 Aug 2014 21:09:07 GMT, Jonathan Marler wrote:

Yes you are correct! ASON's semantics are dependent on the application. That's where the name "Application Specific" came from. The following features make the text "Application Specific"

  1. optional colons/commas
  2. omitting root level braces or brackets
  3. Nameless object fields.

I've noticed that in most applications it's unnecessary to specify the structure of the data you are representing because the application already knows what kind of data it is expecting. This concept is what makes ASON unique from other JSON variants. This is the main concept I wanted to get people's opinions on. There are advantages and disadvantages to it. Thanks for taking the time to read it and understand what makes it unique.

My question is: can we parse any ASON file without application-dependent knowledge? I we can't then it's a show-stopper for any general parser.

Re: Are you sure about SDL?

First, thanks for listing your pluses and minuses, you did a great job in laying out the differences and spelled out some of the reasons I think ASON would be a good fit.

I would like to mention that if DUB were to use the nameless field feature at all, I would only recommend it be supported in the "name" field on some of the objects (like dependency/subPackage). If people thought that even that was too confusing, I would recommend dismissing the feature all together.

My question is: can we parse any ASON file without application-dependent knowledge? I we can't then it's a show-stopper for any general parser.

Well a general purpose parser can accept a schema or provide some way for the application to tell it what types it is expecting. So I think your real question was, is there a way to convert any ASON to JSON without any application dependent knowledge. I believe that as long as there are no nameless fields, the answer is yes. I'll explain by using Nick's previous example.

a b c

If this appeared in the beginning of the file, then this would be an object with it's curly-braces omitted. If it appeared within an explicit curly-brace section, it would represent:

{            // SDL Version 
//...
a b c        // syntax error, field 'c' missing it's value
//...
}        
{            // JSON Version
//...
"a":"b","c"  // syntax error, field 'c' missing it's value
//...
}

Keep in mind even though this is valid ASON, if an object's fields are written this close together, I would use delimiters to make it easier to read:

{a:b,c}        // SDL Version 

If it was within a square-bracket section, it would represent:

[a b c]
["a","b","c"]

Nick's other interpretations of the sequence a b c:

"a":{"b":"c"}
"a":{"b":["c","d"]}
...

wouldn't be valid because every object needs curly-braces and every list needs square-brackets. However, if you have a bunch of crazy nameless fields you could get yourself into trouble, but that's why you only use them when it makes sense. In the case of DUB package files, allowing only one nameless field would require an ASON to JSON converter to have application knowledge, but the DUB parser itself would still be just fine.

Note: the reason ASON supports nameless fields is so that if other applications want to use them they have the option. There are some cases where nameless fields would be very useful, but for build configuration files this feature isn't that important.

Pages: 1 2