RejectedSoftware Forums

Sign up

Pages: 1 2

Re: First impressions

On Fri, 26 Jul 2013 14:40:47 GMT, Manu Evans wrote:

If simple form handling logic is potentially throw-ing all over the place, what is the typical structure/flow to handle errors gracefully?
Are there any idiomatic examples of this sort of code?

In my experience it always boils down to separating validation of user input (which is expected to throw a lot) into separate part of application and process already sanitized input further. You can catch exceptions in top-most block of former and use it to render nice error message to user about invalid input. Any exception thrown after is likely to be some weird internal server error and default handler which renders status code 500 makes perfect sense.

I think designing even small apps in generic and clean way really pays it off and using script-like approach of simply going and processing stuff is fundamentally flawed.

Re: First impressions

On Fri, 26 Jul 2013 16:25:50 GMT, Sönke Ludwig wrote:

I'm right there with you. I also never used exceptions in C++ for various reasons and took a long time until I accepted the fact that so many things in D's runtime and standard library depend on them that it doesn't make sense to actively avoid them (however, I still fear I might regret that when looking at the state of non-x86 platforms). Funnily, the thing that made me actually start using them was to make the code look cleaner. I think I looked at a hand written D parser and was impressed how clean that code looked compared to my already clean C-style code.

The good thing is that usually you don't have to care about those exceptions, as long as important cleanup code properly uses scope(exit) or similar means. I have a few places like the following, though, where an explicit try-catch is used:

void handleSubmittedForm(HTTPServerRequest req, HTTPServerResponse res)
{
	try {
		// read form values
		// validate values
		// perform action
		res.redirect("http://page/displaying/results");
	} catch (Exception e) {
		req.params["error"] = e.msg;
		// prints the error message
		renderFormPage(req, res);
	}
}

It helps a lot to keep the error handling code short and readable. But on the other hand you are absolutely right that if opDispatch throws an exception, the get-like behavior cannot be achieved in a sane way ("insane" meaning catching the exception and using a default value). On the other hand I don't like that failures could easily go undetected if the return value has to be explicitly checked. What do you think of a vibe.http.form-like approach? I somehow like the idea of having a low-level API and a higher level layer for maximum convenience to sidestep such issues.

The form approach is definitely interesting, but again, it implies that I can't handle particular missing args distinctly, which i do. It kinda implies a singular error state for missing args.
Also, what if I'm combining arguments taken from POST and from the URL? How will form deal with that? In my current situation, I'm taking arguments from the url, post, and also "/blah/:arg" path args...

Additionally, all this talk of exceptions, and the form approach, all make one important presumption... that it is an ERROR case that an argument is missing. It's not necessarily an error, that argument might have been optional, maybe it has some default... can be implied?

In my case, it's rarely an actual error case if something is missing, and if it is, it almost certainly requires very context specific error handling/reporting, which doesn't work either in the context of a single outer exception handler. The errors are specific, and need to be handled specifically :/

So... my code ends up looking like:

if(req.args.myArg)
  // do soemthing if myArg is present
else
  // maybe i can default it, or handle missing myArg explicitly

I'm writing a web API, but even in the case of a typical webpage, consider a sign-up form; some fields are always optional, if you miss something that isn't optional, some red text "You forgot this!" appears next to the appropriate missing item for instance... that's quite context specific.
You never get redirected to a webpage that says "you made an error somewhere in the form, please try again" ;)

Re: First impressions

On Fri, 26 Jul 2013 23:40:54 GMT, Dicebot wrote:

On Fri, 26 Jul 2013 14:40:47 GMT, Manu Evans wrote:

If simple form handling logic is potentially throw-ing all over the place, what is the typical structure/flow to handle errors gracefully?
Are there any idiomatic examples of this sort of code?

In my experience it always boils down to separating validation of user input (which is expected to throw a lot) into separate part of application and process already sanitized input further. You can catch exceptions in top-most block of former and use it to render nice error message to user about invalid input. Any exception thrown after is likely to be some weird internal server error and default handler which renders status code 500 makes perfect sense.

I think designing even small apps in generic and clean way really pays it off and using script-like approach of simply going and processing stuff is fundamentally flawed.

Can you demonstrate this approach?
There are a few issues here to my mind...

1) An outer block to catch validation exceptions doesn't customise the error for whatever particular thing is missing.
2) Once past the validation process, an unexpected exception will still be caught by the same outer catch that caught the missing args, unless you start messing with specific exception types, which in turn precludes generic handling of argument exceptions.
3) If you go to the effort of defining and throwing specific exceptions for various cases, and/or catching a general exception and performing a bunch of logic to decide what the error case that threw actually was, either way, that's a lot of logic. I don't imagine that's less lines, or more readable than just handling the missing argument with an 'if' at the point I first grab it...

Can you demonstrate what code would look like that:

  • Validates some args, accepting that some args are optional.
  • May receive the args from either post, query, or path args.
  • Gives the opportunity for customised error output for the particular thing that was missing.
  • Doesn't require additional logic to perform those tasks; should be less code/simpler than an if.

If not, then I don't buy that exceptions are a simplification, they would seem more like a convolution, since the flow of code is split up and becomes harder to follow.

Re: First impressions

Idea is to move from untyped world of user input to strongly typed declarative world of D entities and do all validation in process. That is actually a lot more code but it does scale well. Something like this:

struct UserDescription
{
    string name, password_hash;
    @optional Mail mail;
}

// ...

void handleRegistration(HTTPServerRequest req, HTTPServerResponse res)
{
    UserDescription user;
    try
        user.loadFrom(req.form, req.session, <any possible sources>))
    catch (ValidationException e) // loadFrom stores here information about field that failed validation and description of problem
    {
        render!("register.dt", e)(res); // it is up to Diet template code to chose where to render error message
        return;
    }

    // do stuff, 'user' is guaranteed to be valid here and you only care about application logic       
}

// ...

router.any("/regiser", &handleRegistration);

As you can see, this is fundamentally same exception handling but one may abuse the fact that most web data processing is extremely generic and move all the boilerplate into utility functions (with D introspection power it is damn easy). So that you won't have try-catch for every parameter access but only single one for data access as a whole. It won't be any shorter for small programs but will scale naturally. This snippet may become even better once req.params becomes Variant[string] as because that will allow to split handler into two actual handlers and avoid explicit catching in user code at all.

I actually think vibe.d should be enhanced with such small serialization helpers to make such stuff easy out of the box. vibe.http.form feels too heavy and API-focused for me.

Re: First impressions

Forgot to actually answer :)

On Sun, 28 Jul 2013 11:35:21 GMT, Manu Evans wrote:

1) An outer block to catch validation exceptions doesn't customise the error for whatever particular thing is missing.

It can assuming you store that information in the exception.

2) Once past the validation process, an unexpected exception will still be caught by the same outer catch that caught the missing args, unless you start messing with specific exception types, which in turn precludes generic handling of argument exceptions.

"outer" was meant to be "outer for validation code scope"

3) If you go to the effort of defining and throwing specific exceptions for various cases, and/or catching a general exception and performing a bunch of logic to decide what the error case that threw actually was, either way, that's a lot of logic. I don't imagine that's less lines, or more readable than just handling the missing argument with an 'if' at the point I first grab it...

It is not. I don't care how many lines is trivial app long - it is much more important that those lines won't get copy-pasted once application grows. Sanitizing user input can't be short, it is plain impossible. If it is short, it is most likely to not handle validation properly. However, you can move all that clumsy logic into generic helpers and forget about it, focusing on "business logic" (hate this term).

Can you demonstrate what code would look like that:

  • Validates some args, accepting that some args are optional.
  • May receive the args from either post, query, or path args.
  • Gives the opportunity for customised error output for the particular thing that was missing.
  • Doesn't require additional logic to perform those tasks; should be less code/simpler than an if.

Hope snippet above somewhat answers that. Other than last point but as I have already said, I dismiss it as an improper desire :)

Re: First impressions

On Sun, 28 Jul 2013 12:37:51 GMT, Dicebot wrote:

Idea is to move from untyped world of user input to strongly typed declarative world of D entities and do all validation in process. That is actually a lot more code but it does scale well. Something like this:

struct UserDescription
{
    string name, password_hash;
    @optional Mail mail;
}

// ...

void handleRegistration(HTTPServerRequest req, HTTPServerResponse res)
{
    UserDescription user;
    try
        user.loadFrom(req.form, req.session, <any possible sources>))
    catch (ValidationException e) // loadFrom stores here information about field that failed validation and description of problem
    {
        render!("register.dt", e)(res); // it is up to Diet template code to chose where to render error message
        return;
    }

    // do stuff, 'user' is guaranteed to be valid here and you only care about application logic       
}

// ...

router.any("/regiser", &handleRegistration);

As you can see, this is fundamentally same exception handling but one may abuse the fact that most web data processing is extremely generic and move all the boilerplate into utility functions (with D introspection power it is damn easy). So that you won't have try-catch for every parameter access but only single one for data access as a whole. It won't be any shorter for small programs but will scale naturally. This snippet may become even better once req.params becomes Variant[string] as because that will allow to split handler into two actual handlers and avoid explicit catching in user code at all.

I actually think vibe.d should be enhanced with such small serialization helpers to make such stuff easy out of the box. vibe.http.form feels too heavy and API-focused for me.

Okay, this looks pretty nice.
I still can't see how you can customise the error response though without a bunch of logic in the catch, and as soon as that exists, then it could equally just appear at the point of consumption.

If this API was available though, I would definitely use it, and say it was a good API.

Re: First impressions

On Sun, 28 Jul 2013 12:49:43 GMT, Manu Evans wrote:

I still can't see how you can customise the error response though without a bunch of logic in the catch, and as soon as that exists, then it could equally just appear at the point of consumption.

What kind of customization you keep in mind? I'd expect to use default output in most of cases, probably sometimes tweaking it before passing e to render. Any specific example?

If this API was available though, I would definitely use it, and say it was a good API.

Well, usual story: "one day, when I'll have plenty of spare time" ;) Don't use vibe.d now, can't invest much time here, unfortunately.

Re: First impressions

On Sun, 28 Jul 2013 13:59:13 GMT, Dicebot wrote:

On Sun, 28 Jul 2013 12:49:43 GMT, Manu Evans wrote:

I still can't see how you can customise the error response though without a bunch of logic in the catch, and as soon as that exists, then it could equally just appear at the point of consumption.

What kind of customization you keep in mind? I'd expect to use default output in most of cases, probably sometimes tweaking it before passing e to render. Any specific example?

Consider basically any sign-up form, if something is missing, a customised message almost always appears in red to the right of the missing item. In my case, I'm dealing with an API, where the error reporting becomes rather specific.

If this API was available though, I would definitely use it, and say it was a good API.

Well, usual story: "one day, when I'll have plenty of spare time" ;) Don't use vibe.d now, can't invest much time here, unfortunately.

I hacked together a test of your proposal, it's working well, although my argument structures are too small in many cases to see a benefit in terns of number of lines. The exception handling code is definitely larger than the conditional logic I had before.

I am definitely being lynched by the fact that there is no FLS, no user variables field in HTTPServerRequest (both would be useful). I can't carry variables with me throughout the route.

Re: First impressions

On Mon, 29 Jul 2013 02:21:39 GMT, Manu Evans wrote:

I am definitely being lynched by the fact that there is no FLS, no user variables field in HTTPServerRequest (both would be useful). I can't carry variables with me throughout the route.

Just to be picking nits, there is setTaskLocal and getTaskLocal for Variant based FLS and the HTTPServerRequest.params field for string user variables ;)

But, more importantly, I've hacked together a quick TaskLocal template that stores task local (currently just fiber local, where the only difference is variable life time) variables in a per-fiber linear array (fibers are reused for consecutive tasks). See this commit. I'll look into a transition path for string -> Variant when I'm back from my current journey.

Re: First impressions

On Sun, 28 Jul 2013 11:11:38 GMT
"Manu Evans" turkeyman@gmail.com wrote:

So... my code ends up looking like:

if(req.args.myArg)
  // do soemthing if myArg is present
else
  // maybe i can default it, or handle missing myArg explicitly

I'm writing a web API, but even in the case of a typical webpage,
consider a sign-up form; some fields are always optional, if you miss
something that isn't optional, some red text "You forgot this!"
appears next to the appropriate missing item for instance... that's
quite context specific. You never get redirected to a webpage that
says "you made an error somewhere in the form, please try again" ;)

You're approaching it in what I call "the ASP/PHP way". I used to do
that, too. But after having done a lot of web forms in various
languages I got really freaking tried of what invariably ended up being
lots of duplicated logic, data structures and error handling for each
form and field (no matter what language I was using). So I've migrated
to an approach somewhat more inspired by desktop GUIs:

First off, I have an internal data structure, HtmlForm, that represents
an HTML form and all its fields. This consists of various information
for each field like displayed label, internal name, type of field
(text, textarea, radio, etc), default value, whether or not it's
optional, whether it's a password field, and whether or not it's a
"confirmation" field intended to match some other field (such as
"retype your password" field) I have one instance of this structure for
each form in the application.

But I don't instantiate/configure that form structure manually (I used
to, but it lead to generated HTML that wasn't easy to customize).
Instead, I have one single component that will read in an HTML page
(actually a Mustache-D HTML template [1]), scan it for forms via Adam
Rupee's HTML DOM [2], and then creates a HtmlForm structure from that.
The HTML template is also checked for form errors and missing
information (such as ensuring all fields have a matching label tag with
appropriate "for" attribute). Some non-standard tags/attributes I
invented for these purposes are stripped from the HTML template, and a
little bit of necessary boilerplate for the templating engine is
inserted (again, via the DOM).

The resulting HtmlForm structure and new slightly-modified HTML
template (still a mustache-d template) are stored for later use. This
process is only performed as-needed (on actual production, it's just
once - upon application startup).

Now come the cool parts:

Whenever a form submission comes in, I pass the entire POST data
directly into the appropriate HtmlForm. It does all the validation
against itself and then returns a FormSubmission. This FormSubmission
contains all the submitted data, plus info about what fields, if any,
were invalid and why (missing required field, confirmation fields
didn't match, etc.). It also contains the error message, if any. At
this point, I can optionally perform any additional form-specific
validation and update the FormSubmission accordingly (but I rarely need
to).

Then, if the FormSubmission says everything validated ok, I'm done. I
can just read out the required fields knowing they exist and are
valid, and go on with whatever I need to.

OTOH, if the FormSubmission says there was a problem, then I stash the
FormSubmission into the session data and redirect the user back to the
form page (to avoid breaking back/reload [3]). The form page grabs the
FormSubmission, passes it back to the HtmlForm which then gives the
Mustache-D templating engine all the necessary data to properly render
the form on my HTML template: Showing the error message, highlighting
the labels for all invalid fields (CSS tweakable, of course), and
repopulating the non-password fields with whatever information the user
did submit.

Final result: Anytime I need a web form, all I need to do is:

  1. Define the form right there in my HTML template.

  2. Pass the entire POSTed data straight into the form engine, stashing
    the result into the session data and checking "isValid": Valid:
    Continue on doing whatever. Invalid: Redirect back to the form.

  3. Anytime I display the page with the form, I pass the data stashed in
    the session (if any) back to the form engine.

  4. All validation, field repopulation, error messages and "bad field"
    highlighting are handled automagically.

It's all rather tightly coupled with the rest of my framework (which
sits on top of vibe.d), and it's very newcomer-unfriendly (nearly zero
docs, zero examples, zero guarantees of API stability, more GC usage
than vibe.d itself, and a lot that could probably be cleaned up), but
FWIW it is all OSS and up on github:

https://github.com/Abscissa/SemiTwistWeb

The form engine in particular is in this file if you wanted to take a
look, but again, there's probably some rough edges and such, and
radio/checkbox/listbox/uploads aren't implemented yet.

https://github.com/Abscissa/SemiTwistWeb/blob/master/src/semitwistWeb/form.d

[1] Mustache-D: https://github.com/repeatedly/mustache-d

[2] Adam's HTML DOM:
https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff/blob/master/dom.d

[3]
http://blog.httpwatch.com/2007/10/03/60-of-web-users-cant-be-wrong-dont-break-the-back-button/

Pages: 1 2