MonthNovember 2015

How to support promised predicates in Array find

I’m building ACL support in NodeJS. For example, I’ll allow to write:

if (Can(currentUser, 'edit', theDocument)) {
    buttons.push('Edit');
}

For any given action, my ´Can´ function must check a list of permissions. If one matches, then the action ´Can´ asks for is granted, otherwise (no permission exists) it’s denied. Of course I’m only interested in the first permission to match, and to allow for optimizations, I want permissions to be checked in the order I provide.

It’s easy to see that my problem is solved by the Array.prototype.find method. The problem I have, though, is that it only works with immediate predicates, but my checks can entail both immediate and promised predicates. For example, I allow predicates to access the database.

I googled my problem and found this StackOverflow page. Bergi’s answer gives both recursive and non-recursive solutions. (Side note. There was a time when recursive was opposed to iterative. With promises, that’s no longer the case. In fact, the non-recursive solution is a chain of ´catch´ handlers. An iteration is used to build the chain but promises themselves, throwing exceptions, control the iteration.) Benjamin Gruenbaum’s answer gives a recursive solution.

Here are their issues.

  1. The predicate is hacky because it signals a ´false´ by throwing an exception.
  2. The promise management is hacky because (a) it must cater for the predicate with ´.catch()´, and (b) it signals the “not found” outcome with ´reject()´.
  3. The promise management and the predicate are very coupled.
  4. The contract is different from that of the Array.prototype.find method.

So I came up with this one.

function ArrayFind(array, Predicate, thisArg) {
    function MyPredicate(element, index, arr) {
        return Promise.resolve(Predicate.call(thisArg, element, index, arr))
            .then(function (value) {
                return value ? element : undefined;
            });
    }
    function MyResult(element) {
        this.element = element;
    }

    return array.reduce(function(sequence, element, index, arr) {
        return sequence
            .then(function(found) {
                if (found) {
                    throw new MyResult(found);
                }
                return MyPredicate(element, index, arr);
            });
        }, Promise.resolve())
        .catch(function(reason) {
            if (reason instanceof MyResult) {
                return reason.element;
            }
            throw reason;
        });
}

Apart from being a global function instead of an Array instance method, the contract is exactly the same as that of the Array.prototype.find method.

  1. The only hack I used is to immediately exit when an element is found instead of continuing until the end of the ´.then()´ chain. But how I implemented it is both robust (as in Robustness) and hidden (as in Information Hiding). To make sure I do not mistake a rightful exception with my hack, I throw my own fake exception which is a wrapper around the found element. Thus, my fake exception is caught by the last ´.catch()´ and the element is returned.
  2. The predicate can be both immediate or a promise, thanks to ´Promise.resolve(Predicate…)´. If it’s immediate, it can throw an exception if it has to, not if it doesn’t hold true. If it’s a promise, it can reject() if it has to, not if it doesn’t hold true.

Examples

Here are some examples.

    function ImmediatePredicate(n) {
        console.log('-- ' + n);
        if (n % 3 == 0) throw 'dirty'; 
        return n % 2 == 0;
    }

    function PromisePredicate(n) { 
      return new Promise(function(resolve, reject) { 
        window.setTimeout(function() {
          console.log('-- ' + n);
          if (n % 3 == 0) reject('dirty'); 
          resolve(n % 2 == 0);
        }, Math.random() * 2000 + 1000); 
      }); 
    }

    function MixedPredicate(n) {
        if (n < 10) return ImmediatePredicate(n);
        return PromisePredicate(n);
    }

The result is a promise

Here is how ArrayFind compares to Array.prototype.find when no exceptions are thrown:

[1,13,5,4,7].find(ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
4

ArrayFind([1,13,5,4,7], ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

Of course the result is a Promise instead of the found element, but we can append additional handlers, like

ArrayFind([1,13,5,4,7], ImmediatePredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
VM880:4 4
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

An exception makes the promise reject

Here is how ArrayFind compares to Array.prototype.find when an exception is thrown:

[1,3,5,4,7].find(ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 3
VM834:4 Uncaught dirty

ArrayFind([1,3,5,4,7], ImmediatePredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 3
VM925:7 dirty
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

Immediate and promised predicates work equally well

Here you can see how ArrayFind supports at the same time immediate and promised predicates:

ArrayFind([1,13,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}
VM834:12 -- 13
VM834:3 -- 5
VM834:3 -- 4
VM955:4 4


ArrayFind([1,3,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 3
VM974:7 dirty
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}


ArrayFind([1,12,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}
VM834:12 -- 12
VM975:7 dirty

You can’t appreciate from the output above, but the immediate predicate really outputs immediately. :-)

Also notice that in the second to last example the warned ´dirty´ comes from the immediate predicate because 3 < 10, while in the last example it comes from the promised predicate because 12 >= 10.

How to customize MorganJS

MorganJS is easy to install and works nicely out of the box.

var Express = require('express');
var app = Express();
// ...
var Morgan = require('morgan');
app.use(Morgan('dev'));
// ...
app.listen(port);
console.log('Magic happens on port ' + port + ' since ' + (new Date));

Here is what it looks like. Highlighted HTTP status codes are quite useful.

Screen Shot 2015-11-08 at 14.44.07

Thankfully, it’s possible to customize MorganJS by adding tokens, which are template symbols, like this:

Morgan.token('current-user', function(req, res) {
    var result = req.currentUser ? req.currentUser.name : 'Anonymous';
    return result;
});

which can be later used like this:

// ...
var Morgan = require('morgan');
app.use(Morgan(':date[iso] :current-user :method :url :status :response-time ms - :res[content-length] bytes'));
// ...

to produce something like this:

Screen Shot 2015-11-08 at 14.52.16

Uh-oh!! Where are my colors?

I delve into MorganJS code…

/**
 * dev (colored)
 */

morgan.format('dev', function developmentFormatLine(tokens, req, res) {
  // get the status code if response written
  var status = res._header
    ? res.statusCode
    : undefined;

  // get status color
  var color = status >= 500 ? 31 // red
    : status >= 400 ? 33 // yellow
    : status >= 300 ? 36 // cyan
    : status >= 200 ? 32 // green
    : 0; // no color

  // get colored function
  var fn = developmentFormatLine[color];

  if (!fn) {
    // compile
    fn = developmentFormatLine[color] = compile('\x1b[0m:method :url \x1b['
      + color + 'm:status \x1b[0m:response-time ms - :res[content-length]\x1b[0m');
  }

  return fn(tokens, req, res);
})

As you may have noticed, the above code is hard to understand and quite hard-coded too.

  • Hard-coded because, even if the ´dev´ template is documented as ´:method :url :status :response-time ms – :res[content-length]´, it’s really embedded into the code and mixed up with extraneous bits rather than being declared into some option and used like any other MorganJS template is.
  • Hard to understand because the function object is being used as a cache for its own executions which entail a compilation step whose raison d’être I still have to grasp. I could be wrong, but this one could be a clear example of over-engineering.

However my biggest disappointment was that there is no way of reusing the colored ´:status´ token nor the coloring functionality, neither directly, by calling a method, nor indirectly, by copy-pasting some code. A total fail. :-(

Googling “terminal colors” I eventually got to this Unix StackExchange answer, which I used to write this:

function ColorFactory() {
    var result = {};

    /* beautify preserve:start */
    result.DEFAULT      = '\x1b[0m';
    result.WHITE        = '\x1b[1;37m';
    result.BLACK        = '\x1b[0;30m';
    result.BLUE         = '\x1b[0;34m';
    result.LIGHT_BLUE   = '\x1b[1;34m';
    result.GREEN        = '\x1b[0;32m';
    result.LIGHT_GREEN  = '\x1b[1;32m';
    result.CYAN         = '\x1b[0;36m';
    result.LIGHT_CYAN   = '\x1b[1;36m';
    result.RED          = '\x1b[0;31m';
    result.LIGHT_RED    = '\x1b[1;31m';
    result.PURPLE       = '\x1b[0;35m';
    result.LIGHT_PURPLE = '\x1b[1;35m';
    result.BROWN        = '\x1b[0;33m';
    result.YELLOW       = '\x1b[1;33m';
    result.GRAY         = '\x1b[0;30m';
    result.LIGHT_GRAY   = '\x1b[0;37m';
    /* beautify preserve:end */

    // ['DEFAULT', 'WHITE', 'BLACK', ...]
    result.allNames = Object.keys(result);

    // ['\x1b[0m', '\x1b[1;37m', '\x1b[0;30m', ...]
    result.allValues = result.allNames.map(function(name) {
        return result[name];
    });

    var anyValue = result.allValues.map(function(value) {
        return value.replace(/\W/g, '\\$&');
    }).join('|');
    var anySequenceOfValues = new RegExp('(?:' + result.anyValue + ')+(' + result.anyValue + ')', 'g');
    var lastValueOfSequence = '$1';

    function paint(COLOR) {
        return function(string) {
            var colored = COLOR + string + result.DEFAULT;
            var simplified = colored.replace(anySequenceOfValues, lastValueOfSequence);
            return simplified;
        }
    }

    /* beautify preserve:start */
    result.Default     = paint(result.DEFAULT);
    result.White       = paint(result.WHITE);
    result.Black       = paint(result.BLACK);
    result.Blue        = paint(result.BLUE);
    result.LightBlue   = paint(result.LIGHT_BLUE);
    result.Green       = paint(result.GREEN);
    result.LightGreen  = paint(result.LIGHT_GREEN);
    result.Cyan        = paint(result.CYAN);
    result.LightCyan   = paint(result.LIGHT_CYAN);
    result.Red         = paint(result.RED);
    result.LightRed    = paint(result.LIGHT_RED);
    result.Purple      = paint(result.PURPLE);
    result.LightPurple = paint(result.LIGHT_PURPLE);
    result.Brown       = paint(result.BROWN);
    result.Yellow      = paint(result.YELLOW);
    result.Gray        = paint(result.GRAY);
    result.LightGray   = paint(result.LIGHT_GRAY);
    /* beautify preserve:end */

    return result;
}

A nice collateral about my ´ColorFactory´ function is that I can use it also in the console like this:

var color = ColorFactory();
console.log('Magic happens on port ' + color.Purple(port) + ' since ' + (new Date));

to get something like this:

Screen Shot 2015-11-08 at 16.12.17

Finally, I was able to customize Morgan with this:

function MorganFactory() {
    var color = ColorFactory();

    var Morgan = require('morgan');

    Morgan.token('current-user', function(req, res, type) {
        var result = req.currentUser ? req.currentUser.name : 'Anonymous';
        if ('colored' == type) {
            result = req.currentUser ? color.LightBlue(req.currentUser.name) : color.LightRed('Anonymous');
        }
        return result;
    });

    var defaultStatusToken = Morgan['status'];
    Morgan.token('status', function(req, res, type) {
        var status = defaultStatusToken(req, res);
        if ('colored' == type) {
            /* beautify preserve:start */
            var result =
                  status >= 500 ? color.Red(status)
                : status >= 400 ? color.Yellow(status)
                : status >= 300 ? color.Cyan(status)
                : status >= 200 ? color.Green(status)
                : status;
            /* beautify preserve:end */            
        } else {
            result = status;
        }
        return result;
    });

    return Morgan.apply(null, arguments);
}

and use it like this:

// ...
var Morgan = MorganFactory;
app.use(Morgan(':date[iso] :current-user[colored] :method :url :status[colored] :response-time ms - :res[content-length] bytes'));
// ...

to get something like this:

Screen Shot 2015-11-08 at 16.51.54

 

How to improve filters with promises

I had been programming a filters setup for the node API of a MEAN stack app.

Having this ´User´ model:

// user.model.js (complete)

var mongoose = require('mongoose');

var schema   = new mongoose.Schema({
    name: String,
    admin: Boolean
});

module.exports = mongoose.model('User', schema);

It allowed a ´User´ controller like this:

// user.controller.js (complete)

var fields = [
    'name', 
    function (admin) { 
        return !!admin.length; 
    }
];

var Item = require('./user.model');
var Controller = require(global.absPath + '/app/shared/CRUD.controller');
module.exports = Controller(Item, fields);

The meaning should be straightforward: copy the ´name´ field as is and make the ´admin´ field a proper boolean. That was made possible by this:

// CRUD.controller.js (excerpt)

module.exports = CRUD_Controller;

function CRUD_Controller(Item, fields) {
    //...
    function Create(req, res) {

        var item = new Item();

        CopyFields(fields, req.body, item);

        item.save(function(err) {

            if (err) {
                return res.send(err);
            }

            res.json({
                message: 'Item created!'
            });

        });

    }


    function CopyFields(fields, data, item) {

        (fields || []).forEach(function(field) {

            switch (typeof field) {

                case 'string':
                    item[field] = data[field];
                    break;

                case 'function':
                    var matches = String(field).match(/^function\s*\(\s*(\w+)\s*\)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    var name = matches[1];
                    item[name] = field(data[name]);
                    break;

            }

        });

    }
    //...
}

Then I wanted to add a ´password´ field to the ´User´ model. For storing it I decided to go with Strong Password Hashing with Node.js Standard Library. Properly translated to JavaScript and slightly tweaked I got this:

// hash.js (complete)

var crypto = require('crypto');

module.exports = Hash;

return;

function Hash(options, callback) {

    // Default options.plaintext to a random 8-character string
    if (!options.plaintext) {
        return crypto.randomBytes(8, function(err, buf) {
            if (err) {
                return callback(err);
            }
            options.plaintext = buf.toString('base64');
            Hash(options, callback);
        });
    }

    // Default options.salt to a random 64-character string (512 bits)
    if (!options.salt) {
        return crypto.randomBytes(64, function(err, buf) {
            if (err) {
                return callback(err);
            }
            options.salt = buf.toString('base64');
            Hash(options, callback);
        });
    } 

    // Default options.iterations to 10k
    if (!options.iterations) {
        options.iterations = 10000;
    }

    // Default options.digest to sha1
    if (!options.digest) {
        options.digest = 'sha1';
    }

    crypto.pbkdf2(options.plaintext, options.salt, options.iterations, 64, options.digest, function(err, key) {
        if (err) {
            return callback(err);
        }
        options.algorithm = 'PBDFK2';
        options.key = key.toString('base64');
        callback(null, options);
    });

}

So my ´User´ model became this:

// user.model.js (complete)

var mongoose = require('mongoose');

var schema   = new mongoose.Schema({
    name: String,
    password: {
        algorithm:  String,
        digest:     String,
        iterations: Number,
        salt:       String,
        key:        String
    },
    admin: Boolean
});

module.exports = mongoose.model('User', schema);

Have you noticed that the ´Hash´ function relies on the asynchronous´crypto.pbkdf2´ function? That’s just standard, so I wasn’t going to use the synchronous version on a second thought.

Then my problem was:

How do I make these filters work with deferred values?

Ta-da! Promises:

// user.controller.js (complete)

var Promise = require('es6-promise').Promise;
var fields = [
    'name', 
    function (password) { 
        return new Promise(function (resolve, reject) {
            var Hash = require(global.absPath + '/app/components/auth/hash');
            Hash({plaintext: password}, function (error, result) {
                if (error) {
                    reject(Error(error));
                } else {
                    delete result.plaintext;
                    resolve(result);
                }
            });
        });
    }, 
    function (admin) { 
        return !!admin.length; 
    }
];

var Item = require('./user.model');
var Controller = require(global.absPath + '/app/shared/CRUD.controller');
module.exports = Controller(Item, fields);

To make that work I had to change a bit the ´CRUD´ controller.

The first change was to separate the filtering from the assignment, so that I could later use the ´Promise.all´ method which allows to synchronize promises and values as well. That implied to pass from a ´CopyFields´ function which filters and assigns each value in turn to a ´FilterFields´ function which filters all values at once, thus making the assignments directly in the ´Create´ function.

// CRUD.controller.js (broken excerpt) 
 
module.exports = CRUD_Controller; 
 
function CRUD_Controller(Item, fields) { 
    //... 
    function Create(req, res) {

        FilterFields(fields, req.body, function (fFields) {
            var item = new Item();

            fFields.forEach(function (fField) {
                item[fField.name] = fField.value;
            });

            item.save(function(err) {

                if (err) {
                    return res.send(err);
                }

                res.json({
                    message: 'Item created!'
                });

            });
        });

    }


    function FilterFields(fields, data, callback) {

        Promise
            .all((fields || []).map(Filter))
            .then(callback)
            .catch(function (error) {
                console.log(error);
            });


        function Filter(field) {
            var result;

            switch (typeof field) {

                case 'string':
                    result = {
                        name: field,
                        value: data[field]
                    };
                    break;

                case 'function':
                    var matches = String(field).match(/^function\s*\(\s*(\w+)\s*\)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    result = {
                        name: matches[1],
                        value: field(data[matches[1]])
                    };
                    break;

            }

            return result;
        }

    }
    //... 
}

The second change was to add a needed special treatment for my promises. You may have noticed that, in the ´case ‘function’:´ above, ´result.value´ can be a promise BUT that won’t make ´result´ a promise itself!! So the code above wouldn’t work yet, because it would complete ´Promise.all´ before getting the hashed password. Finally, I got this:

// CRUD.controller.js (working excerpt)

module.exports = CRUD_Controller; 
 
function CRUD_Controller(Item, fields) { 
    //... 
    function Create(req, res) {

        FilterFields(fields, req.body, function (fFields) {
            var item = new Item();

            fFields.forEach(function (fField) {
                item[fField.name] = fField.value;
            });

            item.save(function(err) {

                if (err) {
                    return res.send(err);
                }

                res.json({
                    message: 'Item created!'
                });

            });
        });

    }


    function FilterFields(fields, data, callback) {

        Promise
            .all((fields || []).map(Filter))
            .then(callback)
            .catch(function (error) {
                console.log(error);
            });


        function Filter(field) {
            var result;

            switch (typeof field) {

                case 'string':
                    result = {
                        name: field,
                        value: data[field]
                    };
                    break;

                case 'function':
                    var matches = String(field).match(/^function\s*\(\s*(\w+)\s*\)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    result = {
                        name: matches[1],
                        value: field(data[matches[1]])
                    };
                    if (stuff.isPromise(result.value)) {
                        var promise = new Promise(function (resolve, reject) {
                            var name = result.name;
                            result.value.then(function (value) {
                                resolve({
                                    name: name,
                                    value: value
                                });
                            }).catch(function (error) {
                                reject(Error(error));
                            });
                        });
                        result = promise;
                    }
                    break;

            }

            return result;
        }

    }
    //...
}

The added lines make ´result´ a promise if ´result.value´ is one: ´result´ will eventually resolve to the expected result. BTW, the ´stuff.isPromise´ method is the classical ´object.then && typeof object.then == ‘function’´.

© 2017 Notes Log

Theme by Anders NorenUp ↑