Warning: mysql_connect() [function.mysql-connect]: Headers and client library minor version mismatch. Headers:100518 Library:30120 in /home/citrose/public_html/cjblomqvist.com/index.php on line 18
Carl-Johan Blomqvist

Derby: Enhancements and bugfixes to Racer

24th of August, 2013 04:54 PM

Lately, me/B&B Web has put some serious effort into fixing issues and making enhancements to Racer, and in particular filters. Basically we use a fork of Racer in all our projects which we always keep updated with the latest in the original repo from codeparty, but with our additional fixes on top. You can find the repo and our patch branch at: https://github.com/BBWeb/racer/tree/updated.

What is fixed in our branch?

Bugfixes
There are a lot of issues with filters as it stands at the moment in the codeparty-version of Racer. Among them:

  1. Arrays are completely useless as sources to filters. They should work much better in our branch but I still recommend using collections with proper ids instead of arrays due to refLists (which filters relies on a lot) doesn't seem to handle arrays reliably. We've tried to get refLists to work much better with arrays, but after fixing some stuff we basically thought - hey, why not convert our data to collections instead and save us a ton of headache.
  2. We've added a new feature to refLists which is something we call "autoPatchIds". Basically, this patches the list of ids refLists uses when items in the underlying data is added/removed, BEFORE any events are emitted. This is very important when using filters (which breaks otherwise) because Derby uses the events that are emitted from Racer to update things, and when Derby gets the events (and auto-patching isn't enabled), Derby will grab data from the filters after it has already updated itself. The data in Racer will not be in sync with the state Derby assumes things are, and a lot of issues arises. Basically, this fixes a lot of issues.
  3. Another issue that arises in filters is due to faulty assumptions about when things happen och changes and removals. Basically, if one item in a collection is removed, this will trigger a change of value to void 0 (undefined). If anything happens on this removal, such as if there's a blur event in Derby that triggers an update of a value, this will happen after the filter/sort has shuffled things around already. Since Derby triggers this on a ref, and uses an old reference path, which will not most likely point to a different item than intended. Thus, Derby will trigger an update of the wrong item.

Enhancements:

  1. One can now send in a context object into a filter/sort. Basically this object can be used to do comparisons. E.g. 
    model.filter('collection', function (item, i, items, ctx) { 
  item.attribute === ctx.attribute; 
}, {attribute: 'something I want to filter on'});
    In the above scenario, we will get all the items where there's an attribute called 'attribute' that is equal to 'something I want to filter on' (the string passed into the filter function). Why have this? A workaround for using context-objects are to store the data on the model and then get it from there. However, there are downsides to this. In particular, no refiltering will happen automatically if the data in the model is changed. Further, this is a bad design pattern - the filter should be independent from the model/data. If you want to update the context object you've sent in, there is now also a .context(myNewUpdatedContextObject) method that can be triggered on any filter (or sorts - these are the same object in Racer). This will also ensure everything stays updated according to what the filtering should result in - i.e. it ensures the reactiveness is kept proper.
  2. There are two more new methods that can be called on any filter/sort: skip and limit. Skip works like this: 
    var myFilter = model.filter(...);
console.log(myFilter.get()); // [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] 
myFilter.skip(5); 
console.log(myFilter.get()); // [6, 7, 8, 9, 10] - we've skipped the 5 first
    Limit works like this: 
    var myFilter = model.filter(...); 
console.log(myFilter.get()); // [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] 
myFilter.limit(5); 
console.log(myFilter.get()); // [1, 2, 3 ,4, 5] - we've limited the result to the 5 first
    They can also be used together. Basically, they make pagination very much simpler. Furthermore, you can also always pass in a limit to a call to skip, and the reverse, in other words, the methods actually look like this: Skip: 
    @param {number} skip - the number of items to skip 
@param {number} limit - the number of items to limit the result to - optional 
function skip(skip[, limit]) { ... }
    Limit: 
    @param {number} limit - the number of items to limit the result to 
@param {number} skip - the number of items to skip - optional 
function limit(limit[, skip]) { ... }
    A pagination example: 
    .skip((page - 1) * itemsPerPage, itemsPerPage);
    The reason why you'd want to do both skip and limit in the same call is to reduce the amount of updates (refiltering/resorting) that are needed (from two to one).
  3. We've added a new method to model, called .ats([path]). In this method, the combined path of the current model/scoped model + the (optionally) path sent in must point to an array or a collection. This method will return an array consisting of all the items in this array/collection - but only references/scoped models (the same as you would get from doing model.at()). E.g. 
    model.set('_page.collection, { 
    'a': { /* ... */ } 
  , 'b': { /* ... */ } 
}); 
model.ats('_page.collection') 
  === model.at('_page.collection').ats() 
  === model.at('_page').ats('collection') 
  === [model.at('_page.collection.a'), model.at('_page.collection.b')] // All true
  4. We've added the same functionality to filters - the ats method. This works different depending on the scenario. There are two scenarios, either it's an unref'd filter, or it's a ref'd filter. The unref'd filter will return what is expected, model.ats() but filtered and sorted based upon the filter/sort sent in. However, if ref'd, myFilter.ats() will return a list to all items at the ref-path. In general, .ats() can be useful when one wants to easily loop through a list of items but not get them first (albeit since Racer 0.5, there should be little to no performance penalty doing so, but it looks better)
  5. We've exposed a second argument in model.dereference (forArrayMutator). Basically, dereference doesn't work if it has to go through an array (such as any refList). This functionality already existing internally in Racer, but it was previously not possible to pass this into Racer easily.

Other:

  1. Tests have been converted to pure JS (previously in CoffeeScript)
  2. We've added test-cases to more or less all our new features and bugfixes
  3. Lots of comments regarding each thing fixed can be found in the commits and issues/pull requests opened (in codeparty's repo).
To summarize

The best would of course be if these fixes and enhancements can be accepted into the codeparty repo (a bunch of PRs exists for above). In the meantime however, especially if you want to work with filters (which is one awesome feature in Derby/Racer), I'd recommend using our branch. I hope this will relieve a lot of headache (and stupid workarounds) to people using filters.

How can you know that our fixes are any good and won't break stuff? Well, we can't make you any promises there. We've tried to be very careful when patching Racer, and we've tried to touch as few rows as possible while still fixing the root-cause of issues instead of just patching on top to fix special scenarios. Furthermore, we've tried to make sure we have good test coverage, made sure to not break any other tests on the way, and even further, we're running this branch in production with one project, and will with two more very soon, and they seem quite fine.

Any comments, feedback, issues, etcetera is very welcome!

PS. I can also recommend our tracks repo which basically has existing PRs from the codeparty repo pulled into the updated branch which fixes a few irritating issues like faulty behavior/side-effects of app.exit. 

The Weblog

Carl-Johan Blomqvist

Archive
Components performance26th of March, 2021 Common traps26th of March, 2017 Project Management: The Idea/Feedback Bucket28th of July, 2014 Derby: Enhancements and bugfixes to Racer24th of August, 2013 Derby: Issue with array-like bindings in the view and workaround2nd of July, 2013 Derby: Authentication part #131st of July, 2012 Michael Nelson: 6 things I'm loving about Derby31st of July, 2012 CJBlomqvist.com/blog - now with RSS!31st of July, 2012 Makefile for optimizing css, js and index files10th of July, 2012 DerbyJS: refList-wrapper update27th of June, 2012 DerbyJS: Introducing the onrender method render event and its possibilities (Updated!)22nd of June, 2012 Re-launch of Blog!19th of June, 2012 Effective Usage of Mobile Orientation21st of September, 2011 Image Loading in Today's Browsers7th of September, 2011