Getting Started With Falcor

_ TL;DR: Netflix has just open-sourced its Falcor project. Falcor changes the way we approach data for web applications by introducing conventions such as virtual JSON models and the JSON Graph. These conventions make it possible for us to cache data intelligently on the client's device, which can help us to achieve an improved user experience. You can view the repo for the tutorial to see some ways in which Falcor can be put to use.

You can also integrate Auth0 with your Falcor app. Have a look at the docs and seed project to get started. _

Falcor is an open source library from Netflix that aims to change the way web applications request and handle data.

Outline

In this tutorial, we will look at some of Falcor's basic operations and show how they can be used to construct a virtual JSON model from a data source. The example data we'll use will model a handful of JavaScript conferences. We'll start on the client side and eventually move everything over to be served from a NodeJS back end. Specifically, we will cover:

What Falcor is and what problems it solves
How to set up a Falcor model on the client and prime it with data
How to use Falcor's JSON Graph to avoid having duplicate data
How to set up up a Falcor Router with Falcor Express
How to do a basic search using a Falcor route

What is Falcor?

One of the key problems that Falcor solves is the fact that HTTP wasn't designed and really isn't optimized for how web applications should ideally request data. With HTTP, only a single resource can be retrieved per request. Since web applications tend to need to request many small resources, many HTTP requests are often required to get all the data the application needs.

Another problem that Falcor solves is that JSON objects represent data hierarchically, as trees, whereas data is much more often a graph. In other words, an application's data doesn't always have a strict parent-child relationship, but rather will tend to have many predecessors, or be related to many other pieces of data. For example, suppose that we have several categories for our JavaScript conferences. A conference like ng-conf could fit under the category of "JavaScript", but could also fit under "AngularJS". If we were to model this with JSON, we might get something like this:

categories: {
  javascript: [
    0: {
      name: ng-conf
    }
  ],
  angularjs: [
    0: {
      name: ng-conf
    }
  ]
}

As it is expressed here, ng-conf has a parent-child relationship with both categories, when really it is related to both. Falcor makes it possible to deal with data as a graph, but still output it as a normal JSON object.

“Falcor makes it possible to deal with data as a graph, but still output it as a normal JSON object.”
Tweet This

Falcor also helps to make data retrieval more efficient. Instead of pulling many JSON objects from an API and then only using some parts of them in the view layer, Falcor wires together a single virtual JSON model that provides only the data that is required at the time that it is needed. For example, if we wanted to get the name of a conference and the first names of its attendees, we typically would first have our API serve the user data and then we would use the first name value in our view. However, with Falcor, we can easily say that we want only the first name value for each attendee and then have that be the only piece of user data that is returned. In this way, Falcor essentially customizes data to the application's view. Falcor does this through its JSON Graph convention, which is used to model graph information as a JSON object.

Furthermore, Falcor caches the virtual JSON model so that it can be accessed in-memory. When it becomes necessary to request more or different data, Falcor first consults the cache and if the data is not available there, it makes another request for only the fragment that is missing.

falcor

Getting Started

We'll first need to install our dependencies and create some files to work with.

npm install express falcor falcor-express falcor-router lodash
touch app.js index.js index.html

Creating a Model

Falcor models take JSON Graph information and form it into JSON to be consumed by the application. They are also responsible for maintaining an in-memory cache of that JSON data, which reduces network requests and makes operations more efficient. We'll start by providing some data for a local model that sits on the client, but later we'll move it to the server. Let's create some JavaScript conference data:

// app.js

// We can prime the model cache with a new falcor.Model
var model = new falcor.Model({
  cache: {
    events: [
      {
        name: "ng-conf",
        description: "The worlds best Angular Conference",
        location: { city: "Salt Lake City", state: "Utah" }
      },
      {
        name: "React Rally",
        description: "Conference focusing on Facebook's React",
        location: { city: "Salt Lake City", state: "Utah" }
      },
      {
        name: "ng-Vegas",
        description: "Two days jam-packed with Angular goodness with a focus on Angular 2",
        location: { city: "Las Vegas", state: "Nevada" }
      },
      {
        name: "Midwest JS",
        description: "Midwest JS is a premier technology conference focused on the JavaScript ecosystem.",
        location: { city: "Minneapolis", state: "Minnesota" }
      },
      {
        name: "NodeConf",
        description: "NodeConf is the longest running community driven conference for the Node community.",
        location: { city: "Walker Creek Ranch", state: "California" }
      }

    ]
  }  
});

Here we new-up a

falcor.Model

and place our example data within the

cache

object, which effectively "primes" the local model cache.

Falcor provides several intuitive methods for operating on models, including

get

set

, and

call

. If we wanted to retrieve the JSON that is emitted by a JSON Graph, we would use the

get

method; however, if we only wanted to retrieve a single value from the model, we would use

getValue

instead.

Let's use the

get

method to call some of our data.

// app.js

...

model
  // We want the name and description values for the first three items
  // from the data model
  .get(["events", {from: 0, to: 2}, ["name", "description"]])
  .then(function(response) {
    document.getElementById("event-data").innerHTML = JSON.stringify(response, null, 2);
  });

To use the

get

method, we need the JavaScript paths that point to the specific resource or set of resources that we want. JavaScript paths are simply a sequence of keys that refer to a specific location within a JSON object and can be of type

string

boolean

number

, or

null

. The paths can be passed in as a string to the

get

method, but we can also build up the paths in an array. In this case we are saying that we want to get data from the

events

array within our model and that we only want to retrieve the first three items. In the array that makes up the last part of the path, we are specifying that we're interested in the

name

and

description

keys from the model. Since the API for the model is asynchronous, we can retrieve the values from a

promise

by accessing the

response

in the success handler.

When we run this, we can see that we get the JSON printed to the screen:

falcor

That gives us the

name

and

description

for the events, but let's say we want the

location

as well. We might try to add this on as a JavaScript path:

// app.js

...

// We attempt to get the "location" key from out data but find that it
// doesn't display as expected
.get(["events", {from: 0, to: 2}, ["name", "description", "location"]])

...

However, when we try to run this, we see that the data looks exactly the same and that there's no

location

key. That's because Falcor only allows us to pull primitive JavaScript values from our models, not arrays or objects. Data within arrays or objects often tend to become arbitrarily large over time, so Falcor makes it illegal for us to access them directly, to force us to craft our paths more thoughtfully and only retrieve the data that we actually need. If we want access to the values of the keys on the

location

object within our data, we can pass another

pathSet

that takes us directly to the

city

and

state

values from

location

// app.js

...

// To get the values on the "location" object, we need to pass multiple pathSets and
// query the specifc keys on the location object
.get(["events", {from: 0, to: 2}, ["name", "description"]],['events', {from: 0, to: 2}, 'location', ['city', 'state']])

...

falcor

Avoiding Duplicate Data

If we ever want to make a change to one of the properties of this location, we can do so using the

set

method. Let's say that we want to change the value of

state

from "Utah" to "UT."

// app.js

...

model
  // We set the value of the first occurrence of Utah to UT
  .set(falcor.pathValue(["events", 0, "location", "state"], 'UT'))
  .then(function(response) {
    model
      // What we find afterwards is that the value gets changed in one location, but not both.
      .get(["events", {from: 0, to: 2}, ["name", "description"]],['events', {from: 0, to: 2}, 'location', ['city', 'state']])
      .then(function(response) {
        document.getElementById('event-data').innerHTML = JSON.stringify(response, null, 2);
      });    
  });

...

You'll notice that we now have the value for

state

showing up in two different ways, even though they should really be the same.

falcor

Falcor provides a solution to this common problem of duplicate data with a new primitive value type called

reference

. A JSON Graph reference makes it possible to refer to a single location for a piece of data rather than having multiple copies of that data in many places.

To apply this to our case, we first need to change the way we represent our

location

data, so that it is listed by an ID-based key.

// app.js

// We can use the shorthand for references with a variable
var $ref = falcor.Model.ref;

var model = new falcor.Model({
  cache: {
    locationsById: {
      1: {
        city: "Salt Lake City",
        state: "Utah"
      },
      2: {
        city: "Las Vegas",
        state: "Nevada"
      },
      3: {
        city: "Minneapolis",
        state: "Minnesota"
      },
      4: {
        city: "Walker Creek Ranch",
        state: "California"
      }
    },
    events: [
      {
        name: "ng-conf",
        description: "The worlds best Angular Conference",
        location: $ref('locationsById[1]')
      },
      {
        name: "React Rally",
        description: "Conference focusing on Facebook's React",
        location: $ref('locationsById[1]')
      },
      {
        name: "ng-Vegas",
        description: "Two days jam-packed with Angular goodness with a focus on Angular 2",
        location: $ref('locationsById[2]')
      },
      {
        name: "Midwest JS",
        description: "Midwest JS is a premier technology conference focused on the JavaScript ecosystem.",
        location: $ref('locationsById[3]')
      },
      {
        name: "NodeConf",
        description: "NodeConf is the longest running community driven conference for the Node community.",
        location: $ref('locationsById[4]')
      }
    ]
  }  
});

We now use JSON Graph's

$ref

to associate each event's location with a single place in our data. Even though we've represented the data differently here, JSON graph will convert it so that we will still get the same output for our JSON model. Further, if we now run the same

set

method to change the

state

value on the first

location

object, we see that it also gets changed elsewhere.

falcor

Moving to the Server

Priming the client cache with data is great, but we will of course need to have our data served from a back end. To accomplish this, Falcor provides a DataSource implementation that lets us tie JSON Graph information from the back end to our model. The DataSource implements a Router interface that lets us build routes on the server that we can wire up to respond with data.

We'll be using

falcor-express

along with the

falcor-router

to expose our JSON Graph data. Although we won't actually be pulling data from a persistent data store at this time, you would generally want to set up services to query and return data that can then be handled by Falcor.

app.js

, we'll still need to specify that we want to use a

falcor.Model

, but this time we need to set its data source to be remote. We can do this by setting

source

to a

falcor.HttpDataSource

model.json

. This will set up the necessary connection between our model on the front end and the JSON Graph information coming from the server.

// app.js

// We can set the model to have a data source that is retrieved from the backend
// over HTTP by setting the soure to be a falcor.HttpDataSource.
var model = new falcor.Model({source: new falcor.HttpDataSource('/model.json')});

To set up

index.js

on the server side, we'll

require

the packages we need and set up our

eventsData

object, which we'll use to mimic data from a persistent store.

// index.js - server side

var falcor = require('falcor');
var falcorExpress = require('falcor-express');
var Router = require('falcor-router');

var express = require('express');
var _ = require('lodash');
var app = express();

app.use(express.static('.'));

// Have Express request index.html
var $ref = falcor.Model.ref;

// Same data that was used in the view for our
// events, but this time on a simple object
// and not a Falcor model.
var eventsData = {
  locationsById: {
    1: {
      city: "Salt Lake City",
      state: "Utah"
    },

    ...

We'll set up Express to

use

an endpoint at

model.json

which has Falcor's

dataSourceRoute

return a

new Router

// index.js

...

// We setup a model.json endpoint and pass it a dataSourceRoute which
// allows us to serve a router. Various route requests can be sent to the
// router to grab whatever data is required
app.use('/model.json', falcorExpress.dataSourceRoute(function(req, res) {
  return new Router([
    {
      // Our route needs to match a pattern of integers that
      // are used as eventIds
      route: "events[{integers:eventIds}]['name', 'description']",
      get: function(pathSet) {

        var results = [];

        // Above we specified an eventIds identifier that is an
        // array of ids which we can loop over
        pathSet.eventIds.forEach(function(eventId) {

          // Next, an array of key names that map is held at pathSet[2]
          pathSet[2].map(function(key) {

            // We find the event the cooresponds to the current eventId
            var eventRecord = eventsData.events[eventId];

            // Finally we push a path/value object onto
            // the results array
            results.push({
              path: ['events', eventId, key],
              value: eventRecord[key]
            });
          });          
        });

        return results;
      }      
    }
  ]);
}));

app.listen(3000);
console.log("Listening on http://localhost:3000");

Routes are matched against a KeySet that is provided to the first array in the

route

string. For our first route, we specify that we want to match against integers and that we want to identify by

eventId

. The

pathSet

parameter in the

get

method gives us access to the KeySet that has been provided by the client and allows us to loop or map over the keys to match against data from the

eventsData

object.

The first route is used to get access to the

name

and

description

from a range that the client provides. We can use a simple

get

in our

app.js

file to pull these results. Notice here that the front-end code for getting these results remains the same:

// app.js
model
  .get(["events", {from: 0, to: 2}, ["name", "description"]])
  .then(function(response) {
    document.getElementById('event-data').innerHTML = JSON.stringify(response, null, 2);
  });

falcor

If we also wanted to get the location data associated with each event, we would need to setup another route to handle them specifically. Each path that we request needs to be represented by its own route.

// index.js

...

{
// Our route needs to match a pattern of integers that
// are used as locationId
route: "locationsById[{integers:locationId}]['city', 'state']",
get: function(pathSet) {

  var results = [];

  // Above we specified an locationId identifier that is an
  // array of ids which we can loop over
  pathSet.locationId.forEach(function(locationId) {

    // Next, an array of key names that map is held at pathSet[2]
    pathSet[2].map(function(key) {

      // We find the event the cooresponds to the current locationId
      var location = eventsData.locationsById[locationId];

      // Finally we push a path/value object onto
      // the results array
      results.push({
        path: ['locationsById', locationId, key],
        value: location[key]
      });
    });          
  });

  return results;
}

...

Search for an Event by Name

The client can send integers as a KeySet to pull a range of results, but it can also provide strings. We can use this to setup an "event by name" search route.

// index.js

...

{
    // The search route will match keys that match the names
    // of our conferences
    route: "events.byName[{keys}]['description']",
    get: function(pathSet) {

      var results = [];

      // We want to loop over each of the conference names provided
      pathSet[2].forEach(function(name) {

        // We also want to loop over all the events on the data object
        // and check if conference name is there
        eventsData.events.forEach(function(event) {
          if(_.contains(event, name)) {
            results.push({
              path: ['events','byName', name, 'description'],
              value: event.description                
            });
          }
        });          
      });

      return results;
    }
  }

...

In this route, we accept a key or set of keys that will be strings corresponding to event names. We loop over this set of keys, accessible from the

pathSet

, and then loop over our event data to check whether the name provided by the client exists there. If it does, we push onto the results array a path that will serve the description of the event, if it exists.

We can test this route in

app.js

by providing the name of an event.

// app.js

// Search example - we pass "Midwest JS" which will be looked up
// in the events data on the server and sent back if it exists
.get(["events", "byName", ["Midwest JS"], ['description']])

Which yields:

falcor

Aside: Using Falcor with Auth0

Auth0 issues JSON Web Tokens on every login for your users. This means that you can have a solid identity infrastructure, including Single Sign On, User Management, support for Social (Facebook, Github, Twitter, etc.), Enterprise (Active Directory, LDAP, SAML, etc.) and your own database of users with just a few lines of code.

Auth0 provides an easy way to add authentication to your Falcor app. The set up is simple and relies on the middelware provided by

express-jwt

. Have a look at the documentation and seed project to get started and, as always, feel free to open a support ticket if you have any questions.

Wrapping Up

Through its JSON Graph implementation, Falcor provides an innovative way to pull data into a virtual JSON model and cache it on the client side. It also gives us an easy way to serve routes from a back end through its integration with Express.

The new approach to retrieving data that Falcor provides is intriguing and generally nice to work with. It did, however, seem like a lot of boilerplate code was required to serve data from the Falcor router on the server side. We've reached out to the Falcor team to let them know about our experience with the coding for the router.

The Falcor project has excellent documentation on its website. There you'll find guides, videos, and API documentation that all go into much more detail on the concepts covered in this tutorial.

Getting Started With Falcor

Outline

What is Falcor?

Getting Started

Creating a Model

Avoiding Duplicate Data

Moving to the Server

Search for an Event by Name

Aside: Using Falcor with Auth0

Wrapping Up

Getting Started with Relay

Rise of the High Boilerplate Framework: A Look at Falcor and Relay

Authentication in Golang with JWTs