Trace Node.js Monitoring
Need help with Node.js?
Learn more
Need Node.js support?
Learn more

best practices

10 Best Practices for Writing Node.js REST APIs

10 Best Practices for Writing Node.js REST APIs

In this article we cover best practices for writing Node.js REST APIs, including topics like naming your routes, authentication, black-box testing & using proper cache headers for these resources.

One of the most popular use-cases for Node.js is to write RESTful APIs using it. Still, while we help our customers to find issues in their applications with Trace, our Node.js monitoring tool we constantly experience that developers have a lot of problems with REST APIs.

I hope these best-practices we use at RisingStack can help:

#1 - Use HTTP Methods & API Routes

Imagine, that you are building a Node.js RESTful API for creating, updating, retrieving or deleting users. For these operations HTTP already has the adequate toolset: POST, PUT, GET, PATCH or DELETE.

As a best practice, your API routes should always use nouns as resource identifiers. Speaking of the user's resources, the routing can look like this:

  • POST /user or PUT /user:/id to create a new user,
  • GET /user to retrieve a list of users,
  • GET /user/:id to retrieve a user,
  • PATCH /user/:id to modify an existing user record,
  • DELETE /user/:id to remove a user.

#2 - Use HTTP Status Codes Correctly

If something goes wrong while serving a request, you must set the correct status code for that in the response:

  • 2xx, if everything was okay,
  • 3xx, if the resource was moved,
  • 4xx, if the request cannot be fulfilled because of a client error (like requesting a resource that does not exist),
  • 5xx, if something went wrong on the API side (like an exception happened).

If you are using Express, setting the status code is as easy as res.status(500).send({error: 'Internal server error happened'}). Similarly with Restify: res.status(201).

For a full list, check the list of HTTP status codes

#3 - Use HTTP headers to Send Metadata

To attach metadata about the payload you are about to send, use HTTP headers. Headers like this can be information on:

  • pagination,
  • rate limiting,
  • or authentication.

A list of standardized HTTP headers can be found here.

If you need to set any custom metadata in your headers, it was a best practice to prefix them with X. For example, if you were using CSRF tokens, it was a common (but non-standard) way to name them X-Csrf-Token. However with RFC 6648 they got deprecated. New APIs should make their best effort to not use header names that can conflict with other applications. For example, OpenStack prefixes its headers with OpenStack:

OpenStack-Identity-Account-ID  
OpenStack-Networking-Host-Name  
OpenStack-Object-Storage-Policy  

Note that the HTTP standard does not define any size limit on the headers; however, Node.js (as of writing this article) imposes an 80KB size limit on the headers object for practical reasons.

" Don't allow the total size of the HTTP headers (including the status line) to exceed HTTP_MAX_HEADER_SIZE. This check is here to protect embedders against denial-of-service attacks where the attacker feeds us a never-ending header that the embedder keeps buffering."

From the Node.js HTTP parser

#4 - Pick the right framework for your Node.js REST API

It is important to pick the framework that suits your use-case the most.

Express, Koa or Hapi

Express, Koa and Hapi can be used to create browser applications, and as such, they support templating and rendering - just to name a few features. If your application needs to provide the user-facing side as well, it makes sense to go for them.

Restify

On the other hand, Restify is focusing on helping you build REST services. It exists to let you build "strict" API services that are maintainable and observable. Restify also comes with automatic DTrace support for all your handlers.

Restify is used in production in major applications like npm or Netflix.

#5 - Black-Box Test your Node.js REST APIs

One of the best ways to test your REST APIs is to treat them as black boxes.

Black-box testing is a method of testing where the functionality of an application is examined without the knowledge of its internal structures or workings. So none of the dependencies are mocked or stubbed, but the system is tested as a whole.

One of the modules that can help you with black-box testing Node.js REST APIs is supertest.

A simple test case which checks if a user is returned using the test runner mocha can be implemented like this:

const request = require('supertest')

describe('GET /user/:id', function() {  
  it('returns a user', function() {
    // newer mocha versions accepts promises as well
    return request(app)
      .get('/user')
      .set('Accept', 'application/json')
      .expect(200, {
        id: '1',
        name: 'John Math'
      }, done)
  })
})

You may ask: how does the data gets populated into the database which serves the REST API?

In general, it is a good approach to write your tests in a way that they make as few assumptions about the state of the system as possible. Still, in some scenarios you can find yourself in a spot when you need to know what is the state of the system exactly, so you can make assertions and achieve higher test coverage.

So based on your needs, you can populate the database with test data in one of the following ways:

  • run your black-box test scenarios on a known subset of production data,
  • populate the database with crafted data before the test cases are run.

Of course, black-box testing does not mean that you don't have to do unit testing, you still have to write unit tests for your APIs.


Node.js Monitoring and Debugging from the Experts of RisingStack

Improve your REST APIs with Trace
Learn more


#6 - Do JWT-Based, Stateless Authentication

As your REST APIs must be stateless, so does your authentication layer. For this, JWT (JSON Web Token) is ideal.

JWT consists of three parts:

  • Header, containing the type of the token and the hashing algorithm
  • Payload, containing the claims
  • Signature (JWT does not encrypt the payload, just signs it!)

Adding JWT-based authentication to your application is very straightforward:

const koa = require('koa')  
const jwt = require('koa-jwt')

const app = koa()

app.use(jwt({  
  secret: 'very-secret' 
}))

// Protected middleware
app.use(function *(){  
  // content of the token will be available on this.state.user
  this.body = {
    secret: '42'
  }
})

After that, the API endpoints are protected with JWT. To access the protected endpoints, you have to provide the token in the Authorization header field.

curl --header "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ" my-website.com  

One thing that you could notice is that the JWT module does not depend on any database layer. This is the case because all JWT tokens can be verified on their own, and they can also contain time to live values.

Also, you always have to make sure that all your API endpoints are only accessible through a secure connection using HTTPS.

In a previous article, we explained web authentication methods in details - I recommend to check it out!

#7 - Use Conditional Requests

Conditional requests are HTTP requests which are executed differently depending on specific HTTP headers. You can think of these headers as preconditions: if they are met, the requests will be executed in a different way.

These headers try to check whether a version of a resource stored on the server matches a given version of the same resource. Because of this reason, these headers can be:

  • the timestamp of the last modification,
  • or an entity tag, which differs for each version.

These headers are:

  • Last-Modified (to indicate when the resource was last modified),
  • Etag (to indicate the entity tag),
  • If-Modified-Since (used with the Last-Modified header),
  • If-None-Match (used with the Etag header),

Let's take a look at an example!

The client below did not have any previous versions of the doc resource, so neither the If-Modified-Since, nor the If-None-Match header was applied when the resource was sent. Then, the server responds with the Etag and Last-Modified headers properly set.

Node.js RESTfu API with conditional request, without previous versions

From the MDN Conditional request documentation

The client can set the If-Modified-Since and If-None-Match headers once it tries to request the same resource - since it has a version now. If the response would be the same, the server simply responds with the 304 - Not Modified status and does not send the resource again.

Node.js RESTfu API with conditional request, with previous versions

From the MDN Conditional request documentation

#8 - Embrace Rate Limiting

Rate limiting is used to control how many requests a given consumer can send to the API.

To tell your API users how many requests they have left, set the following headers:

  • X-Rate-Limit-Limit, the number of requests allowed in a given time interval
  • X-Rate-Limit-Remaining, the number of requests remaining in the same interval,
  • X-Rate-Limit-Reset, the time when the rate limit will be reset.

Most HTTP frameworks support it out of the box (or with plugins). For example, if you are using Koa, there is the koa-ratelimit package.

Note, that the time window can vary based on different API providers - for example, GitHub uses an hour for that, while Twitter 15 minutes.

#9 - Create a Proper API Documentation

You write APIs so others can use them, benefit from them. Providing an API documentation for your Node.js REST APIs are crucial.

The following open-source projects can help you with creating documentation for your APIs:

Alternatively, if you want to use a hosted products, you can go for Apiary.

#10 - Don't Miss The Future of APIs

In the past years, two major query languages for APIs arose - namely GraphQL from Facebook and Falcor from Netflix. But why do we even need them?

Imagine the following RESTful resource request:

/org/1/space/2/docs/1/collaborators?include=email&page=1&limit=10

This can get out of hand quite easily - as you'd like to get the same response format for all your models all the time. This is where GraphQL and Falcor can help.

About GraphQL

GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools. - Read more here.

About Falcor

Falcor is the innovative data platform that powers the Netflix UIs. Falcor allows you to model all your backend data as a single Virtual JSON object on your Node server. On the client, you work with your remote JSON object using familiar JavaScript operations like get, set, and call. If you know your data, you know your API. - Read more here.

Amazing REST APIs for Inspiration

If you are about to start developing a Node.js REST API or creating a new version of an older one, we have collected four real-life examples that are worth checking out:

I hope that now you have a better understanding of how APIs should be written using Node.js. Please let me know in the comments if you miss anything!

Node.js Best Practices - How to Become a Better Developer in 2017

Node.js Best Practices - How to Become a Better Developer in 2017

A year ago we wrote a post on How to Become a Better Node.js Developer in 2016 which was a huge success - so we thought now it is time to revisit the topics and prepare for 2017!

In this article, we will go through the most important Node.js best practices for 2017, topics that you should care about and educate yourself in. Let’s start!

Node.js Best Practices for 2017

Use ES2015

Last year we advised you to use ES2015 - however, a lot has changed since.

Back then, Node.js v4 was the LTS version, and it had support for 57% of the ES2015 functionality. A year passed and ES2015 support grew to 99% with Node v6.

If you are on the latest Node.js LTS version you don't need babel anymore to use the whole feature set of ES2015. But even with this said, on the client side you’ll probably still need it!

For more information on which Node.js version supports which ES2015 features, I'd recommend checking out node.green.

Use Promises

Promises are a concurrency primitive, first described in the 80s. Now they are part of most modern programming languages to make your life easier.

Imagine the following example code that reads a file, parses it, and prints the name of the package. Using callbacks, it would look something like this:

fs.readFile('./package.json', 'utf-8', function (err, data) {  
  if (err) {
    return console.log(err)
  }

  try {
    JSON.parse(data)
  } catch (ex) {
    return console.log(ex)
  }
  console.log(data.name)
})

Wouldn't it be nice to rewrite the snippet into something more readable? Promises help you with that:

fs.readFileAsync('./package.json').then(JSON.parse).then((data) => {  
  console.log(data.name)
})
.catch((e) => {
  console.error('error reading/parsing file', e)
})

Of course, for now, the fs API does not have an readFileAsync that returns a Promise. To make it work, you have to wrap it with a module like promisifyAll.

Use the JavaScript Standard Style

When it comes to code style, it is crucial to have a company-wide standard, so when you have to change projects, you can be productive starting from day zero, without having to worry about building the build because of different presets.

At RisingStack we have incorporated the JavaScript Standard Style in all of our projects.

Node.js best practices - The Standard JS Logo

With Standard, there is no decisions to make, no .eslintrc, .jshintrc, or .jscsrc files to manage. It just works. You can find the Standard rules here.



Need help with enterprise-grade Node.js Development?
Hire the experts of RisingStack!


Use Docker - Containers are Production Ready in 2017!

You can think of Docker images as deployment artifacts - Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server.

But why should you start using Docker?

  • it enables you to run your applications in isolation,
  • as a conscience, it makes your deployments more secure,
  • Docker images are lightweight,
  • they enable immutable deployments,
  • and with them, you can mirror production environments locally.

To get started with Docker, head over to the official getting started tutorial. Also, for orchestration we recommend checking out our Kubernetes best practices article.

Monitor your Applications

If something breaks in your Node.js application, you should be the first one to know about it, not your customers.

One of the newer open-source solutions is Prometheus that can help you achieve this. Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. The only downside of Prometheus is that you have to set it up for you and host it for yourself.

If you are looking for on out-of-the-box solution with support, Trace by RisingStack is a great solution developed by us.

Trace will help you with

  • alerting,
  • memory and CPU profiling in production systems,
  • distributed tracing and error searching,
  • performance monitoring,
  • and keeping your npm packages secure!

Node.js Best Practices for 2017 - Use Trace and Profling

Use Messaging for Background Processes

If you are using HTTP for sending messages, then whenever the receiving party is down, all your messages are lost. However, if you pick a persistent transport layer, like a message queue to send messages, you won't have this problem.

If the receiving service is down, the messages will be kept, and can be processed later. If the service is not down, but there is an issue, processing can be retried, so no data gets lost.

An example: you'd like to send out thousands of emails. In this case, you would just have to put some basic information like the target email address and the first name, and a background worker could easily put together the email's content and send them out.

What's really great about this approach is that you can scale it whenever you want, and no traffic will be lost. If you see that there are millions of emails to be sent out, you can add extra workers, and they can consume the very same queue.

You have lots of options for messaging queues:

Use the Latest LTS Node.js version

To get the best of the two worlds (stability and new features) we recommend using the latest LTS (long-term support) version of Node.js. As of writing this article, it is version 6.9.2.

To easily switch Node.js version, you can use nvm. Once you installed it, switching to LTS takes only two commands:

nvm install 6.9.2  
nvm use 6.9.2  


Use Semantic Versioning

We conducted a Node.js Developer Survey a few months ago, which allowed us to get some insights on how people use semantic versioning.

Unfortunately, we found out that only 71% of our respondents uses semantic versioning when publishing/consuming modules. This number should be higher in our opinion - everyone should use it! Why? Because updating packages without semver can easily break Node.js apps.

Node.js Best Practices for 2017 - Semantic versioning survey results

Versioning your application / modules is critical - your consumers must know if a new version of a module is published and what needs to be done on their side to get the new version.

This is where semantic versioning comes into the picture. Given a version number MAJOR.MINOR.PATCH, increment the:

  • MAJOR version when you make incompatible API changes,
  • MINOR version when you add functionality (without breaking the API), and
  • PATCH version when you make backwards-compatible bug fixes.

npm also uses SemVer when installing your dependencies, so when you publish modules, always make sure to respect it. Otherwise, you can break others applications!

Secure Your Applications

Securing your users and customers data should be one of your top priorities in 2017. In 2016 alone, hundreds of millions of user accounts were compromised as a result of low security.

To get started with Node.js Security, read our Node.js Security Checklist, which covers topics like:

  • Security HTTP Headers,
  • Brute Force Protection,
  • Session Management,
  • Insecure Dependencies,
  • or Data Validation.

After you’ve embraced the basics, check out my Node Interactive talk on Surviving Web Security with Node.js!

Learn Serverless

Serverless started with the introduction of AWS Lambda. Since then it is growing fast, with a blooming open-source community.

In the next years, serverless will become a major factor for building new applications. If you'd like to stay on the edge, you should start learning it today.

One of the most popular solutions is the Serverless Framework, which helps in deploying AWS Lambda functions.

Attend and Speak at Conferences and Meetups

Attending conferences and meetups are great ways to learn about new trends, use-cases or best practices. Also, it is a great forum to meet new people.

To take it one step forward, I'd like to encourage you to speak at one of these events as well!

As public speaking is tough, and “imagine everyone's naked” is the worst advice, I'd recommend checking out speaking.io for tips on public speaking!

Become a better Node.js developer in 2017

As 2017 will be the year of Node.js, we’d like to help you getting the most out of it!

We just launched a new study program called "Owning Node.js" which helps you to become confident in:

  • Async Programming with Node.js
  • Creating servers with Express
  • Using Databases with Node
  • Project Structuring and building scalable apps

I want to learn more !


If you have any questions about the article, find me in the comments section!

JavaScript Clean Coding Best Practices - Node.js at Scale

JavaScript Clean Coding Best Practices - Node.js at Scale

Writing clean code is what you must know and do in order to call yourself a professional developer. There is no reasonable excuse for doing anything less than your best.

In this blog post, we will cover general clean coding principles for naming and using variables & functions, as well as some JavaScript specific clean coding best practices.

“Even bad code can function. But if the code isn’t clean, it can bring a development organization to its knees.” — Robert C. Martin (Uncle Bob)


Node.js at Scale is a collection of articles focusing on the needs of companies with bigger Node.js installations and advanced Node developers. Chapters:

First of all, what does clean coding mean?

Clean coding means that in the first place you write code for your later self and for your co-workers and not for the machine.

Your code must be easily understandable for humans.

"Write code for your later self and for your co-workers in the first place - not for the machine." via @RisingStack

Click To Tweet

You know you are working on a clean code when each routine you read turns out to be pretty much what you expected.

JavaSctipr Clean Coding: The only valid measurement of code quality is WTFs/minute

JavaScript Clean Coding Best Practices

Now that we know what every developer should aim for, let’s go through the best practices!

How should I name my variables?

Use intention-revealing names and don't worry if you have long variable names instead of saving a few keyboard strokes.

If you follow this practice, your names become searchable, which helps a lot when you do refactors or you are just looking for something.

// DON'T
let d  
let elapsed  
const ages = arr.map((i) => i.age)

// DO
let daysSinceModification  
const agesOfUsers = users.map((user) => user.age)  

Also, make meaningful distinctions and don't add extra, unnecessary nouns to the variable names, like its type (hungarian notation).

// DON'T
let nameString  
let theUsers

// DO
let name  
let users  

Make your variable names easy to pronounce, because for the human mind it takes less effort to process.

When you are doing code reviews with your fellow developers, these names are easier to reference.

// DON'T
let fName, lName  
let cntr

let full = false  
if (cart.size > 100) {  
  full = true
}

// DO
let firstName, lastName  
let counter

const MAX_CART_SIZE = 100  
// ...
const isFull = cart.size > MAX_CART_SIZE  

In short, don't cause extra mental mapping with your names.

How should I write my functions?

Your functions should do one thing only on one level of abstraction.

Functions should do one thing. They should do it well. They should do it only. — Robert C. Martin (Uncle Bob)

// DON'T
function getUserRouteHandler (req, res) {  
  const { userId } = req.params
  // inline SQL query
  knex('user')
    .where({ id: userId })
    .first()
    .then((user) => res.json(user))
}

// DO
// User model (eg. models/user.js)
const tableName = 'user'  
const User = {  
  getOne (userId) {
    return knex(tableName)
      .where({ id: userId })
      .first()
  }
}

// route handler (eg. server/routes/user/get.js)
function getUserRouteHandler (req, res) {  
  const { userId } = req.params
  User.getOne(userId)
    .then((user) => res.json(user))
}

After you wrote your functions properly, you can test how well you did with CPU profiling - which helps you to find bottlenecks.

Node.js Monitoring and Debugging from the Experts of RisingStack

Find slow functions using Trace
Learn more

Use long, descriptive names

A function name should be a verb or a verb phrase, and it needs to communicate its intent, as well as the order and intent of the arguments.

A long descriptive name is way better than a short, enigmatic name or a long descriptive comment.

// DON'T
/**
 * Invite a new user with its email address
 * @param {String} user email address
 */
function inv (user) { /* implementation */ }

// DO
function inviteUser (emailAddress) { /* implementation */ }  


Avoid long argument list

Use a single object parameter and destructuring assignment instead. It also makes handling optional parameters much easier.

// DON'T
function getRegisteredUsers (fields, include, fromDate, toDate) { /* implementation */ }  
getRegisteredUsers(['firstName', 'lastName', 'email'], ['invitedUsers'], '2016-09-26', '2016-12-13')

// DO
function getRegisteredUsers ({ fields, include, fromDate, toDate }) { /* implementation */ }  
getRegisteredUsers({  
  fields: ['firstName', 'lastName', 'email'],
  include: ['invitedUsers'],
  fromDate: '2016-09-26',
  toDate: '2016-12-13'
})

Reduce side effects

Use pure functions without side effects, whenever you can. They are really easy to use and test.

// DON'T
function addItemToCart (cart, item, quantity = 1) {  
  const alreadyInCart = cart.get(item.id) || 0
  cart.set(item.id, alreadyInCart + quantity)
  return cart
}

// DO
// not modifying the original cart
function addItemToCart (cart, item, quantity = 1) {  
  const cartCopy = new Map(cart)
  const alreadyInCart = cartCopy.get(item.id) || 0
  cartCopy.set(item.id, alreadyInCart + quantity)
  return cartCopy
}

// or by invert the method location
// you can expect that the original object will be mutated
// addItemToCart(cart, item, quantity) -> cart.addItem(item, quantity)
const cart = new Map()  
Object.assign(cart, {  
  addItem (item, quantity = 1) {
    const alreadyInCart = this.get(item.id) || 0
    this.set(item.id, alreadyInCart + quantity)
    return this
  }
})


Organize your functions in a file according to the stepdown rule

Higher level functions should be on top and lower levels below. It makes it natural to read the source code.

// DON'T
// "I need the full name for something..."
function getFullName (user) {  
  return `${user.firstName} ${user.lastName}`
}

function renderEmailTemplate (user) {  
  // "oh, here"
  const fullName = getFullName(user)
  return `Dear ${fullName}, ...`
}

// DO
function renderEmailTemplate (user) {  
  // "I need the full name of the user"
  const fullName = getFullName(user)
  return `Dear ${fullName}, ...`
}

// "I use this for the email template rendering"
function getFullName (user) {  
  return `${user.firstName} ${user.lastName}`
}


Query or modification

Functions should either do something (modify) or answer something (query), but not both.


Everyone likes to write JavaScript differently, what to do?

As JavaScript is dynamic and loosely typed, it is especially prone to programmer errors.

Use project or company wise linter rules and formatting style.

The stricter the rules, the less effort will go into pointing out bad formatting in code reviews. It should cover things like consistent naming, indentation size, whitespace placement and even semicolons.

"The stricter the linter rules, the less effort needed to point out bad formatting in code reviews." by @RisingStack

Click To Tweet

The standard JS style is quite nice to start with, but in my opinion, it isn't strict enough. I can agree most of the rules in the Airbnb style.

How to write nice async code?

Use Promises whenever you can.

Promises are natively available from Node 4. Instead of writing nested callbacks, you can have chainable Promise calls.

// AVOID
asyncFunc1((err, result1) => {  
  asyncFunc2(result1, (err, result2) => {
    asyncFunc3(result2, (err, result3) => {
      console.lor(result3)
    })
  })
})

// PREFER
asyncFuncPromise1()  
  .then(asyncFuncPromise2)
  .then(asyncFuncPromise3)
  .then((result) => console.log(result))
  .catch((err) => console.error(err))

Most of the libraries out there have both callback and promise interfaces, prefer the latter. You can even convert callback APIs to promise based one by wrapping them using packages like es6-promisify.

// AVOID
const fs = require('fs')

function readJSON (filePath, callback) {  
  fs.readFile(filePath, (err, data) => {
    if (err) {
      return callback(err)
    }

    try {
      callback(null, JSON.parse(data))
    } catch (ex) {
      callback(ex)
    }
  })
}

readJSON('./package.json', (err, pkg) => { console.log(err, pkg) })

// PREFER
const fs = require('fs')  
const promisify = require('es6-promisify')

const readFile = promisify(fs.readFile)  
function readJSON (filePath) {  
  return readFile(filePath)
    .then((data) => JSON.parse(data))
}

readJSON('./package.json')  
  .then((pkg) => console.log(pkg))
  .catch((err) => console.error(err))

The next step would be to use async/await (≥ Node 7) or generators with co (≥ Node 4) to achieve synchronous like control flows for your asynchronous code.

const request = require('request-promise-native')

function getExtractFromWikipedia (title) {  
  return request({
    uri: 'https://en.wikipedia.org/w/api.php',
    qs: {
      titles: title,
      action: 'query',
      format: 'json',
      prop: 'extracts',
      exintro: true,
      explaintext: true
    },
    method: 'GET',
    json: true
  })
    .then((body) => Object.keys(body.query.pages).map((key) => body.query.pages[key].extract))
    .then((extracts) => extracts[0])
    .catch((err) => {
      console.error('getExtractFromWikipedia() error:', err)
      throw err
    })
} 

// PREFER
async function getExtractFromWikipedia (title) {  
  let body
  try {
    body = await request({ /* same parameters as above */ })
  } catch (err) {
    console.error('getExtractFromWikipedia() error:', err)
    throw err
  }

  const extracts = Object.keys(body.query.pages).map((key) => body.query.pages[key].extract)
  return extracts[0]
}

// or
const co = require('co')

const getExtractFromWikipedia = co.wrap(function * (title) {  
  let body
  try {
    body = yield request({ /* same parameters as above */ })
  } catch (err) {
    console.error('getExtractFromWikipedia() error:', err)
    throw err
  }

  const extracts = Object.keys(body.query.pages).map((key) => body.query.pages[key].extract)
  return extracts[0]
})

getExtractFromWikipedia('Robert Cecil Martin')  
  .then((robert) => console.log(robert))


How should I write performant code?

In the first place, you should write clean code, then use profiling to find performance bottlenecks.

Never try to write performant and smart code first, instead, optimize the code when you need to and refer to true impact instead of micro-benchmarks.

"Write clean code first and optimize it when you need to. Refer to true impact instead of micro-benchmarks!"

Click To Tweet

Although, there are some straightforward scenarios like eagerly initializing what you can (eg. joi schemas in route handlers, which would be used in every request and adds serious overhead if recreated every time) and using asynchronous instead of blocking code.

Download the whole building with Node.js series as a single pdf

Next up in Node.js at Scale

In the next episode of this series, we’ll discuss advanced Node.js async best practices and avoiding the callback hell!

If you have any questions regarding clean coding, don’t hesitate and let me know in the comments!


React.js Best Practices for 2016

React.js Best Practices for 2016

2015 was the year of React with tons of new releases and developer conferences dedicated to the topic all over the world. For a detailed list of the most important milestones of last year, check out our React in 2015 wrap up.

The most interesting question for 2016: How should we write an application and what are the recommended libraries?

As a developer working for a long time with React.js I have my own answers and best practices, but it's possible that you won’t agree on everything with me. I’m interested in your ideas and opinions: please leave a comment so we can discuss them.

React.js logo - Best Practices for 2016

If you are just getting started with React.js, check out our React.js tutorial, or the React howto by Pete Hunt.

Dealing with data

Handling data in a React.js application is super easy, but challenging at the same time.
It happens because you can pass properties to a React component in a lot of ways to build a rendering tree from it; however it's not always obvious how you should update your view.

2015 started with the releases of different Flux libraries and continued with more functional and reactive solutions.

Node.js Monitoring and Debugging from the Experts of RisingStack

Build performant backend for React appliactions using Trace
Learn more

Let's see where we are now:

Flux

According to our experience, Flux is often overused (meaning that people use it even if they don't even need it).

Flux provides a clean way to store and update your application's state and trigger rendering when it's needed.

Flux can be useful for the app's global states like: managing logged in user, the state of a router or active account but it can turn quickly into pain if you start to manage your temporary or local data with it.

We don’t recommend using Flux for managing route-related data like /items/:itemId. Instead, just fetch it and store it in your component's state. In this case, it will be destroyed when your component goes away.

If you need more info about Flux, The Evolution of Flux Frameworks is a great read.

Use redux

Redux is a predictable state container for JavaScript apps.

If you think you need Flux or a similar solution you should check out redux and Dan Abramov's Getting started with redux course to quickly boost your development skills.

Redux evolves the ideas of Flux but avoids its complexity by taking cues from Elm.

Keep your state flat

API's often return nested resources. It can be hard to deal with them in a Flux or Redux-based architecture. We recommend to flatten them with a library like normalizr and keep your state as flat as possible.

Hint for pros:

const data = normalize(response, arrayOf(schema.user))

state = _.merge(state, data.entities)  

(we use isomorphic-fetch to communicate with our APIs)

Use immutable states

Shared mutable state is the root of all evil - Pete Hunt, React.js Conf 2015

Immutable logo for React.js Best Practices 2016

Immutable object is an object whose state cannot be modified after it is created.

Immutable objects can save us all a headache and improve the rendering performance with their reference-level equality checks. Like in the shouldComponentUpdate:

shouldComponentUpdate(nexProps) {  
 // instead of object deep comparsion
 return this.props.immutableFoo !== nexProps.immutableFoo
}

How to achieve immutability in JavaScript?
The hard way is to be careful and write code like the example below, which you should always check in your unit tests with deep-freeze-node (freeze before the mutation and verify the result after it).

return {  
  ...state,
  foo
}

return arr1.concat(arr2)  

Believe me, these were the pretty obvious examples.

The less complicated way but also less natural one is to use Immutable.js.

import { fromJS } from 'immutable'

const state = fromJS({ bar: 'biz' })  
const newState = foo.set('bar', 'baz')  

Immutable.js is fast, and the idea behind it is beautiful. I recommend watching the Immutable Data and React video by Lee Byron even if you don't want to use it. It will give deep insight to understand how it works.

Observables and reactive solutions

If you don't like Flux/Redux or just want to be more reactive, don't be disappointed! There are other solutions to deal with your data. Here is a short list of libraries what you are probably looking for:

  • cycle.js ("A functional and reactive JavaScript framework for cleaner code")
  • rx-flux ("The Flux architecture with RxJS")
  • redux-rx ("RxJS utilities for Redux.")
  • mobservable ("Observable data. Reactive functions. Simple code.")

Routing

Almost every client side application has some routing. If you are using React.js in a browser, you will reach the point when you should pick a library.

Our chosen one is the react-router by the excellent rackt community. Rackt always ships quality resources for React.js lovers.

To integrate react-router check out their documentation, but what's more important here: if you use Flux/Redux we recommend to keep your router's state in sync with your store/global state.

Synchronized router states will help you to control router behaviors by Flux/Redux actions and read router states and parameters in your components.

Redux users can simply do it with the redux-simple-router library.

Code splitting, lazy loading

Only a few of webpack users know that it's possible to split your application’s code to separate the bundler's output to multiple JavaScript chunks:

require.ensure([], () => {  
  const Profile = require('./Profile.js')
  this.setState({
    currentComponent: Profile
  })
})

It can be extremely useful in large applications because the user's browser doesn't have to download rarely used codes like the profile page after every deploy.

Having more chunks will cause more HTTP requests - but that’s not a problem with HTTP/2 multiplexed.

Combining with chunk hashing you can also optimize your cache hit ratio after code changes.

The next version of react-router will help a lot in code splitting.

For the future of react-router check out this blog post by Ryan Florence: Welcome to Future of Web Application Delivery.

Components

A lot of people are complaining about JSX. First of all, you should know that it’s optional in React.

At the end of the day, it will be compiled to JavaScript with Babel. You can write JavaScript instead of JSX, but it feels more natural to use JSX while you are working with HTML.
Especially because even less technical people could still understand and modify the required parts.

JSX is a JavaScript syntax extension that looks similar to XML. You can use a simple JSX syntactic transform with React. - JSX in depth

If you want to read more about JSX check out the JSX Looks Like An Abomination - But it’s Good for You article.

Use Classes

React works well with ES2015 classes.

class HelloMessage extends React.Component {  
  render() {
    return <div>Hello {this.props.name}</div>
  }
}

We prefer higher order components over mixins so for us leaving createClass was more like a syntactical question rather than a technical one. We believe there is nothing wrong with using createClass over React.Component and vice-versa.

PropType

If you still don't check your properties, you should start 2016 with fixing this. It can save hours for you, believe me.

MyComponent.propTypes = {  
  isLoading: PropTypes.bool.isRequired,
  items: ImmutablePropTypes.listOf(
    ImmutablePropTypes.contains({
      name: PropTypes.string.isRequired,
    })
  ).isRequired
}

Yes, it's possible to validate Immutable.js properties as well with react-immutable-proptypes.

Higher order components

Now that mixins are dead and not supported in ES6 Class components we should look for a different approach.

What is a higher order component?

PassData({ foo: 'bar' })(MyComponent)  

Basically, you compose a new component from your original one and extend its behaviour. You can use it in various situations like authentication: requireAuth({ role: 'admin' })(MyComponent) (check for a user in higher component and redirect if the user is not logged in) or connecting your component with Flux/Redux store.

At RisingStack, we also like to separate data fetching and controller-like logic to higher order components and keep our views as simple as possible.

Testing

Testing with good test coverage must be an important part of your development cycle. Luckily, the React.js community came up with excellent libraries to help us achieve this.

Component testing

One of our favorite library for component testing is enzyme by AirBnb. With it's shallow rendering feature you can test logic and rendering output of your components, which is pretty amazing. It still cannot replace your selenium tests, but you can step up to a new level of frontend testing with it.

it('simulates click events', () => {  
  const onButtonClick = sinon.spy()
  const wrapper = shallow(
    <Foo onButtonClick={onButtonClick} />
  )
  wrapper.find('button').simulate('click')
  expect(onButtonClick.calledOnce).to.be.true
})

Looks neat, isn't it?

Do you use chai as assertion library? You will like chai-enyzime!

Redux testing

Testing a reducer should be easy, it responds to the incoming actions and turns the previous state to a new one:

it('should set token', () => {  
  const nextState = reducer(undefined, {
    type: USER_SET_TOKEN,
    token: 'my-token'
  })

  // immutable.js state output
  expect(nextState.toJS()).to.be.eql({
    token: 'my-token'
  })
})

Testing actions is simple until you start to use async ones. For testing async redux actions we recommend to check out redux-mock-store, it can help a lot.

it('should dispatch action', (done) => {  
  const getState = {}
  const action = { type: 'ADD_TODO' }
  const expectedActions = [action]

  const store = mockStore(getState, expectedActions, done)
  store.dispatch(action)
})

For deeper redux testing visit the official documentation.

Use npm

However React.js works well without code bundling we recommend using Webpack or Browserify to have the power of npm. Npm is full of quality React.js packages, and it can help to manage your dependencies in a nice way.

(Please don’t forget to reuse your own components, it’s an excellent way to optimize your code.)

Bundle size

This question is not React-related but because most people bundle their React application I think it’s important to mention it here.

While you are bundling your source code, always be aware of your bundle’s file size. To keep it at the minimum you should consider how you require/import your dependencies.

Check the following code snippet, the two different way can make a huge difference in the output:

import { concat, sortBy, map, sample } from 'lodash'

// vs.
import concat from 'lodash/concat';  
import sortBy from 'lodash/sortBy';  
import map from 'lodash/map';  
import sample from 'lodash/sample';  

Check out the Reduce Your bundle.js File Size By Doing This One Thing for more details.

We also like to split our code to least vendors.js and app.js because vendors updates less frequently than our code base.
With hashing the output file names (chunk hash in WebPack) and caching them for the long term, we can dramatically reduce the size of the code what needs to be downloaded by returning visitors on the site. Combining it with lazy loading you can imagine how optimal can it be.

If you are new to Webpack, check out this excellent React webpack cookbook.

Component-level hot reload

If you ever wrote a single page application with livereload, probably you know how annoying it is when you are working on something stateful, and the whole page just reloads while you hit a save in your editor. You have to click through the application again, and you will go crazy repeating this a lot.

With React, it's possible to reload a component while keeping its states - boom, no more pain!

To setup hot reload check out the react-transform-boilerplate.

Use ES2015

I mentioned that we use JSX in our React.js components what we transpile with Babel.js.

Babel logo in React.js Best Practices 2016

Babel can do much more and also makes possible to write ES6/ES2015 code for browsers today. At RisingStack, we use ES2015 features on both server and client side which are available in the latest LTS Node.js version.

Linters

Maybe you already use a style guide for your JavaScript code but did you know that there are style guides for React as well? We highly recommend to pick one and start following it.

At RisingStack, we also enforce our linters to run on the CI system and for git push as well. Check out pre-push or pre-commit.

We use JavaScript Standard Style for JavaScript with eslint-plugin-react to lint our React.js code.

(That's right, we do not use semicolons anymore.)

GraphQL and Relay

GraphQL and Relay are relatively new technologies. At RisingStack, we don’t use it in production for now, just keeping our eyes open.

We wrote a library called graffiti which is a MongoDB ORM for Relay and makes it possible to create a GraphQL server from your existing mongoose models.
If you would like to learn these new technologies we recommend to check it out and play with it.

Takeaway from these React.js Best Practices

Some of the highlighted techniques and libraries are not React.js related at all - always keep your eyes open and check what others in the community do. The React community is inspired a lot by the Elm architecture in 2015.

If you know about other essential React.js tools that people should use in 2016, let us know in the comments!

How to Become a Better Node.js Developer in 2016

These tips and best practices are not just for development - but how to operate Node.js infrastructures, how you should do your day-to-day development and other useful pieces of advice.

UPDATE: We published a newer version of this article, which summarizes our Node.js Best Practices for 2017, areas you should focus on if you'd like to become a better Developer.

Use ES2015

During the summer of 2015 the final draft of ES2015 (formerly ES6) was published. With this a number of new language features were added to the JavaScript language, including:

  • arrow functions,
  • template strings,
  • rest operator, argument spreading,
  • generators,
  • promises,
  • maps, sets,
  • symbols,

and a lot more. For a comprehensive list of new features check out ES6 and Beyond by Kyle Simpson. Most of them are added to Node.js v4.

On the client side, you can already use all of them with the help of Babel, a JavaScript compiler. Still, on the server side we prefer to use only the features that are added to the latest stable version, without compiling the source to save us from all the potential headaches.

For more information on ES6 in Node.js, visit the official site: https://nodejs.org/en/docs/es6/.

Callback convention - with Promise support

For the last years, we encouraged you to expose an error-first callback interface for your modules. With the generators functions already being available and with the upcoming async functions your modules (the ones published to NPM) should expose an error-first callback interface with Promise support.

Why? To provide backward compatibility a callback interface has to be provided, and for future compatibility you will need the Promise support as well.

For a demonstration of how to do it, take a look at the script below. In this example the readPackage function reads the package.json file and returns its' content by providing both a Promise and a callback interface.

Async patterns

For a long time in Node.js, you had two choices to manage asynchronous flows: callbacks and streams. For callbacks you could use libraries like async and for streams through, bl or highland.

With the introduction of Promises, generators and the async functions it is changing.

For a more detailed history of asynchronous JavaScript check out The Evolution of Asynchronous JavaScript

Error handling

Error handling is a crucial part of the application to get right: knowing when to crash, or simply just log the error and continue/retry can be hard.

To make it easier, we have to distinguish between programmer errors and operational errors.

Programmer errors are basically bugs so you should crash the process immediately as you won't know in what state your application is.

On the other hand, operational errors are problems with the system itself or a remote service, like request timeout or running out of memory. Based on the nature of the error you can try to solve with retrying, or if a file is missing you may have to create it first.

Error handling in callbacks

If an error occurs during an async operation, the error object will be passed as the first argument of the async function. You always have to check it and handle it.

The code snippet in the Callback convention section above contains an example.

Error handling in Promises

What's going to happen in the following snippet?

  1. It will throw an exception in line 3
  2. The catch will handle it, and print it out to the stdout: [Error: ops]
  3. The execution continues and in line 9 a new error will be thrown
  4. Nothing else

And really nothing else - the last error thrown will be a silent one. Pay extra attention to always add a catch as the last member of the promise chain. It will save you a lot of headaches. So it should look like this:

And now the output will be:

[Error: ops]
[Error: ups]

Use JavaScript Standard Style

In the past years, we had JSLint then JSHint, JSCS, ESLint - all excellent tools trying to automate as much code checking as possible.

Recently, when it comes to code style we use the JavaScript Standard Style by feross.

js-standard-style

The reason is simple: no configuration needed, just drop it in the project. Some rules that are incorporated (taken from the readme):

  • 2 spaces – for indentation
  • Single quotes for strings – except to avoid escaping
  • No unused variables
  • No semicolons
  • Never start a line with ( or [
    • This is the only gotcha with omitting semicolons
  • Space after keywords if (condition) { ... }
  • Space after function name function name (arg) { ... }
  • Always use === instead of == – but obj == null is allowed to check null || undefined.
  • Always handle the node.js err function parameter
  • Always prefix browser globals with window – except document and navigator are okay
    • Prevents accidental use of poorly-named browser globals like open, length, event, and name.

Also, if your editor of choice supports ESLint only, there is an ESLint ruleset as well for the Standard Style, the eslint-plugin-standard. With this plugin installed your .eslintrc will something like this:

{
  "plugins": [
    "standard"
  ],
}

The Twelve-Factor Application

The Twelve-Factor application manifesto describes best practices on how web applications should be written.

  1. One codebase tracked in revision control, many deploys
  2. Explicitly declare and isolate dependencies
  3. Store config in the environment
  4. Treat backing services as attached resources
  5. Strictly separate build and run stages
  6. Execute the app as one or more stateless processes
  7. Export services via port binding
  8. Scale out via the process model
  9. Maximize robustness with fast startup and graceful shutdown
  10. Keep development, staging, and production as similar as possible
  11. Treat logs as event streams
  12. Run admin/management tasks as one-off processes

Starting new projects

Always start a new project with npm init. This will generate a basic package.json for your project.

If you want to skip the initial questions and go with the defaults, just run npm init --yes.

Monitoring your applications

Getting notified as soon as something wrong happened or is going to happen in your system can save your business.

To monitor your applications, you can use open-source software as well as SaaS products.

For open-source, you can take a look at Zabbix, Collectd, ElasticSearch or Logstash.

If you do not want to host them, you can try Trace, our Node.js and microservice monitoring solution.


Trace - Node.js and microservice monitoring

Use a build system

Automate everything you can. There is nothing more annoying and wasteful activity for a developer than to do grunt work.

Nowadays the tooling around JavaScript evolved a lot - Grunt, Gulp, and Webpack, just to name a few.

At RisingStack, most of the new projects use Webpack to aid in front-end development and gulp for other kinds of automated tasks. At first, Webpack can take more time to understand - for newcomers I highly recommend to check out the Webpack Cookbooks.

Use the latest LTS Node.js version

To get both stability and new features we recommend to use the latest LTS (long-term support) version of Node.js - they are the ones with even release numbers. Of course, feel free to experiment with newer versions, called the Stable release line with the odd release numbers.

If you are working on different projects with different Node.js version requirements then start using the Node Version Manager - nvm today.

For more information on Node.js releases check out the official website: https://nodejs.org/en/blog/community/node-v5/.

Update dependencies on a weekly basis

Make it a habit to update dependencies on a weekly basis. For this, you can use npm outdated or the ncu package.

Pick the right database

When talking about Node.js and databases the first technology that usually comes up is MongoDB. While there is nothing wrong with it, don't just jump into using it. Ask yourself and your team questions before doing so. To give some idea:

  • Do you have structured data?
  • Do you have to handle transactions?
  • How long should you store the data?

You may only need Redis, or if you have structured data then you could go for PostgreSQL. If you start developing with SQL in Node.js, check out knex.

Use Semantic Versioning

Semantic Versioning is a formal convention for specifying compatibility using a three-part version number: major version; minor version; and patch.

Major versions are bumped if an API change is not backward-compatible. Minor versions are bumped when new features are added, but the API change is backward compatible. Patch versions are bumped when only bug fixes happened.

Luckily, you can automate the release of your JavaScript modules with semantic-release.

Keep up

It can be challenging to keep up with the latest news and developments in the JavaScript and Node.js world. To make it easier make sure to subscribe to the following media:

Learn more

Want to learn more about how you could implement Node.js at your company? Drop us a message at RisingStack.com!

Hapi on Steroids - Using Generator Functions with Hapi

You may be familiar with our Koa tutorial series: we took a look on how generator functions work and how you can do a basic application using Koa.

Also, we are huge fans of hapi - if you haven't seen our hapi node.js tutorial, now is the time!

So the question came: wouldn't it be great if we could use generator functions with hapi?

The Motivation

Using error-first callbacks fetching users from your database may look like something like this:

Users.find({  
    age: 32
}, function (err, users) {
    if (err) {
        request.server.log([
          'error',
          'users',
          err
        ]);
        return reply(boom.badImplementation());
    }

    return reply(users);
});

The same logic with hapi on steroids looks like this:

var users;

try {  
  users = yield Users.find({age: 32})    
} catch (exception) {
    request.server.log([
      'error',
      'users',
      exception
    ]);
    return reply(boom.badImplementation());
}

return reply(users);  

How to Use Generators With Hapi

In short: we wrapped all of the route handlers with co in hapi. It is just a thin layer, but it enables you to write all your underlying logic using generators.

var co = require('co');

function get (request, reply) {  
  co(function* () {
    return yield db.getUsers();
  })
    .then(function (response) {
      reply(response);
    })
    .catch(function (err) {
      request.server.log([
        'error',
        'login',
        err
      ]);
      return reply(boom.badImplementation());
    });
}

module.exports.get = get;  

Tests

You may ask: okay, okay, but how to test it, I am used to hapi.inject? Gladly the solution is pretty easy. If you are using co-mocha you can do something like this in your before hook:

before(function () {  
  var _this = this;

  _this.server = yield server.create();

  this.inject = function (options) {
    return new Promise(function (resolve) {
      _this.server.inject(options, function (res) {
        return resolve(res);
      });
    });
  };
});

With this solution, a test for an endpoint will become this:

describe('GET /v1/info', function () {  
  it('should return with a proper status code and body', function* () {
    var res = yield this.inject({
      method: 'GET',
      url: '/v1/info'
    });

    expect(res.result).to.eql({
      status: 'ok'
    });
  });
});

The future

It is great to see more and more ES6 features landing in either Node.js or io.js. With tools like Babel one can use an either bigger set of language features.

We are curious to hear your stories on how you use ES6 in production today, what problems you are facing - don't hesitate to comment! :)

Node.js Production Checklist

Intro

Previously we talked about Node.js best practices then best practices again and how to run Node.js in production.

In this post I'd like to give you a general checklist what you should do before going to production with Node.js. Most of these points are not just applying to Node.js, but every production systems.

Disclaimer: this checklist is just scratching the surface - every production deployment is different, so make sure you understand your system, and use these tips accordingly.

Deployment

Even in bigger systems we see lots of manual steps involved in deployment. This approach is very error prone - in case someone forgets something, you will have a bad time. Never do deployment manually.

Instead you can use tools like Codeship or Shippable if you are looking for hosted solutions, or Jenkins if you are going to set up a more complex pipeline.

Speaking of deployment: you may want to take a look at immutable infrastructures as well, and what challenges they can solve for you.

Security

Security is the elephant in the room - but it shouldn't be. Let's take a look at some of the possible security issues, and how you can fix them.

NPM Shrinkwrap

npm shrinkwrap  

This command locks down the versions of a package's dependencies so that you can control exactly which versions of each dependency will be used when your package is installed.

Uhm, ok, but why do you need this? Imagine the following scenario - during development everything works as expected, all your tests pass, but your production environment is broken. One of the reasons can be, that a new version of the package you are using was released which contained breaking changes.

This is why you should use SemVer as well - but still, we make mistakes, so we better prepare for them. Before pushing your changes to production, use npm shrinkwrap.

Node Security Project CLI

Once you have your npm-shrinkwrap.json, you can check if they have known vulnerabilities.

For this you have to install nsp using npm i nsp -g.

After that just use nsp audit-shrinkwrap, and hopefully you will get No vulnerable modules found as a result. If not, you should update your dependencies.

For more on Node.js Security you should watch Adam Baldwin's talk and read our blogpost dealing with Node.js Security.

Use VPNs and Private Networks

Private networks are networks that are only available to a restricted set of users and servers.

A VPN, or virtual private network, is a way to create secure connections between remote computers and present the connection as if it were a local private network. This provides a way to configure your services as if they were on a private network and connect remote servers over secure connections. - Digital Ocean

But why? Using VPNs you can create a private network that only your servers can see. With this solution your communication will be private and secure. You only have to expose the interfaces that your clients actually need - no need to open up a port for Redis or Postgres.

Logging

In short: log everything, all the time - but not just in your Node.js application, but on the operating system level as well. For this one of the most popular solutions is to use Logstash.

But why do you need logging? Just a couple of the use cases: (sure, sometimes they overlap)

  • find problems in applications running in production
  • find security holes
  • oversee your infrastructure

Speaking of Node, you can use either Winston or Bunyan to do logging.

You can check out related blogposts by Pinterest and Cloudgear as well.

Monitoring & Alerting

Monitoring is crucial - but not just in mission critical systems. If something bad happens, like your landing page is not showing up, you want to be notified about it.

There are tons of tools out there coming to the rescue like Zabbix, New Relic, Monit, PagerDuty, etc. The important thing here is to pick what suits you the best, and use that - just make sure you have it set up. Do not have illusions that your system won't fail - I promise you, it will and you are going to have a bad time.

For a more detailed talk on monitoring I strongly suggest you to watch the following video on monitoring the Obama campaign.

Caching Node.js Production Applications

Cache (almost) everything - by caching I don't only mean the classical HTTP caching, but on the database level as well.

The whys:

  • smaller load on your servers -> cost-effective infrastructure
  • faster responses to the clients -> happy users

Speaking of HTTP REST APIs it is really easy to implement caching, but one thing you should keep in mind: GET endpoints can be cached, but PUT, POST, & DELETE endpoints cannot.

For a reference implementation I would suggest you to read API Caching 101 by Fastly.

Outro

This checklist applies to most systems, not just to the ones implemented in Node.js. As this list just scratching the surface, I would like to ask you: what would you add to the list? All comments/feedbacks are very welcomed!

Recommended reading

A step-by-step guide on how to set up your own Node.js production environment.

Node.js Best Practices - Part 2

You may remember our previous post on Node.js best practices. In this article we will continue with more best practices that can help you become a better Node.js developer.

Consistent Style

When developing JavaScript applications in a bigger team, it is important to create a style guide that everyone accepts and adapts to. If you are looking for inspiration, I would recommend checking out the RisingStack Node.js Style Guide.

But this is just the first step - after you set a standard, all of your team members have to write code using that style guide. This is where JSCS comes into the picture.

JSCS is a code style checker for JavaScript. Adding JSCS to your project is a piece of cake:

npm install jscs --save-dev  

The very next step you have to make is to enable it from the package.json file by adding a custom script:

scripts: {  
    "jscs": "jscs index.js"
}

Of course, you can add multiple files/directories to check. But why we have just created the custom script inside the package.json file? We installed jscs as a local dependency only, so we can have multiple versions on the same system. This will work because NPM will put node_modules/.bin on the PATH while executing.

You can set your validation rules in the .jscsrc file, or use a preset. You can find the available presets here, and can use them with --preset=[PRESET_NAME].

Enforce JSHint / JSCS Rules

Your build pipeline should contain JSHint and JSCS as well, but it may be a good idea to run pre-commit checks on the developers' computers as well.

To do this easily you can use the pre-commit NPM package:

npm install --save-dev pre-commit  

and configure it in your package.json file:

pre-commit": [  
    "jshint",
    "jscs"
],

Note, that pre-commit will look up what to run in your package.json's script section. By enabling this, these checks will run before every commit.

JS over JSON for configuration

We see that a lot of project uses JSON files as configuration sources. While this may be a widespread approach, JS files provide more flexibility. For this purpose we encourage you to use a config.js file:

Use NODE_PATH

Have you ever encountered something like the following?

When you end up with a quite complex project structure, requiring modules may get messy. To solve this problem you have two options:

  • symlinking your modules into the node_modules folder
  • use NODE_PATH

At RisingStack we use the NODE_PATH way, as symlinking everything to the node_modules folder takes extra effort, and may not work for various operating systems.

Setting up NODE_PATH

Imagine the following project structure:

Node.js project structure for NODE_PATH

Instead of using relative paths, we can use NODE_PATH which will point to the lib folder. In our package.json's start script we can set it and run the application with npm start.

Dependency Injection

Dependency injection is a software design pattern in which one or more dependencies (or services) are injected, or passed by reference, into a dependent object.

Dependency injection is really helpful when it comes to testing. You can easily mock your modules' dependencies using this pattern.

In the example above we have two different dbs. In the index.js file we have the "real" db module, while in the second we simply create a fake one. This way we made it really easy to inject fake dependencies into the modules we want to test.

Need a helping hand in developing your application?

RisingStack provides JavaScript development and consulting services - ping us if you need a helping hand!

Node.js Security Tips

2015 October Update: we've published an updated & extended version of this article. Learn more about Node.js security here: Node.js Security Checklist

Node.js is getting more and more mature, no doubt - despite this, not a lot of security guidelines are out there.

In this post I will share some points you should keep in mind when it comes to Node.js security.

No eval, or friends

Eval is not the only one you should avoid - in the background each one of the following expressions use eval:

  • setInterval(String, 2)
  • setTimeout(String, 2)
  • new Function(String)

But why should you avoid eval?

It can open up your code for injections attacks (eval of user input - wow, it hurts even to write down, please never do this) and is slow (as it will run the interpreter/compiler) .

Strict mode, please

With this flag you can opt in to use a restricted variant of JavaScript. It eliminates some silent errors and will throw them all the time.

Undeletable properties
'use strict';  
delete Object.prototype; // TypeError  
Object literals must be unique
'use strict';  
var obj = {  
    a: 1, 
    a: 2 
}; 
// syntax error
Prohibits with
var obj = { x: 17 };  
with (obj) // !!! syntax error  
{

}

To get a complete list of these silent errors, visit MDN.

Static code analysis

Use either JSLint, JSHint or ESLint. Static code analysis can catch a lot of potential problems with your code early on.

Testing

I hope it goes without saying: testing, testing and a little bit more testing.

unit testing

Sure, it's not just unit tests - you should shoot for the test pyramid.

Say no to sudo node app.js

I see this a lot: people are running their Node app with superuser rights. Why? Because they want the application to listen on port 80 or 443.

This is just wrong. In case of an error/bug your process can bring down the entire system, as it will have credentials to do anything.

Instead of this, what you can do is to set up an HTTP server/proxy to forward the requests. This can be nginx, Apache, you name it.

Avoid command injection

What is the problem with the following snippet?

child_process.exec('ls', function (err, data) {  
    console.log(data);
});

Under the hood child_process.exec makes a call to execute /bin/sh, so it is a bash interpreter and not a program launcher.

This is problematic when user input is passed to this method - can be either a backtick or $(), a new command can be injected by the attacker.

To overcome this issue simply use child_process.execFile.

For the original blogpost dealing with command injection, please visit LiftSecurity.

Temp files

Pay extra attention when creating files, like handling uploaded files. These files can easily eat up all your disk space.

To deal with this, you should use Streams.

Securing your web application

This part is not just about Node - but about how you should secure your web applications in general.

Reflected Cross Site Scripting

This occurs when an attacker injects executable code to an HTTP response. When an application is vulnerable to this type of attack it will send back unvalidated input to the client (mostly written in JavaScript). It enables the attacker to steal cookies, perform clipboard theft and modify the page itself.

Example

http://example.com/index.php?user=<script>alert(123)</script>

If the user query string is sent back to the client without validation, and is inserted into the DOM, it will be executed.

How to prevent it?
  • never insert untrusted data into the DOM
  • HTML escape before inserting

More info on Reflected Cross Site Scripting and how to avoid it.

Stopping Cookie Theft

By default, cookies can be read by JavaScript on the same domain. This can be dangerous in case of a Cross Site Scripting attack. But not just that: any third-party JavaScript library can read them.

Example

var cookies = document.cookie.split('; ');

How to prevent it?

To prevent this you can set the HttpOnly flag on cookies, which will make your cookies unreachable for Javascript.

Content Security Policy

Content Security Policy (CSP) is an added layer of security that helps to detect and mitigate certain types of attacks, including Cross Site Scripting (XSS) and data injection attacks.

CSP can be enabled by the Content-Security-Policy HTTP header.

Example

Content-Security-Policy: default-src 'self' *.mydomain.com

This will allow content from a trusted domain and its subdomains.

More info and examples on CSP.

Cross-Site Request Forgery

CSRF is an attack which forces an end user to execute unwanted actions on a web application in which he/she is currently authenticated.

It can happen because cookies are sent with every request to a website - even when those requests come from a different site.

Example
<body onload="document.forms[0].submit()">  
  <form method="POST" action="http://yoursite.com/user/delete">
    <input type="hidden" name="id" value="123555.">
  </form>
</body>  

The result of the above snippet can easily result in deleting your user profile.

How to prevent it?

To prevent CSRF, you should implement the synchronizer token pattern - luckily the Node community has already done it for you. In short, this is how it works:

  1. When a GET request is being served check for the CSRF token - if it does not exists, create one
  2. When a user input is showed, make sure to add a hidden input with the CSRF token's value
  3. When the form is sent, make sure that the value coming from the form and from the session are a match.
In practice

To see all this in action you should do the Security Adventure workshopper which will guide you through a real life example on how to secure an Express-based application.

Secure your Express application: Helmet for the rescue

Helmet is a series of middlewares that help secure your Express/Connect apps. Helmet helps with the following middlewares:

  • csp
  • crossdomain
  • xframe
  • xssfilter
  • and much more

For more info and on how to use, check out its repository: https://github.com/evilpacket/helmet.

Tools to use

  • npm shrinkwrap: Locks down dependency versions recursively and creates a npm-shrinkwrap.json file out of it. This can be extremely helpful when creating releases. For more info, pay NPM a visit.

  • retire.js: The goal of Retire.js is to help you detect the use of module versions with known vulnerabilities. Simply install with npm install -g retire. After that, running it with the retire command will look for vulnerabilities in your node_modules directory. (Also note, that retire.js works not only with node modules, but with front end libraries as well.)

Stay updated

If you want to stay updated on potential security vulnerabilities (I hope you do!) then follow the Node Security project. Their goal is to audit every single module in NPM, and if they find issues, fix them.

Need help in developing your application?

RisingStack provides JavaScript development and consulting services - ping us if you need a helping hand!

Node.js Best Practices

We get asked about Node.js best practices, tips all the time - so this post intends to clean things up, and summarizes the basics of how we write Node.js at RisingStack.

Some of these Node.js best practices fall under the category of Coding style, some deal with Developer workflow.

Coding style

Callback convention

Modules should expose an error-first callback interface.

It should be like this:

module.exports = function (dragonName, callback) {  
  // do some stuff here
  var dragon = createDragon(dragonName);

  // note, that the first parameter is the error
  // which is null here
  // but if an error occurs, then a new Error
  // should be passed here
  return callback(null, dragon);
}



Need help with enterprise-grade Node.js Development?
Hire the experts of RisingStack!


Always check for errors in callbacks

To better understand why this is a must, first start with an example that is broken in every possible way, then fix it.

// this example is **BROKEN**, we will fix it soon :)
var fs = require('fs');

function readJSON(filePath, callback) {  
  fs.readFile(filePath, function(err, data) {  
    callback(JSON.parse(data));
  });
}

readJSON('./package.json', function (err, pkg) { ... }

The very first problem with this readJSON function, is that it never checks, if an Error happened during the execution. You should always check for them.

The improved version:

// this example is **STILL BROKEN**, we are fixing it!
function readJSON(filePath, callback) {  
  fs.readFile(filePath, function(err, data) {
    // here we check, if an error happened
    if (err) {
      // yep, pass the error to the callback
      // remember: error-first callbacks
      callback(err);
    }

    // no error, pass a null and the JSON
    callback(null, JSON.parse(data));
  });
}

Return on callbacks

One of the problems that still exists in the above example, is that if an Error occurs, then the execution will not stop in the if statement, but will continue. This can lead to lots of unexpected things. As of a rule of thumb, always return on callbacks.

// this example is **STILL BROKEN**, we are fixing it!
function readJSON(filePath, callback) {  
  fs.readFile(filePath, function(err, data) {
    if (err) {
      return callback(err);
    }

    return callback(null, JSON.parse(data));
  });
}

Use try-catch in sync code only

Almost there! One more thing we have to take care of is the JSON.parse. JSON.parse can throw an exception, if it cannot parse the input string to a valid JSON format.

As JSON.parse will happen synchronously, we can surround it with a try-catch block. Please note, that you can only do this with synchronous codeblocks, but it won't work for callbacks!

// this example **WORKS**! :)
function readJSON(filePath, callback) {  
  fs.readFile(filePath, function(err, data) {
    var parsedJson;

    // Handle error
    if (err) {
       return callback(err);
    }

    // Parse JSON
    try {
      parsedJson = JSON.parse(data);
    } catch (exception) {
      return callback(exception);
    }

    // Everything is ok
    return callback(null, parsedJson);
  });
}

Try to avoid this and new

Binding to a specific context in Node is not a win, because Node involves passing around lots of callbacks, and heavy use of higher-level functions to manage control flow. Using a functional style will save you a lot of trouble.
Of course, there are some cases, when prototypes can be more efficient, but if possible, try to avoid them.

Create small modules

Do it the unix-way:

Developers should build a program out of simple parts connected by well defined interfaces, so problems are local, and parts of the program can be replaced in future versions to support new features.

Do not build Deathstars - keep it simple, a module should do one thing, but that thing well.

Use good async patterns

Use async.

Error handling

Errors can be divided into two main parts: operational errors and programmer errors.

Operational errors

Operational errors can happen in well-written applications as well, because they are not bugs, but problems with the system / a remote service, like:

  • request timeout
  • system is out of memory
  • failed to connect to a remote service
Handling operational errors

Depending on the type of the operational error, you can do the followings:

  • Try to solve the error - if a file is missing, you may need to create one first
  • Retry the operation, when dealing with network communication
  • Tell the client, that something is not ok - can be used, when handling user inputs
  • Crash the process, when the error condition is unlikely to change on its own, like the application cannot read its configuration file

Also, it is true for all the above: log everything.

Programmer errors

Programmer errors are bugs. This is the thing you can avoid, like:

  • called an async function without a callback
  • cannot read property of undefined
Handling programmer errors

Crash immediately - as these errors are bugs, you won't know in which state your application is. A process control system should restart the application when it happens, like: supervisord or monit.

Workflow tips

Start a new project with npm init

The init command helps you create the application's package.json file. It sets some defaults, which can be later modified.

Start writing your fancy new application should begin with:

mkdir my-awesome-new-project  
cd my-awesome-new-project  
npm init  

Specify a start and test script

In your package.json file you can set scripts under the scripts section. By default, npm init generates two, start and test. These can be run with npm start and npm test.

Also, as a bonus point: you can define custom scripts here and can be invoked with npm run-script <SCRIPT_NAME>.

Note, that NPM will set up $PATH to look in node_modules/.bin for executables. This helps avoid global installs of NPM modules.

Environment variables

Production/staging deployments should be done with environment variables. The most common way to do this is to set the NODE_ENV variable to either production or staging.

Depending on your environment variable, you can load your configuration, with modules like nconf.

Of course, you can use other environment variables in your Node.js applications with process.env, which is an object that contains the user environment.

Do not reinvent the wheel

Always look for existing solutions first. NPM has a crazy amount of packages, there is a pretty good chance you will find the functionality that you are looking for.

Use a style guide

It is much easier to understand a large codebase, when all the code is written in a consistent style. It should include indent rules, variable naming conventions, best practices and lots of other things.

For a real example, check out RisingStack's Node.js style guide.

Next up

I hope this post will help you succeed with Node.js, and saves you some headaches.

This post will continue with another one dealing with operational tips and best practices.

You can read about deployment tips here: Continuous Deployment of Node.js Applications.

Recommended reading: Node.js Best Practices - Part 2 - JSHint and JSCS Rules, JS over JSON, dependency injection and more.

Need help in developing your application?

RisingStack provides JavaScript development and consulting services - ping us if you need a helping hand!