Lesson Overview

In this lesson you'll learn what Node.js is and what it's typically used for. I'll give a brief and simplified summary of how it works and why it's to your advantage to learn Node.js.

What is Node.js?

Node.js is a runtime environment that allows you to create server-side programs using JavaScript. It allows you to do full-stack development: full-stack development means writing both the back-end and the front-end of an application using a single language/technology. The back-end refers to the server-side code and the front-end refers to the client-side code. Before Node.js, you would write the front-end using JavaScript, which used to be a client-side technology and didn't run on the server. Then you had to write the back-end using a server-side technology such as PHP. With Node.js, you can write both the front-end and back-end using a single language: JavaScript.

Node.js is not a framework, although many frameworks use Node.js as a foundation.

Node.js is open source, and cross-platform (it can run on any machine without having to recompile it for different machine languages).

Unlike regular JavaScript, which runs in a browser, Node.js runs on the server. It has no browser-specific APIs, but it does have APIs for very basic HTTP communcations, basic encryption, and many other useful components you would need for web applications.

Lots of frameworks are based on Node.js. One of the most common is Express.js, which I'll also cover in these tutorials. Other popular frameworks based on Node.js include Koa, Vue.js, NestJs, React, Sails, Next.js, Mocha, and lots more. A quick web search for "Node.js frameworks" will bring up a long list of possibilities, and the list is constantly changing as new frameworks are created and some increase or decrease in popularity.

What is Express.js?

Express.js is the most popular framework for Node.js. Express.js is the foundation of several other popular frameworks such as Sails, Feathers, NestJs, and many more. It helps you perform tasks Node.js can't do alone, such as handling different http verbs (get, post, etc) and using middleware. It also allows you to create HTML templates into which you can place dynamic data, and even do basic things like setting and using environment variables.

Express.js is unopinionated: you can add any middleware you need in whatever order you want, you can modularize your code in multiple files if you want, using any directory structure you prefer, etc.

To use Express.js, we write a Node.js application, import Express.js, and then use the Express.js modules and functions we are interested in.

How Node.js Works

Typically, you start a Node.JS application by creating a server that responds to http requests. The code in your app (that you write) will create the server, configure it, and handle requests that come in. When your app runs, it will receive requests and send appropriate responses. A lot of this work is done using callback functions.

How does a Node.js Application Work?

Usually JavaScript is client-side: The browser contains JavaScript engine that interprets/executes JavaScript. JavaScript Engines include:

Node.js is built with V8, but since it executes server-side, it doesn't matter to the other browsers that use different engines!

The Event Loop

A thread is a single execution context: it can only execute one task at a time. Some processes require several threads in order to get their tasks done. Most software uses a thread pool: a collection of threads that are available to be assigned to processes waiting to execute. When a process needs a thread, it takes one from the thread pool when it becomes available. In this way, processes can get things done using multiple threads at the same time.

Unlike most software, Node.js uses a single event loop. This event loop consists of a single thread that, like all threads, can only perform one task at a time. It's very similar to a chef working in her kitchen: She can be working on several dishes for several orders in various stages of completion: she could be cooking food for one order while preparing ingredients for a new order. However, she can only work on one specific task at a time.

The chef has specific dishes to complete and each dish consists of a specific set of tasks that need to be performed in order, but the chef is not obligated to complete a task in full before starting a different task. For example, she can be cutting some onions for an order that just came in, and then stop cutting to turn and flip some steaks for an order that is almost completed, then taste a sauce and add some salt, and then turn back to cutting onions.

The chef can even delegate some tasks to other chefs or cooks in the kitchen. For example, she could have an apprentice cut the onions while she tells another to retrieve something from the fridge, and then she can start plating the steaks that are now finished cooking.

The Node.js event loop is similar: there are a series of tasks to get done, and the event loop is always working on one task at a time. Sometimes it offloads larger tasks to other components of the system or to other modules. It does this with callbacks: the component will inform the event loop when the callback has been finished, and the event loop will pick the task back up and work on it before passing it off to something else so that a different task can be worked on.

Here's a simplified explanation of how this works. Note that I'm skimming over some of the finer details. If you want to know more, you can find many resources online that explain the various phases of the event loop and how those phases work together to process many tasks in multiple request/response cycles.

Imagine a couple of requests arrive at the server for processing. These could be requests for a simple web page, or they could be more complex requests that require access to a database or more intensive processing. A request may consist of multiple tasks: For example, a request might require retrieving some database data and then formatting that data in an HTML table before that request's response can be sent back to the client.

  1. In this example, I've imagined 2 requests: one is a bit more complex and has two tasks, and one is simple and has one task. All tasks enter the event queue, which is like a lineup where the tasks wait for their turn to be processed by the event loop.
    tasks wait in the event queue for their turn
    Tasks for incoming requests wait for their turn in the Event Queue
  2. As discussed previously, the event loop is a single thread, so it can only focus on one task at a time. When it's free, it pops the next available task out of the event queue for processing. The remaining tasks move up in the queue to wait for their turn.
    request one's task 1 enters the event loop
    The first task in line enters the event loop
  3. Before the task is processed, it is examined to see what kind of operation it needs to perform: a blocking operation or a non-blocking operation? Blocking operations include things like database access, network requests, reading or writing to the file system, and more intensive computations that need a lot of resources. These kinds of operations will block the event loop if they are processed in the event loop's thread. For example, if the task involves reading a list of books from the database, the event loop has to stop and wait for the database to receive the query, process it, and return the results. Non-blocking operations do not block the event loop because they are very quick and require fewer resources. For example, calculating a result from some form inputs that came in with the request and adding the result to a results.html page that will be sent back to the client.
    is the task a blocking or non-blocking operation
    The first task in line enters the event loop
  4. If the task requires a blocking operation, the task needs to be passed off to another thread so that it doesn't block the event loop. Recall that a thread can only work on one task at a time, so a blocking operation on the event loop means that no other tasks/requests can be processed until the operation has completed. So the event loop passes blocking tasks off to worker threads.
    blocking operations are passed to worker threads
    Blocking operations are passed to worker threads

    Part of Node.js includes a thread pool: a collection of available worker threads who sit and wait until they are needed. These separate worker threads process blocking tasks via callbacks: the callback executes on the worker thread so that the event loop can move on to the next job.

    This means the next task can enter the event loop for processing while the worker thread is busy dealing with the first task. This task has to be evaluated to see if it contains a blocking operation or non-blocking operation.

  5. Meanwhile, as the event loop is processing the second task, the callback for the first task may have finished. The event loop is busy, so the worker thread, which is now free, returns to the thread pool and the first task's result waits in the task queue. It has to wait here until the event loop is free.
    callback finishes its work and moves result to the task queue while second task is being processed
    Callback has finished, Result waits for its turn while next task is processed
  6. If a task contains a non-blocking operation, it can be processed right in the event loop. This is also done with a callback: the callback executes and the result is put back into the event loop to move on to its response/destination.
    non blocking operation is processed in the event loop
    Non-Blocking operations can be processed in the event loop

    Once the second task's non-blocking operation has completed, the event loop sends the result to the area that prepares and sends responses. At this time, a new request with 2 tasks arrives: these tasks sit in the event queue and wait their turn for processing.

  7. Now the event loop is free. It's now able to pick up the waiting Task 1 result in the task queue and if it has no other processing, it can pass that on to the response preparation area. Once that has finished, it can pick up the next task in the event queue.
    both part 1 and part 2 of the first request re-enter the event loop to go to response preparation, task 1 of request 2 enters event loop
    Callback has finished, Result waits for its turn while next task is processed

    Both the tasks for the first request can now be used to prepare and send off a response to the client. Meanwhile, the task for the second request is being processed as a blocking or non-blocking operation. Eventually, the next task in the event queue can enter the event loop for processing.

  8. The event loop is infinte: it will continue to process requests and responses in this way, passing off the more intensive or slower operations to worker threads so that it can keep things moving!
    Response 1 is prepared and sent to client while 2nd request's task is being processed
    Response for the first request is prepared and sent while 2nd request is being processed

Because of the event loop and the way Node.js works, it's a great choice for handling multiple request/response cycles. Extra modules/frameworks can be added to handle both blocking and non-blocking operations. This is why Node.js is able to quickly handle several requests efficiently with fewer resources.

Why Learn Node.js?

Node.js uses the V8 engine, which is widely supported on the internet: V8 can only grow and improve and will be around for a very long time. So learning Node.js is a good investment in your future. Jobs that require Node.js and Node.js frameworks are currently in-demand. Some very familiar web applications are currently using Node.js, such as Mozilla, Paypal, Yahoo, LinkedIn, Netflix, eBay, and Uber.

Node.js is much better at certain things than other technologies, so if you plan on writing any of these types of applications, it's definitely a good idea to learn Node.js. For example, it's much better at applications that require chat functionality, instant messaging, and real-time collaboration tools. It's also great for applications that require continuous data streaming, single-page applications that need a lot of functionality in a single page, and location-based applications that need continuous location updates. It's also the preferred technology for IoT (Internet of Things) systems because of its ability to quickly and efficiently process several tasks at the same time.

Node.js itself has many benefits over other languages: