WIXLIB

Node.js and Backend DevelopmentShan-Hung WuCS, NTHU

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk2

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk3

Clone hello-node npm install --save yargs Yargs– Command line interface (CLI) tools Usage: node src/main.js command [options]4

Javscript runtime engine Why Node.js in the backend?– Event-based and asynchronous I/O; very fast– NPM and large ecosystem5

Node Runtime node app.js node// REPL Global objects:– window global (scope)– document process (.env, .argv(), .exit()etc.)– os, fs, module.exports Node.js supports ES6– No need for Babel– No import (use require() instead) See API Docs6

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk7

WeatherMood Postson Node.jsWhy fast as an backend engine?8

Asynchronous I/O// A.js.// 1fs.readReadFile('file.name', (err, data) {. // 3});// 2http.get('url', res {Time. // 5});// 4CPU A1 A6A5A3.// 6I/O 1 Shortens exectime– If with parallelI/O channelsAsyncA2I/O 2SyncA4TimeCPU A1I/O 1I/O 2A5 A6A3A2A49

Single-Threaded Event Loop// User A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6// User B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1I/O 1I/O 2Process10

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1I/O 1A2I/O 2Process11

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1I/O 1I/O 2ProcessA2A412

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6I/O 1I/O 2ProcessA2A413

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2A414

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6A3I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2A415

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6A3I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B2A416

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6A3I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B2A4B417

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6A3I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6B2A4B418

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3B2A4B419

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6A5I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3B2A4B420

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B3A5CPU A1 A6I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3B2A4B421

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6B3I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3A5B2A4B422

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3A5B3B2A4B423

Single-Threaded Event Loop// A.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6CPU A1 A6I/O 1I/O 2Process// B.js.// 1fs.readReadFile(., () {. // 3});// 2http.get(., () {. // 5});// 4.// 6B1A2B6A3A5B3B5B2A4B424

CPU A1 A6I/O 1I/O 2B1A2B6A3A5B5AdvantagesB2A4B3B4 Interleaved exec of multiple code blocks Short response time Pipelining between CPU and I/Os– High throughput Avoids overhead of concurrency control (e.g.,locks, thread scheduling, etc.)– Higher throughput than muti-threaded engines25

Multi-Core Machines? Cluster module that runs one Node.js processper core Scales up linearly, if– Workload can be partitioned evenly by processes– I/Os are not saturated26

When Not to Use Node.js? CPU-bound tasksCPU A1I/O 1I/O 2A6A2B1B6A3A5B3B5B2A4B4 Or, use child process.exec() (orspawn() for streaming output from child)– Foreground core background cores27

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk28

NPM Package manager npm init npm install --[save save-dev] pkg-name var m require('module'); Packages vs. modules?29

--save vs. --save-dev? Given dependency tree:Your proj Pkg 1 Pkg PkgPkg 2 Pkg PkgPkg 1Pkg 23435 People who clone/fork your package willdownload the following packages:{Pkg 1, Pkg 2, Pkg 3}// via 'npm install'30

Command Line Interface (CLI) Option 1: to parse process.argv yourself Option 2: yargs.argv npm install --save yargs– Defines commands and their options– Help– Sanity checks31

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk32

Debugging// in srcdebugger; nodedebug debug debug debug src/main.js command [options]ncrepl Chrome inspector equivalent?33

Node Inspector (Experimental) node --inspect --debug-brk src/main.js . Then paste “chrome-devtools://.” intoChrome34

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk35

Node.js as Backend Engine Typically, web workloads are I/O bound– High throughput– Low latency If designed well, API requests can be easilypartitioned across multiple processes– Scales linearly36

Clone weathermood-server Checkout the file branch npm install --save express body-parser npm install –save-dev nodemon Express– A web app framework based on Node.js Body-Parser– An Express middleware for parsing request body Nodemon– Auto-restarter (remember “webpack -w”?)37

Web App Backend Development1. Prepare static resources– E.g., *.html, *.css, client-side JS, images, etc.– In dist/ of branch server-file ofweathermood2. Define API for AJAX calls– Dynamic resources, i.e., API for AJAX calls3. Code API4. Deploy web app to hosting server(s)– Web App web server(s)– For now, local machine as development server38

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk39

Defining APIs E.g., listing posts, create a post, vote, etc. But HTTP defines only 4 methods– GET, POST, PUT, DELETE Option 1: define new “verbs”– Always POST to the same URL– Define body (can follow SOAP) Option 2: define new “nouns”– E.g., vote POST vote– Different URLs for different nouns/resources– REST makes your backend simple and scalable40

URL MappingsURLs\MethodsGETPOSTPUTDELETEReplace the entirecollection.Delete theentirecollection.Treat this resource as Update thisa collection andresource or createcreate a new member. one (known ID).Delete thisresource.http:// {host}/ {resource}sList all resources Create a new(satisfying query resource (unknown“?.”).ID).http:// {host}/ {resource}s/ {id}Read a specificresource. Each resource type maps to 2 URL types– Collection URLs vs resource URLs List post:GET /posts?seatchText .&. Create post: POST /posts Vote “clear”: POST /posts/ {id}/clearVotes41

HTTP Response Codes GET:– 200 OK (with body) POST:– 200 OK (with body)– 201 Created (with header Location showing ID)– 204 No Content PUT and DELETE:– 200 OK (with body)– 204 No Content Error:– 400 Bad Request, 401 Unauthorized, 404 Not Found, or500 Internal Server Error42

Requirements Stateless: session state (e.g., shopping cart)cannot be kept in client side– Session state sent with requests using cookies– So, requests from clients can be partitioned easily(scalable web servers) GET requests much have no side effect– Allows proxy nodes in the routing path PUT and DELETE requests must beidempotent: duplicated requests has no effect43

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk44

Express Normal flow: routers model calls– Request parsing: req.query, req.params,req.body– Responses: res.status(), res.json(),res.sendStatus() Error flow: throw error handlermiddleware responses45

Middleware Error handlingServing static filesFilteringLoggingValidationAuthentication and authorizationand more.46

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk47

Debugging Nodemon: npm run watch Postman48

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk49

Cloud Computing Not only services Also platformsor infrastructure– Pay only whatyou used50

IaaS, PaaS, and SaaS51

Web App Backend:PaaS or IaaS?52

Development Efficiency Iterations: bug fixes, updates, finding PMF, etc. Time is money! IaaS: high management cost, low elastisity PaaS: low cost and elastic, but“Well, it works on my computer ”

Outline Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk54

PaaS You define your runtime55

Virtual Machines vs. Containers Lightweight, elastic Consistent runtime56