WIXLIB

Table of ContentsDrawing charts99Axis, scale, range, and domainJoining the data points99101Advantages of directives102Generalizing the chart directive103Testing the directiveSetting up the testing environment105106Unit testing the chart directiveE2E testing of the chart directiveSummary110113115Getting started with KarmaGetting started with Protractor106108Chapter 5: Loading and Parsing DataLoading external dataXHR – the native XMLHttpRequestd3.xhr – the XHR wrapper of D3.jsUseful wrappers for common file typesCreating a simple D3 data loading componentTesting the simple D3 data loading componentSummary of D3.js as a data loading service http – the Angular wrapper for XHRCreating an AngularJS data loading componentTesting the AngularJS data loading componentSummary of AngularJS as a data loading serviceParsing log files to JavaScript objectsParsing log files step by stepParsing the custom date format to a JavaScript dateA reusable data parsing serviceTesting the data parsing serviceGrouping log 34135137139Testing the classifier141Displaying logsSummaryChapter 6: Drawing Curves and ShapesCommon shapes and primitivesCurved lines with the SVG pathDrawing straight linesBézier curvesDrawing ellipses and elliptical arcsGenerating arcs142144147148151154155159164Summary169[ iv ]

Table of ContentsChapter 7: Controlling Transitions and Animations171Chapter 8: Bringing the Chart to Life with Interactions193Chapter 9: Building a Real-time Visualization to MonitorServer Logs227AnimationsTimer in D3.js with d3.timerInterpolation with D3.jsEasy animations with transitionsStaggered transitionsChaining transitionsCreating plausible transitionsInterpolate anything with tweensRealistic animations with easingTransitions in chartsSummaryListen for eventsType of EventsGetting event coordinatesCursorsA simple cursorSnappy cursorsLabeling the cursor positionCustomizing events in D3.jsDisplaying self-updating cursor valuesZooming and panningA simple zoomAn x axis zoomInteractive filtersAdvanced filtering using brushesCreating a simple brushBuilding a custom date filter directiveSummaryBuilding a real-time monitoring serverSetting up a Node.js applicationSetting up a web server that serves static filesAdding server push with WebSocketsReading logs and pushing them to the clientWatching files for 228229231232

Table of ContentsProcessing and visualizing logs on the clientBootstrapping a template with AngularJS and Socket.IOUsing Socket.IO with AngularJSProcessing log filesThe dashboard applicationSummaryIndex[ vi ]234234235237242244247

PrefacePrefaceThe question of how I could better visualize my application server's log files hasalways hovered around in my mind. In this book, we'll materialize this idea. Afterreading the book, you'll be able to analyze the log files of your application server anddisplay them as a time series chart on a web page—a real-time autoupdating chart.I picked my two favorite JavaScript libraries—AngularJS and D3.js—to develop sucha visualization.AngularJS is a web application framework that facilitates the development of modernweb applications with concepts such as the MVC/MVVM pattern, two-way databinding, testable modules, and dependency injection. I especially like the idea that onecan declare templates to extend the HTML language; this enables a whole new worldof applications and the sharing of work between designers and frontend engineers.D3.js is a visualization toolkit that facilitates the generation and manipulationof web-based vector graphics with full access to the underlying SVG standard.Moreover, animations and interactive visualizations change how users perceiveweb applications; D3.js offers everything you need to make a visualizationinteractive out of the box.In this book, I will teach you how to create an interactive real-time visualization forthe Web with AngularJS as the application framework and D3.js as the visualizationtoolkit. I will explain the basic concepts of D3.js, when and why we use it, and whyit's so much better than other DOM-transforming libraries, with a focus on its datadriven approach. Also, I will show you best practices to manipulate datasets andpreprocess data—an everyday job if you want to display data from different sources.Then, we will spend more time on seeing how a visualization library can be bestintegrated into an AngularJS application using modules, directives, and services tomake the components testable and maintainable. Later, we will focus on developingthe visualization.[ vii ]

PrefaceFirst, we need to load real data in the application and parse the text data to aJavaScript array of objects. Then, we need to preprocess and group the dataaccording to the way we want to display it. Now, we can generate a curve or shapein the chart that represents our data. Next, we want to animate this curve—to showa transition between the old and the new state—whenever we update the data ofthe chart. We also want to zoom and pan inside the charts and create filters in orderto clamp the chart to a certain range, for example, a starting time and an endingtime for the time series data. Finally, we want to put everything together on to adashboard and feed it with real server logs of a web server.Displaying data on the web is a common task for developers. This books walks youthrough an exciting project, explaining best practices and common pitfalls in thisdomain using the two most amazing JavaScript libraries: AngularJS and D3.js.What this book coversChapter 1, The Magic of SVG, D3.js, and AngularJS, introduces you to the world ofvector graphics on the web and points out the advantages of D3.js.Chapter 2, Getting Started with D3.js, explains how to draw vector graphics with D3.jsand its data-driven approach.Chapter 3, Manipulating Data, walks you through the important task of manipulatingand preprocessing datasets in JavaScript and D3.js.Chapter 4, Building a Chart Directive, lets you implement a first scatter chart directivein order to integrate the visualization into a simple AngularJS application with afocus on modularization and testability.Chapter 5, Loading and Parsing Data, explains how to load files and parse realdata using AngularJS and D3.js. It also explains how to aggregate informationfor the visualization.Chapter 6, Drawing Curves and Shapes, helps you understand lines, curves, and shapesin SVG and D3.js in order to create different chart types, such as line charts, areacharts, and bar charts.Chapter 7, Controlling Transitions and Animations, shows how animations can be usedto better visualize changes in the dataset and the visualization. You will learn aboutanimations in D3.js and how to create plausible transitions to update chart directives.Chapter 8, Bringing the Chart to Life with Interactions, explains how to add interactionsto the chart directive, which is a big advantage of web-based visualizations. You willalso see how to combine and encapsulate the interactions of the D3 chart and theAngularJS application.[ viii ]

PrefaceChapter 9, Building a Real-time Visualization to Monitor Server Logs, walks you throughan exciting example where you will use all the acquired knowledge of the book tobuild a visualization dashboard to analyze server logs.What you need for this bookTo get started, all you need is a web browser and a text editor, such as Sublime Text.However, I recommend that you install Node.js (http://nodejs.org/) that includesthe npm package manager and the Bower (http://bower.io/) module in order torun the tests. I will walk you through the installation of the required packages in thecorresponding chapters.Starting from Chapter 4, Building a Chart Directive, we will use the http-server module(https://www.npmjs.com/package/http-server) to run the examples from a localweb server. An internet connection is only required for installing tools and libraries,but not for running the examples.Who this book is forData Visualization with D3 and AngularJS is intended for web developers withexperience in AngularJS who want to implement fully customizable interactivevector graphics for all platforms. Whether you already know a bit about SVG andvector graphics in the browser or have never used any visualization library inJavaScript before, you will be able to master the data-driven techniques of D3. Ineither case, this book will get you up and running quickly and also challenges you ifyou have already worked with D3.ConventionsIn this book, you will find a number of text styles that distinguish between differentkinds of information. Here are some examples of these styles and an explanation oftheir meaning.Code words in text, database table names, folder names, filenames, file extensions,pathnames, dummy URLs, user input, and Twitter handles are shown as follows:"The .exit() function will return all elements of the Selection because the datasetcontains no elements."[ ix ]

PrefaceA block of code is set as follows: html head title My App /title /head body h1 class "header My Application /h1 p class "content" Lorem ipsum dolor sit amet, . /p /body /html When we wish to draw your attention to a particular part of a code block, therelevant lines or items are set in bold: html . body object id "vis" width "50" height "30" param name "movie" value "vis.swf" embed src "vis.swf" type "application/x-shockwave-flash" /object /body /html Any command-line input or output is written as follows:npm install --save expressnpm install --save serve-staticNew terms and important words are shown in bold. Words that you see on the screen,for example, in menus or dialog boxes, appear in the text like this: "The t parameterin the range 0 to 1 defines the position on the curve between the starting point P0(in the figure P1) and the end point P3 (in the figure P2); P1 (in the figure C1) and P2(in the figure C2) are called control points and are used to model the shape betweenthese points."Warnings or important notes appear in a box like this.Tips and tricks appear like this.[x]

PrefaceReader feedbackFeedback from our readers is always welcome. Let us know what you think aboutthis book—what you liked or disliked. Reader feedback is important for us as it helpsus develop titles that you will really get the most out of.To send us general feedback, simply e-mail feedback@packtpub.com, and mentionthe book's title in the subject of your message.If there is a topic that you have expertise in and you are interested in either writingor contributing to a book, see our author guide at www.packtpub.com/authors.Customer supportNow that you are the proud owner of a Packt book, we have a number of things tohelp you to get the most from your purchase.Downloading the example codeYou can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If youpurchased this book elsewhere, you can visit http://www.packtpub.com/supportand register to have the files e-mailed directly to you.Downloading the color images of this bookWe also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand thechanges in the output. You can download this file from: oads/8484OS Graphics.pdf.ErrataAlthough we have taken every care to ensure the accuracy of our content, mistakesdo happen. If you find a mistake in one of our books—maybe a mistake in the text orthe code—we would be grateful if you could report this to us. By doing so, you cansave other readers from frustration and help us improve subsequent versions of thisbook. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Formlink, and entering the details of your errata. Once your errata are verified, yoursubmission will be accepted and the errata will be uploaded to our website or addedto any list of existing errata under the Errata section of that title.[ xi ]www.allitebooks.com

PrefaceTo view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The requiredinformation will appear under the Errata section.PiracyPiracy of copyrighted material on the Internet is an ongoing problem across allmedia. At Packt, we take the protection of our copyright and licenses very seriously.If you come across any illegal copies of our works in any form on the Internet, pleaseprovide us with the location address or website name immediately so that we canpursue a remedy.Please contact us at copyright@packtpub.com with a link to the suspected piratedmaterial.We appreciate your help in protecting our authors and our ability to bring youvaluable content.QuestionsIf you have a problem with any aspect of this book, you can contact us atquestions@packtpub.com, and we will do our best to address the problem.[ xii ]

The Magic of SVG, D3.js,and AngularJSIn this book, we will develop an application that combines both frameworks D3.jsand AngularJS into a stunning interactive visualization, an interactive dashboard tovisualize server logs in real time. First, I will discuss the outline and the purpose ofthis application. Then, I will explain how to modularize the visualization so that allcomponents can be seamlessly embedded into one single application.In the second section, we will talk about the terminology and definitions that areused in this book to introduce you to this topic or to refresh your knowledge of somebasics in computer graphics. It will help you to understand why we are using vectorgraphics—especially SVG for the Web—for the visualization that we will develop inthis book (and not, for example, Canvas or WebGL).The next part of the chapter will give an introduction to D3.js and its data-drivenapproach. It will help you to understand the similarities and fundamental differencesof D3.js and other DOM transforming libraries such as jQuery. Then, we will seean introduction to AngularJS and discuss its strengths for modern applicationdevelopment as well as the integration of D3.js into an AngularJS application.The last section of this chapter will help you to understand why we prefer D3.jsover other visualization libraries. It will give a brief outline and comparison of webvisualization tools emphasizing on libraries to create and manipulate vector graphicsand the evolution of web standards. We will discuss the main ideas and advantagesof using D3.js in web applications and the differences between it and other tools forweb visualizations (such as Raphaël or Three.js).[1]

The Magic of SVG, D3.js, and AngularJSIn this chapter, you will learn: How to distinguish vector and pixel graphics and to know theirapplication areas When and why to use vector graphics, especially SVG Why D3.js is superior to other common graphical toolkits When and why to use D3.js for creating graphical content The advantages of integrating D3.js into an AngularJS application How to structure and encapsulate a visualization library written in D3.js touse with an AngularJS applicationBuilding a real-time dashboard tovisualize server logsDid you ever wonder how many users visit your web application or how manyexceptions your server application raised in the last month? The solution to varioussimilar problems is to simply analyze and visualize the log or access files of theparticular application.In this book, we will develop a real-time dashboard to visualize application logsand the server's system status in the browser. Thus, we will build a custom GoogleAnalytics-like visualization dashboard for our own server (see the following figure).Additionally, we want to add custom animation and user interactions—such asselecting data ranges, zooming, and panning—to the visualization.[2]

Chapter 1Google Analytics dashboardThe dashboard (frontend) will be capable of loading, parsing, and grouping log filesof different formats, filtering data based on selected data ranges and displaying,updating, and animating multiple chart types. We will build the visualization librarywith D3.js and the application with AngularJS.While developing this application, we will see all the advantages, problems, andbest practices of D3.js and AngularJS in action. Also, you will learn to combine bothframeworks to one single application.During most of the chapters, we will focus on loading log data from static resources.In the last chapter, we will implement a simple web server (backend) using Node.js and Express to detect log file changes in real time and push these changes to thedashboard using WebSockets and the Socket.IO library.Besides implementing all the functionality, you will also learn how to design andtest (we will use unit and integration tests) the frontend components with the testrunners, Karma and Protractor.[3]

The Magic of SVG, D3.js, and AngularJSTerminology and definitionsLet's start from the beginning. In the following chapters, we will discuss computergraphics and visualizations for the Web. Therefore, it's important to understand thebasic terminology of this domain. In this section, I will refresh your knowledge ofthe two most common image representations (vector and pixel graphics). I will alsodiscuss the web standards that are relevant for graphical applications (DOM, SVG,and so on). If these definitions are not new for you, then nothing can stop you fromjumping directly to the next section.Document Object ModelThe Document Object Model (DOM) is the tree representation of the hierarchicalelements of an HTML document and it was specified by the World Wide WebConsortium (W3C). These elements in the DOM are called nodes (for example, html,head, body, and so on), which can have attributes (for example, class "header")and content (for example, "My Application" is the content of the h1 node). TheDOM provides a public JavaScript interface with which we can access nodes andmanipulate them.Let's look at the source code of a simple HTML page to see an example of theDOM tree: html head title My App /title /head body h1 class "header My Application /h1 p class "content" Lorem ipsum dolor sit amet, . /p /body /html Downloading the example codeYou can download the example code files for allPackt books you have purchased from your accountat http://www.packtpub.com. If you purchasedthis book elsewhere, you can visit h

Setting up an AngularJS application 90 Organizing the directory 90 Installing AngularJS 90 Bootstrapping the index file 91 Adding a module and a controller 92 Integrating D3.js into AngularJS 93 Organizing the directory 93 Wrapping D3.js 94 A chart directive 95 A direc