Transcription
i
Elastic SearchAbout the TutorialElasticsearch is a real-time distributed and open source full-text search and analyticsengine. It is used in Single Page Application (SPA) projects. Elasticsearch is an open sourcedeveloped in Java and used by many big organizations around the world. It is licensedunder the Apache license version 2.0.In this tutorial, you will learn in detail the basics of Elasticsearch and its important features.AudienceThis tutorial is designed for software professionals who want to learn the basics ofElasticsearch and its programming concepts in simple and easy steps. It describes thecomponents of Elasticsearch with suitable examples.This tutorial is designed to configure the HR module of SAP in an easy and systematic way.Packed with plenty of screenshots, it will be useful for consultants as well as end-users.PrerequisitesBefore you begin with this tutorial, you should have a basic understanding of Java, JSON,search engines, and web technologies. The interaction with Elasticsearch is throughRESTful API; therefore, it is always recommended to have knowledge of RESTful API.If you are new to any of these concepts, we suggest you to take the help of tutorials basedon these topics before you start with Elasticsearch.Copyright & Disclaimer@Copyright 2019 by Tutorials Point (I) Pvt. Ltd.All the content and graphics published in this e-book are the property of Tutorials Point (I)Pvt. Ltd. The user of this e-book is prohibited to reuse, retain, copy, distribute or republishany contents or a part of contents of this e-book in any manner without written consentof the publisher.We strive to update the contents of our website and tutorials as timely and as precisely aspossible, however, the contents may contain inaccuracies or errors. Tutorials Point (I) Pvt.Ltd. provides no guarantee regarding the accuracy, timeliness or completeness of ourwebsite or its contents including this tutorial. If you discover any errors on our website orin this tutorial, please notify us at contact@tutorialspoint.comii
Elastic SearchTable of ContentsAbout the Tutorial . iiAudience . iiPrerequisites . iiCopyright & Disclaimer. iiTable of Contents . iii1.ELASTIC SEARCH – BASIC CONCEPTS . 1General Features . 1Key Concepts . 1Advantages . 2Disadvantages . 3Comparison between Elasticsearch and RDBMS. 32.ELASTIC SEARCH – INSTALLATION. 43.ELASTIC SEARCH – POPULATE. 8Create Index . 8Add data . 8Adding Sample Data in Kibana . 104.ELASTIC SEARCH – MIGRATION BETWEEN VERSIONS . 12Steps for Upgrade . 12Upgrading from 6.6 or Earlier . 135.ELASTIC SEARCH – API CONVENTIONS . 14Multiple Indices . 14Comma Separated Notation . 14all Keyword for All Indices . 14Wildcards ( * , , – ) . 15iii
Elastic Searchallow no indices . 17expand wildcards . 18Date Math Support in Index Names . 19Pretty Results . 19Human Readable Output. 20Response Filtering . 206.ELASTIC SEARCH – DOCUMENT APIS . 21Index API . 21Automatic Index Creation . 22Versioning . 22Operation Type . 23Automatic ID generation . 24Get API . 24Delete API . 27Update API . 277.ELASTIC SEARCH – SEARCH APIS . 29Multi-Index . 29URI Search . 30Request Body Search . 318.ELASTIC SEARCH – AGGREGATIONS . 33Metrics Aggregations . 33Avg Aggregation . 33Cardinality Aggregation . 36Extended Stats Aggregation . 37Max Aggregation . 38Min Aggregation . 39Sum Aggregation . 40iv
Elastic SearchStats Aggregations . 41Aggregation Metadata . 429.ELASTIC SEARCH – INDEX APIS . 44Create Index . 44Delete Index . 45Get Index . 45Index Exist . 46Index Settings. 46Index Stats . 47Flush . 4710. ELASTIC SEARCH – CAT APIS . 48Verbose . 48Headers . 48Sort . 49Count . 4911. ELASTIC SEARCH – CLUSTER APIS. 50Cluster Health . 50Cluster State . 51Cluster Stats . 52Cluster Update Settings . 53Node Stats. 53Nodes hot threads. 5412. ELASTIC SEARCH – QUERY DSL. 55Match All Query . 55Full Text Queries . 57Query String Query . 60v
Elastic SearchTerm Level Queries . 61Range Query . 62Compound Queries . 64Geo Queries . 6513. ELASTIC SEARCH – MAPPING . 67Field Data Types . 67Removal of Mapping Types . 6914. ELASTIC SEARCH – ANALYSIS . 70Configuring the Standard analyzer . 71Tokenizers . 7415. ELASTIC SEARCH – MODULES . 77Cluster-Level Routing and Shard Allocation . 77Discovery . 79Gateway . 80HTTP. 80Indices . 81Node . 8216. ELASTIC SEARCH – INDEX MODULES. 83Static Index Settings . 83Dynamic Index Settings . 8317. ELASTIC SEARCH – INGEST NODE. 84Steps Involved . 8418. ELASTIC SEARCH – MANAGING INDEX LIFECYCLE . 87Policy Management APIs . 87Index Management APIs . 87Operation Management APIs . 88vi
Elastic Search19. ELASTIC SEARCH – SQL ACCESS. 89Advantages of Elasticsearch SQL . 89SQL Query . 9120. ELASTIC SEARCH – MONITORING . 92Collectors . 92Exporters. 9221. ELASTIC SEARCH – ROLLUP DATA . 93Create a Rollup Job . 9422. ELASTIC SEARCH – FROZEN INDICES . 96Example for Freezing and Unfreezing . 96Searching a Frozen Index . 96Monitoring Frozen Indices . 9623. ELASTIC SEARCH – TESTING . 97Prerequisites . 97Unit Testing . 97Integration Testing . 97Test Cluster Methods . 98Accessing Clients . 99Randomized Testing . 99Assertions . 10024. ELASTIC SEARCH – KIBANA DASHBOARD . 101Dashboard Creation . 101Inspecting Elements . 104Sharing Dashboard . 10525. ELASTIC SEARCH – FILTERING BY FIELD. 106Filtering by Time . 107vii
Elastic SearchFiltering by Fields . 10726. ELASTIC SEARCH – DATA TABLES . 110Visualize . 110Select Data Table. 111Select Metrics. 11227. ELASTIC SEARCH – REGION MAPS. 114Visualize . 114Choose the Metrics . 11528. ELASTIC SEARCH – PIE CHARTS . 117Visualize . 117Choose the Metrics . 118Pie Chart Options . 12029. ELASTIC SEARCH – AREA AND BAR CHARTS . 121Area Chart . 121Choose the Metrics . 121Horizontal Bar Chart . 123Vertical Bar Chart . 12530. ELASTIC SEARCH – TIME SERIES . 128Choose Metrics . 129Result . 13031. ELASTIC SEARCH – TAG CLOUDS . 131Visualize . 131Choose the Metrics . 132Tag Cloud Options . 13432. ELASTIC SEARCH – HEAT MAPS . 135viii
Elastic SearchChoose the Metrics . 13633. ELASTIC SEARCH – CANVAS . 139Opening a Canvas . 139Cloning A Workpad . 140Modifying the Workpad . 14134. ELASTIC SEARCH – LOGS UI. 142Logstash Logs . 142ix
1. Elastic Search – Basic ConceptsElasticsearch is an Apache Lucene-based search server. It was developed by Shay Banonand published in 2010. It is now maintained by Elasticsearch BV. Its latest version is 7.0.0.Elasticsearch is a real-time distributed and open source full-text search and analyticsengine. It is accessible from RESTful web service interface and uses schema less JSON(JavaScript Object Notation) documents to store data. It is built on Java programminglanguage and hence Elasticsearch can run on different platforms. It enables users toexplore very large amount of data at very high speed.General FeaturesThe general features of Elasticsearch are as follows: Elasticsearch is scalable up to petabytes of structured and unstructured data. Elasticsearch can be used as a replacement of document stores like MongoDB andRavenDB. Elasticsearch uses denormalization to improve the search performance. Elasticsearch is one of the popular enterprise search engines, and is currently beingused by many big organizations like Wikipedia, The Guardian, StackOverflow,GitHub etc. Elasticsearch is an open source and available under the Apache license version 2.0.Key ConceptsThe key concepts of Elasticsearch are as follows:NodeIt refers to a single running instance of Elasticsearch. Single physical and virtual serveraccommodates multiple nodes depending upon the capabilities of their physical resourceslike RAM, storage and processing power.ClusterIt is a collection of one or more nodes. Cluster provides collective indexing and searchcapabilities across all the nodes for entire data.1
Elastic SearchIndexIt is a collection of different type of documents and their properties. Index also uses theconcept of shards to improve the performance. For example, a set of document containsdata of a social networking application.DocumentIt is a collection of fields in a specific manner defined in JSON format. Every documentbelongs to a type and resides inside an index. Every document is associated with a uniqueidentifier called the UID.ShardIndexes are horizontally subdivided into shards. This means each shard contains all theproperties of document but contains less number of JSON objects than index. Thehorizontal separation makes shard an independent node, which can be store in any node.Primary shard is the original horizontal part of an index and then these primary shards arereplicated into replica shards.ReplicasElasticsearch allows a user to create replicas of their indexes and shards. Replication notonly helps in increasing the availability of data in case of failure, but also improves theperformance of searching by carrying out a parallel search operation in these replicas.Advantages Elasticsearch is developed on Java, which makes it compatible on almost everyplatform. Elasticsearch is real time, in other words after one second the added document issearchable in this engine. Elasticsearch is distributed, which makes it easy to scale and integrate in any bigorganization. Creating full backups are easy by using the concept of gateway, which is presentin Elasticsearch. Handling multi-tenancy is very easy in Elasticsearch when compared to ApacheSolr. Elasticsearch uses JSON objects as responses, which makes it possible to invokethe Elasticsearch server with a large number of different programming languages. Elasticsearch supports almost every document type except those that do notsupport text rendering.2
Elastic SearchDisadvantages Elasticsearch does not have multi-language support in terms of handling requestand response data (only possible in JSON) unlike in Apache Solr, where it is possiblein CSV, XML and JSON formats. Occasionally, Elasticsearch has a problem of Split brain situations.Comparison between Elasticsearch and RDBMSIn Elasticsearch, index is similar to tables in RDBMS (Relation Database ManagementSystem). Every table is a collection of rows just as every index is a collection of documentsin Elasticsearch.The following table gives a direct comparison between these ndexTableFieldColumnDocumentRow3
2. Elastic Search – InstallationElastic SearchIn this chapter, we will understand the installation procedure of Elasticsearch in detail.To install Elasticsearch on your local computer, you will have to follow the steps givenbelow:Step 1: Check the version of java installed on your computer. It should be java 7 orhigher. You can check by doing the following:In Windows Operating System (OS) (using co
Elasticsearch is a real-time distributed and open source full-text search and analytics engine. It is used in Single Page Application (SPA) projects. Elasticsearch is an open source . The user of this e-book is p
