WIXLIB

Introduction to OpenStack SwiftCloudOpen Japan 2014Yuji Hagiwarahagiwarayuj@nttdata.co.jpPlatform Engineer, NTT DATA Corp.Copyright 2014 NTT DATA Corporation

Agenda1.What is Swift?2.Swift’s Latest Information3.Swift’s FutureCopyright 2014 NTT DATA Corporation2

Who am IYuji Hagiwara – Platform Engineer, NTT DATA Corp.Since 2011 Using OpenStackSince 2013 Developing Searching on SwiftDemo App for Searching on SwiftCopyright 2014 NTT DATA Corporation3

BackgroundData Explosion on Enterprise – Amount of Unstructured Data has been growing. We need storage with Scalability, Durability, Availability.Amount of Unstructured DataGrowingexponentially2004EB or PB scaleExamples of Unstructured Data Media (Images, Videos, Audios) Web Contents Documents Backups/Archives2007201020132016Where should we store these data?One of the Solutions is Swift.Copyright 2014 NTT DATA Corporation4

What is Swift?Swift is. A storage system with Scalability, Durability, Availability. The REST-ful Distributed Object Storage likely Amazon S3. One of OpenStack Core Components. Implemented by Python. A Open Source Software.① Block Storage (Cinder)② Object Storage (Swift)Copyright 2014 NTT DATA Corporation5

Usageso simple. curl -XPUT --data-binary ccount/container/object curl ntainer/object curl /container/objectCopyright 2014 NTT DATA Corporation6

Use cases of SwiftCopyright 2014 NTT DATA Corporation7

Swift as a storage for a variety of applicationsSystemBackupCMSFTP-like useDigitalDistributionWebApps CyberDuckREST APISwiftCopyright 2014 NTT DATA Corporation 8

OpenStack Swift deployments and use casesName of enterpriseProduct/ serviceDescriptionRackspace（USA）Cloud FilesCloud file share service by Rackspace itself. They use same code asOSS except for features such as authentication, Accounting and CDN（ 500PB）Korean Telecom （SourthKorea）ucloud storage serviceObject storage service using OpenStack/Swift（16PB size)Sina （Republic of China）Sina App Engine（SAE）Public storage service. They moved to OpenStack from anothertechnology MongoDB in 2012.San Diego SupercomputerCenter （USA）SDSC Cloud StorageServicesCloud storage service on SDSC. Users can select Amazon/S3 orRackspace Swift.SME Storage （USA）SMEStorageOpen Cloud PlatformCloud storage service based on Rackspace Cloud FileSoftLayer （USA）SoftLayer Object StoragePublic object storage service. Acquisition by IBMSwiftStack（USA）Swift StackProvide professional service and Operation and management productHP（USA）HP CloudPrivate cloud storage service uses OpenStack.Wikimedia（USA）Wikimedia storageMedia files store for Wikipedia.NII（JAPAN）Academic Cloud serviceAcademic cloud service by National Institute of Informatics in Japan（ＮＩＩ）(Integrated and supported by NTT Data）Copyright 2014 NTT DATA Corporation9

Inside SwiftCopyright 2014 NTT DATA Corporation10

Architecture: NodesSwift consist of 2-type Nodes: Proxy Node and Storage Node.Forward Data to nodeApplicationHTTP Load balancerProxy NodeProxy NodeProxy Node Store dataStorage NodeStorage NodeStorage NodeStorage Node Copyright 2014 NTT DATA Corporation11

Architecture: The RingThe Ring (static table for data allocation on storage node)decide the optimal Storage Node by Name.ApplicationHTTP Load balancerProxy NodeProxy NodeRingStorage NodeStorage NodeProxy Node RingRingStorage NodeStorage Node RingCopyright 2014 NTT DATA CorporationRingRingRing12

Architecture: The role of RingIf you requested to Store the data “A”, 3 Replica nodes store the data “A”.dataApplicationHTTP Load balancerProxy Node“A” must be locatedRingat “1”, “2”, “4”Storage Node 1RingStorage Node 3Data “A”RingCopyright 2014 NTT DATA CorporationProxy NodeRingStorage Node 2Data “A”Data “B”RingProxy NodeStorage Node 4Data “A”Data “B”Data “B”Ring Ring13

Architecture: The role of RingIf you requested to Get the data “A”, One of Nodes reply the data “A”.datadataApplicationHTTP Load balancerProxy NodeProxy NodeRingStorage Node 1RingRingStorage Node 3Data “A”RingCopyright 2014 NTT DATA CorporationProxy NodeRingStorage Node 2Data “A”Data “B”“A” must be locatedat “1”, “2”, “4”Storage Node 4Data “A”Data “B”Data “B”Ring Ring14

Scalability（１） Expand proxy server“Throughput”（２）Expand Storage serversor disks “volume”More xyStorageStorageStorageStorageStorage(Expand)More VolumeCopyright 2014 NTT DATA Corporation15

Many processes working pyright 2014 NTT DATA plicatorcontainerupdatercontainerauditor16

ReplicatorNode 1ReplicatorNode 2Node 3Node 4NormalDefeatRecoveryDisk(1) Each nodes checks data in othersNode 1Node 2Node 3Node 4(2) Disk defeat(3) Detect disk troubleNode 1Node 2Node 3Node 4(4) Copy data to another nodeNode 1Node 2Node 3Node 4(5) Recover disk(6) recover data to original nodeNode 1Node 2Node 3Node 4(7)Delete temporal dataCopyright 2014 NTT DATA Corporation17

ReplicatorNode 1ReplicatorNode 2Node 3Node 4NormalDefeatRecoveryDisk(1) Each nodes checks data in othersNode 1Node 2Node 3Node 4(2) Disk defeat(3) Detect disk troubleNode 1Node 2Node 3Node 4(4) Copy data to another nodeNode 1Node 2Node 3Node 4(5) Recover disk(6) recover data to original nodeNode 1Node 2Node 3Node 4(7)Delete temporal dataCopyright 2014 NTT DATA Corporation18

Normal stateEach Data has replicated.ApplicationHTTP Load balancerProxy NodeProxy NodeRingRingStorage NodeStorage NodeServerServerData “A”Data “B”RingStorage NodeStorage NodeServerData “A” ServerData “A”Data “B”Data catorReplicatorReplicatorRing Ring Copyright 2014 NTT DATA CorporationRing RingProxy Node19

ReplicatorNode 1ReplicatorNode 2Node 3Node 4NormalDefeatRecoveryDisk(1) Each nodes checks data in othersNode 1Node 2Node 3Node 4(2) Disk defeat(3) Detect disk troubleNode 1Node 2Node 3Node 4(4) Copy data to another nodeNode 1Node 2Node 3Node 4(5) Recover disk(6) recover data to original nodeNode 1Node 2Node 3Node 4(7)Delete temporal dataCopyright 2014 NTT DATA Corporation20

Defeat stateIf a disk is broken.ApplicationHTTP Load balancerProxy NodeProxy NodeRingRingStorage NodeStorage NodeServerServerData “A”Data “B”RingStorage Node Storage NodeServerServerData “A”Data “B”Data “A”BrokenData catorReplicatorReplicatorRing Ring Copyright 2014 NTT DATA CorporationRing RingProxy Node21

Defeat stateReplicator detects the lost data and replicates the data to another node fortemporary.ApplicationHTTP Load balancerProxy NodeProxy NodeRingRingStorage NodeStorage NodeServerServerRingStorage Node Storage NodeServerData “B”Data “A”Data “A”Data “B”ServerData “A”Data rReplicatorReplicatorReplicatorTemporary dataCopyright 2014 NTT DATA CorporationRing Ring Ring RingProxy NodeWhen detect a lost data,Replicate the data.22

ReplicatorNode 1ReplicatorNode 2Node 3Node 4NormalDefeatRecoveryDisk(1) Each nodes checks data in othersNode 1Node 2Node 3Node 4(2) Disk defeat(3) Detect disk troubleNode 1Node 2Node 3Node 4(4) Copy data to another nodeNode 1Node 2Node 3Node 4(5) Recover disk(6) recover data to original nodeNode 1Node 2Node 3Node 4(7)Delete temporal dataCopyright 2014 NTT DATA Corporation23

Recovery stateWhen the broken disk is replaced to a fresh disk.ApplicationHTTP Load balancerProxy NodeProxy NodeRingRingStorage NodeStorage NodeServerServerRingStorage Node Storage NodeServerServerData “A”Data “B”Data “B”Data “A”Data “A”Data catorReplicatorReplicatorRing Ring Copyright 2014 NTT DATA CorporationRing RingProxy Node24

Recovery stateReplicator replicates the data and removes the temporary data.ApplicationHTTP Load balancerProxy NodeProxy NodeRingRingStorage NodeStorage NodeServerServerData “A”Data “B”RingStorage NodeStorage NodeServerServerData “A”Data “B”Data “B”Data “A” plicatorReplicatorRemovedCopyright 2014 NTT DATA CorporationRing Ring Ring RingProxy NodeReplicate the data tothe correct node.25

ReplicatorNode 1ReplicatorNode 2Node 3Node 4NormalDefeatRecoveryDisk(1) Each nodes checks data in othersNode 1Node 2Node 3Node 4(2) Disk defeat(3) Detect disk troubleNode 1Node 2Node 3Node 4(4) Copy data to another nodeNode 1Node 2Node 3Node 4(5) Recover disk(6) recover data to original nodeNode 1Node 2Node 3Node 4(7)Delete temporal dataCopyright 2014 NTT DATA Corporation26

Latest InformationCopyright 2014 NTT DATA Corporation27

History and Trend of Community2010.6 Start OpenStack Project2010.10 1st release "Austin"2013.4 “Grizzly”2013.4 “Icehouse”2013.10 “Havana”2014.10 “Juno”NowHistory of OpenStackHot Topics on NowErasure CodingStorage PolicyGlobal ClusterFundamentalDevelopment Trend in SwiftTimeline in each functionsDevelopingSupportedCopyright 2014 NTT DATA Corporation28

Latest info: Erasure CodingReplicationErasure CodingDataDatadistributedistributeDataSize3x originalCopyright 2014 NTT DATA rity22x original29

Latest info: Storage PolicyBeforeAfterDataDataDataDataDataDataPartial PartialDataDataDataDataDataDataSame Policy on clusterCopyright 2014 NTT DATA CorporationDataDataDataParity1Parity2DataVariety Policy on cluster30

2 concepts:Integrated Searchable StorageIntelligent Resource ManagementFuture Direction of SwiftCopyright 2014 NTT DATA Corporation31

Integrated Searchable StorageSwift should be integrated with Searching.It means to need searching as Scalable, Durable, Available as Swift.UsersStoreOperatorsGetSwiftStorage SystemSearchManagersCopyright 2014 NTT DATA Corporation32

Use cases of Search1.Content Search2.Detection for de-duplicationDataHashAlready Stored?Data DataDataHash HashHash3.Tiered storageHot Content – More ModifiedCheap StorageCold Content – Less ModifiedModifed Date is older than 05/20/2014?Copyright 2014 NTT DATA Corporation33

Future: Integrated Searchable StorageHow do we ernalWhere do searchSwift with search library(such as Lucene)Search Engine(such as Solr)RedundancyHighDepend on Search engineAvailabilityHighDepend on Search engineScalabilityHighDepend on Search engineDifficulty of implementationHardEasyCopyright 2014 NTT DATA Corporation34

Our ImplementationInternal Approach Hack Swift to embed the search ary APINew SearchAPIProxy NodeQueryDataDistributed by the erADBContainerBDBContainer AIndexContainer BIndex Copyright 2014 NTT DATA Corporationobjectservercontainer-serverIndexing35

Future: Intelligent Resource ManagementErasure CodingSwift has more and more different functions.Other arbitrary tructorecauditorStorage PolicyMulti-ring accountreaperCopyright 2014 NTT DATA plicatorcontainerupdatercontainerauditor36

Future: Intelligent Resource Management Resources are drained! – IOPS, CPU, Network, Memory Performance Priorities of these functions aredifferent by the Requirement.Ex1) Store performance VS Search performance0Ex2) Service Level on Business HourOutsideVS on Outside Hour18(High-prio tocheck durability)6Business Hour(High-prio to processrequests)12More Intelligent Resource Management is necessary.with cgroupsCopyright 2014 NTT DATA Corporation37

Summary1.What is Swift? Swift is a Great OSS, for storing unstructured data.2.Swift’s Latest Information Erasure Coding Storage Policy3.Swift’s Future Integrated Searchable Storage Intelligent Resource managementCopyright 2014 NTT DATA Corporation38

PR: Demonstration is Now Available!We exhibit the Demo Application(Contents delivery system) built with Swift. On-demand Delivery a lot of contents(Pictures or movies) stored at Swift. Implemented Searching on Swift. (Our original implementation)(map for demo booth)Copyright 2014 NTT DATA Corporation39

Q&A: Do you have any question?Copyright 2011 NTT DATA CorporationCopyright 2014 NTT DATA CorporationThank you for your attention!Please contact tohagiwarayuj@nttdata.co.jp ,if you have any questions or comments.