Transcription
White PaperDell PowerScale OneFS: Release ContentsNew features, functionality, and enhancementsAbstractThis document summarizes the new features and functionality contained ineach Dell PowerScale OneFS release from OneFS 7.1 to the present.April 20221Dell PowerScale OneFS: Release Contents H18031.6
RevisionsDateCommentNovember 2019Updated for OneFS 8.2.1December 2019Updated for OneFS 8.2.2June 2020Updated for OneFS 9.0.0October 2020Updated for OneFS 9.1.0April 2021Updated for OneFS 9.2October 2021Updated for OneFS 9.3April 2022Updated for OneFS 9.4AcknowledgmentsThis paper was produced by the following:Author: Nick TrimbeeThe information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in thispublication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.Use, copying, and distribution of any software described in this publication requires an applicable software license.This document may contain certain words that are not consistent with Dell's current language guidelines. Dell plans to update the document oversubsequent future releases to revise these words accordingly.This document may contain language from third party content that is not under Dell's control and is not consistent with Dell's current guidelines for Dell'sown content. When such third party content is updated by the relevant third parties, this document will be revised accordingly.Copyright 2019-2022 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarksof Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. [3/31/2022] [White Paper] [H18031.6]2Dell PowerScale OneFS: Release Contents H18031.6
Table of contentsIntroduction . 4OneFS 7.1. 4OneFS 7.1.1. 4OneFS 7.2. 5OneFS 7.2.1. 5OneFS 8.0. 6OneFS 8.0.1. 7OneFS 8.1. 8OneFS 8.1.1. 8OneFS 8.1.2. 9OneFS 8.1.3. 9OneFS 8.2. 9OneFS 8.2.1. 11OneFS 8.2.2. 11OneFS 9.0.0. 12OneFS 9.1.0. 12OneFS 9.2.0. 13OneFS 9.2.1. 13OneFS 9.3. 14OneFS 9.4. 15Summary. 163Dell PowerScale OneFS: Release Contents H18031.6
IntroductionThis document lists and briefly describes the new features and functionality delivered in each DellPowerScale OneFS release since OneFS version 7.1. Besides the release number and date, the Dell internalcode name for each release is also provided.OneFS 7.1ReleaseFeatureInfoOneFS 7.1AuditSMB file access and configuration auditing, recording who did what on files andsending out to the Dell Common Event Enabler (CEE). CEE 6.0.3 providessupport and integration with Varonis DatAdvantage for Windows.Cluster management access control system that divides up the powers of the“root” user and allows assignment of specific roles to other non-privileged users.RBAC has a set of integrated roles and the ability to create user-defined roles.The three integrated roles are Audit Administrator, System Administrator, andSecurity Administrator.Post-process deduplication, reducing the physical data footprint by locating andsharing common elements across files using ‘shadow store’ containers. Simultaneous Jobs: Allowing Multiple Jobs to run. Impact Control: Adding new Impact control rules for CPU and DISK usage. Monitoring: Improved monitoring and stats. Job Engine Exclusion Sets.Code name:WaikikiRoles BasedAdministrationRelease date:October 2013SmartDedupeJob Engine v2OneFS 7.1.1ReleaseFeatureInfoOneFS 7.1.1L3 CacheCode name: JawsRoles BasedAdministrationSMB3 MultichannelUsing a nodes SSD as a large, cost-effective, persistent read cache for evictedL2 data and metadata blocks.WebUI interface and additional roles.Release date: July20144Allows SMB3 storage devices to automatically discover and use multiple networkconnections simultaneously, increasing throughput and fault tolerance.MMCIntegrationAllows a cluster’s SMB shares to be managed using the MMC tool by providingsupport for the “Shared Folders” snap-in to MMC.S210 andX410S210 and X410 Gen5 platform node support.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 7.2ReleaseFeatureInfoOneFS 7.2UserspaceNFS NFS AccessZones Code name: MobyRelease date:November 2014ProtocolAuditingSource BasedRoutingNFS v3 and NFS v4 servers ported to userspace.Rewritten and multi-instanced OneFS File System Driver.Added NFS feature: Zone support for exports.Added NFS feature: Aliases for exported paths.Added NFS feature: Audit.Improved NFS statistics reporting and management.Improved NFS WebUI, added support for new features.Improved NFS CLI, added support for new features.Ability to authenticate against multiple authentication providers, includingmultiple instances of: AD, LDAP, NIS, Local User/Group Database. Each access zone will be associated with: One or more SmartConnect Zones that will be allowed access to thiszone. One set of shares that will be accessible through this Access zone. One or more Authentication providers to authenticate users for thisAccess zone. File access and configuration auditing for the NFS and SMB protocols. Recording all file access events (who did what on files) and sending out tothe Dell Common Event Enabler (CEE). CEE 6.3.1.0 provides support and integration with Varonis DatAdvantage forWindows.Routes traffic based on its source IP address.HD400Isilon HD400 Gen5 platform node support.FSAInsightIQ 3.1 support.OneFS 7.2.15ReleaseFeatureInfoOneFS 7.2.1IPv6Support for both IPv6-only and dual-stack configurations.Code name: OrcaX210X210 Gen5 platform node support.Release date:June 2015NL410NL410 Gen5 platform node support.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 8.0ReleaseFeatureInfoOneFS 8.0SMB3ContinuousAvailabilityNFS andSMB FileBlockingNFSv4FailoverSupport for SMB3 continuous availability and witness.Code name:RiptideRelease date:December 2015DNS PerAccess ZoneOverlappingAccess ZonesHDFS AuditCloudPools1.0SmartDedupeJob Control6Control what type of files can be written by NFS or SMB to a cluster, based oninclude of exclude filter lists. NFSv4 clients can now use dynamic IPs which will give them more serviceavailability and non-disruption during IP migration events or NFS serviceinterruption. When a client’s virtual IP address moves, or a OneFS group change eventoccurs, the client application will continue without disruption. As such, nounexpected I/O error will propagate back up to the client application. Ability to specify separate primary and secondary DNS servers for eachaccess zone. DNS servers are configured by a new ‘Groupnet’ network object which livesabove a subnet.Ability to create Access Zones with overlapping datasets.Support for file access and configuration auditing for the HDFS protocol byrecording all file access events (who did what on files) and sending out to theDell Common Event Enabler (CEE). Enables the movement and tiering of data by NFS or SMB from OneFSclusters to Amazon S3, Microsoft Azure, or OneFS to OneFS cloud storage. Archiving files to Cloud Storage. Recalling files from the Cloud Storage. NFS and SMB I/Os to the cloud data (Note: I/O utilizing any otherprotocols, local, HTTP, or FTP is not supported and will return anEIO error). Encryption for data written to the cloud. Compression for data written to the cloud. Garbage collection for orphaned cloud objects whose retentionperiod has expired. Feature interoperability with NDMP, SyncIQ, SnapshotIQ, WORM,and Quota.Allow reconfiguration of priority and impact level for SmartDedupe job defaults.CELOG 2.0Redesign of OneFS cluster event logging and alerting infrastructure.NDU andRollback SD EdgeSoftware-defined virtual OneFS nodes on VMware vSphere.Support for rolling (one node at a time) upgrades.Ability to roll back to earlier OneFS version.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 8.0.1ReleaseFeatureInfoOneFS 8.0.1SyncIQComplianceMode FailoverHDFSSyncIQ failover and failback with compliance mode SmartLock, deliveringautomated disaster recovery for financial services SEC-17a4 regulatorycompliance.Code name:Halfpipe Release date:September 2016 CloudPoolsProxy SupportMac OSXSupport AuditPerformance AIMAPerformance Small FileEfficiency forHealthcarePACS UI Scalability 7Single management point for Ambari operator to manage and monitor aHadoop cluster with OneFS as the storage layer.Ranger authorization integration to ensure proper user access to files isgoverned by Ranger policies and OneFS file system access control.Kerberos encryption to secure and encrypt data from HDFS client to or fromOneFS, preventing rogue users from sniffing HDFS traffic on network.Data-node load balancing, avoiding overloading nodes and increasingcluster resilience.Ambari metrics and alerting.Multiple nodes can now simultaneously tier to the cloud.No direct external network exposure of OneFS systems for CloudPools.No network workarounds necessary to configure CloudPools.Enables macOS specific features like color tagging of files and extendedmetadata.Mac users can leverage powerful Mac features to sort, tag files for efficientfile categorization.Improved directory listing speed from Mac clients, in particular for largedirectories.Audit at scale – up to 50 million audit events.File access and configuration auditing records all file access events (who didwhat on files) and sends them out to the Dell Common Event Enabler (CEE).CEE 6.5 provides support and integration with Varonis DatAdvantage forWindows, specifically allowing: Usable Access Auditing; Recommendations,Analytics and Modeling; and Data Owner Identification and Involvement.Ensuring that proper user access to OneFS is maintained with serverconsolidation (for example: forest or domain collapse) by supported SIDHistory.Negative caching helps reduce the load placed on authentication providersby caching the information that a user or group does not exist in anyauthentication provider.A policy mechanism to control containerization, so that it may be optional,allowing customers explicitly to opt for better small-file storage efficiency inexchange for some small-file performance.A class of specialized shadow stores to act as containers. These shadowstores differ from existing shadow stores in a few ways in order to isolatefragmentation, to support tiering, and to support future optimizations whichwill be specific to single-reference stores.A method to pack, unpack, and repack data into, out of, and betweencontainers.Improved performance and usability for the SMB share and NFS exportpages.WebUI now can handle long lists, and the backing stores are proven up to40,000 records. The WebUI now returns ten items per page.Dell PowerScale OneFS: Release Contents H18031.6
Performance statistics for Job Engine and data services.ReleaseFeatureInfoOneFS 8.1Gen6HardwareSupport for new Isilon F800, H600, H500, H400, A200, and A2000 Gen6platform hardware.Code name:Freight TrainsE LicensingSupport for E Licensing.Release date:May 2017SD EdgeSoftware-defined OneFS on single VMware vSphere host.OneFS 8.1OneFS 8.1.1ReleaseFeatureInfoOneFS 8.1.1SMB3EncryptionAllows OneFS SMB server to be configured for SMB3 encryption on a per share,zone, or cluster-wide loudPoolsfor GCPSetting cluster encoding and export encoding to the same character set will yieldRFC compliant filenames, enabling exports to NFS clients using non-westerncharacters.Code name:NiijimaRelease date:December 20178Support for Cloudera Navigator – lineage and metadata managementintegration.Enables users to use NFS or SMB to move data from OneFS clusters to GoogleCloud Storage and tier it.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 8.1.2ReleaseFeatureInfoOneFS 8.1.2HDFS Code name:KanagawaHadoop 3 SupportOneFS Ambari Management PackWebHDFS supportLLAP supportRanger with SSL supportRelease date:August 2018OneFS 8.1.3ReleaseFeatureInfoOneFS 8.1.3In-lineCompressionSupport for in-line compression on F810 with FPGA offload.F810F810 Gen6 hardware platform support.ReleaseFeatureInfoOneFS 8.2Cluster ScaleSMBPerformanceDatasetMonitoringSmartQuotas252 node cluster support.Performance statistics for SMB.Code name:SeismicRelease date:January 2019OneFS 8.2Code name:PipelineRelease date:May 2019SmartPoolsFilePolicy JobHDFSEncryptionSupport for Hadoop data encryption (TDE).SyncIQ SmartConnect9 Support for 500 K quotas Percentage-based quotas Default directory quotas Client-side reporting improvements Application logical quotasEfficient changelist-based SmartPools file pool policy job.SyncIQ encryption for secure replication over the WAN and other untrustednetworks. SyncIQ bandwidth throttling.SmartConnect and Flexnet update.Dell PowerScale OneFS: Release Contents H18031.6
ReleaseFeatureInfoSnapshotIQDomainsRe-architecting of SnapshotIQ framework to avoid costly snapshot painting anddrive scalability and other efficiencies.Small FileEfficiencyDefragmenterIntroduction of small file efficiency job for defragmenting shadow stores andopening up SFSE to low-write archive workloads beyond healthcare PACS.SmartLockIntroduction of ComplianceStoreDelete job.AuditSupport for granular audit event selection, avoiding the collection of unneededaudit events that third-party audit applications do not register for.Multi-tenant, access zone aware, role-based access control.Zone-awareRBACCloudPools2.0 NDMP 2-wayFC Adapter SSH Multifactor AuthH5600 Support for simultaneous firmware upgrade Ability to pause and resume OneFS upgrade Patch installation during OneFS upgrade Removal of dependency on Flash from WebUI. Improved internal network config pages.Isilon H5600 Gen6 deep chassis introduction.Leaf/spineback end Upgrade orNDUWebUI10 Support for AWS signature authentication version 4.Deliver network statistics per CloudPools account or file pool policy.Support for Alibaba Cloud and Amazon C2S public cloud providers.Full integration of CloudPools and data services like Snapshot, Sparse filehandling, Quota, AVScan, and WORM.2-way NDMP support for Gen6 platforms with combo Fibre Channel/10GbEinterface card.Support of NDMP Redirector and Throttler.Support of ComboCopy for CloudPools.Multi-factor authentication support through integration with Duo service,using phone call-back and SMS passcode.Dell Z9100 switches supporting leaf/spine architecture.144 nodes max cluster with leaf/spine back end.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 8.2.1ReleaseFeatureInfoOneFS 8.2.1In-lineDeduplication Support for in-line deduplication on the F810 platform.Data Reduction reporting enhancements.ServiceabilityEnhancements Instant secure erase.Improved Healthchecks.Patch Framework improvements.PlatformAPIEnhancements for CommVault and DataIQ.Code name: AcelaRelease date:September 2019OneFS 8.2.2ReleaseFeatureInfoOneFS 8.2.2H5600 In-lineDataReductionSupport for in-line compression and deduplication on the deep-hybrid H5600platform.Large filesupportSupport for 16 TB files.25 GbEsupportSupport for front-end 25 Gb Ethernet formance statistics for NFS protocol.Code name:BeachcomberRelease date:January 202011Parallel upgrades.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 9.0.0ReleaseFeatureInfoOneFS 9.0.0F600 & F200OneFS support for the new PowerScale F600-NVMe and F200 platform.Code name:CascadesIn-line DataReductionSupport for inline compress and deduplication on the all-flash PowerScaleF600-NVMe and F200 platforms.S3 ProtocolSupport for the S3 protocol, expanding the data lake to native file and object.IPMIIPMI support for remote console and power control of nodes.HDFSHDFS Ranger NANON SupportRelease date:June 2020OneFS 9.1.0ReleaseFeatureInfoOneFS 9.1.0CAVAAntivirusSupportFaster ClientFailoversCAVA Antivirus support for SMB clients using Common Event Enabler (CEE)plus integration with most AV software vendors to scan files, providing betterperformance than current ICAP.CloudPoolsCloud ObjectCacheAudit LogPurgingCloudPools traffic optimization, reducing the number of network round-tripswhen recalling data from a CloudPools target.CustomizableCELOG AlertsConfigurable event thresholds allow users to adjust the value at which eventsare triggered for various cluster statistics.In-productActivationActivate product licenses with ELMS over Secure Remote Services.NDMP Multistream BRERestartable backup support for multi-stream NDMP.Code name:DeccanRelease date:October 202012Improved group change time for planned shutdowns. Shrinks the window fordata unavailability due to group changes to less than 5 seconds.Ability to automatically and periodically purge audit logs.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 9.2.0ReleaseFeatureInfoOneFS 9.2.0PowerScale F900OneFS support for the new PowerScale F900-NVMe platform.Code name:EmpireIn-line Data ReductionSupport for in-line compress and deduplication on the all-flashPowerScale F900-NVMe platform.NFSoRDMAEnterprise KeyManagement for SEDsNFS over RDMA support for NFS v3 for PowerScale and Isilon nodeswhich contain supported Ethernet interface cards.Support for external KMIP compliant key managers such as SKLM,SafeNet, and Vormetric for storing encryption keys on SED clusters.Full IPv6 SupportFull IPv6 support that meets the requirements for USGv6 compliance.CELOG WebUIAddition of a web-based user interface for CELOG allowing events to bedisplayed, searched, categorized, filtered, ignored, and resolved.IB Backend forPowerScaleSupport for back-end InfiniBand network interfaces on PowerScalenodes, for compatibility with legacy IB clusters.Cluster ConfigurationExport or ImportAbility to export cluster configuration, which can be used to restore aconfiguration to the original cluster or a different cluster.Drain Based UpgradeNodes are prevented from rebooting or restarting protocol services untilall SMB clients have disconnected from the node.Release date:April 2021OneFS 9.2.1ReleaseFeatureInfoOneFS 9.2.1PowerScale PortfolioSupportOneFS support for the new PowerScale H700, H7000, A300, and A3000platforms.Code name:Flying ScotsmanRelease date:May 202113Dell PowerScale OneFS: Release Contents H18031.6
OneFS 9.3ReleaseFeatureInfoOneFS 9.3PowerScale PortfolioSupportOneFS support for the new PowerScale P100 and B100 performanceand backup accelerator nodes.Writable SnapshotsEnables the creation and management of a space and time efficient,modifiable copy of a regular OneFS snapshot.Connectivity support for versions 4.1 and 4.2 of the NFS file accessprotocol.Code name:GothamRelease date:October 2021NFS v4.1 and v4.2Support14Long filename supportProvision for file names up to 1024 bytes, allowing support for longnames in UTF-8 multi-byte character sets.Inline data in inodesData efficiency feature allowing a small file’s data to be stored in unusedspace within its inode block.HDFS ACLsProvide support for HDFS-4685 access control lists, allowing users tomanage permissions on their datasets from Hadoop clients.Job engine exclusionsAllow Job engine jobs to be run on a defined subset of a cluster’s nodes.CloudPools RecallImproved CloudPools file recall and rehydrate performance.Safe SMB clientdisconnectsAllows SMB clients the opportunity to flush their caches before beingdisconnected.S3 protocolenhancementsAdded support for S3 chunked upload, multi-object delete, and nonslash delimiters for lists.Dell PowerScale OneFS: Release Contents H18031.6
OneFS 9.4ReleaseFeatureInfoOneFS 9.4SmartSync Data MoverIntroduction of a new OneFS SmartSync data mover, allowing flexible datamovement and copying, incremental resyncs, push and pull data transfer,and one-time file to object copy.IB to Ethernet MigrationNon-disruptive rolling InfiniBand to Ethernet back-end network migration forlegacy Gen6 clusters.Secure boot support is extended to include the F900, F600, F200,H700/7000, and A700/7000 platforms.Code name:HexieReleasedate: April2022Secure Boot15Smarter SmartConnectDiagnosticsIdentifies non-resolvable nodes and provides their detailed status, allowingthe root cause to be easily pinpointed.QLC Drive SupportSupport is added in OneFS 9.4 to accommodate quad-level cell (QLC) NVMedrives in the F900 and F600 platform nodes.In-line DedupeIn-line deduplication will be enabled by default on new OneFS 9.4 clusters.Clusters upgraded to OneFS 9.4 will maintain their current deduplicationconfiguration.Healthcheck AutoupdatesAutomatic monitoring, download, and installation of new healthcheckpackages as they are released.CloudIQ ProtocolStatisticsNew protocol statistics ‘count’ keys are added, allowing performance to bemeasured over a specified time window and providing point-in-time protocolinformation.Secure Remote ServicesAlerts and CELOGEvent LimitingCloudPools StatisticsPrevents CELOG from sending 151 event types (out of 433 total) to DellSecure Remote Services and restricts CELOG alerts from customer-createdchannels.Automated statistics gathering on CloudPools accounts and policiesproviding insights for planning and troubleshooting CloudPools-relatedactivities.Dell PowerScale OneFS: Release Contents H18031.6
SummaryThis document summarizes the new features and functionality contained in each OneFS release, from OneFS 7.1 inOctober 2013 to the present time. OneFS releases are assigned a release number (for example, 8.2.1) based on thefollowing criteria:Release digitRelease typeDescriptionFirstMajorA release with functionality that provides a substantial move forward in the productSecondMajorRelease with functionality in the same family and theme as the parent major release.ThirdMinorNew features and functionality added between major releases.Major and minor releases also receive internal code names, which are agreed upon by engineering and productmarketing. For example, Acela is the code name for OneFS 8.2.1.Technical support and resourcesDell.com/support is focused on meeting customer needs with proven services and support.The Dell Technologies Info Hub provides expertise that helps to ensure customer success on Dell storage platforms.16Dell PowerScale OneFS: Release Contents H18031.6
Data-node load balancing, avoiding overloading nodes and increasing cluster resilience. Ambari metrics and alerting. CloudPools Proxy Support Multiple nodes can now simultaneously tier to the cloud. No direct external network exposure of OneFS systems for CloudPools. No network workarounds necessary to configure CloudPools.
