PowerScale OneFS NFS Design Considerations - Dell USA

Transcription

Technical White PaperDell EMC PowerScale: OneFS NFS DesignConsiderations and Best PracticesAbstractThis document shows how to implement the Network File System (NFS) serviceon Dell EMC PowerScale OneFS and provides key considerations and bestpractices when using PowerScale to provide NFS storage service. This papercovers OneFS 8.0.x and later.December 2021H17240.3

RevisionsRevisionsDateDescriptionMay 2018Initial releaseJune 2020PowerScale rebrandingMay 2021NFSv3 over RDMA new feature in OneFS 9.2.0October 2021NFSv4.1 support in OneFS 9.3.0AcknowledgmentsAuthor: Lieven LinThe information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in thispublication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.Use, copying, and distribution of any software described in this publication requires an applicable software license.Copyright 2018--2021 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarksof Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. [12/3/2021] [Technical White Paper] [H17240.3]2Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

Table of contentsTable of contentsRevisions.2Acknowledgments .2Table of contents .3Executive summary.5Audience .51NFS protocol and OneFS .61.1OneFS overview .61.2NFS protocol introduction .61.2.1 NFSv3 .71.2.2 NFSv4.x .71.2.3 Advantages of NFSv4.x .81.3NFS compatibility with OneFS .81.3.1 NFSv4.1 support .9234563Implementing NFS .102.1Identity management and authentication .102.2Create NFS export .112.3Mount export over NFSv3/NFSv4.0/NFSv4.1/NFSv4.2 .12PowerScale OneFS considerations .133.1NFS export considerations .133.2SmartConnect .133.3Access zone .153.4AIMA (Authentication, Identity Management, Access) .163.5OneFS protocol audit .18NFS client considerations.204.1Linux client .204.2macOS client .23NFS security considerations.245.1Network security considerations .245.2Authentication .255.3NFSv4.x ACL .265.4NFSv4.x pseudo-file system .30NFSv3 over RDMA .326.1NFSv3 over RDMA overview .326.2Management options .33Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

Table of contents6.2.1 Enable/disable NFS over RDMA globally .336.2.2 Filter RoCEv2 capable network interfaces for an IP pool .346.2.3 Check ROCEv2 capability of network interfaces .356.37AUseful NFS commands and tools .377.1isi statistics command .377.2Packet capture tool and analysis .39Technical support and resources .41A.14Key considerations .36Related resources .41Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

Executive summaryExecutive summaryThis document provides common configurations and considerations to help you implement, configure, andmanage NFS storage service on Dell EMC PowerScale products. This content includes the following: NFS protocol introduction and its compatibility with OneFSQuick start implementation guide to use NFS service on OneFSNFS considerations on OneFSNFS considerations on clientNFS security considerationsAudienceThis document is intended for administrators who are using NFS storage service on PowerScale OneFS. Thedocument assumes you have knowledge of the following: Network Attached Storage (NAS) systemsNetwork File System (NFS) protocolPowerScale OneFS distributed file system and scale-out architectureDirectory service such as Active Directory and LDAPYou should also be familiar with Dell EMC PowerScale documentation resources, including: 5Dell EMC PowerScale OneFS: A Technical OverviewPowerScale OneFS Web Administration GuidePowerScale OneFS CLI Administration GuideCurrent PowerScale Software ReleasesOneFS Security Configuration GuideDell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

NFS protocol and OneFS1NFS protocol and OneFS1.1OneFS overviewOneFS is a fundamental component for Dell EMC PowerScale storage. It is used to power all Dell EMCPowerScale NAS storage solutions and offers the following key advantages: Scale-out architecture: OneFS is designed to scale in terms of machine, by adding more PowerScalenodes to a cluster, both performance and capacity is scaled. It enables PowerScale scale to a multipetabyte large cluster which contains a maximum of 252 nodes.Single file system and namespace: Traditional storage model contains file system, volume manager,and data protection. OneFS combines all of them into a single intelligent distributed file system, andprovides a single namespace that is accessible through multi-protocol (NFS, SMB, HDFS, S3, HTTP,and FTP) simultaneously.Efficiency and ease of management: OneFS provides unique features to improve PowerScale NASstorage system efficiency and ease of management. For example, with OneFS SmartPools, you cantier your cold data to lower-cost PowerScale nodes automatically.OneFS is not only the operating system but also the underlying file system that drives and stores data in thePowerScale scale-out NAS cluster. For more details, see the document Dell EMC PowerScale OneFS: ATechnical Overview.1.2NFS protocol introductionThe Network File System (NFS) protocol allows users to mount remote file systems transparently and accessto shared files across networks. It uses a client/server model based on Remote Procedure Call Protocol(RFC5531), so NFS is portable across different machines, operating systems, network architecture, andtransport protocols. NFS eliminates the need to keep copies of files on several machines by letting the clientsall share a single copy of a file on the server.There are three major NFS versions. Each of them is defined in an RFC specification as shown in the Table1. This chapter will provide a brief summary about NFSv3 and NFSv4.x as they are implemented in mostenterprise environments. For more details, you can refer to the links in the table.NFS versions and evolutionVersionRFCStatusNFSv2RFC1094 (published on 1989)ObsoleteNFSv3RFC1813 (published on 1995)Most popularNFSv4.0RFC3010 (published on 2000, obsolete)Slowly replacing NFSv3RFC3530 (published on 2003, obsolete)RFC7530 (published on 2015)6NFSv4.1RFC5661 (published on 2010)NFSv4.2RFC7862 (published on 2016)Adopted gradualllyDell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

NFS protocol and OneFS1.2.1NFSv3The NFSv3 is a stateless protocol. Statelessness means that the server does not need to maintain stateabout any of its clients in order to function correctly. The NFSv3 has following key enhancements comparedwith NFSv2: The file handle has been increased to a variable length up to 64 bytes, instead of 32 bytes.Supports files larger than 2 GB. Maximum file size depends on the NFS server’s local filesystems.Eliminates the 8 KB limit of the maximum size of an on-the-wire NFS read or write operation.Introduces the concept of Weak Cache Consistency. A server reply to a read or write operationreturns extra attribute information which can be used by clients to decide whether its data andattribute caches are stale. So NFSv3 clients will detect changes to files faster that are modified byother client.Introduces safe asynchronous writes to improve performance. New COMMIT procedure is addedto flush data to stable storage for an asynchronous write and to detect if retransmit the data isneeded.To function correctly, NFSv3 also relies on several auxiliary protocols.Mount protocolThe mount protocol is separate from, but related to, the NFS protocol. It provides operating system-specificservices to get NFS off the ground - looking up export pathnames, validating user identity, and checkingaccess permissions. Clients use the mount protocol to get the first file handle, which allows them entry into aremote file system.Network Lock Manager (NLM) protocol and Network Status Monitor (NSM) protocolBecause NFS is a stateless service, auxiliary protocols are needed to offer inherently stateful services suchas file locking and access control synchronization. So the RPC-based NLM protocol and NSM protocol areprovided to work together to provide file locking and access control capability.Binding protocol (RFC1833)As NFS protocol and its auxiliary protocols mentioned above are all based on RPC, it is necessary to mapRPC program number/version number pairs to the network port number (TCP/UDP port number) for thatversion of that program. Binding protocol provides such a lookup service to find a network port number for aspecific RPC program number/version. There are three versions of a lookup service: RPCBIND (Versions 3and 4) which uses a transport-independent format for the transport address, and Port Mapper (Version 2)which is an older protocol only specific for TCP and UDP transport.1.2.2NFSv4.xThe biggest change in NFSv4.x is that it is designed as a stateful protocol. Unlike earlier NFS versions whichneeds auxiliary protocols to provide additional functions, the NFSv4.x integrates the file locking (NLM/NSM)and the mount protocol. Besides, the NFSv4.x also provides some of the key features as follows: 7Introduces the COMPOUND procedure as a wrapper to coalesce one or more operations into asingle RPC request.Introduces the NFSv4.x Access Control Lists (ACL) to support more expressive and granularaccess control while clients accessing the NFS shared files.Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

NFS protocol and OneFS 1.2.3The server can grant a read or write file delegation to the clients, which enables the clients toaggressively cache file data.Session model is introduced since NFSv4.1 for better connection management.Advantages of NFSv4.xThe NFSv4.x retains the essential characteristics of previous version, such as independent of operatingsystems, simplicity, and good performance. It also has more advantages compared with older versions.COMPOUND procedureThe compound procedure will combine multiple individual operations into a single request to reduce thenumber of RPC packets transmitted over the network. The client can avoid the cumulative latency of multipleRPCs. So the NFSv4.x will perform better in a potentially high latency network like Internet.Firewall friendlyTo access an NFSv3 server, the client needs to involve NFS protocol and its auxiliary protocols (port mapper,mount, NLM/NSM), each of them needs TCP/UDP ports which would not be the well-known ports listening onthe network. This will cause problem for using NFS through firewall. In the NFSv4.x, there are no auxiliaryprotocols and it is designed as a single protocol using a single TCP port, usually listening on port 2049. So ittraverses firewalls and network address translation (NAT) devices easily, and makes the networkmanagement and configuration easily. More details and considerations are discussed in 5.1 Network securityconsiderations.Stronger securityThe NFSv4.x ACL file attribute is added to enable more expressive access control. The NFSv4.x ACL modelis quite rich. With the use of NFSv4.x ACL, the server does all access control based on the server’sinterpretation of the ACL although the client can manipulate the ACL attributes. More details andconsiderations are discussed in 5.3 NFSv4.x ACL.1.3NFS compatibility with OneFSThis document focuses on OneFS 8.0.0 and above. However, to provide an overview of the NFS compatibilitywith OneFS families, a compatibility table is shown in Table 2.NFS compatibility with OneFSOneFS Sv3 NFSv4.0 NFSv4.1 Note: NFSv4.2 just adds additional new features based on NFSv4.1, therefore, OneFS allows clients tomount NFS export with NFSv4.2, but OneFS does not implement NFSv4.2 features. ** indicates that theversion of OneFS is EOSL. For information about the support and service life-cycle dates for PowerScalehardware and software products, see the PowerScale Product Availability Guide.8Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

NFS protocol and OneFS1.3.1NFSv4.1 supportNFSv4.1 introduced several new features to the NFSv4 protocol standard, as covered in RFC-5661. Thesedifferences are covered in Section 1.8 of the RFC.Some features are listed as required, which means that the feature must be implemented in or supported bythe NFS server to claim RFC standard. Other features are listed as recommended or optional features andare supported ad hoc by the NFS server but are not required to claim RFC compliance. Starting from OneFS9.3.0, OneFS supports NFSv4.1 and NFSv4.2 to access data by implementing all the required featuresdefined in RFC-5661. This support excludes the Secret State Verifier (SSV) which is not implemented by anyopen-source Linux clients. OneFS 9.3.0 implements the session model which allows NFSv4.1 and NFSv4.2clients to leverage trunking. Trunking is the use of multiple connections between a client and server in orderto widen the I/O path. OneFS 9.3.0 supports both session trunking and client ID trunking. Session trunking is the association of multiple TCP connections to the same session. Most Linuxsupports session trunking by using nconnect option which is included in Linux kernel version higherthan 5.3.Client ID trunking is the association of multiple sessions to the same client ID. Not observed anyopen-source Linux clients support client ID trunking.Figure 1 shows the supported NFS operations in OneFS 9.3.0. Both NFSv4.1 and NFSv4.2 use the existingNFSv4.0 I/O stack in OneFS. NFSv4.2 is a superset of NFSv4.1, with all new features being optional. OneFSstill allows clients to mount NFS export of OneFS with NFSv4.2 and access OneFS data even OneFS doesnot implement any NFSv4.2 features.NFSv4.2 SpecNFSv4.1 SpecNFSv4.0 SpecOperation 20: OPEN CONFIRMOperation 30: RENEW*Operation 35: SETCLIENTID*Operation 36:SETCLIENTID CONFIRMOperation 39: RELEASE LOCKOWNEROperation 3: ACCESSOperation 4: CLOSEOperation 5: COMMITOperation 6: CREATE*Operation 7: DELEGPURGE*Operation 8: DELEGRETURNOperation 9: GETATTROperation 10: GETFHOperation 11: LINKOperation 12: LOCKOperation 13: LOCKTOperation 14: LOCKUOperation 15: LOOKUPOperation 16: LOOKUPPOperation 17: NVERIFYOperation 18: OPENOperation 19: OPENATTROperation 21: OPEN DOWNGRADEOperation 22: PUTFHOperation 23: PUTPUBFHOperation 24: PUTROOTFHOperation 25: READOperation 26: READDIROperation 27: READLINKOperation 28: REMOVEOperation 29: RENAMEOperation 31: RESTOREFHOperation 32: SAVEFHOperation 33: SECINFOOperation 34: SETATTROperation 37: VERIFYOperation 38: WRITEOperation 40: BACKCHANNEL CTLOperation 41: BIND CONN TO SESSIONOperation 42: EXCHANGE IDOperation 43: CREATE SESSIONOperation 44: DESTROY SESSIONOperation 45: FREE STATEID*Operation 46: GET DIR DELEGATION*Operation 47: GETDEVICEINFO*Operation 48: GETDEVICELIST*Operation 49: LAYOUTCOMMIT*Operation 50: LAYOUTGET*Operation 51: LAYOUTRETURNOperation 52: SECINFO NO NAMEOperation 53: SEQUENCEOperation 54: SET SSVOperation 55: TEST STATEID*Operation 56: WANT DELEGATIONOperation 57: DESTROY CLIENTIDOperation 58: RECLAIM COMPLETESupported NFS operations in OneFS 9.3.09Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3*Operation 59: ALLOCATE*Operation 60: COPY*Operation 61: COPY NOTIFY*Operation 62: DEALLOCATE*Operation 63: IO ADVISE*Operation 64: LAYOUTERROR*Operation 65: LAYOUTSTATS*Operation 66: OFFLOAD CANCEL*Operation 67: OFFLOAD STATUS*Operation 68: READ PLUS*Operation 69: SEEK*Operation 70: WRITE SAME*Operation 71: CLONE*Represent no support inOneFS 9.3.0

Implementing NFS2Implementing NFSThe Dell EMC PowerScale OneFS operating system can enable seamless multiprotocol data access withunified security model. NFS is one of the protocols that gives UNIX and Linux system access to OneFS. Thischapter will provide a quick start guide to implement NFS as your data access protocol to OneFS in yourenvironment: identity management and authentication, create NFS export and mount export to clients.2.1Identity management and authenticationIt is important to understand the identity management and authentication methods before implementing NFSto your environment. Identity management will provide a location to store user information, tell where a useris. Authentication is a process that validates the identity of a user. In OneFS, the identity management andauthentication is offered by authentication providers. OneFS supports the following methods for authenticatinguser: Active Directory (AD)Lightweight Directory Access Protocol (LDAP)Network Information Service (NIS)Local users and groupsIt is recommended to use Active Directory service and LDAP for ease of user identity and accessmanagement.Active Directory (AD)Active Directory is implemented by Microsoft that provides several services: LDAP, Kerberos, and DNS. Theprimary reason for a PowerScale cluster to join an AD domain is to provide user/group identity managementand authentication. Active Directory service is used to authenticate all Windows clients and users. OneFS iscompliant with RFC2307, therefore in a multiprotocol environment it is recommended to integrate AD withOneFS to provide a centralized identity management and authentication.RFC2307 allows you to implement unified authentication for UNIX and Windows Active Directory accounts byassociating a user ID (UID), group ID (GID), home directory, and shell with an Active Directory object.Windows Server supported some variations of these schema extensions in versions before Windows 2003 R2with Microsoft Services for UNIX. Windows 2003 R2 and later versions provide full RFC 2307 support. Thismeans that, when configured, the standard UNIX attributes exist in the schemas of domains created withWindows 2003 R2 and later.To use Active Directory as authentication provider for NFS service. You need to configure the OneFS andActive Directory for RFC2307 compliance, and integration with AD for NFS is also needed on the NFS clientside. For more details about how to enable RFC2307 for OneFS and Active Directory, refer to the blog article.For more details about how to integrate Linux client with AD for NFS, refer to the associated officialdocumentations, for example, refer to Red Hat Windows Integration Guide for Red Hat Linux distribution.Lightweight Directory Access Protocol (LDAP)OneFS cluster can also be configured to use LDAP as the authentication provider. The LDAP service in aOneFS cluster supports the following features: 10Users, groups, and netgroupsCustomized LDAP attribute mappingDell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

Implementing NFS Simple BIND authenticationRedundancy and load balancing across all server with identical directory dataEncrypted passwordsFor more details on configure OneFS cluster integrate with LDAP, refer to OneFS Web Administration Guide.To enable a Linux client using LDAP, you can refer to the corresponding Linux distribution officialdocumentation, for example, refer to Red Hat System-level Authentication Guide for Red Hat Linuxdistribution.Network Information Service (NIS)The NIS is a directory services protocol designed by Sun Microsystems. It has inherent limitations, especiallyin the areas of scalability and security. So it is usually replaced by LDAP unless your environment has beenusing NIS for a long time.Local users and groupsThe OneFS cluster supports local users and groups for authentication. You can create local users and groupsaccounts directly on the cluster. Local authentication can be useful for a test environment or if there is nodirectory service available.In a multi-protocol environment, there are usually multi-authentication providers with Active Directory forWindows client’s access and LDAP for Linux or UNIX client’s access. If a user exists in both Active Directoryand LDAP, it is required to configure a user-mapping rule for the user to have enough permission to accessfiles. You can use isi auth mapping create/view to create or view the user mapping rules.2.2Create NFS exportOneFS supports NFSv3, NFSv4.0, NFSv4.1 and NFSv4.2. OneFS does not implement NFSv4.2 newfeatures, but it allows clients to mount NFS export with NFSv4.2. By default, OneFS has NFS servicedisabled. You need to enable NFS service first from WebUI or using isi command. Enabling NFSv4.x is nondisruptive on a OneFS cluster, and it will run concurrently with NFSv3. Any existing NFSv3 clients will not beimpacted by enabling NFSv4.x.OneFS NFS export is zone-aware, every NFS export is associated with an Access Zone. By default, there isan access zone named “System” which will be used if you do not specify an Access Zone name when theexport is created. More details about access zones are discussed in 3.3 Access zone.The NFS export can be created using both WebUI and isi command. For details about creating NFS exports,refer to OneFS Web Administration Guide and OneFS CLI Administration Guide.By default, the NFS service applies a root-squashing rule (map the root user to nobody user) for the NFSexport. This will prevent the client gain root privileges to the server despite the user’s credential. It isrecommended to keep the rule as the default setting since the root account is the super user in Linux andUNIX environments.Note: If you are creating an NFSv4.x export, you need to configure a same NFSv4.x domain on both theOneFS cluster and NFSv4.x clients. You can configure the NFSv4.x domain for the OneFS from WebUI orusing isi command isi nfs settings zone modify. To configure the NFSv4.x domain for NFSv4.x11Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

Implementing NFSclient, you can edit the Domain example.local to your NFSv4.x domain in the /etc/idmapd.conf fileon the client.2.3Mount export over NFSv3/NFSv4.0/NFSv4.1/NFSv4.2NFS v3/v4 are supported on Linux 2.6 kernels and later. In this white paper, we will use Centos 6.9 as NFSclient to illustrate the client side configuration.NFS exports are mounted on the client using the mount command. The format of the command shown asbelow:# mount -t nfs -o options server:/remote/export /local/directoryWhen you mount an NFS export, the NFS protocol version is determined at mount time, and can be modifiedby specifying the version of the NFS protocol using mount options nfsvers or vers. For example, thecommand mount -t nfs -o nfsvers 4.1 server:/remote/export /local/directory willmount the export with NFSv4.1.The drawback of using mount command is that the mounted NFS export is not persistent across clientreboots. So you can use /etc/fstab and autofs to mount the NFS file system. For more details aboutmount, /etc/fstab and autofs, refer to Red Hat Storage Administration Guide. More mount options andconsiderations are discussed in 3.3 NFS Client Considerations.Note: mount options nfsvers and vers have the same meaning in the mount command. The option vers iscompatible with NFS implementations on Solaris and other vendors.12Dell EMC PowerScale: OneFS NFS Design Considerations and Best Practices H17240.3

PowerScale OneFS considerations3PowerScale OneFS considerations3.1NFS export considerationsNFS export read size and write size on OneFSWhile mounting export to a client, you can specify the read size (rsize) and write size (wsize) options,larger rsize and wsize will improve the throughput performance. By default in OneFS, the rsize of 128 KBand wsize of 512 KB are advertised for NFSv3 Linux clients, but can be set as high as 1 MB. NFSv4.xdefaults to 1 MB for both rsize and wsize. Explicitly setting these values too small will override the defaultvalue and may result in slower than optimal performance.NFS aliasesIn general, it is recommended to create an export with short path when possible. If the long directory pathmust be used, you can create an NFS alias for the directory path. An NFS alias is designed to give functionalparity with SMB share name within the context of NFS. It provides a shortcut for the path. Like exports, NFSaliases are also access zone-ware. You can specify the

The Network File System (NFS) protocol allows users to mount remote file systems transparently and access to shared files across networks. It uses a client/server model based on Remote Procedure Call Protocol (RFC5531), so NFS is portable across different machines, operating systems, network architecture, and transport protocols.