INTEGRATING EMC ISILON WITH KERBERISED NFS AND WINDOWS ACTIVE . - Dell

Transcription

INTEGRATING EMC ISILON WITHKERBERISED NFS AND WINDOWSACTIVE DIRECTORYA Practical Guide for Implementation andConfigurationABSTRACTThis white paper provides technical configuration information for theimplementation of a secure Kerberos NFS environment where Microsoft WindowsActive Directory is the authentication method. The paper contains configurationsnippets and procedures for various Linux distributions to provide a workingsolution.February, 2015

To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact yourlocal representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC StoreCopyright 2015 EMC Corporation. All Rights Reserved.EMC believes the information in this publication is accurate as of its publication date. The information is subject to changewithout notice.The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind withrespect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for aparticular purpose.Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.EMC², EMC, the EMC logo, Isilon, OneFS and SmartConnect are registered trademarks or trademarks of EMC Corporation in theUnited States and other countries.VMware is a registered trademark or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All othertrademarks used herein are the property of their respective owners.Part Number H139912

TABLE OF CONTENTSEXECUTIVE SUMMARY . 5Intended Audience . 5Audience Assumptions . 5Prerequisites . 5Revision History . 5WINDOWS ACTIVE DIRECTORY . 6Creating Service Accounts . 6Creating Service Accounts from the Command Line . 6Configuring Kerberos Keytabs for Linux . 7LINUX CONFIGURATION . 8Kerberos . 8Krb5.conf . 8Krb5.keytab . 8Porting Windows Keytabs for Linux . 8System Security Services Daemon (SSSD) . 9Sssd.conf . 9Idmapd.conf . 10Red Hat Enterprise Linux 6 . 10Enabling SSSD . 10NFS Configuration . 10Authconfig. 10ISILON ONEFS CONFIGURATION . 11Prerequisites . 113

OneFS Active Directory Settings . 11NFSv4 . 12Kerberos . 12Check for missing SPNs . 12Creating missing SPNs . 12General Kerberos Settings . 13Reverse DNS . 13Windows DNS . 13Bind DNS . 13EXPORTING DIRECTORIES . 14MOUNTING AND TESTING DIRECTORIES . 14APPENDIX A: TROUBLESHOOTING . 15Time. 15Isilon . 15Linux . 15NFSv4 Domain Setting . 15Isilon . 15Linux . 16Review Cluster Authentication . 16Isilon . 16Check DNS PTR Records . 164

EXECUTIVE SUMMARYOrganisations are increasingly challenged to find ways to integrate their UNIX/Linux environments with their corporate Windowsenvironment. The need for a single sign-on (SSO) infrastructure and centralised storage go hand in hand, having the same dataaccessible and controlled by centrally managed access control lists (ACLs) across varying platforms is more efficient and secure.This document aims to be a collective source of techniques and configurations required to integrate EMC Isilon with KerberisedNFS where Microsoft Windows Active Directory is the authentication medium. This document explores the setup needed to coversome popular flavours of Linux such as: Red Hat Enterprise Linux and its community equivalent CentOS as well as UbuntU.INTENDED AUDIENCEThis guide is intended for experienced system and storage administrators who are familiar with network storage administration,Windows Active Directory and Linux administration.AUDIENCE ASSUMPTIONSThe guide assumes the reader has an understanding and working knowledge of the following: NFS and SMB storage protocols, as appropriate for the specific organisational requirements. Isilon scale-out storage architecture and the EMC Isilon OneFS operating system. Microsoft Windows Active Directory security architecture and authentication. Linux operating system.PREREQUISITESTechniques described within this document make reference to an established and working Windows Active Directorydomain/forest with an established relationship with an EMC Isilon cluster. Additional information on this topic may be located inthe Isilon Administration Guide.This paper has been written based on the use of the following architectures and the versions described:EMC OneFS 7.2.0.1Red Hat Enterprise Linux 6Microsoft Windows Server 2008 R2REVISION HISTORYDateVersionAuthorChange SummaryInitial Document including:April 20150.4Darren Miller Universal Linux configurationRed Hat Enterprise Linux configurationIsilon configurationActive Directory procedures and adjustments5

WINDOWS ACTIVE DIRECTORYIn order to provide single sign-on service for Linux and subsequent authentication services, the Windows architecture must firstbe prepared. Additional supported features are needed to extend the schema making the Kerberos implementation capable ofauthenticating UNIX based users and groups. This in turn unifies account attributes, placing UIDs/GIDs and SIDs within thesame database, which overall avoids the need for a further user mapping utility.Required Roles: Identity Management for UNIXWithin the Identity Management for UNIX MMC create a NIS server with an appropriate domain name. This extends the schemaand enables UNIX attributes to be stored for user accounts.Existing and new user and group accounts will need to be modified to take advantage of the new features. A new tab labelled“UNIX Attributes” will appear in the User properties window within Active Directory Users and Computers MMC. Starting withrequired groups as this is a prerequisite for users.Select the newly created NIS domain from the drop down menu will unlock theadditional attributes.The GID and for user accounts the UID, Login Shell, Home Directory andPrimary group name will all default. In the case for user accounts Select theappropriate group from the drop down menu and complete the modification.The example shown is for a user account called “isilonuser1”Note: The NIS Domain in this example is labelled “isilon” this is not the storagecluster name.Additional attributes have defaulted to:UID: 10000Login Shell: /bin/shHome Directory: /home/isilonuser1Primary group name/GID: Domain UsersThese settings will reference variables localised to the UNIX/Linux system andnot to the Isilon cluster.CREATING SERVICE ACCOUNTSLinux will require a number of service accounts: NFS computer account per Linux machineHOST computer account per Linux machineNFS and HOST accounts are machine specific and are tied to the Kerberos authentication system directly. Create the aboveaccounts by following Microsoft standard procedures. For the machine specific accounts use a suitable naming scheme similarto:RHEL6 – HOST computer accountnfsRHEL6 – NFS computer accountIn the example “RHEL6” is the hostname and the NFS account name is the hostname name prefixed by “nfs” to simply mark itout as the NFS account.CREATING SERVICE ACCOUNTS FROM THE COMMAND LINEFrom an administrator command prompt:C:\ dsadd computer CN RHEL6,CN Computers,DC isilon,DC localThe example shows the hostname RHEL6 being added to the Computers container in the Active Directory “Isilon.local”. Theobject may be verified by using the command:C:\ dsquery computerResults should appear similar to:“CN RHEL6,CN Computers,DC isilon,DC local”6

CONFIGURING KERBEROS KEYTABS FOR LINUXThe accounts previously created will need to be aligned to support a Linux Kerberos client deployment. To achieve this, ServicePrincipal Names (SPN) will need to be aligned to those being used in Linux. ArcFour (RC4-HMAC-NT) is the highest encryption supported by Isilon OneFS 7.2.0.1Open a command prompt:C:\ setspn -A nfs/rhel6.isilon.local nfsRHEL6C:\ setspn -A host/rhel6.isilon.local RHEL6Generate the keytabs:C:\ ktpass -princ host/rhel6.isilon.local@ISILON.LOCAL -mapuser ISILON\RHEL6 -crypto RC4-HMAC-NT rndPass out rhel6.krbC:\ ktpass -princ nfs/rhel6.isilon.local@ISILON.LOCAL -mapuser ISILON\nfsRHEL6 -crypto RC4-HMAC-NT rndPass out nfsrhel6.krbThe output files default to the local user profile path. If performing these commands as the administrator the files may belocated in c:\users\administrator. These files need to be copied to the relevant Linux host, in most cases using an SFTP client isthe easiest.7

LINUX CONFIGURATIONThere are many methods which can be deployed in order to achieve a single sign-on agenda with Linux. This document utilisesthe System Security Services Daemon (SSSD) as well as general Kerberos configurations.KERBEROSKerberos is a computer network authentication protocol which works on the basis of 'tickets' to allow nodes communicating overa non-secure network to prove their identity to one another in a secure manner. There are two parts to the Kerberosconfiguration in Linux, the main configuration file and keytab database.KRB5.CONFThis is the main Kerberos configuration file usually located “/etc/krb5.conf”. Kerberos utilises this file to provide requiredinformation to interact with the Active Directory securely./etc/krb5.conf[logging]default FILE:/var/log/krb5libs.log[libdefaults]default realm ISILON.LOCALdns lookup realm truedns lookup kdc trueticket lifetime 24hrenew lifetime 7dforwardable truerdns falseallow weak crypto truedefault tkt enctypes arcfour-hmacdefault tgs enctypes arcfour-hmacpermitted enctypes arcfour-hmac[realms]ISILON.LOCAL {kdc isilonad.isilon.localadmin server isilonad.isilon.localdefault domain isilon.local}[domain realm].isilon.local ISILON.LOCALisilon.local ISILON.LOCAL[logging] - Contains relations which determine how Kerberos entities are toperform their logging.[libdefaults] - Contains various default values used by the Kerberos V5library.default realm - This relation identifies the default realm to be used in aclient host's Kerberos activity.dns lookup realm - Indicate whether DNS TXT records should be used todetermine the Kerberos realm of a host.dns lookup kdc - Indicate whether DNS SRV records should be used tolocate the KDCs and other servers for a realm, if they are not listed in theinformation for the realm.forwardable - Initial tickets by default will be forwardable.rdns - Prevent the use of reverse DNS resolution when translating hostnamesinto service principal names.*enctypes - List of default and permitted session encryption types[realms] - Contains subsections keyed by Kerberos realm names whichdescribe where to find the Kerberos servers for a particular realm and otherrealm-specific information. Each tag in the [realms] section names aKerberos realm. The value of the tag is a subsection where the relations inthat subsection define the properties of that particular realm.kdc - The value of this relation is the name of a host running a KDC for thatrealm.admin server - his relation identifies the host where the administrationserver is running.default domain - This relation identifies the default domain for which hostsin this realm are assumed to be in.[domain realm] - The section provides a translation from a hostname tothe Kerberos realm name for the services provided by that host.KRB5.KEYTABA keytab is a file containing pairs of Kerberos principals and encrypted keys. The file is usually referenced as “/etc/krb5.keytab”.PORTING WINDOWS KEYTABS FOR LINUXIn the previous section “WINDOWS ACTIVE DIRECTORY - CONFIGURING KERBEROS KEYTABS FOR LINUX”, two files werecreated:rhel6.krbnfsrhel6.krbThese individual files need to be merged into one certificate database file “/etc/krb5.keytab”Import the keytabs:# ktutilktutil: rkt /root/rhel6.krbktutil: rkt /root/nfsrhel6.krbktutil: wkt /etc/krb5.keytabktutil: exitVerify the keytab:# klist -k -eklist -k -e outputKeytab name: FILE:/etc/krb5.keytabKVNO Principal---- -----------------------3 host/rhel6.isilon.local@ISILON.LOCAL (arcfour-hmac)3 nfs/rhel6.isilon.local@ISILON.LOCAL (arcfour-hmac)The output should look similar to the example.8

Initialise the keytab:# kinit -k# klist -eThe output should look similar to theexample.klist -e outputTicket cache: FILE:/tmp/krb5cc 0Default principal: host/rhel6.isilon.local@ISILON.LOCALValid startingExpiresService principal02/24/15 16:04:47 02/25/15 02:04:52 krbtgt/ISILON.LOCAL@ISILON.LOCALrenew until 03/03/15 16:04:47, Etype (skey, tkt): arcfour-hmacSYSTEM SECURITY SERVICES DAEMON (SSSD)The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote authenticationmechanisms. Its master configuration file can be referenced as “/etc/sssd/sssd.conf”.SSSD.CONFIndividual pieces of SSSD functionality are provided by special SSSD services that are started and stopped together with SSSD.The services are managed by a special service frequently called "monitor"./etc/sssd/sssd.conf[sssd]config file version 2reconnection retries 3sbus timeout 30services nss, pamdomains ISILON[nss]filter groups rootfilter users rootreconnection retries 3[pam]reconnection retries 3[domain/ISILON]description AD serverenumerate trueid provider adauth provider adaccess provider adchpass provider adad server isilonad.isilon.localad domain isilon.localdefault shell /bin/bashfallback homedir /home/%uldap schema ADldap idmap default domain ISILON.LOCALldap id mapping falseldap user object class userldap user name sAMAccountNameldap user uid number uidNumberldap user gid number gidNumberldap user home directory unixHomeDirectoryldap user shell loginShellldap user principal userPrincipalNameldap group object class groupldap group name nameldap group member memberldap group gid number gidNumberldap force upper case realm true[sssd] – This section is used to configure the monitor as wellas some other important options like the identity domains.domains - A domain is a database containing userinformation. SSSD can use more domains at the same time,but at least one must be configured or SSSD won't start. Thisparameter describes the list of domains in the order to bequeried.[nss] – Configuration options specific to the Name ServiceSwitch (NSS) service.[pam] – Configuration options specific to the PluggableAuthentication Module (PAM) service.[domain/ID] – Configuration options specific to targetenvironment, designated by a unique identifier. Activedomain configurations are triggered by the “domains” optionwithin the [sssd] section.id provider – The identification provider used for thedomain. Set to Active Directory (ad)auth provider – The authentication provider used for thedomain. Set to Active Directory (ad)access provider - The access control provider used for thedomain. Set to Active Directory (ad)chpass provider – Designates the provider which shouldhandle change password operations for the domain. Set toActive Directory (ad)ad server – Hostnames or IP addresses of the domaincontrollers, comma separated in order of preference.ad domain – Specifies the name of the Active Directorydefault shell – Specifies an operating shell in the event avalue is not supplied by the domain provider.fallback homedir – Specifies a home directory path in theevent a value is not supplied by the domain provider.ldap schema – Specifies the schema type to use. Set toActive Directory (AD).ldap idmap default domain – Specify the name of thedefault domain, must be the same as used in the Kerberosconfiguration.ldap id mapping – If using Identity Management for UNIXthis setting must be disabled.The remaining options represent formatting for the Linuxauthentication system.9

IDMAPD.CONFThe configuration file is for libnfsidmap. This is used by idmapd and svcgssd to map NFSv4 names to and from IDs. Theparameter “Domain” must be the same as that previously referenced as “ldap idmap default domain” in the sssd.conf. i.e:Domain ISILON.LOCALRED HAT ENTERPRISE LINUX 6Specific configuration required for Red Hat Enterprise Linux 6ENABLING SSSDCommands specific to Red Hat Enterprise Linux for enabling and starting SSSD:# chkconfig sssd on# service sssd startNFS CONFIGURATIONEnable secure NFS within the Red Hat NFS configuration file referenced in “/etc/sysconfig/nfs”, uncomment the following:SECURE NFS "yes"Commands specific to Red Hat Enterprise Linux for enabling and starting GSSD:# chkconfig rpcgssd on# service rpcgssd startCommands specific to Red Hat Enterprise Linux for enabling and starting IDMAPD:# chkconfig rpcidmapd on# service rpcidmapd startAUTHCONFIGEnable SSSD as an authentication provider on Red Hat Enterprise Linux:authconfig --enablesssd --enablesssdauth --update10

ISILON ONEFS CONFIGURATIONThis section covers the essential configuration required for OneFS to respond to requests for secure kerberised NFSauthenticated by Active Directory.PREREQUISITES The cluster must be joined correctly to the target Active Directory.All IP addresses within the required SmartConnect Zone must be added to the reverse DNS with the same FQDN forthe cluster delegation.ONEFS ACTIVE DIRECTORY SETTINGSFor mixed mode authentication operations where Active Directory is the authentication authority a number of advanced optionswill need to be enabled.Access Authentication ProvidersActive DirectoryView DetailsServices for UNIX: rfc2307 – This leverages the Identity Management for UNIX services in the Active Directory schemaMap user/group into primary domain: Yes – Without this setting a the domain name will need to be prefixed during userlogin.Auto-Assign UIDs: No – OneFS by default will generate pseudo UIDs for users it cannot match to SIDs this can causepotential user mapping issues.Auto-Assign GIDs: No – OneFS by default will generate pseudo GIDs for group it cannot match to SIDs as with the usermapping equally a group mapping mismatch could occur.The example below shows the advanced active directory settings utilised for the test domain. If the status indicator appears inany colour other than green the active directory is out of synchronisation with OneFS and will need to be repaired beforecontinuing.Isilon Web UI - Authentication Providers11

NFSV4OneFS does not enable NFSv4 by default and therefore setting associated with it will need configuring before any exports can bemounted. Located in:ProtocolsUNIX Sharing (NFS)Global SettingsEnable NFSv4 protocol and set the NFSv4 domain. The domain should match those used previously within the Linux section ofthis document.Isilon Web UI – UNIX Sharing (NFS)KERBEROSAs with Linux; Isilon OneFS uses Kerberos keytabs generated by the Active Directory to securely authenticate. However whenOneFS joins the domain it will automatically generate the SPNs and Kerberos certificates necessary to complete theauthentication process for all the SmartConnect zones that are configured. If SmartConnect zones are added after the domainjoin, additional SPNs will be required to complete the authentication process.CHECK FOR MISSING SPNSMissing SPNs will usually be reported as a warning event in the web UI, alternatively it is also possible to check using the OneFScommand line interface (CLI) accessible via an established SSH session to any node.# isi auth ads spn check --domain isilon.localThe output of the command will return any missing SPNs for the Active Directory domain “isilon.local”.CREATING MISSING SPNSMissing SPNs will need to be added to the Active Directory configuration. If the SmartConnect zone is missing from the SPNdatabase associated with the cluster, users will not be permitted to authenticate for that zone.# isi auth ads spn create --domain isilon.local --spn ”HOST/data.isilon.local” --user administratorThe example command will create an SPN for the SmartConnect zone “data.isilon.local” on the “isilon.local” Active Directoryusing the domain user “administrator”. Repeat the command for all the missing SPNs.Once complete restart the NFS services:PreOneFS 7.2:# isi for array ‘killall nfsd'# isi for array ‘killall gssd'OneFS 7.2 and greater:# /usr/likewise/bin/lwsm restart onefs nfs12

GENERAL KERBEROS SETTINGSNew to OneFS 7.2 is the Kerberos tab in the authentication providers section of the web UI.AccessAuthentication ProvidersKerberos SettingsSet the default domain to match the target as performed in previous steps.Isilon Web UI - Authentication ProvidersREVERSE DNSFor every network interface card (NIC) that is added to the SmartConnect zone associated with NFSv4, the assigned IPaddressing will need a DNS Pointer Record (PTR) to be created in reverse DNS directed at the DNS delegation FQDN. TheKerberos authentication sequence must be able to reverse lookup the cluster to an authorised SPN in active directory.WINDOWS DNSThe example shows the PTR record properties for the test cluster “cluster.isilon.local”, the IP address 192.168.40.31 is anaddress which is within the SmartConnect zone pool of addresses.Microsoft Windows - DNS ManagerBIND DNSIn the required reverse subnet file; enter a line, similarto the following, for each address required.192.168.40.31INPTRcluster.isilon.local.13

EXPORTING DIRECTORIESIt is important to note: As of OneFS 7.2 the NFS services have moved to the user space known as uNFS. This move wasperformed to provide multi-tenancy and as a result shares are now Access Zone centric. For the purpose of this paper, thedefault “System” access zone is utilised.Within the OneFS Web UI, export the required directories with Kerberos (KRB5) security only. The default security method is touse traditional UNIX system (SYS).ProtocolsUNIX Sharing (NFS)NFS ExportsMOUNTING AND TESTING DIRECTORIESThere are two methods to physically mount the exported directories. The first, and most obvious, is the manual method. Thesecond is automatically in fstab. First of all, test the configuration manually:# mount –t nfs4 –o sec krb5 cluster.isilon.local:/ifs/data/mixed /mntLogin as an Active Directory user with UNIX attributes enabled.Check that a Kerberos certificate has beenapplied: klist –eIf successful the results should be similar tothe example. cd /mnt touch test.txt lsklist -e outputTicket cache: FILE:/tmp/krb5cc 10001 OyzkCkDefault principal: isilonuser1@ISILON.LOCALValid startingExpires04/08/15 10:31:04 04/08/15 20:31:04renew until 04/15/15 10:31:04, Etypehmac-sha1-9604/08/15 10:31:08 04/08/15 20:31:04renew until 04/15/15 10:31:04, EtypeService principalkrbtgt/ISILON.LOCAL@ISILON.LOCAL(skey, tkt): arcfour-hmac, ey, tkt): arcfour-hmac, arcfour-hmacSuccessful if able to change directory to thekerberised mount point and list or create files.On the principal that the manual mount was successful, edit the /etc/fstab file and append the information for the required cluster.isilon.local:/ifs/data/mixed /datanfs4nfs4defaults,sec krb5defaults,sec krb50000Mount the directories via the fstab:# mount –aSuccessful if the NFSv4 exported directories mount correctly.This can be verified using the command:# dfThe output should appear similar to theexample.df outputFilesystem1K-blocks Used/dev/sda316307112 lon.local:/ifs/home37219328 672768cluster.isilon.local:/ifs/data/mixed37219328 672768Available Use%12300140 21%9573440%21837823%Mounted on//dev/shm/boot365465602%/home365465602%/data14

APPENDIX A: TROUBLESHOOTINGTIMEKerberos is very time sensitive; if any system in the chain strays beyond the threshold the whole sequence will fail. Errorssimilar to the following may appear in log files:KRB5KRB AP ERR SKEWThe default threshold is 5 minutes.It is highly recommended that all systems be linked to a reliable NTP source. All time must match that of the authenticationauthority, which in this environment is Active Directory.ISILONFrom the Isilon CLI execute the command:# isi for array –s ‘date’The command will report the time on each node; any deviations will need to be addressed.If not using NTP then resolve the skew using the date command. Refer to “man date” for more information:# isi for array –s ‘date MMDDhhmm’LINUXExecuting the kinit command will result in a clock skew error similar to:# kinit –kkinit: Clock skew too great while getting initial credentialsThis can be verified by executing the date command:# dateIf not using NTP then resolve the skew using the date command. Refer to “man date” for more information:# date MMDDhhmmNFSV4 DOMAIN SETTINGThe NFSv4 domain makes reference to the setting specifically for IDMAPD. amenamenamenameis Case-Sensitiveis Access Zone specificmust match on the client and the clusterdefaults to 'localdomain' on the clusterdefaults to 'dnsdomainname' on many Linux distributionsThroughout this paper it is referenced in uppercase, this is purely as a reference to Active Directory and is optional.ISILONThe domain name can be verified by executing the following command from the Isilon CLI:# isi nfs settings zone view --zone SystemNFSv4 Domain: ISILON.LOCALNFSv4 Replace Domain: YesNFSv4 No Domain: NoNFSv4 No Domain UIDs: YesNFSv4 No Names: NoNFSv4 Allow Numeric Ids: YesThe command shows the zone “System” if using multiple access zones substitute this name for the required zone name.For more information refer to the Isilon configuration section on page 12.15

LINUXThe domain name can be verified by examining the file /etc/idmapd.conf:# more /etc/idmapd.conf[General]#Verbosity 0# The following should be set to the local NFSv4 domain name# The default is the host's DNS domain name.Domain ISILON.LOCALConfirm rpc.gssd and rpc.idmapd is running:# ps –ef grep rpcrpc16961rpcuser 18351root18552root18621root188210 11:49 ?0 11:49 ?0 11:49 ?0 11:49 ?0 11:49 ?00:00:00 rpcbind00:00:00 rpc.statd00:00:00 [rpciod/0]00:00:00 rpc.gssd00:00:00 rpc.idmapdREVIEW CLUSTER AUTHENTICATIONReview cluster authentication setup for "ID coherency". Coherency refers to the user/group consistency on the cluster versusthe clients. Authentication sources come into play with NFSv3 using a

INTEGRATING EMC ISILON WITH KERBERISED NFS AND WINDOWS ACTIVE DIRECTORY A Practical Guide for Implementation and Configuration ABSTRACT This white paper provides technical configuration information for the implementation of a secure Kerberos NFS environment where Microsoft Windows Active Directory is the authentication method.