WIXLIB

Manage File System Space in Oracle ManagedFile Transfer Cloud ServiceORACLE WHITE PAPER AUGUST 2018

DisclaimerThe following is intended to outline our general product direction. It is intended for informationpurposes only, and may not be incorporated into any contract. It is not a commitment to deliver anymaterial, code, or functionality, and should not be relied upon in making purchasing decisions. Thedevelopment, release, and timing of any features or functionality described for Oracle’s productsremains at the sole discretion of Oracle.MANAGING FILE SYSTEM SPACE IN MFTCS

Table of ContentsDisclaimer1Introduction1Shared Filesystem on MFTCS1How is the sharing accomplished?1Structure of MFT root directory1DBFS Details1DBCS (Database Cloud Service)1DBFS (Database File System)1DBFS Overhead3Autoextend’s Effect on Total and Free Space3Monitoring File System Usage on MFT PodHow to Run the ScriptManaging Space on the MFT Pod334Remove Unwanted Files from the Embedded SFTP Server4Purge Completed Transfers4Scale up DBCS Storage5MANAGING FILE SYSTEM SPACE IN MFTCS

IntroductionThis document explains how Oracle Managed File Transfer Cloud Service uses a shared Database File System(DBFS), how to monitor the available space and see what is using that space, and how to reduce usage or addstorage as needed.Shared Filesystem on MFTCSOracle Managed File Transfer (MFT) Cloud Service uses shared file system across the nodes in the cluster tostore payloads, the SFTP server files, custom callouts, and JCA control files. On each node of the MFT CloudService instance, the files are stored at /u01/soacs/mftroot/mft. This is also known as MFT root. All filesstored within the MFT root directory are visible across all nodes in the cluster.How is the sharing accomplished?The file sharing on MFT Cloud Service cluster is accomplished using Oracle DBFS (Database File System). Tostart sharing the files among different MFT Cloud Service nodes in the cluster, a mount is created on the MFT rootdirectory. The DBFS client on each node maps the shared filesystem to a big file tablespace that exists in MFT’sassociated DBCS database. Directory and file modifications within the DBFS mount are stored in the DBFStablespace. When the files are listed using the ls command in linux, the files information are fetched from theDBFS tablespace.Structure of MFT root directoryThe MFT root directory is located at: /u01/soacs/mftroot/mft. There are 4 different directories present insidethe MFT root directory, each directory has a specific purpose:» ftp rootThe ftp root directory is used to store data coming to the embedded SFTP server and also for the SOAP typetargets with delivery preference as SFTP.» storageThe storage directory is used for payloads coming from all other protocols except the embedded SFTP server.» control dirThe control dir directory is used for creating and maintaining marker files (0 byte size files) and in progress filesfor JCA sources (File remote FTP and remote SFTP).» calloutsThe callouts directory is used to store the executables/jar files. The jars stored under callouts can be used toexecute pre-processing or post-processing operation on the files transferred.DBFS DetailsDBCS (Database Cloud Service)The details of the database can be found from the SOA Cloud Service console. The details page of the MFT CloudService pod contains the information of the DBCS instance to which it is associated. This is found under theAssociations tab.Instructions for accessing and querying the DBCS instance can be found in the DBCS Administration Guide. Seethe Database Cloud Service guide.DBFS (Database File System)The DBFS tablespace can be found by using the following query:SELECT TABLESPACE NAME, FILE NAME, STATUS, AUTOEXTENSIBLE FROM DBA DATA FILES WHERETABLESPACE NAME LIKE '%DBFS%';1 MANAGING FILE SYSTEM SPACE IN MFTCS

DBFS TABLESPACETABLESPACE NAMESTATUSAUTOEXTENSIBLESP1531457313 DBFSAVAILABLEYESThe user associated with the respective tablespace can be found using the following query:SELECT * FROM ALL USERS WHERE USERNAME LIKE '%DBFS%';USER ASSOCIATED WITH DBFS TABLESPACEUSERNAMEUSER IDCREATEDCOMMONORACLE MAINTAINEDSP1531457313 DBFS13613-JUL-18NONTo identify the total size of the DBFS tablespace, used apace, free space and auto extendable property, use thefollowing query:column "Tablespace" format a13column "Used MB"format 99,999,999column "Free MB"format 99,999,999column "Total MB"format 99,999,999column "Auto Extend" format a13selectfs.tablespace name(df.totalspace - fs.freespace)fs.freespacedf.totalspaceround(100 * (fs.freespace / ace name,round(sum(bytes) / 1048576) TotalSpace ,autoextensiblefromdba data filesgroup bytablespace name,autoextensible) df,(selecttablespace name,round(sum(bytes) / 1048576) FreeSpacefromdba free spacegroup bytablespace name) fswheredf.tablespace name fs.tablespace nameand df.tablespace name like '%DBFS%';2 MANAGING FILE SYSTEM SPACE IN MFTCS"Tablespace","Used MB","Free MB","Total MB","Pct. Free",

EXAMPLE OUTPUTUser NameUsed MBSP1531457313 DBFS 6809Free MB95591Total MB102400Pct. Free93DBFS OverheadIn addition to files, DBFS also stores directory and file metadata, and this metadata itself uses space in the DBFStablespace. The DBFS metadata overhead is normally about 15%. This overhead should be considered whenmonitoring DBFS space utilization. For example, 100 GB of files stored in the DBFS mount will actually use 115GB of the DBFS tablespace.Autoextend’s Effect on Total and Free SpaceIt is important to understand that file system commands like du do not reflect total and freespace accurately withinthe DBFS mount on the MFT Cloud Service nodes. This is because the DBFS mount client program calculatestotal and freepsace based on the space currently allocated to the DBFS tablespace. The DBFS tablespace is set toautoextend by default in 100MB increments. As files are added, the tablespace will grow to approximately 90% fulland then extend automatically. The DBFS client does not consider the potential free space the DBCS instanceallocates to the tablespace as needed, and therefore commands like du usually report much lower total and freespace than are actually available.Total and free space should be monitored using the sql script above and discounted by the overhead factor. Thetotal space can be increased as discussed in the Scale up DBCS storage section below.The free space in the DBCS instance should be monitored regularly to avoid running out of room to expandthe DBFS tablespace which will result in shutting down the MFT instance.Monitoring File System Usage on MFT PodSpace utilization on the MFTCS pod can be monitored using standard linux file system commands by evaluatingusage in the four MFT root subdirectories mentioned above. Keep in mind the total and free space is not accurateas mentioned in the section above.The following script reports the five largest directories in the MFT root and from each of its subdirectories.How to Run the ScriptThis script can be executed on the actual MFTCS VM.To run the script:1. Connect to the MFTCS VM - ssh -i pvt key opc@ ip of the MFTCS machine 2. Switch as Oracle user : sudo su- oracle3. Copy the script mentioned below in a temp file : vi /tmp/temp.sh4. Change the permission on the temp file : chmod 755 /tmp/temp.sh5. Execute the script: ./tmp/temp.sh#!/bin/sh#Listing the space usage for all directories inside mft applicationecho "List the directory usage inside MFT root directory"cd /u01/soacs/mftroot/mftdu -hs * sort -rh head -5echo "List the directory usage inside MFT/FTP ROOT directory"cd /u01/soacs/mftroot/mft/ftp rootdu -hs * sort -rh head -5echo "List the directory usage inside MFT/STORAGE directory"3 MANAGING FILE SYSTEM SPACE IN MFTCS

cd /u01/soacs/mftroot/mft/storagedu -hs * sort -rh head -5echo "List the directory usage inside MFT/CONTROL DIR directory"cd /u01/soacs/mftroot/mft/control dirdu -hs * sort -rh head -5echo "List the directory usage inside MFT/CALLOUTS directory"cd /u01/soacs/mftroot/mft/calloutsdu -hs * sort -rh head -5Managing Space on the MFT PodThere are two ways to reduce space utilization within the MFT root directory and one way to expand the totalavailable space.Remove Unwanted Files from the Embedded SFTP ServerThis can be done by browsing the SFTP server with any SFTP client, looking for files that are no longer needed,and removing them. It can also be done directly on any of the MFT managed nodes by using standard linux filesystem commands within the /u01/soacs/mftroot/mft/ftp root directory.Caution: No files outside of the ftp root directory should be modified directly by the user. These aremaintained by MFT and direct modifications will cause problems with the overall system.Purge Completed TransfersPurging older completed transfers will reduce the size of the storage directory. In MFT Release 12.2.1.2 or later,completed transfer purges can be scheduled with retention dates using the MFTCS administrative UI (for version12.1.3 use WLST commands as documented in MFT Purge Commands.The data on MFT pod can be managed using the purge feature provided on the MFT app. It can be used toschedule the frequency of data purge. The data retention date can also be scheduled on the intuitive MFT Purgeuser interface as shown below (under Administration tab):Figure 1. MFT Console showing Purge user interface4 MANAGING FILE SYSTEM SPACE IN MFTCS

Scale up DBCS StorageThe total space allocated to DBFS can be increased by scaling up DBCS storage. For the instructions on scalingup DBCS storage, see Scaling a Database Deployment.DBCS storage should be expanded in large increments (1 or 2 TB) as there are only five total increasespossible in OCI classic.“Oracle Cloud Infrastructure Classic supports 10 block storage volumes attachedto a compute node, of which 5 are used when the database deployment iscreated. Thus, you have only 5 opportunities to scale up storage. Consequently,each scale-up operation you perform can dramatically affect the maximum sizeyour database can grow to.”5 MANAGING FILE SYSTEM SPACE IN MFTCS

Oracle Corporation, World HeadquartersWorldwide Inquiries500 Oracle ParkwayPhone: 1.650.506.7000Redwood Shores, CA 94065, USAFax: 1.650.506.7200CONNECT W ITH r.com/oracleoracle.comCopyright 2018, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and thecontents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any otherwarranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability orfitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations areformed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means,electronic or mechanical, for any purpose, without our prior written permission.Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license andare trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo aretrademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0818Manage File System Space in Oracle MFT Cloud ServiceAugust 2018Author: Milton SinghContributing Authors: Saurav Sao, Mark Overton

Oracle Managed File Transfer (MFT) Cloud Service uses shared file system across the nodes in the cluster to store payloads, the SFTP server files, custom callouts, and JCA control files. On each node of the MFT Cloud Service instance, the files are stored at /u01/soacs/mftroot/mft. This is also known as MFT root. All files