WIXLIB

Preface Oracle Machine Learning Technologies Oracle Autonomous DatabaseConventionsThe following text conventions are used in this document:ConventionMeaningboldfaceBoldface type indicates graphical user interface elements associated with anaction, or terms defined in text or the glossary.italicItalic type indicates book titles, emphasis, or placeholder variables for whichyou supply particular values.monospaceMonospace type indicates commands within a paragraph, URLs, code inexamples, text that appears on the screen, or text that you enter.ix

1About Oracle Machine Learning for PythonThe following topics describe Oracle Machine Learning for Python (OML4Py) and itsadvantages for the Python user. What Is Oracle Machine Learning for Python? Advantages of Oracle Machine Learning for Python Transparently Convert Python to SQL About the Python Components and Libraries in OML4PyWhat Is Oracle Machine Learning for Python?Oracle Machine Learning for Python (OML4Py) enables you to run Python commands fordata transformations and for statistical, machine learning, and graphical analysis on datastored in or accessible through an Oracle database using a Python API.OML4Py is a Python module that enables Python users to manipulate data in databasetables and views using Python syntax. OML4Py functions and methods transparentlytranslate a select set of Python functions into SQL for in-database execution.OML4Py is available in the following Oracle database environments: The Python interpreter in Oracle Machine Learning Notebooks in your OracleAutonomous Database. For more information, see Get Started with Notebooks for DataAnalysis and Data Visualization in Using Oracle Machine Learning Notebooks.In this environment, all the required components are included, including Python, requiredPython libraries, and the Python interpreter in Notebooks. An OML4Py client connection to OML4Py in an on-premises Oracle Database instance.For this environment, you must install Python, the required Python libraries, and theOML4Py server components in the database, and you must install the OML4Py client.See Install OML4Py for On-Premises Databases.Designed for problems involving both large and small volumes of data, OML4Py integratesPython with the database. With OML4Py, you can do the following: Develop, refine, and deploy user-defined Python functions and machine learning modelsthat leverage the parallelism and scalability of the database to automate data preparationand machine learning. Run overloaded Python functions and use native Python syntax to manipulate indatabase data, without having to learn SQL. Use Automated Machine Learning (AutoML) to enhance user productivity and machinelearning results through automated algorithm and feature selection, as well as modeltuning and selection. Use Embedded Python Execution to run user-defined Python functions in Python enginesspawned and managed by the database environment. The user-defined functions and1-1

Chapter 1Advantages of Oracle Machine Learning for Pythondata are automatically loaded to the engines as required, and when data-paralleland task-parallel execution is enabled.Advantages of Oracle Machine Learning for PythonUsing OML4Py to prepare and analyze data in or accessible to an Oracle databasehas many advantages for a Python user.With OML4Py, you can do the following: Operate on database data without using SQLOML4Py transparently translates many standard Python functions into SQL. WithOML4Py, you can create Python proxy objects that access, analyze, andmanipulate data that resides in the database. OML4Py can automatically optimizethe SQL by taking advantage of column indexes, query optimization, tablepartitioning, and database parallelism.OML4Py overloaded functions are available for many commonly used Pythonfunctions, including those on Pandas data frames for in-database execution.See Also: Transparently Convert Python to SQL Automate common machine learning tasksBy using Oracle’s advanced Automated Machine Learning (AutoML) technology,both data scientists and beginner machine learning users can automate commonmachine learning modeling tasks such as algorithm selection and featureselection, and model tuning and selection, all of which leverage the parallelprocessing and scalability of the database.See Also: About Automated Machine Learning Minimize data movementBy keeping the data in the database whenever possible, you eliminate the timeinvolved in transferring the data to your client Python engine and the need to storethe data locally. You also eliminate the need to manage the locally stored data,which includes tasks such as distributing the data files to the appropriate locations,synchronizing the data with changes that are made in the production database,and so on.See Also: About Moving Data Between the Database and a Python Session Keep data secureBy keeping the data in the database, you have the security, scalability, reliability,and backup features of the database for managing the data. Use the power of the databaseBy operating directly on data in the database, you can use the memory andprocessing power of the database and avoid the memory constraints of your clientPython engine. Use current dataAs data is refreshed in the database, you have immediate access to current data. Save Python objects to a datastore in the databaseYou can save Python objects to an OML4Py datastore for future use and for useby others.1-2

Chapter 1Advantages of Oracle Machine Learning for PythonSee Also: About OML4Py Datastores Build and store models in the databaseUsing Embedded Python Execution, you can build native Python models and store andmanage them in an OML4Py datastore.You can also build in-database models, with, for example, an oml class such as theDecision Tree class oml.dt. These in-database models have proxy objects that referencethe actual models. Keeping with normal Python behavior, when the Python engineterminates, all in-memory objects, including models, are lost. To prevent an in-databasemodel created using OML4Py from being deleted when the database connection isterminated, you must store its proxy object in a datastore.See Also: About Machine Learning Classes and Algorithms Score dataFor most of the OML4Py machine learning classes, you can use the predict andpredict proba methods of the model object to score new data.For these OML4Py in-database models, you can also use the SQL PREDICTION functionon the model proxy objects, which scores directly in the database. You can use indatabase models directly from SQL if you prepare the data properly. For open sourcemodels, you can use Embedded Python Execution and enable data-parallel execution forperformance and scalability. Run user-defined Python functions in embedded Python enginesUsing OML4Py Embedded Python Execution, you can store user-defined Pythonfunctions in the OML4Py script repository, and run those functions in Python enginesspawned by the database environment. When a user-defined Python function runs, thedatabase starts, controls, and manages one or more Python engines that can run inparallel. With the Embedded Python Execution functionality, you can do the following:–Use a select set of Python packages in user-defined functions that run in embeddedPython engines–Use other Python packages in user-defined Python functions that run in embeddedPython engines–Operationalize user-defined Python functions for use in production applications andeliminate porting Python code and models into other languages; avoid reinventingcode to integrate Python results into existing applications–Seamlessly leverage your Oracle database as a high-performance computingenvironment for user-defined Python functions, providing data parallelism andresource management–Perform parallel simulations, for example, Monte Carlo analysis, using theoml.index apply function–Generate PNG images and XML representations of both structured and image data,which can be used by Python clients and SQL-based applications. PNG images andstructured data can be used for Python clients and applications that use REST APIs.See Also: About Embedded Python Execution1-3

Chapter 1Transparently Convert Python to SQLTransparently Convert Python to SQLWith the transparency layer classes, you can convert select Python objects to Oracledatabase objects and also invoke a range of familiar Python functions that areoverloaded to invoke the corresponding SQL on tables in the database.The OML4Py transparency layer does the following: Contains functions that convert Python pandas.DataFrame objects to databasetables Overloads Python functions, translating their functionality into SQL Leverages proxy objects for database data Uses familiar Python syntax to manipulate database dataThe following table lists the transparency layer functions.Table 1-1Transparency Layer FunctionsFunctionDescriptionoml.createCreates a table in a the database schema from a Python data set.oml object.pullCreates a local Python object that contains a copy of data referencedby the oml object.oml.pushPushes data from a Python session into an object in a databaseschema.oml.syncCreates a DataFrame proxy object in Python that represents adatabase table or view.oml.dirReturn the names of oml objects in the Python session workspace.oml.dropDrops a persistent database table or view.Transparency layer proxy classes map SQL data types or objects to correspondingPython types. The classes provide Python functions and operators that are the sameas those on the mapped Python types. The following table lists the transparency layerdata type classes.Table 1-2Transparency Layer Data Type ClassesClassDescriptionoml.BooleanA boolean series data class that represents a single column of 0, 1, andNULL values in database data.oml.BytesA binary series data class that represents a single column of RAW or BLOBdatabase data types.oml.FloatA numeric series data class that represents a single column of NUMBER,BINARY DOUBLE, or BINARY FLOAT database data types.oml.StringA character series data class that represents a single column of VARCHAR2,CHAR, or CLOB database data types.oml.DataFrame A tabular DataFrame class that represents multiple columns ofoml.Boolean, oml.Bytes, oml.Float, and oml.String data.1-4

Chapter 1About the Python Components and Libraries in OML4PyThe following table lists the mappings of OML4Py data types for both the reading and writingof data between Python and the database.Table 1-3Python and SQL Data Type EquivalenciesDatabase ReadPython Data TypesDatabase WriteN/ABooleanIf oranumber True, then NUMBER (thedefault), else BINARY DOUBLE.BLOBRAWbytesBLOBRAWBINARY DOUBLEBINARY FLOATNUMBERfloatIf oranumber True, then NUMBER (thedefault), else BINARY DOUBLE.CHARCLOBVARCHAR2strCHARCLOBVARCHAR2About the Python Components and Libraries in OML4PyOML4Py requires an installation of Python, a number of Python libraries, as well as theOML4Py components. In Oracle Autonomous Database, OML4Py is already installed. The OML4Py installationincludes Python, additional required Python libraries, and the OML4Py servercomponents. A Python interpreter is included with Oracle Machine Learning Notebooks inAutonomous Database. You can install OML4Py in an on-premises Oracle Database. In this case, you mustinstall Python, the additional required Python libraries, the OML4Py server components,and an OML4Py client. See Install OML4Py for On-Premises Databases.Python Version in Current Release of OML4PyThe current release of OML4Py is based on Python 3.9.5.This version is in the current release of Oracle Autonomous Database. You must install itmanually when installing OML4Py on an on-premises Oracle Database.Required Python LibrariesThe following Python libraries must be included. cx Oracle 8.1.0 cycler 0.10.0 joblib 1.1.0 kiwisolver 1.1.0 matplotlib 3.3.3 numpy 1.21.5 pandas 1.3.41-5

Chapter 1About the Python Components and Libraries in OML4Py Pillow-8.2.0 pyparsing 2.4.0 python-dateutil 2.8.1 pytz 2019.3 scikit-learn 1.0.1 scipy 1.7.3 six 1.13.0 threadpoolctl 2.1.0All the above libraries are included with Python in the current release of OracleAutonomous Database.For an installation of OML4Py in an on-premises Oracle Database, you must installPython and additionally the libraries listed here. See Install OML4Py for On-PremisesDatabases.1-6

2Install OML4Py Client for Linux for Use WithAutonomous DatabaseYou can install and use the OML4Py client for Linux to work with OML4Py in an OracleAutonomous database.OML4Py on premises runs on 64-bit platforms only. For supported platforms see OML4Py OnPremises System Requirements.The following instructions tell you how to download install Python, configure yourenvironment, install manage your client credentials, install Oracle Instant Client, and installthe OML4Py client:1.Download the Python 3.9.5 source and untar it:wget .5.tar.xztar xvf Python-3.9.5.tar.xz2.OML4Py requires the presence of the perl-Env, libffi-devel, openssl, openssl-devel,tk-devel, xz-devel, zlib-devel, bzip2-devel, readline-devel, libuuid-devel andncurses-devel libraries. Confirm that they exist and install any that are missing, velreadline-devellibuuid-develncurses-develIf nothing is returned, install via yum:sudo yum install perl-Env libffi-devel openssl openssl-devel tk-devel xzdevel zlib-devel bzip2-devel readline-devel libuuid-devel ncurses-develNote:RPMs must be installed under sudo, or root.2-1

Chapter 23.To build Python, enter the following commands, where PREFIX is the directory inwhich you installed Python-3.9.5. Use make altinstall to avoid overriding thesystem default's Python installation.export PREFIX pwd /Python-3.9.5cd PREFIX./configure --prefix PREFIX --enable-sharedmake clean; makemake altinstall4.Set environment variable PYTHONHOME and add it to your PATH, and set environmentvariable LD LIBRARY PATH:export PYTHONHOME PREFIXexport PATH PYTHONHOME/bin: PATHexport LD LIBRARY PATH PYTHONHOME/lib: LD LIBRARY PATHCreate a symbolic link in your PYTHONHOME/bin directory. You need to link it toyour python3.9 executable, which you can do with the following commands:cd PYTHONHOME/binln -s python3.9 python3You can now start Python with the python3 script: python3pip will return warnings during package installation if the latest version is notinstalled. You can upgrade the version of pip to avoid these warnings:python3 -m pip install --upgrade pip5.Install the Oracle Instant Client for Autonomous Database, as follows:Download the Oracle Instant Client for your system. Go to the Oracle Instant ClientDownloads page and select Instant Client for Linux x86-64. For more instructionsee Install Oracle Instant Client for Linux for On-Premises Databases.For instruction on installing the Oracle instant client for on-premises see InstallOML4Py Client for On-Premises Databases.If you have root access to install an RPM on the client system. Alternatively, youcan also download the zip file installer, unzip the file, and add the location of theunzipped file to LD LIBRARY PATH as done in next section.wget https://download.oracle.com/otn tclient19.14-basic-19.14.0.0.0-1.x86 64.rpmrpm -ivh oracle-instantclient19.14-basic-19.14.0.0.0-1.x86 64.rpmexport LD LIBRARY PATH /usr/lib/oracle/19.14/client64/lib: LD LIBRARY PATH2-2

Chapter 2If you do not have root access to install an RPM on the client system.wget https://download.oracle.com/otn -basic-linux.x64-19.14.0.0.0dbru.zipunzip xport LD LIBRARY PATH /path/to/instantclient 19 4: LD LIBRARY PATH6.Download the client credentials (wallet) from your Autonomous database. Create adirectory for the Wallet contents. Unzip the wallet zip file to the newly created directory:Note:An mTLS connection using the client Wallet is required. TLS connections arenot currently supported.mkdir -p mywalletdirunzip Wallet.name.zip -d mywalletdir cd mywalletdir/ lsREADMEewallet.p12ojdbc.propertiescwallet.sso keystore.jks sqlnet.ora7.tnsnames.oratruststore.jksUpdate sqlnet.ora with the wallet location. If you're working behind a proxy firewall, setthe SQLNET.USE HTTPS PROXY environment variable to on:WALLET LOCATION (SOURCE (METHOD file) (METHOD DATA (DIRECTORY "mywalletdir")))SSL SERVER DN MATCH yesSQLNET.USE HTTPS PROXY on8.Add proxy address information to all service levels in tnsnames.ora, and add theconnection pools for all service levels. If you are behind a firewall, enter the proxyaddress and port number to all service levels in tnsnames.ora. You will also need to addthree new entries for the AutoML connection pools as shown below.Note:If the proxy server contains a firewall to terminate connections within a set timeperiod, the database connection will also be terminated.For example, myadb medium pool is another alias for the connection string withSERVER POOLED added to the corresponding one for myadb medium.myadb low (description (retry count 20)(retry delay 3)(address (https proxy your proxy address here)(https proxy port 80)(protocol tcps)(port 1522)(host adb.us-sanjose-1.oraclecloud.com))(connect data (service name qtraya2braestch myadb medium.adb.oraclecloud.com2-3

Chapter 2))(security (ssl server cert dn "CN adb.ussanjose-1.oraclecloud.com,OU Oracle ADB SANJOSE,O OracleCorporation,L Redwood City,ST California,C US")))myadb medium (description (retry count 20)(retry delay 3)(address (https proxy your proxy address here)(https proxy port 80)(protocol tcps)(port 1522)(host adb.us-sanjose-1.oraclecloud.com))(connect data (service name qtraya2braestch myadb medium.adb.oraclecloud.com))(security (ssl server cert dn "CN adb.ussanjose-1.oraclecloud.com,OU Oracle ADB SANJOSE,O OracleCorporation,L Redwood City,ST California,C US")))myadb high (description (retry count 20)(retry delay 3)(address (https proxy your proxy address here)(https proxy port 80)(protocol tcps)(port 1522)(host adb.us-sanjose-1.oraclecloud.com))(connect dat

About Oracle Machine Learning for Python. What Is Oracle Machine Learning for Python?1-1. Advantages of Oracle Machine Learning for Python1-2. Transparently Convert Python to SQL1-4. About the Python Components and Libraries in OML4Py1-5. Install OML4Py Client for Linux for Use With Autonomous Database. Install OML4Py for On-Premises Databases