CONNX INTEGRATION WITH MICROSOFT SQL SERVER - Generix

Transcription

ISSUE 2 · APRIL 2018CONNXINTEGRATION WITHMICROSOFT SQL SERVER Larry McGhaw, VP, R&D CONNX, Software AGTechie DeepDiveSQL Server linked serversSQL Server’s linked server feature enables fast and easyintegration of SQL Server data and non-SQL Server data, directlyin the SQL Server engine itself. It has many technical features.Each is individually described in various locations in SQL Serverdocumentation or knowledge base articles. This guide is anattempt to provide context to these items. Also, we describe thereasons why certain options exist and their ramifications.Building a linked server connectionBuilding a linked server connection requires a couple ofingredients:1. SQL Server2. A third-party driver, OLE DB or ODBC for the database to beintegrated with SQL Server, such as CONNXUsing CONNX, SQL Server easily integrates with all of the datasources supported, including Adabas. Linked servers can becreated in two ways: either via the SQL Server ManagementStudio; or via a stored procedure called sp addlinkedserver. Ifthere is a need to dynamically create linked servers via a script,then the stored procedure route is better. Please refer to theexisting detailed SQL Server documentation on this storedprocedure if desired. Using the SQL Server Management Studio,linked servers can be found under “Security Objects." Add anew linked server by right-clicking on “Linked Servers." And thenselect “New Linked Server ”Figure 1: Creating a new linked server from the Management StudioAfter selecting the name of your linked server connection, youmust decide what type of driver to use to access the non-SQLServer database.Microsoft is moving away from OLE DB and standardizing onODBC again for access to external data sources. Select the“Microsoft OLE DB Provider for ODBC Driver."Once the driver has been selected, fill in the remaining fields. The“Product Name” field is not used for any technical purpose. Butit is required to complete the linked server connection. Pleaserefer to the documentation of the third-party driver for details onwhat to supply for the remaining fields. In the provider string field,specify the CONNX driver, along with the name of the CDD, andany desired application filters.SOFTWARE AG TECHNIQUES

CONNX Integration with SQL ServerFigure 2: Specifying the linked server driver connection propertiesFigure 3: Specifying the authentication method for the linked server connectionDon’t press the “OK” button yet. Important authentication andsecurity options are located on the “Security” page. This canbe selected by clicking Security on the upper left hand side ofthe dialog. Some databases do not require a username andpassword to connect. In these cases, select the option “Be madewithout using a security context." However, most databasesrequire a username and password. This makes the process ofconnecting to them through SQL server a bit challenging. Acase in point: a SQL Server is configured to use Microsoft ActiveDirectory logons. But the other database does not supportthis type of logon. If the desired data source does not supportactive directory authentication, the mismatch in authenticationcredentials would make it impossible to connect. The solution tothe problem is provided on the following dialog. Here, a staticset of credentials for the other database will be used for all linkedserver connections. Select “Be made using this security context."Then provide the appropriate username and password for theother database.If at all possible, do not use a single set of authenticationcredentials for all linked server connections. Instead, a separatemapping is established for each SQL Server logon by pressing the“Add” button on the Security page. The “impersonate” optionis just another word for using Active Directory authenticationfor the other database – do not select this checkbox. Additionalimportant options can be found by clicking “Server Options” inthe upper left area of the “New Linked Server” dialog.Figure 4: Specifying the linked server “server options”2

CONNX Integration with SQL ServerThe following table lists each of the options and its function:Collation CompatibleIf the codepage and sort order of theother database is the same a SQLServer, enable this option for betterperformance. By default this optionis not enabled. This means that SQLServer will re-sort all data returnedfrom the third-party driver. Thisoption should be turned on.Data AccessEnables or disables the use ofqueries with the third-party driver.Obviously, this needs to be turnedon.RPC & RPC OUTEnables the use of storedprocedures stored within the otherdatabase. This option should beturned on.Use Remote CollationIf the collation of a column in theother database is different than thedefault collation on the SQL Server,SQL Server will honor the columncollation as specified by the linkedserver instead of re-sorting thedata using the default SQL Servercollation if this option is enabled.Collation NameThe collation of the other databasecan be specified here if it differs fromSQL Server.Connection Timeout &Query TimeoutTimeout optionsDistributor, Publisher,SubscriberThese options are typically used inSQL server to SQL Server or Oracle to SQL Server replication.Lazy Schema ValidationIf set to true: schema validationoccurs at execution time versesprepare time, when it is set to false.For most third-party drivers, this hasno performance impact.Enable promotion ofdistributed transactionsfor RPCWhen enabled, SQL Server willautomatically start a distributedtransaction when a stored procedureis executed via the third-party driver.Accessing your non-SQL server dataThere are four ways to access data with CONNX with SQL Server:two ways use linked servers, and two ways do not.Figure 5: Tree view of tables available via a linked server connectionThe entire process of creating a linked server can be skippedby using OPENDATASOURCE and OPENROWSET. Both ofthese options enable direct access to the CONNX driver viaSQL without any prior linked server creation. When using thesetwo methods, there are many driver configuration options thatare not available for configuration such as collation compatibly.Additionally, both of these ad-hoc query methods are disabled bydefault in SQL Server. To enable these ad-hoc methods executethe following commands:sp configure 'show advanced options', 1;RECONFIGURE;Gosp configure 'Ad Hoc Distributed Queries', 1;RECONFIGURE;GoThe potential danger in enabling this option is that now any usercan access data from any OLE DB Provider or ODBC Driver onThe important options here are “Collation Compatible” andthe SQL Server not just the ones that have been specifically“Data Access." “Collation Compatible” improves performance. It defined as linked servers. However the option is available and it isshould be enabled. “Data Access” must be enabled for the linked up to each admin to determine the risk of these features based onserver to work with queries.the project and security needs.Once all options for the new linked server have been set, pressOK to create the linked server. To validate connectivity to thelinked server, double click on the linked server name in SQLServer Management Studio. Then, expand the catalog until youcan see your tables.Here is an example of the use of OPENDATASOURCE:SELECT *FROM OPENDATASOURCE('MSDASQL','driver connx32;DD c:\cdd\demo4.cdd;application adabas;UID xxxxx;PWD xxxxx').adalocal.connx75.employees3

CONNX Integration with SQL ServerIf your third-party driver does not support catalog or ownernames, leave those portions empty. For example if a providerdoes not support catalog names, but does support owner names,the syntax would look like this:Select * from MyLinkedServer.richard.productsFor a driver that does not support either catalog names, or ownernames, it would look like this:Select * from MyLinkedServer productsAnd for a fully compliant driver that supports both catalog andowner names, like CONNX, it would look like this:Select * from MyLinkedServer.oregon.richard.productsFigure 6: Example of the OpenDatasource statementAnd a corresponding example of the use of OPENROWSET:SELECT a.*FROM OPENROWSET('MSDASQL','driver connx32;DD c:\cdd\demo4.cdd;APPLICATION ADABAS;UID XXX;PWD XXX;','SELECT e.first name, e.name, en.LANGfrom adalocal.employees e inner joinadalocal.employees lang enon (e.isn en.isn)where e.city ''PARIS''ORDER BY e.name DESC') AS a;The advantage of the four-part name syntax is that it is the easiestand most compact to write. Users can leverage their existingknowledge of SQL Server syntax, and mix SQL Server and nonSQL Server tables in a single SQL statement with ease. The hugedisadvantage of using the four-part name syntax is performance.When performing any type of complex SQL operation on multipletables in the linked server connection, SQL Server will typicallydownload a snapshot each table into SQL server. And, then itperforms the filtering and joins operations on those snapshots.For large tables this is a performance bottleneck. The way toavoid this is to use the fourth and final option of data access – theOPENQUERY syntax.Using the OPENQUERY syntax, a complex sub-component of alarger SQL statement can be sent directly to the third-party driver.This increases performance. An example of the syntax is shownhere:SELECT a.*FROM OPENQUERY(ADABAS,'SELECT e.first name, e.name, en.LANGfrom adalocal.employees e inner joinadalocal.employees lang enon (e.isn en.isn)where e.city ''PARIS''ORDER BY e.name DESC ') AS a;Figure 7: Example of the OpenRowset statementAccessing data with linked serverobjectsThe remaining two ways to query data via a third-party driverinvolve using linked server objects. The first method involvesusing a four-part name with the following syntax: Linked Server Name . Object Catalog . ObjectOwner . Object Name 4Figure 8: Example of the OpenQuery statement

CONNX Integration with SQL ServerAlternatively, the exec syntax can be used, providing the ability tospecify parameters:exec ('SELECT e.first name, e.name, en.LANGfrom adalocal.employees e inner joinadalocal.employees lang enon (e.isn en.isn)where e.city ?ORDER BY e.name DESC ', 'PARIS') at ADABAS;Global OLE DB provider optionsDynamic ParameterEnables SQL server to sendparameterized queries to the OLEDBprovider. It can result in greaterperformance. This option should beselected.Nested QueriesEnables SQL Server to send “nested”queries. The table name is actuallya sub-select, offloading the queryprocessing to the third-party provider.Many third-party providers do notsupport nested queries. So, typically,this option is turned off.Level Zero OnlyZero level refers to “basic” OLEDB query interfaces. Most OLE DBproviders only implement these basicquery interfaces. This option shouldnot be selected.Allow In ProcessThis option affects both memoryusage and performance to a largeextent. “In process” means the OLEDB provider will be loaded in thesame process as the SQL Serverengine. The advantage of In processis performance. The disadvantageis that SQL Server and the OLEDB provider are fighting for thesame process virtual memory. SQLServer typically reserves all but asmall amount of memory for itself.If your OLE DB provider is runningout of memory, you may need toadd or adjust the “-g” SQL Servercommand line option. The “-g”memory to reserve option indicates,in megabytes, how much memorySQL Server will leave alone for thirdparty drivers and components touse. The default amount is 256M.Depending on how heavily linkedservers are used, this may need to beincreased. If the “Allow In Process”is not enabled, then SQL Server willcreate a surrogate process and loadthe third-party driver in that processinstead. This gives the third-partydriver its own separate memoryspace. But, it also results in slowerperformance due to the time it takesto transfer data from the surrogateprocess to the main SQL Serverprocess and also the load time of thedriver in the surrogate process. Thesurrogate process provides completeisolation of the third-party driver andthe SQL Server database engine. Soany instability in the driver will notaffect SQL Server itself.Each OLE DB provider has eight global options that can beconfigured for all linked servers for that provider.Figure 9: Accessing global properties for an OLE DB ProviderSelect MSDASQL, as this is the short name for the OLEDBprovider for ODBC drivers.Figure 10: Global properties for an OLE DB provider5

CONNX Integration with SQL ServerAllow In ProcessAdditionally, from a Windows security perspective, drivers that areloaded under the surrogate processhave the impersonated securitycontext of the SQL server user. Thedriver may not have access to thephysical resources on the SQL Servercomputer required of the driver.Drivers loaded in process have aWindows security context of the SQLServer engine itself. This is usuallysufficient to access any resourcesrequired by the driver.Non TransactedUpdatesBy default, SQL Server uses MicrosoftDistributed Transaction Coordinator(MSTDS) to ensure that any updatesmade to date in the third party driverare properly synced to SQL Servertransactions. This is typically calleda two-phase commit. If your thirdparty does not support distributedtransactions (this option is onlyavailable with OLE DB providers,not ODBC driver), and you still wantto allow updates and take the riskof SQL Server commits not being insync with commits to you third-partydriver, enable this option.Index as Access PathThis option is typically for very basicOLE DB providers that don’t supportSQL. But they do support accessingdata via indexes. This option isnormally not enabled.Disallow AdhocAccessEnable this option if you want toprevent access to the linked server.Supports ‘Like’OperatorEnable this option if your third partydriver supports the like operator.Most providers for modern databasessupport this. This option should beselected.In summary, SQL Server provides many ways of integratingCONNX data sources (such as Adabas) and SQL Server datatogether. Each has its own set of advantages and disadvantages.The security and business needs of each project should beevaluated against the available options to determine what is bestfor the specified use case.To learn more, visit www.connx.com.ABOUT SOFTWARE AGSoftware AG (Frankfurt TecDAX: SOW) helps companies with their digital transformation. With Software AG’s Digital Business Platform, companies can better interact with their customersand bring them on new ‘digital’ journeys, promote unique value propositions, and create new business opportunities. In the Internet of Things (IoT) market, Software AG enables enterprisesto integrate, connect and manage IoT components as well as analyze data and predict future events based on Artificial Intelligence (AI). The Digital Business Platform is built on decades ofuncompromising software development, IT experience and technological leadership. Software AG has more than 4,500 employees, is active in 70 countries and had revenues of 879 millionin 2017. To learn more, visit www.softwareag.com. 2018 Software AG. All rights reserved. Software AG and all Software AG products are either trademarks or registered trademarks of Software AG.Other product and company names mentioned herein may be the trademarks of their respective owners.SAG CONNX Integration With SQL TECHniques Apr18

SQL Server linked servers SQL Server's linked server feature enables fast and easy integration of SQL Server data and non-SQL Server data, directly in the SQL Server engine itself. It has many technical features. Each is individually described in various locations in SQL Server documentation or knowledge base articles. This guide is an