WIXLIB

Chapter 1Workflow to Create and Add a Apache Kafka Adapter Connection to an IntegrationWorkflow to Create and Add a Apache Kafka AdapterConnection to an IntegrationYou follow a very simple workflow to create a connection with an adapter and include theconnection in an integration in Oracle Integration.This table lists the workflow steps for both adapter tasks and overall integration tasks, andprovides links to instructions for each step.StepDescriptionMore Information1Access Oracle Integration.Go to https://instance name/ic/home.2Create the adapter connections for Create an Apache Kafka Adapter Connectionthe applications you want tointegrate. The connections can bereused in multiple integrations andare typically created by theadministrator.3Create the integration. When youdo this, you add trigger (source)and invoke (target) connections tothe integration.Create Integrations in Using Integrations in OracleIntegration 3 and Add the Apache Kafka AdapterConnection to an Integration.4Map data between the triggerconnection data structure and theinvoke connection data structure.Map Data in Using Integrations in Oracle Integration 35(Optional) Create lookups thatmap the different values used bythose applications to identify thesame type of object (such asgender codes or country codes).Manage Lookups in Using Integrations in OracleIntegration 36Activate the integration.Activate Integrations in Using Integrations in OracleIntegration 37Monitor the integration on thedashboard.Monitor Integrations in Using Integrations in OracleIntegration 38Track payload fields in messagesduring runtime.Assign Business Identifiers for Tracking Fields inMessages and Manage Business Identifiers for TrackingFields in Messages in Using Integrations in OracleIntegration 39Manage errors at the integrationlevel, connection level, or specificintegration instance level.Manage Errors in Using Integrations in OracleIntegration 3Note: The Apache Kafka Adapter can only be used asan invoke connection to produce and consumeoperations.1-3

2Create an Apache Kafka Adapter ConnectionA connection is based on an adapter. You define connections to the specific cloudapplications that you want to integrate. The following topics describe how to defineconnections.Topics Prerequisites for Creating a Connection Create a ConnectionPrerequisites for Creating a ConnectionYou must satisfy the following prerequisites to create a connection with the Apache KafkaAdapter. Know the Host and Port of the Bootstrap Server Obtain Security Policy Details Configure Confluent Kafka with the Apache Kafka AdapterKnow the Host and Port of the Bootstrap ServerKnow the host and port of the bootstrap server to use to connect to a list of Kafka brokers.Obtain Security Policy DetailsObtain the following security policy details for the Apache Kafka Adapter. If using the Simple Authentication and Security Layer (SASL) Plain over SSL or SASLPlain security policy, know the SASL username and password. To use SASL Plain over SSL, TLS, or Mutual TLS policies, have the required certificates.Configure Confluent Kafka with the Apache Kafka AdapterTo configure Confluent Kafka with the Apache Kafka Adapter, you must obtain the followinginformation to successfully configure the Apache Kafka Adapter on the Connections page.1.Generate the username and password required for the Connections page at the ents/env-dvgny/clusters/lkc-rjn91/api-keys Click Add Key.2-1

Chapter 2Create a ConnectionYou must enter the key in the SASL Username field and the secret key in theSASL Password field on the Connections page. See Configure ConnectionSecurity.2.Generate the truststore:a.Generate the certificate.echo -n openssl s client -connect host:port sed -ne '/BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' /tmp/server.certWhere host:port is the combination of the bootstrap server and port.b.Generate the truststore from the certificate created in Step a:keytool -keystore conf 2.jks -alias ConfRoot -import -fileconf server.certFor this example, conf 2.jks is the name of the truststore file to upload in theTrustStore field on the Connections page.c.When prompted, enter a password. Remember the password because youmust enter it in the Truststore password field on the Connections page.Note the following restrictions: The Apache Kafka Adapter supports Apache Kafka serializers/deserializers(String/ByteArray). It doesn't support Confluent or any other serializers/deserializers. Supports only the SASL PLAIN over SSL security policy. Supports the XML/JSON and AVRO message structures. Other structures/formatsare not supported. The schema registry is not supported with the Apache Kafka Adapter.Create a ConnectionBefore you can build an integration, you have to create the connections to theapplications with which you want to share data.To create a connection in Oracle Integration:1.In the navigation pane, click Design, then Connections.2.Click Create.Note:You can also create a connection in the integration canvas. See DefineInbound Triggers and Outbound Invokes.3.In the Create Connection panel, select the adapter to use for this connection. Tofind the adapter, scroll through the list, or enter a partial or full name in the Searchfield.2-2

Chapter 2Create a Connection4.5.Enter the information that describes this connection.a.Enter a meaningful name to help others find your connection when they begin tocreate their own integrations. The name you enter is automatically added in capitalletters to the Identifier field. If you modify the identifier name, don't include blankspaces (for example, SALES OPPORTUNITY).b.Select the role (direction) in which to use this connection (trigger, invoke, or both).Only the roles supported by the adapter are displayed for selection. When you selecta role, only the connection properties and security policies appropriate to that role aredisplayed on the Connections page. If you select an adapter that supports bothinvoke and trigger, but select only one of those roles, you'll get an error when you tryto drag the adapter into the section you didn't select. For example, assume youconfigure a connection for the Oracle Service Cloud (RightNow) Adapter as only aninvoke. Dragging the adapter to a trigger section in the integration produces anerror.c.Enter optional keywords (tags). You can search on the connection keywords on theConnections page.d.Enter an optional description of the connection.Click Create.Your connection is created. You're now ready to configure the connection details, such asconnection properties, security policies, connection login credentials, and (for certainconnections) agent group.Configure Connection PropertiesEnter connection information so your application can process requests.1.Go to the Properties section.2.In the Bootstrap Servers field, specify the host and port to use to connect to a list ofKafka brokers. A Kafka cluster consists of one or more servers (Kafka brokers) runningKafka. Producers are processes that publish data (push messages) to Kafka topics withinthe broker. A consumer of topics pulls messages from a Kafka topic.Configure Connection SecurityConfigure security for your Apache Kafka Adapter connection by selecting the security policyand security token.1.Go to the Security section.2.Select the security policy.Security PolicyDescriptionMutual TLSMutual Transport Layer Security (TLS) is asecurity practice that uses client TLS certificatesto provide an additional layer of protection thatallows client information to be cryptographicallyverified. Mutual TLS enables the server toauthenticate the identity of the client.2-3

Chapter 2Create a Connection3.Security PolicyDescriptionSASL PLAIN over SSLSimple Authentication and Security Layer(SASL) is a framework for authentication anddata security in Internet protocols. It separatesauthentication mechanisms from applicationprotocols to enable any authenticationmechanism supported by SASL to be used inany application protocol that uses SASL. Plaintext authentication assumes that the user nameand password are submitted to the server inclear text. Therefore, this authentication methodis only considered secure when using anencrypted connection. This security policyenables you to use SASL Plain with SSLencryption.SASL PLAINUse SASL Plain without SSL encryption.TLSTLS is a cryptographic protocol that providesend-to-end security of data sent betweenapplications over the Internet.No Security PolicyDo not use any security policy.Based on your security policy selection, enter the following details:If You Selected.Specify These Details.Mutual TLS SASL PLAIN over SSLThis option enables you to use directconnectivity and eliminates the need to perform the procedures described Configurean Agent Group. TrustStore: Select the check box, thenclick Upload to upload the truststore.KeyStore: Select the check box, thenclick Upload to upload the keystore.TrustStore password and ConfirmTrustStore password: Enter thepassword, then enter it a second time toconfirm.KeyStore password and ConfirmKeyStore password: Enter thepassword, then enter it a second time toconfirm.Key password and Confirm Keypassword: Enter the password, thenenter it a second time to confirm.SASL Username: Enter the SASLusername.SASL Password and Confirm SASLPassword: Enter the password, thenenter it a second time to confirm.TrustStore: Select the check box, thenclick Upload to upload the truststore.KeyStore: Select the check box, thenclick Upload to upload the keystore.TrustStore password and ConfirmTrustStore password: Enter thepassword, then enter it a second time toconfirm.KeyStore password and ConfirmKeyStore password: Enter thepassword, then enter it a second time toconfirm.2-4

Chapter 2Create a ConnectionIf You Selected.Specify These Details.SASL PLAIN TLS SASL Username: Enter the SASLusername.SASL Password and Confirm SASLPassword: Enter the password, thenenter it a second time to confirm.TrustStore: Select the check box, thenclick Upload to upload the truststore.TrustStore password and ConfirmTrustStore password: Enter thepassword, then enter it a second time toconfirm.Configure an Agent GroupConfigure an agent group for accessing the service hosted on your premises behind the firewall.1.Click Configure Agents.The Select an Agent Group page appears.2.Click the name of the agent group.3.Click Use.To configure an agent group, you must download and install the on-premises connectivityagent. See Download and Run the Connectivity Agent Installer and About ConnectivityAgents and Integrations Between On-Premises Applications and Oracle Integration in UsingIntegrations in Oracle Integration 3.Test the ConnectionTest your connection to ensure that it's configured successfully.1.In the page title bar, click Test. What happens next depends on whether your adapterconnection uses a Web Services Description Language (WSDL) file. Only some adapterconnections use WSDLs.If Your Connection.Then.Doesn't use a WSDLThe test starts automatically and validates the inputs you provided for theconnection.Uses a WSDLA dialog prompts you to select the type of connection testing to perform: 2.Validate and Test: Performs a full validation of the WSDL, includingprocessing of the imported schemas and WSDLs. Completevalidation can take several minutes depending on the number ofimported schemas and WSDLs. No requests are sent to theoperations exposed in the WSDL.Test: Connects to the WSDL URL and performs a syntax check onthe WSDL. No requests are sent to the operations exposed in theWSDL.Wait for a message about the results of the connection test. If the test was successful, then the connection is configured properly.2-5

Chapter 2Create a Connection 3.If the test failed, then edit the configuration details you entered. Check fortypos, verify URLs and credentials, and download the diagnostic logs foradditional details. Continue to test until the connection is successful.When complete, click Save.2-6

3Add the Apache Kafka Adapter Connection toan IntegrationWhen you drag the Apache Kafka Adapter into the trigger or invoke area of an integration,the Adapter Endpoint Configuration Wizard is invoked. This wizard guides you throughconfiguration of the Apache Kafka Adapter endpoint properties.The following sections describe the wizard pages that guide you through configuration of theApache Kafka Adapter as a trigger and an invoke in an integration.Topics Basic Info Page Operations Page Topic & Partition Page Message Structure Page Headers Page Summary PageBasic Info PageYou can enter a name and description on the Basic Info page of each adapter in yourintegration.ElementDescriptionWhat do you want to callyour endpoint?Provide a meaningful name so that others can understand theresponsibilities of this connection. You can include English alphabeticcharacters, numbers, underscores, and hyphens in the name. You can'tinclude the following characters: What does this endpointdo?No blank spaces (for example, My Inbound Connection)No special characters (for example, #;83& or righ(t)now4)except underscores and hyphensNo multibyte charactersEnter an optional description of the connection’s responsibilities. Forexample:This connection receives an inbound request tosynchronize account information with the cloudapplication.3-1

Chapter 3Operations PageOperations PageSelect the operation to perform.ElementDescription What operation do youwant to perform on a Kafka topic? Publish records to a Kafka topicConsume records from a Kafka topicConsume records from a Kafka topic by specifyingoffsetTopic & Partition PageSelect the operation and topic on which to perform the operation and optionally specifythe message structure. Configure the Apache Kafka Adapter as an Invoke Configure the Apache Kafka Adapter as a TriggerConfigure the Apache Kafka Adapter as an Invoke ConnectionElementDescriptionSelect a TopicSelect the topic on which to perform the operation. You can alsoenter the beginning letters of the topic to filter the display oftopics. A topic is a category in which applications can add,process, and reprocess messages. You subscribe to messagesin topics.Specify the Partition(This field is only displayed ifyou select Publish recordsto a Kafka topic orConsume records from aKafka topic.)Specify the partition to which to push the selected topic. Kafkatopics are divided into partitions that enable you to split dataacross multiple brokers. If you do not select a specific partitionand use the Default selection, Kafka considers all availablepartitions and decides which one to use.Specify the consumer group to attach.Consumer Group(This field is only displayed if Consumers join a group by using the same group ID. Kafkayou select Consume records assigns the partitions of a topic to the consumers in a group.from a Kafka topic.) Specify the option forconsuming messages(This field is only displayed if you select Consume recordsfrom a Kafka topic.)Read latest: Reads the latest messages starting at thetime at which the integration was activated.Read from beginning: Select to read messages from thebeginning.As an example, if you select to read from the beginningand have activated the integration, the first scheduled runpicks up 20 records and the next scheduled run picks upthe next 20 records. If the integration is then deactivated,edited, and reactivated, the next 20 records are picked up.3-2

Chapter 3Topic & Partition PageElementDescriptionSpecify the number of messages to read. The threshold for theMaximum Number ofRecords to be fetchedcomplete message payload is 10 MB.(This field is only displayed ifyou select Consume recordsfrom a Kafka topic orConsume records from aKafka topic by specifyingoffset.)Do you want to specify themessage structure?Select Yes if you want to define the message structure to useon the Message Structure page of the wizard. Otherwise, selectNo.Do you want to specify theheaders for the message?Select Yes if you want to define the message headers to use onthe Headers page of the wizard. Otherwise, select No.Review and updateadvanced configurationsClick Edit to open the Advanced Options section to enable ordisable the transactional producer.1.2.Transaction Producer: This field is only displayed if youselect Publish records to a Kafka topic. This optionprovides the following capabilities:a.If selected, the transactional producer enables anapplication to send messages to multiple partitionsatomically.b.If not selected, the Apache Kafka Adapter isconfigured as a nontransactional producer.Message Type: This option defines the message type.Available options are String or Bytes. It defines theserializers to use for the message. This selection isapplicable for the message key and value.Configure the Apache Kafka Adapter as a Trigger ConnectionElementDescriptionSelect a TopicSelect the topic on which to perform the operation. You can alsoenter the beginning letters of the topic to filter the display of topics. Atopic is a category in which applications can add, process, andreprocess messages. You subscribe to messages in topics.Specify the PartitionSpecify the partition to which to push the selected topic. Kafka topicsare divided into partitions that enable you to split data acrossmultiple brokers. If you do not select a specific partition and use theDefault selection, Kafka considers all available partitions anddecides which one to use.Consumer GroupSpecify the consumer group to attach.Consumers join a group by using the same group ID. Kafka assignsthe partitions of a topic to the consumers in a group.Polling Frequency (Sec)Specify the frequency at which to fetch records.Maximum Number of Records Specify the number of messages to read. The threshold for theto be fetchedcomplete message payload is 10 MB.Do you want to specify themessage structure?Select Yes if you want to define the message structure to use on theMessage Structure page of the wizard. Otherwise, select No.Do you want to specify theheaders for the message?Select Yes if you want to define the message headers to use on theHeaders page of the wizard. Otherwise, select No.3-3

Chapter 3Message Structure PageElementDescriptionReview and update advancedconfigurationsClick Edit to open the Advanced Options section.1.Message Type: This option defines the message type. Availableoptions are String or Bytes. It defines the serializers to use forthe message. This selection is applicable for the message keyand value.Message Structure PageSelect the message structure to use. This page is displayed if you selected Yes for theDo you want to specify the message structure? field on the Topic & Partition page.ElementDescriptionHow would you like tospecify the messagestructure? Select FileClick Browse to select the file. Once selected, the file name i

Apache Kafka distributed publish-subscribe messaging system from Oracle Integration and allows for the publishing and consumption of messages from a Kafka topic. The Apache Kafka Adapter provides the following benefits: Establishes a connection to the Apache Kafka messaging system to enable messages to be published and consumed.