Transcription
Replication in Firebird 4:concepts and usageDmitry Yemanovdimitr@firebirdsql.orgFirebird Projectwww.firebirdsql.org
2Firebird Conference 2019Berlin, 17-19 OctoberFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN2
3ConceptsInitial goals Built-in replacement for (3rd party) trigger-based solutions Logical (aka record level) replication (*) No need for journal table(s) and triggers (intact schema) Better performance (small overhead, no GC problems) Native support for sequences and DDL operationsFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN3
4ConceptsInitial goals Built-in replacement for (3rd party) trigger-based solutions Logical (aka record level) replication (*) No need for journal table(s) and triggers (intact schema) Better performance (small overhead, no GC problems) Native support for sequences and DDL operations(*) Statement-level for sequence and DDL operationsFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN4
5ConceptsKey features Logical uni-directional replication «PUSH» approach, different transport options Synchronous and asynchronous Simple configuration Customizable replication set Conflict detection, reporting and correction Load balancing (read-only)FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN5
6ConceptsSynchronous replicationDatabaseTransaction 1Transaction 2Transaction NBuffer 1Buffer 2Buffer NTxn 1Txn 2Txn NReplicaFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN6
7ConceptsAsynchronous replication (primary side)DatabaseTransaction 1Transaction 2Transaction NBuffer 1Buffer 2Buffer NJournalArchiveFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN7
8ConceptsAsynchronous replication (replica side)ArchiveTxn 1Txn 2Txn NReplicaFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN8
9ArchitectureUnder the hood Replication is transaction-awareEvery transaction has internal replication buffer(size is configurable, should be balanced)Buffer is flushed upon either: Size overflow Transaction commit / rollback Savepoint rollbackEvery «flush» produces a replication packet (aka «change block»)Blocks are transferred to replica database(s)or written to the journalFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN9
10ArchitectureUnder the hood Buffers are not always flushed synchronously Replication background thread per database Queue of «overflow» blocks to be flushed Lagging is limited to keep the throughput stableFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN10
11ArchitectureSpecifics Both «changes» and «undos» are replicated Savepoint stack is preserved «Undos» are frame-based (using savepoints)Optimization Small rolled back transactions are not flushed,just discardedFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN11
12ArchitectureNever replicated «De facto» read-only transactions External tables Virtual tables Temporary tables Any garbage collection activity System sequences, except RDB BACKUP HISTORY Some DDL commands:ALTER DATABASE, DROP DATABASECREATE SHADOW, DROP SHADOWCREATE USER, ALTER USER, DROP USERFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN12
13ArchitectureError handling Every error is written to replication.log,prefixed with (primary replica) side and database pathnameFor synchronous replication, errors may be duplicatedon the both sidesIf error is persistent and affects user operations,replication is automatically disabled (at least partially)replication.log may also contain warnings,they do not affect the replication flowFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN13
14ArchitectureSynchronous replication Every primary database keeps active connectionsto all the synchronous replica databasesReplication packets are transferredvia native remote protocol and Firebird APIFailed synchronous replica is excluded from replication,others remain workingFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN14
15ArchitectureAsynchronous replication Journal — contiguous sequence of segments Linked to its corresponding database via UUID Segments are uniquely (and sequentially) numbered Change blocks are written one after another,every block has an associated flush timestampOperational and archive journalsFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN15
16ArchitectureOperational journal One or more segments on the primary sidethat are being written to Segments are rotated (with renaming) Segments may have multiple states: FREE — empty segment ready for reuse USED — segment being currently written FULL — segment ready for archiving ARCHIVE — segment being archivedArchieving is a process of copying full segments elsewhere(to apply them to the replica database later)Archive segments are persistent, read-only and not rotatableFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN16
17ArchitectureHow segments are applied to the replica Firebird process creates an embedded connection Journal directory is periodically scanned for new files Found segments are read and processedone after another in the sequence order Segments are removed automatically after applying Replica may be disconnected and reconnected after timeoutFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN17
18ArchitectureHow segments are applied to the replica Replica may be disconnected and reconnected after timeoutSegments containing changes from not yet committed transactionsare preseved until those transactions are finished Markers: Oldest Sequence and Next Sequence : Offset Current state is stored in the {UUID} file After reconnection changes from «unfinished» transactionsare re-applied, other changes up no Next:Offset are skippedThen replication continues in the usual modeFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN18
19ArchitectureLoad balancing on the replica side Replica may be read-only or read-writeIn read-only replica, all transactions started by regular usersare forced to be read-onlyThanks to MGA, readers do not conflict with writers — concurrentreads by users (e.g. reporting) are possible But conflicts are still possible (DDL changes) Read-write replica allows concurrent writes by users Conflicts must be avoided by usersFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN19
20ConfigurationConcepts replication.conf —all settings for both primary and replica sidesPrimary side: parsed and cached when the first connectionattaches to the databaseReplica side: parsed and cached when Firebird is startedCore settings for the primary side include filter, exclude filter —regular expressions for table customizationbuffer size — per transaction caching thresholdFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN20
21ConfigurationSynchronous mode sync replica — connection string to the replica database Multiple entries are allowed:sync replica john:smith@backup1:/my/replica/db1.fdbsync replica ase /your/db.fdb{sync replica sysdba:masterkey@otherhost:/db.fdb}FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN21
22ConfigurationAsynchronous mode (master side) Many options — read replication.conf for detailslog directory, log file prefix, log segment size, log segment count,log group flush delay, log archive directory, log archive command, log archive timeout log directory is required Either log archive directory or log archive command is required Other options are used for tuningExampledatabase /your/db.fdb{log directory /your/db/operlog/log archive directory /your/db/archlog/}FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN22
23ConfigurationAsynchronous mode (replica side) Only log source directory is required Other options are used for tuningExampledatabase /your/db.fdb{log source directory /your/db/incominglog/}FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN23
24Usage patternsHow to start synchronous replication1) Set up replication.conf for your database2) Restart Firebird or reconnect all users3) Check replication.logFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN24
25Usage patternsHow to start asynchronous replication1) Create directory for operational and archive journals(better on a different storage)2) Set up replication.conf for your database3) Restart Firebird or reconnect all users4) Check replication.log5) Ensure journal files are being created and archived properlyFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN25
26Usage patternsHow to set up replica1) Make a file-level copy of the primary database2) Gfix database -replica read onlyIf asynchronous replication is used3) Set up replication.conf for the replica database4) Restart Firebird service5) Ensure journal files are received and processed6) Try verbose logging true for better understandingFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN26
27Usage patternsHow to fix broken replication1) Make a file-level copy of the primary database2) Gfix database -replica read only3) Shutdown the broken replica4) Copy the broken replica elsewhere (or remove it)5) Rename the new replica to the old name6) Do not touch anything else ;-)FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN27
28Usage patternsHow to recover from failure1) For asynchronous replication decide whether to recover right now(but lose some recent changes) or wait for replication to catch up2) Stop replication on the replica side: Shutdown replica database Shutdown Firebird service3) Disable replica side settings in replication.conf4) Gfix database -replica none5) Copy replica to become the new primaryFIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN28
29Questions?FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN29
3 FIREBIRD INTERNATIONAL CONFERENCE '2019 - BERLIN 3 Concepts Initial goals Built-in replacement for (3rd party) trigger-based solutions Logical (aka record level) replication (*) No need for journal table(s) and triggers (intact schema) Better performance (small overhead, no GC problems) Native support for sequences and DDL operations
