Skip to main content

Feature Details

The Cross-Cluster Replication (CCR) feature in Doris is primarily used to efficiently synchronize data across multiple clusters, enhancing business continuity and disaster recovery. CCR supports various operations in Doris, ensuring data consistency across different clusters. Below are the details of the main Doris operations supported by CCR.

note
  1. The - in Doris Version indicates Doris 2.0 or higher versions, supporting all CCR versions. It is recommended to use Doris 2.0.15 or 2.1.6 or later versions.
  2. CCR Syncer and Doris version requirements: Syncer Version >= downstream Doris Version >= upstream Doris Version. Therefore, first upgrade Syncer, then the downstream Doris, and finally the upstream Doris.
  3. CCR currently does not support the storage-compute decoupling mode.

Database​

Database Properties​

Database-level tasks synchronize database properties during Full Sync.

PropertySupportedDoris VersionSync MethodDescription
replication_allocationSupported-Full Sync
data quotaNot Supported
replica quotaNot Supported

Modifying Database Properties​

CCR tasks do not synchronize database property modification operations.

PropertySupportedUpstream Can ModifyDownstream Can ModifyDescription
replication_allocationNot SupportedNoNoModifications cause CCR task failure
data quotaNot SupportedYesYes
replica quotaNot SupportedYesYes

Renaming Database​

Renaming is not supported for upstream and downstream. Doing so might cause views to stop working.

Table​

Table Properties​

PropertySupportedDoris VersionSync MethodDescription
Table model (duplicate, unique, aggregate)Supported-SQL
Partition and bucketSupported-SQL
replication_numSupported-SQL
replication_allocation (resource group)Supported-SQLUpstream and downstream must match; BE tags must match, otherwise the CCR task will fail
colocate_withNot Supported
storage_policyNot Supported
dynamic_partitionSupported-SQL
storage_mediumSupported-SQL
auto_bucketSupported-SQL
group_commit seriesSupported-SQL
enable_unique_key_merge_on_writeSupported-SQL
enable_single_replica_compactionSupported-SQL
disable_auto_compactionSupported-SQL
compaction_policySupported-SQL
time_series_compaction seriesSupported-SQL
binlog seriesSupported-SQL
variant_enable_flatten_nestedSupported-SQL
skip_write_index_on_loadSupported-SQL
row_store seriesSupported-SQL
seq columnSupported-SQL
enable_light_schema_changeSupported-SQL
compression_typeSupported-SQL
indexSupported-SQL
bloom_filter_columnsSupported-SQL
bloom_filter_fppSupported
storage_cooldown_timeNot Supported
generated columnSupported-SQL
Auto-increment IDNot SupportedIssues with this feature

Basic Table Operations​

OperationSupportedDoris VersionSync MethodDownstream Can Operate IndependentlyDescription
create tableSupported-SQL/Partial SyncNoTable creation synced by SQL or partial sync depending on certain settings
drop tableSupported-SQL/Full SyncNo2.0.15/2.1.6 and earlier use Full Sync, later use SQL
rename tableNot Supported2.1.8/3.0.4SQLNoRenaming will stop the CCR task
replace tableSupported2.1.8/3.0.4SQL/Full SyncNoFull Sync triggered for table-level tasks
truncate tableSupported-SQLNo
restore tableNot Supported

Modifying Table Properties​

Sync method is SQL.

PropertySupportedDoris VersionUpstream Can ModifyDownstream Can ModifyDescription
colocateNot SupportedYesNoTriggering Full Sync causes data loss at downstream
distribution typeNot SupportedNoNo
dynamic partitionNot SupportedYesNo
replication_numNot SupportedNoNo
replication_allocationNot SupportedNo
storage policyNot SupportedNoNo
enable_light_schema_changeNot SupportedCCR can only sync light schema changes
row_storeSupported2.1.8/3.0.4Through Partial Sync
bloom_filter_columnsSupported2.1.8/3.0.4Through Partial Sync
bloom_filter_fppSupported2.1.8/3.0.4Through Partial Sync
bucket numNot SupportedYesNo
isBeingSyncedNot SupportedNoNo
compaction seriesNot SupportedYesNo
skip_write_index_on_loadNot SupportedYesNo
seq columnSupported-YesNo
delete sign columnSupported-YesNo
commentSupported2.1.8/3.0.4YesNo

Column Operations​

Base Index Column Operations

OperationSupportedDoris VersionSync MethodDownstream Can Operate IndependentlyDescription
add key columnSupported-Partial SyncNo
add value columnSupported-SQLNo
drop key columnSupported-Partial SyncNo
drop value columnSupported-SQLNo
modify columnSupported-Partial SyncNo
order bySupported-Partial SyncNo
renameSupported2.1.8/3.0.4SQLNo
commentSupported2.1.8/3.0.4SQLNo
note

add/drop value column requires setting "light_schema_change" = "true" during table creation.

Rollup Index Column Operations

OperationSupportedDoris VersionSync MethodDescription
add key columnSupported2.1.8/3.0.4Partial Sync
add value columnSupported2.1.8/3.0.4SQLNeeds light schema change enabled
drop columnSupported2.1.8/3.0.4Partial Sync
modify columnUnknown2.1.8/3.0.4Partial SyncDoris does not support directly modifying rollup column types
order bySupported2.1.8/3.0.4Partial Sync

Rollup Operations​

OperationSupportedDoris VersionSync MethodDescription
add rollupSupported2.1.8/3.0.4

| Partial Sync | | | drop rollup | Supported | 2.1.8/3.0.4 | SQL | | | rename rollup | Supported | 2.1.8/3.0.4 | SQL | |

Index Operations​

Inverted Index

OperationSupportedDoris VersionSync MethodDescription
create indexSupported2.1.8/3.0.4Partial Sync
drop indexSupported2.1.8/3.0.4SQL
build indexSupported2.1.8/3.0.4SQL

Bloom Filter

OperationSupportedDoris VersionSync MethodDescription
add bloom filterSupported2.1.8/3.0.4Partial Sync
alter bloom filterSupported2.1.8/3.0.4Partial SyncRefers to modifying bloom_filter_columns
drop bloom filterSupported2.1.8/3.0.4Partial Sync

Data​

Import Operations​

Import MethodSupportedDoris VersionSync MethodDownstream Can Operate IndependentlyDescription
stream loadSupported (except temp partitions)-TXNNoData is visible when the upstream transaction is committed
broker loadSupported (except temp partitions)-TXNNoSame as stream load
routine loadSupported (except temp partitions)-TXNNoSame as stream load
mysql loadSupported (except temp partitions)-TXNNoSame as stream load
group commitSupported (except temp partitions)2.1TXNNoSame as stream load

Data Operations​

OperationSupportedDoris VersionSync MethodDownstream Can Operate IndependentlyDescription
deleteSupported-TXNNoData is visible in binlog when committed, downstream starts syncing
updateSupported-TXNNoSame as delete
insertSupported-TXNNoSame as delete
insert into overwriteSupported (except temp partitions)2.1.6Partial SyncNoSame as insert
insert into overwriteSupported (except temp partitions)2.0Full SyncNoSame as insert

Partition Operations​

OperationSupportedDoris VersionSync MethodDownstream Can Operate IndependentlyDescription
add partitionSupported-SQLNoTriggers Full Sync or Partial Sync, causing data loss in downstream
add temp partitionNot SupportedNoTemporary partitions not supported for backup; workaround in newer versions
drop partitionSupported-SQL/Full SyncNo2.0.15/2.1.6 and earlier use Full Sync, later use SQL
replace partitionSupported2.1.7/3.0.3Partial SyncNoOnly supports strict range and non-tmp partitions for replace
modify partitionNot SupportedNoModifying partition properties is not supported
rename partitionSupported2.1.8/3.0.4SQLNo

View Operations​

OperationSupportedDoris VersionSync MethodDescription
create viewSupported-SQLWorks if upstream and downstream names match
alter viewSupported2.1.8/3.0.4SQL
drop viewSupported2.1.8/3.0.4SQL
note

Due to Doris limitations, column names or view names cannot match the database name.

Materialized Views​

Synchronized Materialized View

OperationSupportedDoris VersionSync MethodDescription
create materialized viewSupported2.1.8/3.0.4Partial SyncSame name at upstream and downstream works; different names require manual rebuild at downstream
drop materialized viewSupported2.1.8/3.0.4SQL

Asynchronous Materialized View

OperationSupported
create async materialized viewNot Supported
alter async materialized viewNot Supported
drop async materialized viewNot Supported
refreshNot Supported
pauseNot Supported
resumeNot Supported

Statistics​

Statistics are not synchronized between upstream and downstream, and work independently.

Other Operations​

OperationSupported
external tableNot Supported
recycle binNot Supported
catalogNot Supported
workload groupNot Supported
jobNot Supported
functionNot Supported
policyNot Supported
userNot Supported
cancel alter jobSupported