Skip to main content
Skip to main content

Export Overview

The data export function is used to write the query result set or Doris table data into the specified storage system in the specified file format.

The differences between the export function and the data backup function are as follows:

 Data ExportData Backup
Final Storage LocationHDFS, Object Storage, Local File SystemHDFS, Object Storage
Data FormatOpen file formats such as Parquet, ORC, CSVDoris internal storage format
Execution SpeedModerate (requires reading data and converting to the target data format)Fast (no parsing and conversion required, directly upload Doris data files)
FlexibilityCan flexibly define the data to be exported through SQL statementsOnly supports table-level full backup
Use CasesResult set download, data exchange between different systemsData backup, data migration between Doris clusters

Choosing Export Methods​

Doris provides three different data export methods:

  • SELECT INTO OUTFILE: Supports the export of any SQL result set.
  • EXPORT: Supports the export of partial or full table data.
  • MySQL DUMP: Compatible with the MySQL dump command for data export.

The similarities and differences between the three export methods are as follows:

 SELECT INTO OUTFILEEXPORTMySQL DUMP
Synchronous/AsynchronousSynchronousAsynchronous (submit EXPORT tasks and check task progress via SHOW EXPORT command)Synchronous
Supports any SQLYesNoNo
Export specific partitionsYesYesNo
Export specific tabletsYesNoNo
Concurrent exportSupported with high concurrency (depends on whether the SQL statement has operators such as ORDER BY that need to be processed on a single node)Supported with high concurrency (supports tablet-level concurrent export)Not supported, single-threaded export only
Supported export data formatsParquet, ORC, CSVParquet, ORC, CSVMySQL Dump proprietary format
Supports exporting external tablesYesPartially supportedNo
Supports exporting viewsYesYesYes
Supported export locationsS3, HDFS, LOCALS3, HDFS, LOCALLOCAL

SELECT INTO OUTFILE​

Suitable for the following scenarios:

  • Data needs to be exported after complex calculations, such as filtering, aggregation, joins, etc.
  • Suitable for scenarios that require synchronous tasks.

EXPORT​

Suitable for the following scenarios:

  • Large-scale single table export, with simple filtering conditions.
  • Scenarios that require asynchronous task submission.

MySQL Dump​

Suitable for the following scenarios:

  • Compatible with the MySQL ecosystem, requires exporting both table structure and data.
  • Only for development testing or scenarios with very small data volumes.

Export File Column Type Mapping​

Parquet and ORC file formats have their own data types. Doris's export function can automatically map Doris's data types to the corresponding data types in Parquet and ORC file formats. The CSV format does not have types, all data is output as text.

The following table shows the mapping between Doris data types and Parquet, ORC file format data types:

  1. Doris export to ORC file format data type mapping table:

    Doris TypeOrc Type
    booleanboolean
    tinyinttinyint
    smallintsmallint
    intint
    bigintbigint
    largeIntstring
    datestring
    datev2string
    datetimestring
    datetimev2timestamp
    floatfloat
    doubledouble
    char / varchar / stringstring
    decimaldecimal
    structstruct
    mapmap
    arrayarray
    jsonNot supported
  2. When Doris exports to Parquet file format, it first converts Doris in-memory data to Arrow in-memory data format, then writes out to Parquet file format. The mapping relationship between Doris data types and Arrow data types is:

    Doris TypeArrow Type
    booleanboolean
    tinyintint8
    smallintint16
    intint32
    bigintint64
    largeIntutf8
    dateutf8
    datev2Date32Type
    datetimeutf8
    datetimev2TimestampType
    floatfloat32
    doublefloat64
    char / varchar / stringutf8
    decimaldecimal128
    structstruct
    mapmap
    arraylist