HUDI_META

Description

hudi_meta table-valued-function(tvf), using for read hudi metadata, operation history, timeline of table, instant state etc.

Syntax

hudi_meta(
  "table" = "ctl.db.tbl", 
  "query_type" = "timeline"
  ...
  );

parameter description

Each parameter in hudi_meta tvf is a pair of "key"="value".

Related parameters:

• table： (required) Use hudi table name the format catlog.database.table.
• query_type： (required) The type of hudi metadata. Only timeline is currently supported.

Example

Read and access the hudi tabular metadata for timeline.

select * from hudi_meta("table" = "ctl.db.tbl", "query_type" = "timeline");

Can be used with desc function :

desc function hudi_meta("table" = "ctl.db.tbl", "query_type" = "timeline");

Keywords

hudi_meta, table-valued-function, tvf

Best Practice

Inspect the hudi table timeline :

select * from hudi_meta("table" = "hudi_ctl.test_db.test_tbl", "query_type" = "timeline");
+-------------------+--------+--------------------------+-----------+-----------------------+
| timestamp         | action | file_name                | state     | state_transition_time |
+-------------------+--------+--------------------------+-----------+-----------------------+
| 20240724195843565 | commit | 20240724195843565.commit | COMPLETED | 20240724195844269     |
| 20240724195845718 | commit | 20240724195845718.commit | COMPLETED | 20240724195846653     |
| 20240724195848377 | commit | 20240724195848377.commit | COMPLETED | 20240724195849337     |
| 20240724195850799 | commit | 20240724195850799.commit | COMPLETED | 20240724195851676     |
+-------------------+--------+--------------------------+-----------+-----------------------+

Filtered by timestamp :

select * from hudi_meta("table" = "hudi_ctl.test_db.test_tbl", "query_type" = "timeline") 
where timestamp = 20240724195843565;
+-------------------+--------+--------------------------+-----------+-----------------------+
| timestamp         | action | file_name                | state     | state_transition_time |
+-------------------+--------+--------------------------+-----------+-----------------------+
| 20240724195843565 | commit | 20240724195843565.commit | COMPLETED | 20240724195844269     |
+-------------------+--------+--------------------------+-----------+-----------------------+