org.apache.hudi.table.upgrade

Class UpgradeDowngrade

java.lang.Object

org.apache.hudi.table.upgrade.UpgradeDowngrade

public class UpgradeDowngrade
extends Object

Helper class to assist in upgrading/downgrading Hoodie when there is a version change.

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieWriteConfig	config
protected HoodieEngineContext	context
static String	HOODIE_UPDATED_PROPERTY_FILE

Constructor Summary

Constructors

Constructor and Description
UpgradeDowngrade(HoodieTableMetaClient metaClient, HoodieWriteConfig config, HoodieEngineContext context, SupportsUpgradeDowngrade upgradeDowngradeHelper)

Method Summary

Modifier and Type	Method and Description
protected Map<ConfigProperty,String>	downgrade(HoodieTableVersion fromVersion, HoodieTableVersion toVersion, String instantTime)
boolean	needsUpgradeOrDowngrade(HoodieTableVersion toWriteVersion)
void	run(HoodieTableVersion toVersion, String instantTime) Perform Upgrade or Downgrade steps if required and updated table version if need be.
protected Map<ConfigProperty,String>	upgrade(HoodieTableVersion fromVersion, HoodieTableVersion toVersion, String instantTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HOODIE_UPDATED_PROPERTY_FILE

public static final String HOODIE_UPDATED_PROPERTY_FILE

See Also:

Constant Field Values

config

protected HoodieWriteConfig config

context

protected HoodieEngineContext context

Constructor Detail

UpgradeDowngrade

public UpgradeDowngrade(HoodieTableMetaClient metaClient,
                        HoodieWriteConfig config,
                        HoodieEngineContext context,
                        SupportsUpgradeDowngrade upgradeDowngradeHelper)

Method Detail

needsUpgradeOrDowngrade

public boolean needsUpgradeOrDowngrade(HoodieTableVersion toWriteVersion)

run

public void run(HoodieTableVersion toVersion,
                String instantTime)

Perform Upgrade or Downgrade steps if required and updated table version if need be.

Starting from version 0.6.0, this upgrade/downgrade step will be added in all write paths.

Essentially, if a dataset was created using an previous table version in an older release, and Hoodie version was upgraded to a new release with new table version supported, Hoodie table version gets bumped to the new version and there are some upgrade steps need to be executed before doing any writes.

Similarly, if a dataset was created using an newer table version in an newer release, and then hoodie was downgraded to an older release or to older Hoodie table version, then some downgrade steps need to be executed before proceeding w/ any writes.

Below shows the table version corresponding to the Hudi release: Hudi release -> table version pre 0.6.0 -> v0 0.6.0 to 0.8.0 -> v1 0.9.0 -> v2 0.10.0 -> v3 0.11.0 -> v4 0.12.0 to 0.13.0 -> v5 0.14.0 to current -> v6

On a high level, these are the steps performed

Step1 : Understand current hoodie table version and table version from hoodie.properties file Step2 : Delete any left over .updated from previous upgrade/downgrade Step3 : If version are different, perform upgrade/downgrade. Step4 : Copy hoodie.properties -> hoodie.properties.updated with the version updated Step6 : Rename hoodie.properties.updated to hoodie.properties

Parameters:

toVersion - version to which upgrade or downgrade has to be done.

instantTime - current instant time that should not be touched.

upgrade

protected Map<ConfigProperty,String> upgrade(HoodieTableVersion fromVersion,
                                             HoodieTableVersion toVersion,
                                             String instantTime)

downgrade

protected Map<ConfigProperty,String> downgrade(HoodieTableVersion fromVersion,
                                               HoodieTableVersion toVersion,
                                               String instantTime)