net.heartsavior.spark.sql.checkpoint
CheckpointUtil
Related Doc:
package checkpoint
Create savepoint from existing checkpoint.
Create savepoint from existing checkpoint.
OffsetLog and CommitLog will be purged based on newLastBatchId.
Use additionalMetadataConf to modify metadata configuration: you may want to modify it
when rescaling state, or migrate state format version.
e.g. when rescaling, pass Map(SQLConf.SHUFFLE_PARTITIONS.key -> newShufflePartitions.toString)
spark session
the root path of existing checkpoint
the root path of new savepoint - target directory should be empty
the new last batch ID - it needs to be one of committed batch ID
the configuration to add to existing metadata configuration
whether to exclude state directory
Providing features to deal with checkpoint, like creating savepoint.