Packages

  • package root
    Definition Classes
    root
  • package ai
    Definition Classes
    root
  • package catboost
    Definition Classes
    ai
  • package spark

    CatBoost is a machine learning algorithm that uses gradient boosting on decision trees.

    CatBoost is a machine learning algorithm that uses gradient boosting on decision trees.

    Overview

    This package provides classes that implement interfaces from Apache Spark Machine Learning Library (MLLib).

    For binary and multi- classification problems use CatBoostClassifier, for regression use CatBoostRegressor.

    These classes implement usual fit method of org.apache.spark.ml.Predictor that accept a single org.apache.spark.sql.DataFrame for training, but you can also use other fit method that accepts additional datasets for computing evaluation metrics and overfitting detection similarily to CatBoost's other APIs.

    This package also contains Pool class that is CatBoost's abstraction of a dataset. It contains additional information compared to simple org.apache.spark.sql.DataFrame.

    It is also possible to create Pool with quantized features before training by calling quantize method. This is useful if this dataset is used for training multiple times and quantization parameters do not change. Pre-quantized Pool allows to cache quantized features data and so do not re-run feature quantization step at the start of an each training.

    Detailed documentation is available on https://catboost.ai/docs/

    Definition Classes
    catboost
  • package params
    Definition Classes
    spark
  • CatBoostClassificationModel
  • CatBoostClassifier
  • CatBoostPredictorTrait
  • CatBoostRegressionModel
  • CatBoostRegressor
  • Pool

class CatBoostRegressionModel extends RegressionModel[Vector, CatBoostRegressionModel] with CatBoostModelTrait[CatBoostRegressionModel]

Regression model trained by CatBoost. Use CatBoostRegressor to train it

Serialization

Supports standard Spark MLLib serialization. Data can be saved to distributed filesystem like HDFS or local files. When saved to path two files are created: -<path>/metadata which contains Spark-specific metadata in JSON format -<path>/model which contains model in usual CatBoost format which can be read using other local CatBoost APIs (if stored in a distributed filesystem it has to be copied to the local filesystem first).

Examples:

  1. Save model

    val trainPool : Pool = ... init Pool ...
val regressor = new CatBoostRegressor
val model = regressor.fit(trainPool)
val path = "/home/user/catboost_spark_models/model0"
model.write.save(path)
    2. ,

  2. Load model

    val dataFrameForPrediction : DataFrame = ... init DataFrame ...
val path = "/home/user/catboost_spark_models/model0"
val model = CatBoostRegressionModel.load(path)
val predictions = model.transform(dataFrameForPrediction)
predictions.show()
Linear Supertypes
CatBoostModelTrait[CatBoostRegressionModel], MLWritable, RegressionModel[Vector, CatBoostRegressionModel], PredictionModel[Vector, CatBoostRegressionModel], PredictorParams, HasPredictionCol, HasFeaturesCol, HasLabelCol, Model[CatBoostRegressionModel], Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CatBoostRegressionModel
  2. CatBoostModelTrait
  3. MLWritable
  4. RegressionModel
  5. PredictionModel
  6. PredictorParams
  7. HasPredictionCol
  8. HasFeaturesCol
  9. HasLabelCol
  10. Model
  11. Transformer
  12. PipelineStage
  13. Logging
  14. Params
  15. Serializable
  16. Serializable
  17. Identifiable
  18. AnyRef
  19. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CatBoostRegressionModel(nativeModel: TFullModel)
  2. new CatBoostRegressionModel(uid: String, nativeModel: TFullModel = null, nativeDimension: Int)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T
    Attributes
    protected
    Definition Classes
    Params
  4. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. final def clear(param: Param[_]): CatBoostRegressionModel.this.type
    Definition Classes
    Params
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  8. def copy(extra: ParamMap): CatBoostRegressionModel
    Definition Classes
    CatBoostRegressionModel → Model → Transformer → PipelineStage → Params
  9. def copyValues[T <: Params](to: T, extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  10. final def defaultCopy[T <: Params](extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  14. def explainParams(): String
    Definition Classes
    Params
  15. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  16. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  17. final val featuresCol: Param[String]
    Definition Classes
    HasFeaturesCol
  18. def featuresDataType: DataType
    Attributes
    protected
    Definition Classes
    PredictionModel
  19. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  20. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  21. def getAdditionalColumnsForApply: Seq[StructField]
    Attributes
    protected
    Definition Classes
    CatBoostRegressionModel → CatBoostModelTrait
  22. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  23. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  24. final def getFeaturesCol: String
    Definition Classes
    HasFeaturesCol
  25. final def getLabelCol: String
    Definition Classes
    HasLabelCol
  26. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  27. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  28. final def getPredictionCol: String
    Definition Classes
    HasPredictionCol
  29. def getResultIteratorForApply(rawObjectsDataProvider: SWIGTYPE_p_NCB__TRawObjectsDataProviderPtr, dstRows: ArrayBuffer[Array[Any]], threadCountForTask: Int): Iterator[Row]
    Attributes
    protected
    Definition Classes
    CatBoostRegressionModel → CatBoostModelTrait
  30. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  31. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  32. def hasParent: Boolean
    Definition Classes
    Model
  33. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  34. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  35. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  36. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  37. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  38. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  39. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  40. final val labelCol: Param[String]
    Definition Classes
    HasLabelCol
  41. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  42. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  43. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  44. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  45. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  46. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  47. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  48. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  49. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  50. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  51. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  52. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  53. var nativeDimension: Int
    Attributes
    protected
    Definition Classes
    CatBoostRegressionModel → CatBoostModelTrait
  54. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  55. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  56. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  57. def numFeatures: Int
    Definition Classes
    PredictionModel
    Annotations
    @Since( "1.6.0" )
  58. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  59. var parent: Estimator[CatBoostRegressionModel]
    Definition Classes
    Model
  60. def predict(features: Vector): Double

    Prefer batch computations operating on datasets as a whole for efficiency

    Prefer batch computations operating on datasets as a whole for efficiency

    Definition Classes
    CatBoostRegressionModel → PredictionModel
  61. final def predictRawImpl(features: Vector): Array[Double]

    Prefer batch computations operating on datasets as a whole for efficiency

    Prefer batch computations operating on datasets as a whole for efficiency

    Definition Classes
    CatBoostModelTrait
  62. final val predictionCol: Param[String]
    Definition Classes
    HasPredictionCol
  63. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  64. final def set(paramPair: ParamPair[_]): CatBoostRegressionModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  65. final def set(param: String, value: Any): CatBoostRegressionModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  66. final def set[T](param: Param[T], value: T): CatBoostRegressionModel.this.type
    Definition Classes
    Params
  67. final def setDefault(paramPairs: ParamPair[_]*): CatBoostRegressionModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  68. final def setDefault[T](param: Param[T], value: T): CatBoostRegressionModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  69. def setFeaturesCol(value: String): CatBoostRegressionModel
    Definition Classes
    PredictionModel
  70. def setParent(parent: Estimator[CatBoostRegressionModel]): CatBoostRegressionModel
    Definition Classes
    Model
  71. def setPredictionCol(value: String): CatBoostRegressionModel
    Definition Classes
    PredictionModel
  72. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  73. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  74. def transform(dataset: Dataset[_]): DataFrame
    Definition Classes
    PredictionModel → Transformer
  75. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  76. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  77. def transformImpl(dataset: Dataset[_]): DataFrame
    Definition Classes
    CatBoostModelTrait → PredictionModel
  78. def transformSchema(schema: StructType): StructType
    Definition Classes
    PredictionModel → PipelineStage
  79. def transformSchema(schema: StructType, logging: Boolean): StructType
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()
  80. val uid: String
    Definition Classes
    CatBoostRegressionModel → Identifiable
  81. def validateAndTransformSchema(schema: StructType, fitting: Boolean, featuresDataType: DataType): StructType
    Attributes
    protected
    Definition Classes
    PredictorParams
  82. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  83. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  84. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  85. def write: MLWriter
    Definition Classes
    CatBoostModelTrait → MLWritable

Inherited from CatBoostModelTrait[CatBoostRegressionModel]

Inherited from MLWritable

Inherited from RegressionModel[Vector, CatBoostRegressionModel]

Inherited from PredictionModel[Vector, CatBoostRegressionModel]

Inherited from PredictorParams

Inherited from HasPredictionCol

Inherited from HasFeaturesCol

Inherited from HasLabelCol

Inherited from Model[CatBoostRegressionModel]

Inherited from Transformer

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Ungrouped