net.snowflake.spark.snowflake.Parameters

MergedParameters

case class MergedParameters(parameters: Map[String, String]) extends Product with Serializable

Adds validators and accessors to string map

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. MergedParameters
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new MergedParameters(parameters: Map[String, String])

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def checkBucketConfiguration: Boolean

    Returns true if bucket lifecycle configuration should be checked

  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def createPerQueryTempDir(): String

    Creates a per-query subdirectory in the rootTempDir, with a random UUID.

  10. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  11. def extraCopyOptions: String

    Extra options to append to the Snowflake COPY command (e.

    Extra options to append to the Snowflake COPY command (e.g. "MAXERROR 100").

  12. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  14. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  15. def isTimezoneSnowflake: Boolean

  16. def isTimezoneSnowflakeDefault: Boolean

  17. def isTimezoneSpark: Boolean

  18. def jdbcDriver: Option[String]

    The JDBC driver class name.

    The JDBC driver class name. This is used to make sure the driver is registered before connecting over JDBC.

  19. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  22. lazy val parallelism: Option[Int]

    Number of threads used for PUT/GET.

  23. val parameters: Map[String, String]

  24. def postActions: Array[String]

    List of semi-colon separated SQL statements to run after successful write operations.

    List of semi-colon separated SQL statements to run after successful write operations. This can be useful for running GRANT operations to make your new tables readable to other users and groups.

    If the action string contains %s, the table name will be substituted in, in case a staging table is being used.

    Defaults to empty.

  25. def preActions: Array[String]

    List of semi-colon separated SQL statements to run before write operations.

    List of semi-colon separated SQL statements to run before write operations. This can be useful for running DELETE operations to clean up data

    If the action string contains %s, the table name will be substituted in, in case a staging table is being used.

    Defaults to empty.

  26. def query: Option[String]

    The Snowflake query to be used as the target when loading data.

  27. def rootTempDir: String

    A root directory to be used for intermediate data exchange, expected to be on cloud storage (S3 or Azure storage), or somewhere that can be written to and read from by Snowflake.

    A root directory to be used for intermediate data exchange, expected to be on cloud storage (S3 or Azure storage), or somewhere that can be written to and read from by Snowflake. Make sure that credentials are available for this cloud provider.

  28. lazy val rootTempDirStorageType: FSType

  29. def s3maxfilesize: String

    Max file size used to move data out from Snowflake

  30. def sfAccount: Option[String]

    Snowflake account - optional

  31. def sfCompress: Boolean

    Snowflake use compression on/off - "on" by default

  32. def sfDatabase: String

    Snowflake database name

  33. def sfExtraOptions: Map[String, AnyRef]

    Returns a map of options that are not known to the connector, and are passed verbosely to the JDBC driver

  34. def sfPassword: String

    Snowflake password

  35. def sfRole: Option[String]

    Snowflake role - optional

  36. def sfSSL: String

    Snowflake SSL on/off - "on" by default

  37. def sfSchema: String

    Snowflake schema

  38. def sfTimezone: Option[String]

    Snowflake timezone- optional

  39. def sfURL: String

    URL pointing to the snowflake database, simply host:port

  40. def sfUser: String

    Snowflake user

  41. def sfWarehouse: Option[String]

    Snowflake warehouse

  42. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  43. def table: Option[TableName]

    The Snowflake table to be used as the target when loading or writing data.

  44. def temporaryAWSCredentials: Option[AWSCredentials]

    Temporary AWS credentials which are passed to Snowflake.

    Temporary AWS credentials which are passed to Snowflake. These only need to be supplied by the user when Hadoop is configured to authenticate to S3 via IAM roles assigned to EC2 instances.

  45. def temporaryAzureStorageCredentials: Option[StorageCredentialsSharedAccessSignature]

    SAS Token to be passed to Snowflake to access data in Azure storage.

    SAS Token to be passed to Snowflake to access data in Azure storage. We currently don't support full storage account key so this has to be provided if customer would like to load data through their storage account directly.

  46. def toString(): String

    Definition Classes
    MergedParameters → AnyRef → Any
  47. def useStagingTable: Boolean

    When true, data is always loaded into a new temporary table when performing an overwrite.

    When true, data is always loaded into a new temporary table when performing an overwrite. This is to ensure that the whole load process succeeds before dropping any data from Snowflake, which can be useful if, in the event of failures, stale data is better than no data for your systems.

    Defaults to true.

  48. lazy val usingExternalStage: Boolean

  49. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  50. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  51. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped