CreateRunDetails

class oci.data_flow.models.CreateRunDetails(**kwargs)

Bases: object

The create run details. The following properties are optional and override the default values set in the associated application: - applicationId - archiveUri - arguments - configuration - definedTags - displayName - driverShape - execute - executorShape - freeformTags - logsBucketUri - metastoreId - numExecutors - parameters - sparkVersion - warehouseBucketUri It is expected that either the applicationId or the execute parameter is specified; but not both. If both or none are set, a Bad Request (HTTP 400) status will be sent as the response. If an appicationId is not specified, then a value for the execute parameter is expected. Using data parsed from the value, a new application will be created and assicated with the new run. See information on the execute parameter for details on the format of this parameter.

The optional parameter spark version can only be specified when using the execute parameter. If it is not specified when using the execute parameter, the latest version will be used as default. If the execute parameter is not used, the spark version will be taken from the associated application.

If displayName is not specified, it will be derived from the displayName of associated application or set by API using fileUri’s application file name. Once a run is created, its properties (except for definedTags and freeformTags) cannot be changed. If the parent application’s properties (including definedTags and freeformTags) are updated, the corresponding properties of the run will not update.

Methods

__init__(**kwargs) Initializes a new CreateRunDetails object with values from keyword arguments.

Attributes

application_id Gets the application_id of this CreateRunDetails.
archive_uri Gets the archive_uri of this CreateRunDetails.
arguments Gets the arguments of this CreateRunDetails.
compartment_id [Required] Gets the compartment_id of this CreateRunDetails.
configuration Gets the configuration of this CreateRunDetails.
defined_tags Gets the defined_tags of this CreateRunDetails.
display_name Gets the display_name of this CreateRunDetails.
driver_shape Gets the driver_shape of this CreateRunDetails.
execute Gets the execute of this CreateRunDetails.
executor_shape Gets the executor_shape of this CreateRunDetails.
freeform_tags Gets the freeform_tags of this CreateRunDetails.
logs_bucket_uri Gets the logs_bucket_uri of this CreateRunDetails.
metastore_id Gets the metastore_id of this CreateRunDetails.
num_executors Gets the num_executors of this CreateRunDetails.
parameters Gets the parameters of this CreateRunDetails.
spark_version Gets the spark_version of this CreateRunDetails.
warehouse_bucket_uri Gets the warehouse_bucket_uri of this CreateRunDetails.
__init__(**kwargs)

Initializes a new CreateRunDetails object with values from keyword arguments. The following keyword arguments are supported (corresponding to the getters/setters of this class):

Parameters:
  • application_id (str) – The value to assign to the application_id property of this CreateRunDetails.
  • archive_uri (str) – The value to assign to the archive_uri property of this CreateRunDetails.
  • arguments (list[str]) – The value to assign to the arguments property of this CreateRunDetails.
  • compartment_id (str) – The value to assign to the compartment_id property of this CreateRunDetails.
  • configuration (dict(str, str)) – The value to assign to the configuration property of this CreateRunDetails.
  • defined_tags (dict(str, dict(str, object))) – The value to assign to the defined_tags property of this CreateRunDetails.
  • display_name (str) – The value to assign to the display_name property of this CreateRunDetails.
  • driver_shape (str) – The value to assign to the driver_shape property of this CreateRunDetails.
  • execute (str) – The value to assign to the execute property of this CreateRunDetails.
  • executor_shape (str) – The value to assign to the executor_shape property of this CreateRunDetails.
  • freeform_tags (dict(str, str)) – The value to assign to the freeform_tags property of this CreateRunDetails.
  • logs_bucket_uri (str) – The value to assign to the logs_bucket_uri property of this CreateRunDetails.
  • metastore_id (str) – The value to assign to the metastore_id property of this CreateRunDetails.
  • num_executors (int) – The value to assign to the num_executors property of this CreateRunDetails.
  • parameters (list[oci.data_flow.models.ApplicationParameter]) – The value to assign to the parameters property of this CreateRunDetails.
  • spark_version (str) – The value to assign to the spark_version property of this CreateRunDetails.
  • warehouse_bucket_uri (str) – The value to assign to the warehouse_bucket_uri property of this CreateRunDetails.
application_id

Gets the application_id of this CreateRunDetails. The OCID of the associated application. If this value is set, then no value for the execute parameter is required. If this value is not set, then a value for the execute parameter is required, and a new application is created and associated with the new run.

Returns:The application_id of this CreateRunDetails.
Return type:str
archive_uri

Gets the archive_uri of this CreateRunDetails. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:The archive_uri of this CreateRunDetails.
Return type:str
arguments

Gets the arguments of this CreateRunDetails. The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as Service Api Spec, where name is the name of the parameter. Example: [ “–input”, “${input_file}”, “–name”, “John Doe” ] If “input_file” has a value of “mydata.xml”, then the value above will be translated to –input mydata.xml –name “John Doe”

Returns:The arguments of this CreateRunDetails.
Return type:list[str]
compartment_id

[Required] Gets the compartment_id of this CreateRunDetails. The OCID of a compartment.

Returns:The compartment_id of this CreateRunDetails.
Return type:str
configuration

Gets the configuration of this CreateRunDetails. The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { “spark.app.name” : “My App Name”, “spark.shuffle.io.maxRetries” : “4” } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

Returns:The configuration of this CreateRunDetails.
Return type:dict(str, str)
defined_tags

Gets the defined_tags of this CreateRunDetails. Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: {“Operations”: {“CostCenter”: “42”}}

Returns:The defined_tags of this CreateRunDetails.
Return type:dict(str, dict(str, object))
display_name

Gets the display_name of this CreateRunDetails. A user-friendly name that does not have to be unique. Avoid entering confidential information. If this value is not specified, it will be derived from the associated application’s displayName or set by API using fileUri’s application file name.

Returns:The display_name of this CreateRunDetails.
Return type:str
driver_shape

Gets the driver_shape of this CreateRunDetails. The VM shape for the driver. Sets the driver cores and memory.

Returns:The driver_shape of this CreateRunDetails.
Return type:str
execute

Gets the execute of this CreateRunDetails. The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include --class, --file, --jars, --conf, --py-files, and main application file with arguments. Example: --jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10 Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

Returns:The execute of this CreateRunDetails.
Return type:str
executor_shape

Gets the executor_shape of this CreateRunDetails. The VM shape for the executors. Sets the executor cores and memory.

Returns:The executor_shape of this CreateRunDetails.
Return type:str
freeform_tags

Gets the freeform_tags of this CreateRunDetails. Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: {“Department”: “Finance”}

Returns:The freeform_tags of this CreateRunDetails.
Return type:dict(str, str)
logs_bucket_uri

Gets the logs_bucket_uri of this CreateRunDetails. An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:The logs_bucket_uri of this CreateRunDetails.
Return type:str
metastore_id

Gets the metastore_id of this CreateRunDetails. The OCID of OCI Hive Metastore.

Returns:The metastore_id of this CreateRunDetails.
Return type:str
num_executors

Gets the num_executors of this CreateRunDetails. The number of executor VMs requested.

Returns:The num_executors of this CreateRunDetails.
Return type:int
parameters

Gets the parameters of this CreateRunDetails. An array of name/value pairs used to fill placeholders found in properties like Application.arguments. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: “iterations”, value: “10”}, { name: “input_file”, value: “mydata.xml” }, { name: “variable_x”, value: “${x}”} ]

Returns:The parameters of this CreateRunDetails.
Return type:list[oci.data_flow.models.ApplicationParameter]
spark_version

Gets the spark_version of this CreateRunDetails. The Spark version utilized to run the application. This value may be set if applicationId is not since the Spark version will be taken from the associated application.

Returns:The spark_version of this CreateRunDetails.
Return type:str
warehouse_bucket_uri

Gets the warehouse_bucket_uri of this CreateRunDetails. An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:The warehouse_bucket_uri of this CreateRunDetails.
Return type:str