Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

1 Overview

1. Execution Configuration

StreamExecutionEnvironment IncludedExecutionConfig, which allows job-specific configuration values ​​to be set at runtime.

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
ExecutionConfig executionConfig = env.getConfig();

The following configuration options are available: (defaults are in bold)

• setClosureCleanerLevel()The closure cleaner level is set by default to ClosureCleanerLevel.RECURSIVEThe closure cleaner removes unnecessary references to anonymous function call classes in Flink programs. After disabling the closure cleaner, the user's anonymous function may be referencing some non-serializable call classes. This will cause the serializer to have an exception. The configurable values ​​are: NONE: Completely disable the closure cleaner,TOP_LEVEL: Only clean top-level classes without recursing into fields,RECURSIVE: Recursively clean all fields.
• getParallelism() / setParallelism(int parallelism). Sets the default parallelism for jobs.
• getMaxParallelism() / setMaxParallelism(int parallelism)Sets the default maximum parallelism for jobs. This setting determines the maximum parallelism and specifies an upper limit for dynamic scaling.
• getNumberOfExecutionRetries() / setNumberOfExecutionRetries(int numberOfExecutionRetries)Sets the number of times a failed task is re-executed. A value of zero effectively disables fault tolerance.-1 Indicates that the system default value (defined in the configuration) is used.This configuration is deprecated, please use restart policy instead.
• getExecutionRetryDelay() / setExecutionRetryDelay(long executionRetryDelay)Sets the delay (in milliseconds) that the system waits before re-executing a job after a job failure. The delay is calculated after all tasks have been successfully stopped on the TaskManagers, and once the delay has passed, the task is restarted.This configuration has been deprecated, please use restart policy instead.
• getExecutionMode() / setExecutionMode()The default execution mode is PIPELINED. Set the execution mode to execute the program. The execution mode defines whether the data exchange is performed in batch mode or in streaming mode.
• enableForceKryo() / disableForceKryo. Do not force the use of Kryo by default. Force GenericTypeInformation to use the Kryo serializer for POJOs, even if they can be parsed as POJOs. In some cases, this configuration should be enabled in preference to Kryo. For example, when Flink's internal serializer cannot handle POJOs correctly.
• enableForceAvro() / disableForceAvro(). Do not force the use of Avro by default. Force Flink AvroTypeInfo to use the Avro serializer instead of Kryo to serialize Avro POJOs.
• enableObjectReuse() / disableObjectReuse()By default, objects are not reused in Flink. Enabling object reuse mode instructs the runtime to reuse user objects for better performance. Note that this may lead to bugs.
• getGlobalJobParameters() / setGlobalJobParameters()This method allows the user to set a custom object as a global configuration for the job. ExecutionConfig Accessible in all user defined functions, so this is an easy way to make configuration globally available across a job.
• addDefaultKryoSerializer(Class type, Serializer serializer). Registers a Kryo serializer instance for the specified type.
• addDefaultKryoSerializer(Class type, Class
  上一篇 下一篇