Version: 1.4.0

Limit Parameter Adjustment

This article describes how to adjust parameters when resources such as memory and CPU are insufficient.

1. Insufficient queue resources

An error is reported during task execution: error code 12001, the error message queue has insufficient CPU resources, you can adjust the number of Spark executors

Click on the management console--parameter configuration--IDE--Spark--display advanced settings--woker engine resource settings (2)--adjust the concurrent number of executors

Or set the global setting to adjust the upper limit of queue resource usage.

2. The number of queue instances exceeds the limit

An error is reported during task execution: error code 12003, the number of error message queue instances exceeds the limit

Click Management Console--Parameter Configuration--Global Settings--Queue Resources--Maximum Number of Yarn Queue Instances [wds.linkis.rm.yarnqueue.instance.max]

3. Global driver memory usage limit

An error is reported during task execution: error code 12004, error message upper limit of global driver memory usage, lower driver memory can be set

Click Management Console--Parameter Configuration--Global Settings--Queue Resources--Maximum Number of Yarn Queue Memory 队列内存使用上限[wds.linkis.rm.yarnqueue.memory.max]

4. Exceeded the upper limit of the CPU number of the global driver

An error is reported during task execution: error code 12005, the error message exceeds the upper limit of the number of CPUs in the global drive, and idle engines can be cleaned up

Click Management Console - Parameter Configuration - Global Settings - Queue Resources - Upper Limit of The Number of Engine Cores[wds.linkis.rm.client.core.max]

Or solve it by cleaning idle engines.

5. Exceeded the upper limit of the engine's maximum concurrent number

An error is reported during task execution: error code 12006, the error message exceeds the maximum concurrency limit of the engine, and idle engines can be cleaned up

Click Management Console--Parameter Configuration--Global Settings--Queue Resources--Maximum concurrent number of each engine globally [wds.linkis.rm.instance]