Enabling Overload Control for a Cluster
Cluster overload occurs when system load such as request volume or resource usage exceeds the system's processing capacity, leading to degraded performance or system failure. Overload control is a dynamic mechanism that limits concurrent external requests based on the resource pressure of the master node, ensuring the stability and reliability of the master node and the cluster. In a CCE standard or Turbo cluster, you can enable cluster overload control, monitor cluster overload, and configure overload alarms to reduce the risk of cluster overload. For details, see Figure 1.
Notes and Constraints
The cluster version must be 1.23 or later.
Enabling Overload Control
After overload control is enabled, CCE will proportionally reject external traffic based on the overload level when the master node is overloaded. This effectively reduces the load on the master node and ensures the stability and availability of the cluster.
Method 1: Enable overload control when creating a cluster.
- When creating a cluster of v1.23 or later, enable Overload Control in (Optional) Advanced Settings > Cluster Scalability on the Buy Cluster page. After this function is enabled, CCE will dynamically adjust concurrent requests based on the master node's resource pressure to ensure the stability and reliability of the master node and the cluster.
Figure 2 Enabling overload control during cluster creation
- After the cluster is created, click the cluster name. The cluster Overview page is displayed. In the Master Nodes area at the lower right of the page, overload control has been enabled if the following information is displayed: Overload control enabled. An overloaded master node rejects external traffic proportionally based on the overload level.
Figure 3 Overload control enabled
Method 2: Enable overload control in an existing cluster.
- Log in to the CCE console and click the name of the cluster of v1.23 or later. The cluster Overview page is displayed.
- In the Master Nodes area at the lower right of the page, click Enable.
Figure 4 Enabling overload control for an existing cluster
- In the Master Nodes area, overload control has been enabled if the following information is displayed: Overload control enabled. An overloaded master node rejects external traffic proportionally based on the overload level.
Figure 5 Overload control enabled
Overload Monitoring
You can monitor cluster metrics to promptly check the cluster overload status.
Method 1: Using the CCE console
- Log in to the CCE console and click the name of an existing cluster whose version is v1.23 or later.
- On the Overview page, check the master node information. The overload level metric will be displayed.
The overload levels are as follows:
- Circuit breaking: Rejects all external traffic.
- Severe overload: Rejects 75% external traffic.
- Moderate overload: Rejects 50% external traffic.
- Slight overload: Rejects 25% external traffic.
- Normal: Does not reject external traffic.
Method 2: Using AOM
You can log in to the AOM console, create a dashboard, and add the metric named vein_overload_level. For details, see Dashboard.
The meanings of the monitoring metrics are as follows:
- 0: Circuit breaking: Rejects all external traffic.
- 1: Severe overload: Rejects 75% external traffic.
- 2: Moderate overload: Rejects 50% external traffic.
- 3: Slight overload: Rejects 25% external traffic.
- 4: Normal: Does not reject external traffic.
The following uses the operations performed on AOM as an example.
- Log in to the AOM console, choose Dashboard in the navigation pane and click Add Dashboard.
Figure 6 Creating a dashboard
- After the dashboard is created, add a metric graph.
- Graph Name: Cluster Overload Metrics
- Add By: All metrics
- Metric: vein_overload_level
- Scope: Optional
Figure 7 Adding a metric graph - After configuring parameters, click Save in the upper right corner.
- Click
in the upper right corner to save the dashboard.
Figure 8 Saving a dashboard
Overload Alarm
The following uses AOM as an example. In this section, only certain example parameters are provided. For more details, see Creating an AOM Metric Alarm Rule.
- Log in to the AOM console. In the navigation pane, choose Alarm Center > Alarm Rules. Click Create.
- Set the following parameters:
- Rule Name: Cluster overload alarms
- Enterprise Project: Enter default.
- Rule Type: Metric alarm rule
- Configuration Mode: Select from all metrics
- Alarm Rule Details: Select vein_overload_level and configure the corresponding scope.
- If the metric value is less than or equal to 1, the cluster is heavily overloaded. You are advised to set a critical alarm.
- If the metric value is less than or equal to 2, the cluster is moderately overloaded. You are advised to set a major alarm.
- If the metric value is less than or equal to 3, the cluster is slightly overloaded. You are advised to set a minor alarm.
Set other parameters as required.
Figure 9 Configuring an alarm rule for an overloaded cluster - Click Create Now.
Disabling Cluster Overload Control
You can disable overload control when it is no longer needed. After overload control is disabled, the master node will no longer reject external traffic when overloaded. Exercise caution when performing this operation.
- Log in to the CCE console and click the name of an existing cluster whose version is v1.23 or later.
- In the navigation pane, choose Settings.
- On the Cluster Access tab page, disable overload control.
- Click OK.
Related Operations
In addition to enabling overload control, you can take other measures to prevent cluster overload, such as limiting the cluster resource data volume and clearing invalid cluster resources. For details, see Protecting a CCE Cluster Against Overload.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot