Optimizing E-Commerce Servers for High Traffic

Introduction

Tuning the servers for handling high traffic is a crucial step for the success of any e-commerce application. Let’s discuss a few Linux kernel parameters that must be checked and configured to handle high available traffic situations, before you go and look at scaling solutions.

Kernel parameters

What are kernel parameters?

They are a set of properties that can be used to change the behaviour of the Linux kernel.

Kernel parameters and performance tuning

By adjusting the kernel parameters value one can alter the behaviour of the kernel during runtime without recompiling the kernel. It can improve the performance of the applications and reduce unnecessary scaling and resource wastages.

Tunable class

The tuneable kernel parameters are also called “Tunables”, and it is divided into classes by the kernel subsystem and this classification is called “Tunable class”.

We will be concentrating on the following “Tunable classes”.

Tuning and management of memory, buffers, and cache

How to configure the kernel parameters?

We can make changes to the kernel parameters using the following methods.

1. Using the “sysctl” command to temporarily set kernel parameters at runtime.
2. Use the “sysctl” command to permanently set kernel parameters.
3. Configuration files in “/etc/sysctl.d/” to adjust kernel parameters.
4. Configuring kernel parameters temporarily through “/proc/sys/“

For the simplicity of the article, we’ll concentrate on the first method of updating the kernel parameters temporarily using the “sysctl” command.

How to check kernel parameters?

Using the command “sysctl -a” one can list all the kernel parameters.

To configure a parameter temporarily

Run the following command.

sysctl <TUNABLE_CLASS>.<PARAMETER>=<TARGET_VALUE>

 For example, the following command sets the limit of the maximum number of incoming connections queued to 1024

sysctl net.core.somaxconn=1024

Few important kernel parameters for an e-commerce application

Networking parameters (net)

Fine tuning the networking parameters (Tunable class: net) is crucial for a web-based application for handling high volumes of connections and data traffic. 

Max socket connection

Set the max number of connections that can be handled by a socket. 

In older systems, it has a default value of 128 which was insufficient but in modern systems it is set to 4096. If the number of connections received is more than this value, the “connection is reset” error message is sent, and the connection is dropped by the system on the specific ports.

$ sysctl net.core.somaxconn
net.core.somaxconn = 4096
$ sysctl net.core.somaxconn=8192

Max network device backlog

Set the maximum number of packets allowed to queue when an interface receives packets faster than the kernel can process them. If the number of requests is greater than the limit the interfaces start dropping the packages.

$ sysctl net.core.netdev_max_backlog
net.core.netdev_max_backlog = 1000
$ systctl net.core.netdev_max_backlog=5000

Max syn connections

These are the maximum number of remembered connection requests, which still do not receive an acknowledgment from the connecting client. The default value is 1024 for systems with more than 128Mb of memory, and 128 for low memory machines.

$ sysctl net.ipv4.tcp_max_syn_backlog
net.ipv4.tcp_max_syn_backlog = 1024
$ sysctl net.ipv4.tcp_max_syn_backlog=4096

System IP port limit

Increase system IP port limits to allow for more connections. This will increase available ports for connection beyond the ephemeral ports.

$ sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768        60999
$ sysctl net.ipv4.ip_local_port_range=1024 65535

Network read and write buffer

The tcp send window, increasing the tcp send and receive buffers will increase the performance a lot if (and only if) you have a lot of large files to send.

$ sysctl net.ipv4.tcp_wmem
net.ipv4.tcp_wmem = 4096   16384  4194304
$ sysctl net.core.wmem_max
net.core.wmem_max = 212992
$ sysctl net.ipv4.tcp_wmem=4096 65536 524288
$ sysctl net.core.wmem_max=1048576

The tcp receive window, if you have a lot of large file uploads, increasing the receive buffers will help.

$ sysctl net.ipv4.tcp_rmem
net.ipv4.tcp_rmem = 4096      131072            6291456
$ sysctl net.core.rmem_max
net.core.rmem_max = 212992
$ sysctl net.ipv4.tcp_rmem=4096 87380 524288
$ sysctl net.core.rmem_max=1048576

Disable IPV6

It disables all ipv6 features. This will save valuable resources that are being processed for Ipv6 even when we do not use ipv6.

$ sysctl net.ipv6.conf.all.disable_ipv6
net.ipv6.conf.all.disable_ipv6 = 0
$ sysctl net.ipv6.conf.all.disable_ipv6 = 1

File System Parameters (fs)

Fine tuning the files system parameter plays a vital role in web content serving.

Max file handles

The maximum number of file descriptors that the kernel can allocate. By default, this value corresponds to your available storage and storage block.

$ sysctl fs.file-max
fs.file-max = 9223372036854775807
$ sysctl fs.file-max = 2097152

Max Inotify handles

Sets maximum number of files that can be monitored with inotify.

$ sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 65536
$ sysctl fs.inotify.max_user_watches = 524288

Memory Management

Fine tuning memory management parameters are essential to ensure that the application has enough resources available.

Swappiness 

How often RAM data is dumped into the disk? The less often the memory swapping happens the faster application performs. Reading and writing the memory to disk is often a bottleneck in application performance. The less often the better. In modern Linux machines having huge RAM often swapping data from RAM is turned off.

$ sysctl vm.swappiness
vm.swappiness = 60
$ sysctl vm.swappiness = 10

Dirty pages in page cache

Dirty pages referred to the stale page caches in the RAM. it decides how often to check and replace stale page caches. It is like swappiness, the less dirty page and less often the check the faster the application can perform. 

$ sysctl vm.dirty_ratio
vm.dirty_ratio = 20
$ sysctl vm.dirty_background_ratio 
vm.dirty_background_ratio = 10
$ sysctl vm.dirty_ratio=15
$ sysctl vm.dirty_background_ratio=5

Overcommit Memory

To allocate more memory than available to an application this parameter can be used.

$ sysctl vm.overcommit_memory
vm.overcommit_memory = 0
$ sysctl vm.overcommit_memory=1

Final Thoughts

By effectively adjusting the kernel parameters we can improve our high traffic ecommerce application’s performance and reduce infrastructure cost spend of maintaining the same.

The above parameters are just scratching the surface of the very basics of optimising the Linux kernel parameters for a high traffic ecommerce application.

Ref :

[1]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/configuring-kernel-parameters-at-runtime_managing-monitoring-and-updating-the-kernel#configuring-kernel-parameters-temporarily-through-proc-sys_configuring-kernel-parameters-at-runtime

[2]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/monitoring_and_managing_system_status_and_performance/tuning-the-network-performance_monitoring-and-managing-system-status-and-performance#increasing-the-transmit-queue-length-of-a-nic-to-reduce-the-number-of-transmit-errors_tuning-network-adapter-settings

[3] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html-single/performance_tuning_guide/index

[4]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/tuning_and_optimizing_red_hat_enterprise_linux_for_oracle_9i_and_10g_databases/sect-oracle_9i_and_10g_tuning_guide-pre_installation_preparation_and_verification-verifying_kernel_parameters#sect-Oracle_9i_and_10g_Tuning_Guide-Pre_Installation_Preparation_and_Verification-Verifying_Kernel_Parameters

[5] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/system_design_guide/configuring-kernel-parameters-at-runtime_system-design-guide

[6]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/adjusting-kernel-parameters-for-database-servers_managing-monitoring-and-updating-the-kernel

[7]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/managing_monitoring_and_updating_the_kernel/adjusting-kernel-parameters-for-database-servers_managing-monitoring-and-updating-the-kernel#parameters-affecting-performance-of-database-applications_adjusting-kernel-parameters-for-database-servers

[8] https://www.ibm.com/docs/en/tnpm/1.4.5?topic=linux-modifying-kernel-parameters

[9]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/configuring-kernel-parameters-at-runtime_managing-monitoring-and-updating-the-kernel#what-are-kernel-parameters_configuring-kernel-parameters-at-runtime

[10] https://www.linux.org/threads/how-can-one-optimize-kernel-parameters-to-enhance-performance.48570/

[11] https://xitoring.com/kb/how-to-fine-tune-linux-kernel-parameters/

[12] https://medium.com/@moaminsharifi/tuning-your-linux-kernel-for-io-intensive-applications-2e059dd5f813

[13] https://docs.oracle.com/en/enterprise-manager/cloud-control/enterprise-manager-cloud-control/13.4/emrel/configuring-kernel-parameters-linux-1.html

[14] https://www.percona.com/blog/tune-linux-kernel-parameters-for-postgresql-optimization/

[15] https://stackoverflow.com/questions/12234050/nginx-high-volume-traffic-load-balancing