Running Magento on AWS has a lot of advantages for sure, but in some cases you'll probably face problems which will be hard to achive.

This happens some weeks ago in a project where the solution partner needs to import 2.1 million products to setup the Magento 2 Commerce instance. Struggling with the default Magento 2 Import/Export functionality, they'll end up with M2IF. So far so good.

The 2.1 million products are separated in bunches with 100k SKUs. Start importing the files one after another, NOT using the bunch functionality, after several files of 100k items per file, the speed slows down from ~2.000/min to 250/min. Now running the previous files again M2IF is still performing at 250/min (see image above).

After contacting us on our Gitter Support Channel, our first assumption was something like a memory problem. After checking all the basic stuff and have some discussions with our DevOps, we came to the conclusion that the problem has to be originated in the AWS infrastructure. During the day Brent, one of the solution partners DEVs, and his collegues figured out that the problem was the EBS Volume Type of the DB node, whereas the web node was a t3.large with unlimited enabled and the DB server a r5a.2xlarge instance, also with unlimited enabled.

In the Amazon AWS EC2 User Guide we found the following chapter that explains the problem

Each volume receives an initial I/O credit balance of 5.4 million I/O credits, which is enough to sustain the maximum burst performance of 3,000 IOPS for 30 minutes. This initial credit balance is designed to provide a fast initial boot cycle for boot volumes and to provide a good bootstrapping experience for other applications. Volumes earn I/O credits at the baseline performance rate of 3 IOPS per GiB of volume size. For example, a 100 GiB gp2 volume has a baseline performance of 300 IOPS.

Understanding that, the simple solution was to add additional storage to the DB server with 15K provisioned IOPS, this solved the problem.

Next Post Previous Post

Load Comments

Please note that by clicking "Load Comments", your browser will establish a connection with servers from Disqus.
For more information, please visit our privacy statement.