[[linux:akamai]]

Export page to Open Document format

Surviving without Akamai

Short history of a change in the technology used by Stratalogica.

For the new version of Stratalogica, Akamai was necessary to climb to around 80GB to NetStorage service hired to accommodate the contents.

Because the amount of information we started this process a week before the limit to release the new version time, we did not have that Akamai, would be very slow to upload the content.

We tried it is standard procedure rsync over ssh, and this failed after 8 hours returning the following error: “rsync error: unexplained error (code 130) rsync.c at (549) [sender = 3.0.9]”

Then we tried to use the FTP protocol to upload 80 GB and we realized that it could take around 300 hours.

The following figure shows an attempt was made to upload all content using the FTP protocol. This was a failure due to the reported time.

We understand that moving many small files for any type of network is a problem, but did not expect it to be so strong in Akamai

We decided to try the technical support from 27 August with different questions to prepare for uploading content.

The route was as follows: 1)

1. First ask for help to enter the File System to review the status Akamai NetStorage, the response was the usual support, Akamai manual at this point seems to me well as my knowledge of Akamai at that time were very low, with the manual I could create an account in order to enter the NetStorage via FTP, although the set to enter via SSH also, that never worked.

2. After many attempts to upload the contents of different forms Akamai try to contact the September 3 to to guide me how to upload these files, the response was twofold:

  • A) We purchased an additional software package with Akamai to accelerate increases
  • B) That attempt simultaneous uploads, so that not explained how, in the manual does not explain Akamai and try to follow what I explained on the phone but it was a long way
  • C) Packing all in .zip, but Akamai is the limit of only handle 2GB .zip files, so I had to compress files to upload them into small pieces, a task that took around 6 hours. After compress files I could get on the NetStorage Akamai archives 2GB pieces which took a good 20 minutes as uploading large files over the network is easier. But I found the difficulty to unzip these files was very slow, I auditioned to leave decompressing a file and 6 hours for delayed too cancel it. The answer to this part of the NetStorage Akamai is unable to decompress so fast, and do the math would take almost as long to decompress the .ZIP files you upload as was trying from the beginning. Doing this process I found that via SSH console NetStorage Akamai service is extremely limited and did not allow a good management to try to better manipulate the .ZIP. And that only allows .ZIP files is very slow with large files, ideally Akamai .TAR allow handling files.

3. Because of the problems with the traditional way of upload content to Akamai September 5 I decided to create a tool that allowed me to go up simultaneously using LFTP and an SSH tunnel was able to accelerate the process a little, but I found that the NetStorage only allows 25 simultaneous connections to upload files, I pretended to upload 100 simultaneously, this process achieves upload around 60% of the content in less than 24 hours, yet on Monday September 7 and I had not finished to interrupt the process and it consumed too many resources of RAM and processor needed to attend connections Stratalogica users to the new platform.

Because of different problems presented September 6 we decided to disable the service of Akamai and make the necessary changes on DNS, with this the requests to www.stratalogica.com are not going to Akamai.

But remove Akamai represented a clear challenge. We had to do that the server on OVH can support all the LOAD and Traffic of the entire application.

What I did on Sunday 6 septimebre concerning the technological infrastructure was to design and configure a new server architecture to support the load, but to understand that I will show you the past and present.

Before applying Stratalogica4 kept running just as handed Herff & Jones, customers were doing www.stratalogica.com requests and Akamai was receiving all traffic, our servers responded with part of the Java application, and then Akamai delivered the static content.

The following chart shows in detail as it was distributed server configuration in OVH, and the different virtual machines that are handling the application. For the previous version had 12 instances of Jboss over 3 Virtual Servers, 1 virtual server with static content and a load balancer that is a web server that handles all clients receive and distribute in instances of JBOSS.



In this scenario as a result of several measurements made during the last months, I can say that 55% of traffic was served by Akamai and 45% is server by our servers corresponding to the dynamic content generated by the application in operation Normal.

Stratalogica4 currently, is being delivered to our users entirely from the server you have at OVH, to remove the DNS configurations that made the network traffic being diverted to the NetStorage Akamai servers, all traffic reaches the server.

Now we receive all requests with a balancer very modern cargo called HAProxy (Used by Twitter and GitHub and other companies) this balancer is responsible for routing requests to the instances of JBOSS and serve requests for static content depending on how you go requesting .

The following chart shows in detail the arrangement of servers, distributed as follows one load balancer, sixteen instances of JBoss on two virtual servers, four Web servers for static content on three virtual servers, 1 virtual server for database.

With this configuration we have begun to assume all traffic without relying on Akamai, deploy from Thursday September 10 to the evening of September 11 the load balancer delivered the following results:

In a day and two hours the new server configuration I give about 26 Gigabytes of content, distributed between static content and dynamic content, so far the new distribution server proves he can handle a fairly large number of connections, allowing us survived so far without Akamai, we have to wait for a week not to do any new Deploy to a larger measurement, and also measure user consumption is something that allows us to this new infrastructure.

Conclusions

Regarding Akamai:
1. Akamai technical support is lousy and assistance to problem solving is slow and most of the answers conclude the purchase of new products, Akamai.

2. The service provided by Akamai NetStorage, serves to accommodate contents, but when uploading very large new files is too slow to load, your way of working had to start uploading files stratalogica4 a month before the launch New versions which is absurd.

3. The way the NetStorage Akamai works today is worn when leaving because all traffic is redirected via DNS round robin balancing and use, which is not 100% efficient and at times our server finished delivering all static content as Akamai's response was slow.

4. The management console Akamai has many usability issues and provides little information on traffic and files that are housed.

5. The content replication is slow, and managing SSL certificates is not 100% clear, I installed our SSL certificate and was delayed about 4 days in retreat to the Akamai network and could not test well, but tests showed that I was not properly installed.

Generally:

1. We can assume all network traffic and load the application with the current settings displayed for the new version.

2. The future of infrastructure for this in Stratalogica implemented solutions that enable load balancing and use different storage services if necessary.

3. From the point of view of technological autonomy is better to have servers like we have today.

4. Migrating to a service like Amazon is very good option and is the path we are testing, with the current configuration we are closer to reaching a cloud infrastructure.


1)
All this can be contrasted in the emails we came across Akamai's platform and support
  • linux/akamai.txt
  • Última modificación: 2015/09/12 06:38
  • por kleper