Notoriously, the top sources of cost in the Cloud are Compute, Database, Storage, and Big Data Analysis (e.g. BigQuery, AWS EMR, Azure Synapse). Another significant source of cost is Data Transfer (“Networking” in GCP, “Bandwidth” in Azure), which can be divided to two types - inter-AZ costs, and egress costs from the cloud environment to the Internet. For some companies these costs can accumulate significantly. Each type of traffic requires a different solution:
Egress from the cloud environment to the Internet is often a necessity driven by the company's business model and cannot be avoided. The best way to mitigate these costs is by adding an external Internet endpoint capable of accepting compressed data. While this may not be a simple undertaking and requires resources outside of the cloud environment, it can result in substantial cost savings, with only a minor increase in the compute cost of compression.
On the other hand, inter-AZ costs are generally unnecessary and tend to accrue due to suboptimal deployments of instances or services. The challenge lies in identifying these data transfer "leaks" since cloud providers do not make it easy to determine which service and instance are responsible for data transmission across AZ lines. We have witnessed several organizations invest significant time attempting to understand the causes of high costs, such as AWS Data Transfer USE1-APS1-AWS-Out-Bytes, without success.
At Underspend, we have developed a free Data Transfer tool that operates within the cloud environment, analyzing flow logs and deployed services. This tool precisely identifies which instance, database, and service is communicating with whom and provides cost information for each interaction. The results obtained from the tool often empower DevOps teams to reallocate resources to the appropriate AZ, leading to substantial cost savings. In cases where resource relocation is not feasible, compressing the data before transferring it between AZs proves to be an effective solution. While compression may seem straightforward, its impact is remarkable.
Using the Underspend Data Transfer tool, we have been able to save our customers millions of dollars annually, and we would be delighted to assist more companies. If you want to reduce your Data Transfer costs, please feel free to reach out to us at datatransfer@underspend.com or simply click on "Get A Demo" on our website.
Comments