Regardless of how you’re using Amazon Web Services (AWS), an important first step – after enabling multi-factor authentication – is setting up budget and billing alerts. Last month I shared the costly consequences of not taking these steps. This month, I’ll share another expensive lesson learned that underscores the importance of AWS billing alerts, along with some recommendations to help increase visibility into your cloud services spending.
Read the fine print
After I burned nearly $2k on my AWS billing mistake, I encouraged leadership at that previous employer to set up billing alerts immediately. I’d seen team members recklessly spinning and abandoning AWS resources. When I voiced concerns about unchecked AWS spending before (and even recommending that developers shouldn’t have carte blanche access), I always got a room full of cross-eyed stares.
It wasn’t until the company was socked with an additional fee of $35,000, on top of their already hefty AWS monthly bill – more than double the average monthly spend – that billing alerts finally became a priority.
Here’s what happened.
Another team was using a service that billed around 10 cents per use for computational time on the service instance. That’s pretty cheap – negligible in the scope of the project, really – so the team ran “jobs” as needed, ad hoc style, with no concern for cost.
But they missed something in the fine print: The minimum billable expense was actually $10 per use. Each 10-cent “expected” use was billed at $10 minimal billing increment—as much as 300 times the anticipated cost. The company ended up on the hook for tens of thousands of dollars for something a billing alert or review would have flagged and prevented.
When billing alerts make a difference
That’s an extreme example, but billing alerts really come in handy in other situations as well:
- Auto-scaling services running beyond intended limits (DDoS attack or scraper/bots triggering scale options)
- Poor configuration of instances, resulting in overspending or billing confusion
- Internal or external abuse of the account and services
- Accounts have been hacked or compromised, such as via social engineering or weak security practice
- Understanding traffic, trends, adoption or other use patterns with respect to your resource pool
- Newly introduced code defect causes service to be called in a loop
- Service that’s left on that is burning money
By now, you should be convinced AWS billing alerts are critical. Read on for a few more of our tips and recommendations.
Test the alerts
Set a very low budget (1% above your current spend, for example) to test the billing alerts and make sure they’re whitelisted in your email provider and flagged correctly. Be sure you’re receiving the notices, regardless of how billing alerts are sent.
Include others
Always include more than one set of eyes on your AWS billing alerts and spend. Add in additional emails or use a broadcast email for your organization.
Don’t lose sleep
If, like 27Global, your business operates in multiple regions, concerns over your AWS bill may have caused you some sleepless nights. Fear not. As this screenshot illustrates, your region or regions don’t impact your billing.
Go further!
You can slice up your billing alerts however necessary, including per service or even by Amazon Resource Name (ARN). Our goal here is to provide basic visibility; anything beyond simple alerting should be handled by someone with deeper knowledge of the services, or ideally, with your SRE team.
It’s easy to make mistakes and oversights like the ones I’ve shared, especially if your company is moving to the cloud or replatforming. Amazon services are wholesale in price, intended to be used in bulk, and require a certain level of expertise to configure and set the right balance of expenses and performance. Learn from my mistake: Advocate for control of services and monitor spending carefully through AWS billing alerts. Your bottom line will thank you.
Marc Weinberg is a Consultant in 27Global’s Denver office. Founded in 2008, 27Global is an AWS certified partner that designs, builds and operates custom software solutions for businesses of all sizes. The perfect pairing of a local leadership with offshore pricing, 27Global has the business acumen to understand your vision and the expertise to build your software solution. To learn more, visit 27Global.com or connect with us on LinkedIn and Twitter.
