In part one of our series on Spot instances, we explored the fundamental concepts behind Spot instances, their benefits, and the key considerations for leveraging them effectively.
Building on this, we’re going to explore how to optimize Spot instance utilization with Auto Scaling groups (ASGs), and the nuances of Spot allocation strategies, providing insight into which strategy suits different scenarios best and why.
Finally we’re going to cover how to maximize your savings with DoiT Spot Scaling, which simplifies the complex process of configuring and managing ASGs for Spot instance utilization.
Auto Scaling groups (ASGs) explained
ASGs are a mechanism for managing groups of instances. They make your workloads more elastic by automatically adjusting the number of instances deployed based on demand, and help you enhance fault tolerance.
Scaling out when demand increases ensures performance and responsiveness, while scaling in during periods of lower demand helps reduce unnecessary costs
You can configure them for allowed instance types and availability zones, minimum and maximum limits on number of instances, and mixed instance policy (percent Spot instances vs. percent on-demand instances), among others. But once configured, ASGs automate much of the resource management.
For example, if you ran an ecommerce website, it would be much easier to respond appropriately to changes in web traffic with Auto Scaling groups.
Manually dealing with changes in traffic would require constant monitoring and quick responses to avoid crashes or slow performance, potentially impacting user experience and resulting in revenue loss.
ASGs and Spot instances
ASGs are especially important in the context of Spot instances because they help you handle Spot interruptions and optimize their utilization by automatically replacing interrupted Spot instances with new ones.
While ASGs don't directly distribute workloads among instances, they ensure the desired number of instances is available. The distribution of incoming workloads or web requests across these instances is typically managed by services like Elastic Load Balancers. ASGs work with these load balancers to automatically direct traffic to each new instance they create, helping balance the workload distribution efficiently.
Additionally, they provide the flexibility of using different instance types within the same group. This helps you utilize various Spot instance types based on availability and cost.
Spot Allocation Strategies
When configuring your Auto Scaling groups, one of the settings you must figure is the Spot Allocation Strategy. Here are different approaches you can take for bidding on and obtaining Spot instances:
Lowest price
The Lowest price strategy targets the cheapest available instance pools. Because it prioritizes price over capacity, there is no guarantee that there will be ample capacity in these pools, increasing your risk of interruption.
You might use this if you are deploying a data analytics pipeline that processes large datasets. This pipeline can schedule its jobs during periods of low spot prices using the lowest-price strategy. Interruptions during higher spot price periods have minimal impact as the pipeline can resume processing when spot prices drop again.
Capacity-optimized
The Capacity-optimized strategy prioritizes instance pools with higher spare capacity, which in turn means lower interruption risk.
Use this strategy when high availability is crucial, and your workload requires a more stable instance pool with fewer interruptions.
For example, if you run a streaming service, you’ll place a priority on availability, not wanting to degrade your users’ streaming experience.
Diversified
A Diversified strategy spreads out your Spot instance allocation across all available pools.
Go with this strategy when you have enough flexibility to use various instance types or across multiple availability zones to increase the chances of obtaining instances.
This strategy might be best if you’re operating across multiple regions or if you run several applications each with different resource requirements.
Price & capacity-optimized
A Price & capacity-optimized allocation strategy balances both price and availability, offering the best chances for spot capacity without compromising on availability.
It’s great in situations where you experience varying demand throughout the day, but offer services that require higher availability — for instance, if you’re operating an ecommerce store.
What can go wrong with Spot instances in ASGs?
There are two primary concerns among AWS customers when it comes to including Spot instances — or expanding their usage — in their ASGs: 1) reliability and 2) configuration complexity.
As Spot instances can be interrupted by AWS with a two-minute notification, it potentially impacts application reliability. As mentioned above, this may not matter as much depending on the use case.
More importantly, implementing Spot Instances in Auto Scaling Groups can be challenging due to the need for proper configuration of launch templates, mixed instance policies, and other settings.
Furthermore, reaching the most suitable configuration for your ASGs to run reliably and cost-effectively with Spot instances requires complex analysis of multiple variables. You need to select the right instance types, AZs, VPCs, and allocation strategies among other variables to ensure optimal capacity and cost savings with Spot Instances. This may involve specifying multiple Availability Zones, setting up scaling policies, and integrating with other AWS services.
Spot Scaling
DoiT Spot Scaling helps you better adopt — and thus save with — Spot instances by recommending best practice configurations to your ASGs.
It manages the instance composition of your Auto Scaling Groups (ASGs) so that you can leverage Spot instances without worrying about interruptions.
For example, below we have an Auto Scaling group that could be configured better to maximize Spot savings and availability. In the current configuration, it only allows for one instance type and only allows for on-demand instances.
However, Spot Scaling recommends additional instance types that have similar attributes to the selected instance types, expanding the number of potential instance pools to tap into.
By allowing for more instance types and availability zones, you decrease the chances that there will be no Spot instances
In the event that there is no spare capacity in any of the instance pools, Spot Scaling will automatically revert to using on-demand instances (even if your mixed instance policy is 100% Spot), until Spot instances are available again.
All you have to do is review your recommendations, modify them as you see fit, and then click “Apply Recommendations”. From then on, Spot Scaling will manage that ASG for you.
How CattleEye reduced EC2 spend by over 60% with Spot Scaling
Running inference pipelines on AWS Batch, CattleEye applies computer vision technology to off-the-shelf cameras, monitoring cows and analyzing footage to improve the welfare of over a million cows.
But real-time analysis of cows is a compute-intensive job, and AWS EC2 drives the majority of CattleEye’s costs. At first they were using GPU instances, which made it difficult to leverage Spot instances. Additionally, they didn’t have time to research other suitable instances that would’ve allowed them to leverage Spot instances more.
CattleEye leveraged Spot Scaling, to better adopt Spot instances. Initially relying solely on on-demand instances, now their batch processing runs 100% on Spot instances – and CattleEye saves over 60% on EC2 as a result.
Take an interactive tour of Spot Scaling here
Conclusion
The complexity of setting up and managing spot instances within ASGs can be daunting, but tools like DoiT Spot Scaling streamline the process. By recommending optimal configurations and managing instance composition dynamically, Spot Scaling ensures maximum spot savings while maintaining availability.
If you’re deploying EC2 instances with Auto Scaling groups, learn more about Spot Scaling here or get in touch with us to learn more about how DoiT can help you optimize compute costs with Spot instances and your cloud bill overall.
