Save more in Elasticache RI with Botmetric

Reserved Instances (RI) have brought in significant opportunities in cost reducing strategies in the AWS ecosystem. AWS started with EC2 Reserved Instance (RI)  and EC2 is the most reserved AWS service. Did you know that apart from EC2 there are other six services that can be reserved in AWS?

Currently Botmetric supports AWS RI for EC2 and RDS under Cost & Governance. A lot of Botmetric customers have been using Elasticache and wanted to seek better recommendations for reservations. As reservations are crucial in cost savings, we as Botmetric wanted to empower our customers with reservation planner for Elasticache.

Botmetric Cost & Governance has now extended its support for Elasticache RI’s. Currently in  Cost & Governance RI tab you can manage, track and regulate your Elasticache RI’s. Elasticache RI under Cost & Governance only supports high utilization RI’s at the moment.

Elasticache RI Dashboard

Elasticache RI has a holistic dashboard that is similar to EC2 and RDS RI, which lists out RI summary and savings recommendations for variant tenure.

Elasticache

Elasticache RI has a holistic dashboard that is similar to EC2 and RDS RI, which lists out RI summary and savings recommendations for variant tenure.

Unused Elasticache RI

Provides a list of reservations which aren’t being used by any running instance, so that you can launch a matching instance class for the reservation.

Plan RI

Provides a list of all the recommended ElastiCache reservations. Botmetric analyzes your ElastiCache on-demand utilization and generates recommendation after matching it with your current set of reservations. You can change the tenure from 1 to 3 year to understand variant savings. You also change the confidence factor to view more differentiated recommendations.

Elastic Cache Plan RI
Existing RI

This is a complete portfolio of your Elasticache RI. It lists all Elasticache RI in your account.
The chief advantage of existing RI portfolio is to identify near-expiry RI and plan for renewal or renew already expired RI.

existing ri

RI Utilization Graph

Here you can identify and understand the performance of Reserved vs On-demand Elasticache RI. You can look at the graph and identify if your On-demand hours are constant , then there is more scope for reservations.

RI Utilization We as Botmetric want to make cost savings in AWS cloud more simpler and better. We have not stopped with EC2, RDS and ElastiCache, we are working on adopting Botmetric Smart Recommendations for Redshift and DynamodB as well. Meanwhile if you wish you perform any kind of cost analysis how much ever detailed, you can achieve this through Custom Reports.

Introducing Enterprise Budgeting – Every CFO's Success Formula in Cloud

Cost budgeting in a large company is an exhaustive process. A tremendous amount of detail and input goes into this iterative procedure where each senior team member brings a cost budget from his or her team and the finance leader integrates it and then negotiates with the senior team members to get the numbers where they need to be. Budgeting is a collective process in which each individual operating units or Cost Centers prepare their own budget in conformity with company goals published by top management. Since cloud is quite scalable and often teams exceed their budget or don’t have a clear visibility over projected spend which leads to budget mismanagement and overall havoc for IT Directors to re-evaluate budget and get the approval of the Finance department. Also, at times IT Directors wish if they were able to set budgets at a very granular level that could diminish any kind of uncertainty. This is where Botmetric’s Budgeting can help you create a comprehensive budget model.

So, what is Enterprise Budgeting?

Botmetric’s new feature ‘Budgeting’ under Cost & Governance, will empower the financial leaders in your organization to set the budget and track it with seamless workflows and processes. The two inputs imperative to the budgeting process in a large enterprise are, a detailed cost model for the entire payer account and a comprehensive cost model for individual Cost Center based on linked account(s) and tags.

Who will benefit from Enterprise Budgeting?

Enterprise budgeting is a powerful tool which will be helpful to senior level professionals such as CFOs, CTOs, IT-Directors, Head of Infrastructure & Engg, Senior IT Managers and more.

Which Botmetric subscription plans have access to Budgeting?

Currently, we are enabling the Budgeting feature for Professional, Premium and Enterprise plans only, on request basis.

Botmetric Workflows Used in Budgeting:

The following workflow can be assigned to the people using budgeting:

  • User: User workflows with write permission will be allowed to only set the budget which will then be sent to a financial admin for approval.
  • Admin: Admin workflows/roles can provide the user with read and write access to budgeting. An admin can set the budget but only a financial admin can approve it.
  • Financial Admin: A Botmetric admin can also be a financial admin whose role will be to define the budget goal in Budgeting and approve the budget set by other users. By default, the owner of a Botmetric account will also be a financial admin.  

Add New User Smart Budgeting

Understanding Botmetric’s New Smart Cost Center

A Cost Center can be a department or any business unit in the company whose performance is usually evaluated through the comparison of budgeted to actual costs. Previously, Botmetric allowed you to create a Cost Center using tag-keys like ‘owner’, ‘customer’, ’role’, ‘team’ etc. Now, as per extensive budgeting requirements, Cost Centers in Botmetric can be defined in two ways-  based on tag keys alone and based on accounts and associated tag key-value pairs.

  • Based on Tag-Key

Here, you can choose the tag key which corresponds to your cost center. Based on the chosen tag key, Botmetric will create all possible cost centers for the tag values corresponding it.

add-cost-center-smart-budgeting

  • Based on account(s) or combination of multiple account(s) and tags

You can also create Cost Centers based on account(s) and customize them based on multiple grouping of tag keys. You can create a cost center group such as  account1->team1->role1.

add-cost-center2-enterprise-budgeting

Let’s say you have different nomenclature for the same tag-keys such as user:TEAM, user:team, user:Team, then you can multi-select these tags and get complete clarity on your cost center group.

add-cost-center3-enterprise-budgeting

Please note that you can only choose one option at a time. You cannot have a few cost centers created based on tags and few on account and tags combination.

How to set, track and monitor the budget?

  1. Allocate & Review

  •  Budget Goal:

Botmetric budgeting enables the financial leader to define a budget goal for the entire payer account as per his estimations for the financial year. You can either enter the budget inputs manually or you can use Botmetric’s estimate to populate the budget inputs across months, quarter and year. Botmetric looks at the data for the last 12 months for yearly budget tracking.

Based on your company size it can take upto 72 hours of time to enable, process and crunch your data.

  •    Assigning Budget to Individual Cost Center:

Individual Cost Center owner(s) or financial admin(s) can set/edit budget goals for their respective units. The owner or financial admin(s) can either enter the budget inputs manually or make use of Botmetric’s estimate to populate the budget inputs across months, quarter and year. If a non financial admin or user is creating the budget for his Cost Center, it will be sent to a financial admin for approval. The new roles provided for Budgeting are helpful for providing clear demarcation between users and financial admin(s). This will allow financial admin(s) to have control over the approval of budget while providing enough flexibility to the other roles to manage their Cost Centers effectively.

enterprise-budgeted-cost-center

  1. Budget Overview

Botmetric’s Budgeting Overview provides a summarised view where you can see a snapshot of your financial year performance at the payer account level. You can compare the actual, allocated and projected spends for the current month, current quarter and financial year. You can alse see a list of top spending Cost Centers for the current month and current quarter. Moreover, a complete trend graph comparing your actual, allocated and budgeted spend performance at a payer account level for 12 months and 4 quarters will help you evaluate Budgeting with a quick glance.

enterprise-budgeting-allocated-yearly-budget

  1. Cost Center View

Botmetric’s Cost Center Overview provides a comprehensive view to track the performance for each Cost Center. Fine grained resources and service details provide a deeper and instantaneous understanding of where a certain Cost Center is incurring more cost. Ability to shuffle the view among monthly, quarterly and yearly options will allow the user to understand the budget variance over time. Each Cost Center will be evaluated to determine whether its incurred cost is within the allocated budget or it has exceeded the defined budget limit.

Moreover, each Cost Center has a corresponding budget trend graph to show the comparison between actual,  allocated and estimated spend. If you have a huge list of Cost Centers in your cloud, the search bar will help you to quickly find the desired Cost Center.

enterprise-budgeting-cost-center-overview
Botmetric’s Enterprise Budgeting will empower IT budget owners to define and track budgets at every granular level. This will also streamline budget processes in your organization and bring composure in the chaotic world of budget goals setting. Signup for 14 days free trial and check how it can help your organisation in cloud cost saving.

Botmetric Brings Unique Click to Fix for AWS Security Group Audit

In today’s day and age, deploying solutions and products on the cloud has become the new norm. However, managing your cloud infrastructure, implementing critical cloud security controls, and preventing vulnerabilities can become quite challenging.

Security & Compliance

Botmetric’s Security & Compliance simplifies the process of discovering and rectifying the threats as well as shortcomings in your AWS infrastructure by providing a comprehensive set of audits and recommendations, which saves a lot of time and makes eliminating unused Security Groups easy.

Botmetric’s Security & Compliance imbibes culture of continuous security and DevSecOps by automating industry best practices for cloud compliance and security. For an AWS user this simplifies the process of discovering and rectifying the threats.

Remediation of Security Threats with Botmetric

At Botmetric, we believe in simplifying cloud management for our customers. To amplify this, we provide the ‘click to fix’ feature for many of our Security & Compliance audits. This feature enables users to implement the best practices recommended by Botmetric simply with the click of a button. While saving a lot of time and effort, it also eliminates the possibility of human error. Moreover, rather than manually fixing each and every resource, Botmetric allows you to select multiple resources and fix them all at once.  

Click to Fix Security Group Audit

In an effort to allow our users to easily secure their cloud, we have recently added ‘click to fix’ feature for all Botmetric security group audits.

Why Botmetric Built Click to Fix for AWS Security Group Audits?

Security groups in AWS provide an efficient way to assign access to resources on your network. The rules that you define in security groups should be scrutinized. For a simple reason that you could end up giving a wide open access resulting in an increased risk of security breaches. The security group audits provided by Botmetric discover issues, such as as security groups having rules with TCP/UDP ports open to public, servers open to public, port ranges open to public,  so on and so forth. These are serious security loopholes that could leave your cloud open to malicious attacks.

Botmetric’s ‘click to fix’ feature for AWS security group audits deletes the vulnerable security group rule, thereby securing access to your cloud resources and protecting your cloud infrastructure.

Botmetric- Click to Fix

List of AWS Security Group Audits provided by Botmetric

  • Database Ports

Protecting database ports is crucial as you wouldn’t want access leaks or open ports to your Database ports. Botmetric scans your database ports open to public, IP and private subnet. Securing these would ensure your database ports safety in a security group.

  • Server Ports

Very essential as a lot of security issues and vulnerabilities have been caused due to server ports. Botmetric secures ports open to public, IP and private subnet.

  • TCP UDP  and ICMP Ports

Relies everything we do on the internet, here Botmetric secures open ports to both public and IP.

There are few more controls for Security Group such as All Traffic and Port Range also secured by the audits.

How to Enable Click to Fix for AWS Security Group Audits?

To use the click to fix for security group audits, please ensure that you have added “ec2:RevokeSecurityGroupIngress” permission to the policy of the role whose ARN is configured for Security and Compliance.

The Bottom line:

At Botmetric, we will continue to add more AWS cloud security and compliance features. We will soon come up with a detailed post on Click to Fix feature for several key AWS Security Audits. Until then stay tuned with us.

This is a newly launched feature by Botmetric. To explore this feature, take up a 14 day trial . If you have any questions on AWS security or AWS security best practices, just drop in a line below in the comment section or Tweet to us at @BotmetricHQ.

Top 5 Cloud Migration Challenges- An Analysis by Minjar

When your enterprise is migrating to the public cloud, you might be faced with various challenges at each stage of migration. In the course of its ever-evolving partnership with AWS and recently with Azure, Minjar has analyzed that there are 5 major challenges that any enterprise will face while they are undergoing the migration process.

These challenges are:

  1. Time/duration
  2. Complexity
  3. Risk
  4. Cost
  5. Security

AWS Cloud Migration Analysis by MinjarFig.1. Minjar Analysis

Time/Duration: The process of migration or rather the replication process, faces a huge challenge during the process of transferring large data sets to the cloud. Once migration is complete, it is important to make sure that the processes are consistent and in tandem. The less time-taken to automate services on the cloud, the more happy clients you get.

For example; SingPost SAM was migrated to AWS Cloud infrastructure by Minjar. Singapore Post approached Minjar with the requirement of automating their cloud deployments, auto scaling their business solution and tighten their security. The company also required 24×7 monitoring, managing and continuous improvement of their AWS infrastructure.

Upon onboarding Minjar re-architected SP eCommerce application infrastructure for high availability, enhanced performance and lower latency. Minjar enforced AWS security best practices, created a foolproof environment by implementing DDoS, VAPT, WAF and data security. It created a VPN setup to connect AWS environment with Singapore post’s network. Minjar automated their application deployment and achieved a 10x reduction in launch time which further reduced the cost by implementing auto scaling and optimized provisioning.

Complexity: The complexity of migration can be reduced with the help of tools. It provides ground to support cloud migration in order to avoid the complexities of the tasks. Minjar helped Cleartrip to migrate to cloud by using the services of AWS products like Virtual Private Cloud (VPC), Elastic Compute Cloud (EC2), Elastic MapReduce (EMR), Relational Database Service (RDS) – MySQL, Route 53, Lambda, Snowball, Simple Storage Service (S3)

Risk: A tip for all the enterprises that plan to migrate to cloud is to first test the tools. This helps to avoid uninvited surprises which can incur cost and time of your business. This can happen due to human intervention. Automating day to day processes with the help of Botmetric Ops and Automation, a premium product by Minjar is a one-stop-solution to overcome the risk of migration.

For example; Shaadi.com evaluated options and decided to migrate its website infrastructure from its legacy cloud managed service to Amazon Web Service (AWS) Cloud, as AWS offers a comprehensive portfolio of services, competitive pricing and allows rapid innovation. However migrating to AWS seemed overwhelming due to the inherent risk involved in migrating its complete online business without impacting its customers. Consequently, Shaadi.com team selected Minjar as its AWS experts to execute the migration.

Cost: The cost and performance of your migration are mutually dependent. Agility of migration in a secure and cost-effective way can transform the enterprise’s business. Through proper management of cost, provisioning of servers and continuous checking of underutilized resources you can reduce the spend on the cloud in an effective way. Botmetric Cost and Governance has been helping Minjar since its inception, to get along well with cost management. With this product, Minjar could easily help companies to adopt cloud environment in a seamless way.

For Bigbasket the services offered in Deployment Automation was done within 2 weeks. Migration of infrastructure was done from Singapore to Mumbai Region which was done per plan in under a month. Automation done by MSP team on production deployment which brought down the deployment task from hours to minutes and the automation done during migration which brought down the total migration downtime to less than an hour helped them.

Security: This is the crucial point where many companies will think twice before cloud migration activity. Minjar uses Botmetric Compliance and Security to overcome this. It helps in finding security loopholes and rectifying it through the product. According to LinkedIn Information Security Community survey, 49% of CIOs and CSOs feel thatone of the major barriers to cloud adoption is the fear of data loss and leakage… 59% believe that traditional network security tools/appliances worked only somewhat or not at all” in the cloud.

Samsung, MAPP, Cleartrip, Godrej, Shaadi.com and now Bigbasket have relied on Minjar for their Cloud migration and automation processes. From the analysis it is understood that Minjar has enhanced businesses by synergising cloud platform to their operations.

Wrapping Up:

Any change invites huge challenge and migrating to cloud is one among those challenges faced by your enterprise during the course of time. We have analyzed and put down a table that speaks a lot when you Do Cloud Magic with Minjar.

Challenges Migration Minjar
Duration More Less
Complexity High Low
Risk High Low
Cost High Low
Security Low High

Cloud migration needs to be secure, agile and seamless. It should synergize the operations of your enterprise by automating the activities and helping business to reduce the total spend on resources.

Botmetric Cost & Governance for Advanced Budget Alerting

IT budgeting can start with a painful process but end in crafting a better strategy and road mapping. Post cloud adoption you grew and so did your cloud spend and keeping budget spend at par was always required, so that you have the money for resources that are need of the hour and required reservations. There are various mechanism to control budgets, and alerts are the easiest way to control your budgets.

Botmetric’s Cost & Governance has a crisp budget alerting where when amount exceeded on payer account will trigger alerts. A lot of customer requests for more filtered budget alerting for a much-focused cost management.

With new budget alerts, now Botmetric users can:

Set budget alert for linked accounts

Now you can configure budget alerts for linked accounts along daily, weekly and monthly filters. So from now, if any of the linked accounts are exceeding your set accepted budget figure, you will get notified instantly.

Set budget alert for cost center

A lot of businesses have an understanding of their silos in cloud infrastructure in the form of cost centers. They gauge spend and addition of resources in terms of cost centers. For them budget alerts for cost center will empower in setting set accepted budget figure, which when exceeded will be notified instantly

Set budget alert for custom group

Very powerful budgeting feature where you can create a group with different rules and filters to create a custom defined budget alerts.

Custom budget alert has rules for:

  •  Linked accounts
  •  AWS services
  •  Reserved Instances
  •  AWS Tags

Accompanied across filters for:

  • Daily
  • Weekly
  • Monthly

Example: Suppose you are looking for EC2 RI’s in a linked account for <tag:Production> if exceeds $10000 to send an alert, you can create a custom group for this alert and get updated whenever set budget exceeds.

Budget alerts are crucial to keep your cloud finance in place and keep you always informed.

Dynamically Increase AWS EBS Capacity On-the-Go Now with New Elastic Volumes

Say goodbye to scheduling downtime while modifying Elastic Block Storage (EBS) volumes. No more bottlenecks. Modify these EBS volumes on-the-go. Here’s why: AWS announces new feature to its EBS portfolio, called Elastic Volumes, which will help you automate changes to your EBS workloads without going offline or impacting your operations. Plus, grow your volume, change your IOPS, or change your volume types too, as your requirements evolve. All without the need for scheduling downtime. And with today’s 24×7 operating models, it is more important than ever to have no room for that downtime.

Elastic Volumes: What is it about

EBS workloads are known to optimize capacity, performance, or cost by allowing you to increase volume size, adjust performance, and change volume type as and when the need arises. Primarily, due to its dynamic nature and the ability to offer persistence high-performance block storage for AWS EC2.

Prior to the launch of Elastic Volumes, you had to schedule a downtime to that end, perform several steps like create a snapshot, restore it to a new volume, and attach this snapshot to a EC2 instance as and when your data volume grows.

Now, with the launch of Elastic Volumes, AWS has simplified the process of modifying EBS volumes drastically. You can also use CloudWatch or CloudFormation, along with AWS Lambda,  to automate EBS volume modifications, without any down time.

AWS, in one of its blogs, says that Elastic Volumes reduce the amount of work and planning needed when managing space for EC2 instances. Instead of a traditional provisioning cycle that can take weeks or months, you can make changes to your storage infrastructure instantaneously, with a simple API call.

Essentially with AWS Elastic Volumes, as per AWS, you can:

  1. Change workloads: For instance, at some point, you realize that Throughput Optimized volumes are a better fit and need to change the type of the volume. You can do so easily with this new feature, without any downtime.
  2. Better handle the spiking demands: Assume, you’re running a relational database on a Provisioned IOPS volume that is set to handle a moderate amount of traffic during the month. You observe ten fold increase in traffic during the final three days of each month due to month-end processing. In this scenario, you can use this new feature provision right, handle the spike, and then dial it down once the spike tones down.
  3. Increase storage: Suppose, you need to provision a volume for 100 GiB. An alert alarm goes off indicating that it is now at 90% of capacity (disk-almost-full). Using this new feature, you can increase size of the volume and expand file system to match, with no downtime, and in a fully automated fashion. You can also use Botmetric Ops & Automation’s Incidents, Actions & Triggers app, which can help you automate increase in size of the volume as soon as this disk-almost-full alert gets triggered. Instead of manually working on it, Botmetric will help you right-size the volume based on the criterion decreed in the respective Actions and Triggers. To know more about Botmetric Incidents, Actions & Trigger, read here.

How to go about it:

It’s very simple to configure:

  1. Sign in to AWS Console
  2. Select Amazon EBS
  3. Right click on the Volume you wish to modify

create volume in aws

Image Source: Amazon Web Services

  1. Modify Volume

Modify Volume AWS

Image Source: Amazon Web Services

  1. Check the progress, whether modified, optimized, or completed.

Modify Volume in aws

Image Source: Amazon Web Services

Limitations:

While the new feature helps increase capacity, tune performance, and change volume types on-the-fly, without disruption, and with single-click, it comes with certain restrictions:

  • Your volume needs to be detached or the instance stopped for modification to proceed, if you encounter an error message while attempting to apply a modification to an EBS volume, or if you are modifying an EBS volume attached to a previous-generation instance type
  • The previous generation Magnetic volume type is not supported by the volume modification methods
  • Decreasing the size of an EBS volume is not supported. However, you can create a smaller volume and then migrate your data to it using application-level tools such as robocopy
  • Modifying a volume, you need to wait at least six hours before applying further modifications to the same volume
  • medium instances are treated as current generation. M3.large, m3.xlarge, and m3.2xl instances are treated as previous generation.

Conclusion:

With the launch of Elastic Volumes, AWS EBS is now more elastic. The best part, you can change an EBS volume’s size or performance characteristics when it’s still attached to and in use by an EC2 instance.

Check out the AWS video to know more:

Containerized Application Deployment on AWS using Docker Cloud

­­­­­­

Docker Cloud, Docker Inc.’s hosted service that helps deploy and manage Dockerized applications, is widely used by DevOps engineers across the world. For the fact that it provides a registry with build and testing facilities for Dockerized application images. It also helps set up and manage host infrastructure, and deployment features to help automate deploying images to an infrastructure.

If you are a DevOps engineer looking to deploy containerized application on Amazon Web Service (AWS) using Docker Cloud, you are at the right place. Here’s a detailed step-by-step guide on how to go about it.

Your Docker Cloud Account and Docker ID

To start with, you need to log in to Docker Cloud using your free Docker ID. Your Docker ID is the same set of credentials you used to log in to the Docker Hub, and this allows you to access your Docker Hub repositories from Docker Cloud.

Images, Builds, and Testing

Docker Cloud uses Docker Hub as an online registry service. This allows you to publish Dockerized images on the internet either publicly or privately. Along with the ability to store pre-built images, Docker Cloud can link to your source code repositories and manage building and testing your images before pushing the images.

Infrastructure Management

Before you can do anything with images, you need to run them somewhere. So, Docker Cloud allows you to link to your infrastructure or cloud service provider, which lets you provision new nodes automatically, and thus help deploy images directly from your Docker Cloud repositories onto your infrastructure hosts.

For illustration, in this post, I am using Amazon Web Services (AWS) as the cloud service provider.

Step1. Link Docker Cloud with your Cloud Infrastructure

When you login to Docker Cloud, the below page appears.

Containerized Application Deployment on AWS using Docker Cloud

Click on first box (Link Provider), which is asking you to link to a hosted cloud service provider like DigitalOcean or AWS.

A page appears as shown below.

Containerized Application Deployment on AWS using Docker Cloud

Click on the plug shaped icon and fill your AWS Secret Key and Access Key (The Key associated to that user who have EC2 Full Access Policy attached).

Now, you’re ready to provision the nodes on AWS cloud.

Step2. Create a Node

Go back to the welcome page and click on the second box, which requests you to create a cluster node.

Click on Create and fill every details: Name of the node, details of the provider like AWS, Region, VPC, subnet-id, sec-group, instance type,IAM role, and node disk size, as well as the number of nodes you want.

And then click on Launch node cluster.

After some time you will get to see a page as shown below.

Now, when you take a peek into your AWS console, you will see three instances launched.

Step3: Create a Service

Go back to the welcome page and click on the third box, which requests you to create a service.

Click on Create Service button to open the Services/Wizard page as shown below.

Go to Miscellaneous section and click on dockercloud/hello-world. This will take you to the settings page as shown below.

Fill in the requisite details and go to Container configuration where you can set the entry point, memory limit, CPU limit and the command you want to run to spin up the container.

Now, publish the exposed port 80.

Note: If you want to link this container with another container, then you can use Links. And if you want to set any environment variable or want to attach or mount any volume, you can use Environment variable and Volume section. Once done, click on Create and Deploy. The below image will give you an idea.

When you click on Create and Deploy, your application service will deploy on all the three nodes created in Step 2.

At this moment, your containerized application is deployed on AWS. You can verify it by clicking on endpoints. The below image says it all!

And, voila! There you go.

See its very simple to deploy the containerized application on AWS in no time with Docker Cloud  ?

There are lot other features inside it like monitoring, autoscale, load balancing, etc. You can use these features when the deployment strategy needs to focus on high availability. Need help in deploying containerized application on AWS with the Docker Cloud? We at Minjar have your back! Do share your comments and thought with us on Twitter, Facebook or LinkedIn. You can drop in a comment below too.

P.S: Read my other blog post, Blue Whale Docker on AWS as CaaS, and How Enterprises Can Use it! to know how to leverage DDC as CaaS on AWS.

Top 11 Hard-Won Lessons We’ve Learned about AWS Auto Scaling

Auto scaling, as we know today, is one of the most powerful tools leveraging the elasticity feature of public cloud – Amazon Web Services (AWS). Its ability to improve the availability of an application or a service, while still keeping cloud infrastructure costs under check, has been applauded by many enterprises across verticals, be it fleet management services or NASA’s research base.

However, at times, AWS Auto Scaling can be a double-edged sword. For the reason that, it introduces higher level complexity in the technology architecture and daily operations management. Without the proper configuration and testing, it might do more harm than good. Even so, all these challenges can be nullified with few precautions. To this end, we’ve collated few lessons we learned over a period – to help you make the most of Auto Scaling capabilities on AWS.

  1. Use Auto Scaling, whether your application is stateful or dynamic

There is a myth among many AWS users that AWS Auto Scaling is hard to use and not so useful with stateful applications. However, the fact is that it is not hard to use. You can get started in minutes, with few precautionary measures like using sticky sessions, keeping provisioning time to minimum, etc. Plus, AWS Auto Scaling helps monitor the instances and heals them if they become unhealthy.

Here’s how: Once the Auto Scaling is activated, it automatically creates an Auto Scaling Group, and provisions the instances accordingly behind the load balancer. This maintains the performance of the application. In addition, Auto Scaling’s Rebalance feature ensures that your capacity is automatically distributed among several availability zone to maximize the resilience of the application. So, whether your application is stateful or dynamic, AWS Auto Scaling helps maintain its performance irrespective of compute capacity demands.

  1. Identify the metrics that impact the performance, during capacity planning

Identify the metrics for the constraining resources, like CPU utilization, memory utilization, of an application. By doing so, it will help track how the resources are impacting the performance of the application. And the result of this analysis will provide the threshold values that will help scale up and scale down the resources perfectly.

  1. Configure AWS CloudWatch to track the identified metrics

The best way forward is to configure Auto Scaling with AWS CloudWatch so that you can fetch these metrics, as and when needed. Using CloudWatch, you can track the metrics in real-time. CloudWatch can be configured to launch the provisioning of an auto scaling group based on the state of a particular metric. 

  1. Understand functionality of Auto Scaling Groups while using Dynamic Auto Scaling

The resource configurations have to be specified in Auto Scaling groups feature provided by AWS. Auto scaling groups would also include rules defining circumstances under which the resources will be launched dynamically. AWS allows assigning the of autoscale groups to the Elastic Load Balancers (ELBs) so that the requests coming to the load balancers are routed to the newly deployed resources whenever they are commissioned.

  1. Use Custom Metrics for Complex Auto Scaling Policies

A practical auto-scaling policy must include multiple metrics, instead of just one allowed by CloudWatch. The best approach to circumvent this restriction is to code a custom metric as a Boolean function using Python and the Boto framework. You can use application specific metric as well along with default metrics like memory utilization or CPU, network, etc.

  1. Use Simple Queuing Services

As an alternative to writing complex code for the custom metric, you can also architect your applications to take requests from a Simple Queuing Service and enable CloudWatch to monitor the length of the queues to decide the scale of the computing environment based on the amount of items in the queue at a given time.

  1. Create Custom Amazon Machine Images (AMIs)

To reduce the time taken to provision instances that contain many custom software (not included in the standard AMIs), you can create a custom AMI that contains the software components and libraries required to create the server instance.

  1. Scaling up other AWS services other than EC2, like AWS DynamoDB

Along with AWS EC2, other resources such as AWS DynamoDB, can also be scaled up and scaled down using Auto Scaling. However, the implementation of the policies are different. Since storage is the second most important service other than computing service, efforts to optimize storage will yield good performance as well as cost benefits.

  1. Predictive Analytics for Proactive Management

Setting up thresholds as described above is reactive. Hence, you can leverage time-series prediction analytics to identify patterns within the traffic logs and ensure that the resources are scaled up at pre-defined time, before events take place.

  1. Custom define Auto Scaling policies & provision AZs capacity accordingly

Auto scaling policies must be defined based on the capacity needs as per Availability Zone (AZ) to save on cost spikes. Because pricing of the resources are based on different regions that encompass these AZs. This is critical especially for Auto Scaling groups configured to leverage multiple AZs along with a percent-based scaling policy.

  1. Use Reactive Scaling policies on top of schedule scaling feature

By using Reactive Scaling policies on top of schedule scaling feature will give you the ability to really respond to the dynamic changing conditions in your application.

Conclusion:

Embrace an intelligent cloud management platform.

Here’s why: Despite configuring CloudWatch and other features of Auto Scaling, you cannot always get everything you need. Further automating various Auto Scaling features using key data-driven insights is the way forward. So, sign-up for an intelligent cloud management platform like Botmetric, which throws key insights to manage AWS Auto Scaling, provides detailed predictive analytics and helps you leapfrog your business towards digital disruption.

Also, do listen to Andre Dufour’s recent keynote on AWS Auto Scaling during the recent 2016 re:Invent, where he reveals that Auto Scaling feature will be available to Amazon EMR (Elastic Map Reduce) service as well along with AWS ECS Container service, and Spot Fleet in regards to dynamic scheduling policies.

It is evident. Automation in every field is upon us. There will soon be a time when we will reach the NoOps state. If you have any questions in regards to AWS Auto Scaling or how you can minimize Ops work with scheduled Auto Scaling or anything about cloud management, just comment below or give us a shout out on Twitter, Facebook, or LinkedIn. We’re all ears! Botmetric Cloud Geeks are here to help.

Here’s Botmetric Wishing Thank You for a Fantastic 2016 & Happy 2017

Dear You,

We are thrilled to close this year with a bang! It’s hard to sum up 2016 in just a few scribbles, especially when we made many new friends and rolled-out so many enhancements & new features: a new platform, new audits, more ingrained intelligence, more cloud optimization options, a revamped website with new UI, and much much more.

And as the New Year sets in slowly across the world, minute by minute, second by second, and continues its journey with the same charm and diligence, we at Botmetric, likewise, will continue our journey towards making cloud management and optimization a breeze for you. With rolled sleeves. With more focus. With more features. And with more zeal. Everything for you, dear customer.

Here’s 2016 Year-in-Review: The Best Year so far, for Botmetric

Continues to be the Highest Rated Product on AWS Marketplace

We attribute our success to our dear customers. Thank you for the timely feedback, and those wonderful testimonials bedecked with five stars. Based on the feedback and our learning, we revamped Botmetric into a platform of three products that are use-case targeted instead of a ‘one-size-fits-all’ approach:

1. Cost management & Governance: This Botmetric product helps you control your cloud spend, save that extra bit, optimize spend through intelligent analytics and allocate cloud resources wisely for maximum ROI. It is built for businesses and CIO teams to enable them in decision making & maximizing cloud ROI.

2. Security & Compliance: This Botmetric product helps you get compliant and keeps your cloud secure with automated audits, health checks, and best practices. It provides the most comprehensive list of automated health checks. It is built for CISO and Security Engineers to proactively identify issues and fix vulnerabilities before they become problems.

3. Ops & Automation: This Botmetric product helps you save time and effort you invest in automating cloud operations. It has built-in operational intelligence that can analyse problems and fix events in seconds. Above all, speeds-up DevOps. It is built for CTOs and DevOps Engineers seeking alert diagnostics, event intelligence, and out of the box automation.

Chose any of the above, or any two of the above, or choose all. Your wish, your products, tailored for your AWS cloud. With these three products, you can realize the full potential of your cloud without any information overload. Find insights that matter to you, in just one-click.

There’s More:

To celebrate what you’ve helped us achieve this year, we have put together few 2016 Botmetric facts:2016 Botmetric Facts

New and Key Botmetric Product Features Rolled-Out in 2016:

1. Data Transfer Analysis for AWS Cost Management: Provides insights into your bandwidth costs on AWS cloud.

2. EC2 Cost Analysis to Optimize AWS Spend: Helps you understand your AWS EC2 spend easily and efficiently.

3. Internal Chargeback: Allocate cost easily across multiple cost centers and bring in the required parity in your AWS cost management.

4. Compliance Audit Policies for Heightened Security: Helps mitigate vulnerabilities in real time and adopt a comprehensive security management.

5. Cloud Reporting Module: Helps you quickly find your AWS cloud reports from one centralized module without scrolling, and to counter endless searching for what you need.

6. Reserved Instance Planner: Provides reservation recommendations at instance level. This revisited RI planner allows you to filter the recommendations, look at the details of the instance being recommended, and accordingly add it to a RI plan. You can also download the plan and work on budget approvals and actual reservations offline.

Work-in-progress

An Advanced DevOps Intelligence Feature: Assuages alert fatigue mess, helps easily understand the alert events through intelligence, and tells you why is it happening. It also checks for pattern in the problems.

We have much more coming up in 2017. So stay tuned with us.

Here’s Botmetric wishing you a very Happy New Year.

 

From,

Team Botmetric

 

Let’s blow the heartiest kisses to the cloud in 2017

Cloud is the new black. Let’s together embrace it more, with dexterity, in 2017.

Share all your 2016 cloud musings, learning, and accomplishments with Botmetric on Twitter, Facebook, or LinkedIn. We’re all ears, and we’ve your back.

Let’s make cloud an easier and a better place to grow our business.

 

P.S. If you have not signed-up with Botmetric yet, then go ahead and take a 14-day free trial. As an AWS user, we’re sure you’ll love it!

Introducing EC2 Cost Analysis in New Botmetric: A Game Plan to Optimize AWS Spend

Elastic Cloud Compute (EC2) is one of the most popular services of AWS and used by almost every Amazon cloud customer. And, in general,  EC2 usage accounts for 70 to 75% of AWS bill for an average AWS user. Moreover, most of the underlying services like EBS, EIP, ELB, NAT, etc. are used in conjunction with EC2 service for deploying applications on AWS cloud.

So, several unique EC2-related line items can show up on your AWS bill, which will further make it even more difficult to comprehend what’s driving all that spending.  A high-level view of the spend will not suffice. Because of this, it is critical to analyze EC2 usage and its spend breakdown by various dimensions like resource, instance type, services, accounts, and more while optimizing AWS costs.

To cater to this need and help our customers understand their AWS EC2 spend easily and efficiently, we have introduced the support for “EC2 Cost Analysis” in the ‘All-New’ Botmetric platform as part of its Cost Management and Governance’ Analyze feature.

Here’re the top features that the new Botmetric EC2 Cost Analysis offers:

1. Know your EC2 spend by instance type: You can quickly drill down and understand your total EC2 cost on AWS cloud split across different EC2 instance families. You can filter this further by various AWS accounts.

 

EC2 Cost Breakdown by instance type

 

2. EC2 cost breakdown by sub services: We have brought together the cost of EBS, EIP, ELB, Data Transfer, NAT Gateway under EC2 cost analysis module so you can easily see what is your mix of total spend across various EC2 related services. You can filter this cost further for any AWS account or month so you can drill into specific details. We also encourage you to drill down this analysis for a particular instance family.

 

EC2 cost breakdown by sub services

 

3. EC2 cost breakdown across different AWS accounts: You can split the EC2 cost across different AWS accounts in your master payer account so you can categorize them based on your usage.  

 

EC2 cost across different AWS account

 

4. Data export in CSV: We allow you to export different breakdown of EC2 cost into CSV file so you can use it for any internal analysis. The data export option allows you to see the cost breakdown by instance types, AWS accounts, related services, specific EC2 resources, etc.

 

Export data in CSV

You can access this feature in Botmetric Cloud Management Platform under Cost & Governance application in the Analyze Module. Please write to us with your feedback on what we can do better and where we can improve it further.

If you want to know some of the AWS EC2 cost saving tips that pro AWS users follow, read this Botmetric blog post. And if you want to know what are the other new features available in the new release of Botmetric, then take an exclusive 14-day trial or read the expert blog post by Botmetric Customer Success Manager, Richa Nath. Until our next blog post, do stay tuned with us on Twitter, Facebook, and  LinkedIn for other interesting news from us! 

Ace Your AWS Cost Management with New Botmetric’s Chargeback Feature

Editor’s Note: This exclusive feature launch blog post is authored by Botmetric CEO Vijay Rayapati.

How awesome would it be if you, as an AWS user, could define, control and understand the cost allocation by different cost centers in your organization, while providing an ability to generate chargeback invoices. It’s possible with the new Botmetric Cost Management & Governance application’s Chargeback module.

This Chargeback module has the same Cost Allocation and Internal Chargeback features from the earlier version of Botmetric, however, packaged in a new UI with augmented features. These features together will help you manage costs associated with  AWS infra in a smart way. Plus, you can manage budget controls better across projects and departments. More than that, users can manage complexities involved in treating chargebacks when dealing with multiple linked accounts from a central payer account. Additionally, you can gain access to many insightful reports for effective tracking and managing of AWS cloud costs.

Key benefits of the new Chargeback module in Botmetric includes:

  • Define Cost Allocation: We have implemented support for defining cost centers within your company by either department or team or business unit so you can split the overall spend into specific cost centers. This works based on your AWS tags for cost allocation and further defines a grouping within Botmetric for creating a cost centers for your business. You can have multiple centers for cost allocation in Botmetric.

Define Cost Allocation with Botmetric Cost & Governance

 

  • Control Unallocated Spend: From the Chargeback module dashboard, you can quickly identify the cost that is unallocated to any cost center in your business. This will allow you to split or allocate the cost into different cost centers and also inform your cloud team on missing tags for cost management.

 

ontrol Unallocated Spend with Botmetric Cost & Governance

 

  • Spend View within a Cost Center : We launched a drill down view so that you can understand the spend view by various AWS services within a cost center. This will allow you to inform the specific cost center teams if they are about to surpass their allocated budget for the month and let them know what is causing the increase in spend.

 

Spend View within a Cost Center

 

  • Download Chargeback and Cost Allocation Data: You can download the chargeback data for any cost center in CSV format. We also allow you to download the unallocated cost as a CSV file, so that you can share it with your IT team for inputs on how  cost should be allocated for different cost centers.

 

Download Chargeback and Cost Allocation Data

We are excited to release the general availability of enterprise cost allocation and chargeback support in Botmetric for AWS cloud. Now Botmetric customers can better manage their budgeting and internal reporting processes.

Please let us know if we can improve anything in Botmetric Chargeback module that can be more useful for your business. Do take a 14-day test drive, if you have not yet tried Botmetric.  

If you want to know several other new features available in the new release, then read the expert blog post by Botmetric Customer Success Manager, Richa Nath. For other interesting news from us, stay tuned on TwitterFacebook, and LinkedIn!

Make AWS Cloud Management a Breeze with New Botmetric Cloud Reporting Module

In an effort to help you quickly find your AWS cloud reports from one centralized module without scrolling, and to counter endless searching for what you need, we came up with a completely revamped AWS cloud reporting module in the ‘All-New’ Botmetric.

Over the years, we’ve become more and more aware of the fact that the key to optimizing the cloud infrastructure is to avoid common mistakes and implement the best practices using a right-sized cloud management console  that aggregates all the key data in one place. With the new Cloud Reporting feature, you can get an aggregated view of your AWS cloud health in a single pane, thus making AWS cloud management a breeze.   

Key benefits of Botmetric’s new AWS cloud reporting module include:

  • A centralized reporting module

We brought together all reports in one centralized module for Botmetric’s new suite of applications so that you can find everything you need in one place. You can now quickly filter the reports by application type (Cost & Governance, Security & Compliance or Ops & Automation) providing more clarity into your AWS’ spend, security, governance, and more. In essence, the new Cloud Reporting module will save time and reduce the manual search-overhead for you.

Centralized reporting module in 'all-new' Botmetric

  • Daily, weekly, and monthly split of reports

Under the new reporting module, you can quickly locate all daily, weekly, and monthly reports for Cost & Governance, Security & Compliance or Ops & Automation using the simple left side navigation filtering. You no longer need to scroll through all reports to find what you need.

View Split of Daily, Weekly, and Monthly Reports in Botmetric AWS Cloud Management

 

  • Filter reports by AWS accounts or month for deep drill down of data:

The new Cloud Reporting module now includes a quick filtering support too so that you can find the reports for specific AWS accounts or a month. This will help you navigate to historic reports and switch between different AWS accounts from a single pane.

Filter Reports by AWS Accounts or Month with Botmetric AWS Cloud Management Platform

What can you expect in the near future in Botmetric Cloud Reporting module for AWS cloud?

  • Create your own reports: Many of our customers asked us to give them the ability to create custom reports. We are working on this request. To this end, we’ll be rolling-out a custom reporting designer in Q1 of 2017.
  • Share reports via email: We will be soon launching report sharing support so that you can send specific reports to different stakeholders in your company, in a snap.

You can access the new Cloud Reporting module in the top right-hand corner of the Botmetric Cloud Management Platform, next to Notifications bar. We would love to hear how useful this module has been for you and how we can make it better to suit your needs.

We’ve also rolled-out the new compliance audit policies and improved security assessment in Botmetric to further simplify and help you resolve the most critical audit vulnerabilities. Give a quick shout out to us about the ‘All-New’ Botmetric. If you have not tried it, then sign-up for 14-day test drive  to experience how easy AWS cloud management is with Botmetric.  

Do read the  expert blog post  by Botmetric Customer Success Manager, Richa Nath, to unearth several other new features available in the new release. Stay tuned with us on  Twitter,  Facebook, and   LinkedIn  for other interesting news from us!