Tag: AWS Cost Management

4 ways AIOps will bring joy to Cloud Engineers

Posted on by Vijay Rayapati

The world of IT has evolved exponentially over the last decade with cloud being the new normal for enterprise businesses. From on-premise data centers to the rise of cloud and converged architecture of the IT operations has undergone a wave of process and procedural changes with DevOps philosophy. The not-so-IT companies like Amazon, Microsoft and Google have disrupted traditional infrastructure and IT operations by removing the heavy lifting of installing data centers, managing servers, networks and storage etc. so that engineers can put their focus back on applications and business operations instead of IT plumbing.

Above all, the DevOps philosophy is to save time and improve performance by bridging the gap between engineers and IT operations, however DevOps hasn’t truly delivered what was expected out of it as engineers still had to handle all the issues and events in their infrastructure whether in Cloud or on-prem data center.
There is a new philosophy emerging around “what if humans could solve new complex problems while we let machines resolve known, repetitive, and identifiable problems in cloud infrastructure management?” – this is known as “the AIOps philosophy” that is slowly taking root in cloud-native and cloud-first companies to reduce the dependency on engineers to resolve problems.  

Many enterprises have already adopted Cloud as key component of their IT and have limited their DevOps to configuration management and automated application deployments. Nurturing the AIOps philosophy will further eliminate the repetitive need for engineers to manage everyday operations and save precious engineering time to focus on business problems. While Cloud has made automation easy for engineers, it’s the lack of intelligence powering their day to day operations that is still causing operational fatigue for engineers even in the cloud world.

The adoption of Cloud and emergence of AI, ML technologies are allowing companies to use intelligent software automation rather than vanilla scripting to make decisions on known problems, predict issues and provide diagnostic information for the new problems to reduce the operational overhead for engineers. The era of pagers to wake up engineers in the middle of the night for down times and known issues will be a by-gone over the next 18 to 24 months.

In the traditional IT world, the main focus of operation engineers was to keep lights ON but in the world cloud, there are new dimensions like Variable Costs, API Driven Infrastructure Provisioning, No Centralised Security and Dynamic Changes that further increase the work burden.  The only way to help companies reduce their cloud costs, improve security compliance for on-demand provisioning, reduce alerts fatigue for engineers and bring intelligent machine operations to handle problems due to dynamic changes is through AIOps – Put AI to make Cloud work for your business.

Managing Enterprise Cloud Costs 

According to RightScale “State of Cloud 2017 Report”, managing cloud costs is the #1 priority for companies that are using Cloud computing. The cloud cost challenges are causing massive headache for finance, product and engineering teams within the organisations due to dynamic provisioning, auto-scaling support and lack of unused cloud resources garbage collection.  When hundreds of engineers within an enterprise use Cloud platforms like AWS, AZURE and Google for their applications, it will be impossible for one person to keep track of spend or deploy any centralised approval processes. Many companies like Botmetric are using machine intelligence and AI technologies to detect the cost spikes, provide deep visibility into who used what and help companies deploy intelligent automation to reduce unused resources and auto resize over provisioned servers, storage etc. in the cloud.

As IT infrastructure is an important factor to your business success, so is its need to understand the optimal usage limitations for your organizational IT infrastructure needs. In comparison on-prem cloud looks pretty easy because of “it’s pay-as-you-go” model, however when you grow exponentially you scale your cloud the same way and this gives you a bill shock. A lot of teams put in place tagging policies, rigorous monitoring but still as controlling cost is not the engineer’s way you still lack the edge. The process of continuous automation will help in reducing those misses in saving cloud cost. AIOps will put across the process of continuous saving in your cloud paradigm. For example: You can automate the process of purchase of Reserved Instances in AWS cloud through simple code with help of AWS Lambda. Another most favourable and most common use case is that of turning off dev instances over weekends and auto-turn back on while the start of weekdays, this saves upto 36% savings for most of cloud users.

Ensuring Cloud Security Compliance 

When any engineer within the organisation can provision a cloud resource through API call, how can businesses ensure that every cloud resource is provisioned with the right security compliance configuration needed for their business and satisfy regulatory requirements like PCI-DSS, ISO 27001, HIPPAA etc. This again requires a real time security compliance detection, informing the right user who provisioned the resource and take actions like shutting down machines if not complied to ensure your business stays protected. The most important part of security these days is continuous monitoring, and this can be achieved if you have a mechanism in place that detects and reports the next millisecond when the alert is received. A lot of organization are developing tools that not only detects security vulnerabilities but auto-resolves them. By leveraging AIOps and using the real time event configuration management data from cloud providers, companies can stay compliant and reduce their business risk.

Reduce Alert Fatigue

The problem of too many alerts is a known issue in the data center world, and is popularly called as Ops fatigue. The traditional NOC team (look at alert emails), IT support team (review tickets & respond) and then engineers looking into the critical problems was broken in the cloud world with DevOps Engineers managing all these tasks.

Also, anybody who managed production infrastructure, business services, applications and architected systems, knows that most of the problems are caused by the known events or identifiable patterns. Noisy Alerts are the common denominator in any IT operations management. With swarm of alerts flooding inboxes, it becomes highly difficult to manage which ones really matter or are ones to be looked upon by engineers. A great solution powered by anomaly detection would be to filter out unnecessary alerts or suppress duplicate alerts for a more concise alert management to detect real issues and predict problems. The engineers already have an idea on what to do when certain events or symptom occur in their application or production infrastructure. When events or alerts are triggered, most of the current tools just provide a text of what happened instead of providing a context of what is happening or why it’s happening? So as DevOps engineers, it’s important for you to create diagnostic scripts or programs so you can get a context of why CPU spiked? Why an application went down? Or why API latency increased? Essentially, to get to the root cause faster powered by intelligence. You should encourage them to deploy anomaly detection powered by machine intelligent and smart automated actions (response mechanisms) for known events with business logic embedded so team can sleep peacefully and never sweat again.

Intelligent Automation For Operations

The engineers responsible for managing the production operations (from ITOps to DevOps era) have been frustrated with the static tooling that’s mostly not intelligent. With the rise of machine intelligence and adoption of deep learning, we will see more of dynamic tooling that can help them in day to day operations.  In the Cloud world, the only magic wand for solving operational problems is to use code and automation as a weapon. Without using intelligent automation to operate your cloud infrastructure would only increase complexity for your DevOps teams. You can create everything from automated remediation actions to alert diagnostics. As a team and DevOps engineer, you need to focus on using CODE as a mechanism for resolving problems. If you are building the CI/CD today then you should certainly deploy a trigger as part of your CI/CD pipeline that can monitor deployment for health metrics and invoke a rollback if it detects performance or SLA issues. Simple remedies like this can save hours of time after every deployment and handle failures gracefully.

We will also see various ITSM vendors bringing AI & ML into their offerings like Intelligent Monitoring (without static thresholds for alerts instead of dynamic alerts), Intelligent Deployment (with cluster management and auto-healing tooling), Intelligent APM (not just what’s happening but why it’s happening due to what), Intelligent Log Management (real time streaming of log events and auto detection of relevant anomaly events based on application stack) and Intelligent Incident Management (suppression of noise from different alerting systems and providing diagnostics for engineers to get to the root cause faster).

The state of Cloud platforms and ITSM offerings is evolving at rapid pace, we are still to see newer concepts powered by AI and ML that revolve around disrupting cloud operations and infrastructure management to ease the pain for engineers to let them sleep peacefully in the night and not worry every time a pager goes off!

You can also read the original post here.

Introducing Enterprise Budgeting – Every CFO's Success Formula in Cloud

Posted on by Rajeev Kumar

Cost budgeting in a large company is an exhaustive process. A tremendous amount of detail and input goes into this iterative procedure where each senior team member brings a cost budget from his or her team and the finance leader integrates it and then negotiates with the senior team members to get the numbers where they need to be. Budgeting is a collective process in which each individual operating units or Cost Centers prepare their own budget in conformity with company goals published by top management. Since cloud is quite scalable and often teams exceed their budget or don’t have a clear visibility over projected spend which leads to budget mismanagement and overall havoc for IT Directors to re-evaluate budget and get the approval of the Finance department. Also, at times IT Directors wish if they were able to set budgets at a very granular level that could diminish any kind of uncertainty. This is where Botmetric’s Budgeting can help you create a comprehensive budget model.

So, what is Enterprise Budgeting?

Botmetric’s new feature ‘Budgeting’ under Cost & Governance, will empower the financial leaders in your organization to set the budget and track it with seamless workflows and processes. The two inputs imperative to the budgeting process in a large enterprise are, a detailed cost model for the entire payer account and a comprehensive cost model for individual Cost Center based on linked account(s) and tags.

Who will benefit from Enterprise Budgeting?

Enterprise budgeting is a powerful tool which will be helpful to senior level professionals such as CFOs, CTOs, IT-Directors, Head of Infrastructure & Engg, Senior IT Managers and more.

Which Botmetric subscription plans have access to Budgeting?

Currently, we are enabling the Budgeting feature for Professional, Premium and Enterprise plans only, on request basis.

Botmetric Workflows Used in Budgeting:

The following workflow can be assigned to the people using budgeting:

  • User: User workflows with write permission will be allowed to only set the budget which will then be sent to a financial admin for approval.
  • Admin: Admin workflows/roles can provide the user with read and write access to budgeting. An admin can set the budget but only a financial admin can approve it.
  • Financial Admin: A Botmetric admin can also be a financial admin whose role will be to define the budget goal in Budgeting and approve the budget set by other users. By default, the owner of a Botmetric account will also be a financial admin.  

Add New User Smart Budgeting

Understanding Botmetric’s New Smart Cost Center

A Cost Center can be a department or any business unit in the company whose performance is usually evaluated through the comparison of budgeted to actual costs. Previously, Botmetric allowed you to create a Cost Center using tag-keys like ‘owner’, ‘customer’, ’role’, ‘team’ etc. Now, as per extensive budgeting requirements, Cost Centers in Botmetric can be defined in two ways-  based on tag keys alone and based on accounts and associated tag key-value pairs.

  • Based on Tag-Key

Here, you can choose the tag key which corresponds to your cost center. Based on the chosen tag key, Botmetric will create all possible cost centers for the tag values corresponding it.

add-cost-center-smart-budgeting

  • Based on account(s) or combination of multiple account(s) and tags

You can also create Cost Centers based on account(s) and customize them based on multiple grouping of tag keys. You can create a cost center group such as  account1->team1->role1.

add-cost-center2-enterprise-budgeting

Let’s say you have different nomenclature for the same tag-keys such as user:TEAM, user:team, user:Team, then you can multi-select these tags and get complete clarity on your cost center group.

add-cost-center3-enterprise-budgeting

Please note that you can only choose one option at a time. You cannot have a few cost centers created based on tags and few on account and tags combination.

How to set, track and monitor the budget?

  1. Allocate & Review

  •  Budget Goal:

Botmetric budgeting enables the financial leader to define a budget goal for the entire payer account as per his estimations for the financial year. You can either enter the budget inputs manually or you can use Botmetric’s estimate to populate the budget inputs across months, quarter and year. Botmetric looks at the data for the last 12 months for yearly budget tracking.

Based on your company size it can take upto 72 hours of time to enable, process and crunch your data.

  •    Assigning Budget to Individual Cost Center:

Individual Cost Center owner(s) or financial admin(s) can set/edit budget goals for their respective units. The owner or financial admin(s) can either enter the budget inputs manually or make use of Botmetric’s estimate to populate the budget inputs across months, quarter and year. If a non financial admin or user is creating the budget for his Cost Center, it will be sent to a financial admin for approval. The new roles provided for Budgeting are helpful for providing clear demarcation between users and financial admin(s). This will allow financial admin(s) to have control over the approval of budget while providing enough flexibility to the other roles to manage their Cost Centers effectively.

enterprise-budgeted-cost-center

  1. Budget Overview

Botmetric’s Budgeting Overview provides a summarised view where you can see a snapshot of your financial year performance at the payer account level. You can compare the actual, allocated and projected spends for the current month, current quarter and financial year. You can alse see a list of top spending Cost Centers for the current month and current quarter. Moreover, a complete trend graph comparing your actual, allocated and budgeted spend performance at a payer account level for 12 months and 4 quarters will help you evaluate Budgeting with a quick glance.

enterprise-budgeting-allocated-yearly-budget

  1. Cost Center View

Botmetric’s Cost Center Overview provides a comprehensive view to track the performance for each Cost Center. Fine grained resources and service details provide a deeper and instantaneous understanding of where a certain Cost Center is incurring more cost. Ability to shuffle the view among monthly, quarterly and yearly options will allow the user to understand the budget variance over time. Each Cost Center will be evaluated to determine whether its incurred cost is within the allocated budget or it has exceeded the defined budget limit.

Moreover, each Cost Center has a corresponding budget trend graph to show the comparison between actual,  allocated and estimated spend. If you have a huge list of Cost Centers in your cloud, the search bar will help you to quickly find the desired Cost Center.

enterprise-budgeting-cost-center-overview
Botmetric’s Enterprise Budgeting will empower IT budget owners to define and track budgets at every granular level. This will also streamline budget processes in your organization and bring composure in the chaotic world of budget goals setting. Signup for 14 days free trial and check how it can help your organisation in cloud cost saving.

Cost Allocation for AWS EBS Snapshots Made Easy, Get Deeper AWS Cost Analysis

Posted on by Manav Narang

All AWS EBS snapshots (which allow you to create persistent block storage volumes for your AWS EC2 instances), including the untagged/underused/unused volumes, cost money. AWS has been evolving the custom tagging support for most of the services like EC2, RDS, ELB, BeanStalk etc. And now it has introduced Cost Allocation for EBS snapshots.

This new feature allows you to use Cost Allocation Tags for your EBS snapshots so that you can assign costs to your customers, applications, teams, departments, or billing codes at the level of individual resources. With this new feature you can analyze your EBS snapshot costs as well as usage easily.

Botmetric, quickly acting on this new AWS announcement, incorporated this cost allocation and cost analysis for EBS snapshots. Of course you can use AWS’ console to activate EBS snapshot tagging and get EBS cost analysis (read this detailed post by AWS to know how). However, when you take this approach, you are required to download the cost and and usage report and analyze it using excel sheets. This get’s tedious. But with this feature now available on Botmetric, you need not juggle through complex excel sheets.

Importance of Tagging EBS Snapshots for Cost Allocation and Analysis

Tagging has been an age old practice with AWS enthusiasts. Not every AWS service permits customer-defined tags for every AWS service. Some that do can be tagged only using API command line access. Among several AWS services, EBS snapshot storage too is one of the metrics that AWS accounts are charged for. So, tagging the EBS snapshots is pivotal for proper cost allocation.

More than that, as an AWS user, you can now see exactly how much data changes have been made between each snapshot, thus giving visibility on how much you can save by copying the snapshots to Glacier instead.

This new feature of will be of greatest interest to enterprise customers seeking to track their cost associated with EBS snapshots, which generally add few thousands of dollars to their AWS bill.  

Earlier, it was a huge challenge for enterprises to track snapshot cost as they could not tag EBS snapshots for cost allocation. But with the availability of this new feature from AWS complemented with Botmetric’s capability to provide cost analysis for EBS snapshots, they get to drill-drown deeper into cost allocation and get a consolidated cost analysis view too.

Even Jeff Barr recounts this fact in his blog post that this feature will be very useful for enterprises, even for AWS customers of all shapes and sizes. He also adds the fact that managed service providers, some of whom manage AWS footprints that encompass thousands of EBS volumes and many more EBS snapshots, will be able to map snapshot costs back to customer accounts and applications.

Analyzing and Generating Cost Reports of Tagged EBS Snapshots in Botmetric

Botmetric, since long, has been offering cost allocation and cost analysis feature. Right from helping customers with proper tagging policies, tagging resources that have not been properly tagged to providing them an edge to allocate costs for those items for which AWS does not allow, analyzing costs of resources for which tagging was not possible earlier, and much more.

If you have to manage your AWS cloud budget like a pro, your AWS cost allocation & chargeback must be perfect. Thanks to Botmetric Cost & Governance’s Chargeback and Analyze, many AWS customers have been able to define, control, allocate, and understand their AWS cost allocation by different cost centers in their organization, while also having an ability to generate internal chargeback invoices. Now with AWS releasing the capability to tag EBS snapshot, you will have a better visibility into your AWs spend.

Cost Allocation for EBS Snapshots in Botmetric

Using Botmetric Cost & Governance’ Chargeback, you can allocate cloud resources with IDs, including the EBS snapshots. Please refer the image below:

Cost Allocation for EBS Snapshots in Botmetric

Cost Analysis of EBS Snapshots in Botmetric

Using Botmetric Cost & Governance’ Analyze, you can analyze the total cost incurred by the EBS snapshots for a particular day or the month using the filter ‘EC2-EBS-Snapshot.’

Cost Analysis of EBS Snapshots in Botmetric

 

You can even analyze the cost for each resource for a particular time stamp, so that you can get complete visibility into your EBS snapshot.

 

Cost Analysis of EBS Snapshots in Botmetric

 

Report Scheduling and Shareability in Botmetric

With Botmetric, you can even schedule alerts and share the cost reports with a set of recipients, so that other members too have visibility into cost allocation and cost analysis.

Report Scheduling and Shareability in Botmetric

 

With Botmetric, you can even share the cost allocation and analysis reports directly with the intended recipients.

Report Scheduling and Shareability in Botmetric

P.S: According to AWS, snapshots are created incrementally and that the first snapshot will look expensive. In regards to a particular EBS volume, deleting the snapshot with the highest cost may simply move some of the costs to a more recent snapshot. Because when you delete a snapshot that contains blocks that are being used by a later snapshot, the space referenced by the blocks will be attributed to the later snapshot.

To Conclude

Since long, Botmetric has the feature to automate taking EBS snapshots based on instance tags, and volume tags, at regular intervals and at any time of day/week/month. With this feature, you can easily perform AWS EC2-EBS cost allocation and analysis.

And to bring cloud cost accounting under control, you need to build a cost reporting strategy for your cloud deployments. Having said that, this can be a daunting task. If you are looking for an easier way to track your cloud spend, the best way forward is to plug-in your AWS to Botmetric Cost & Governance cloud cost management console. Read this post if you want to know how to schedule interval job to capture EBS snapshots based on Instance tags. Until our next blog, stay tuned with us.

A CFO’s Roadmap to AWS Cloud Cost Forecasting and Budgeting

Posted on by Vijay Rayapati

For today’s CFOs, IT is at the top of their agenda. With 26 percent of IT investments requiring direct authorization of a CFO in an organization, several CFOs have either embraced or are ready to adopt cloud due its OPEX model. To this end, Gartner estimates that by 2020,  the aggregate amount of cloud shift might reach up to $216B. And with AWS topping the charts among the CSPs,  Wikibon estimates that by 2022 AWS will reach $43B in revenue, and will be 8.2% of all cloud spending. Despite exponential increase in the adoption, there is one major fear attached to AWS, for that matter all the cloud’s adoption — how to be on top of cloud sprawl, and how to perfect AWS cost forecasting and budgeting as an enterprise business.

If  you are a CFO trying to up your game and seeking to build a roadmap for AWS cloud cost modelling, spend forecasting and cloud budgeting, and above all assuage cloud sprawl?  Here’s how:

AWS Cost Modeling and Calculating TCO of the Cloud

Unlike owned and self-operated data centers, cost modeling for procuring an instances over a specific cloud is different. Hence, total cost of ownership (TCO) is different in both the cases. When it comes to calculating the TCO of your cloud infrastructure, the real challenge is figuring out the pricing models of cloud service providers and planning the accurate capacity requirements for a period of at least two quarters to one full year. And when it comes to AWS cloud, all the services need to be taken into account including data, storage, networking on top of compute infrastructure, etc.

As a CFO, you also need an  application or workload specific cost modeling on AWS cloud. In this case,  the first step should be to take business demand variations into account as per periodic seasonal cycles. Your cost estimations will definitely take a hit if you do not consider the impact of the seasonal fluctuations on service-level agreement, usage patterns, storage requirements, and cloud environment configurations.

Plus, you need to detail the capacity plan to the level of individual application workloads instead of other dimensions like departments. Because the impact will vary from workload to workload. There are chances, you might be over planning or under planning the capacities.

While centralized planning at the application or workload level for all departments is unrealistic and hard to do, it can be an apt approach for realistic budgeting and forecasting if it’s already followed for data centers. However, when departments or business units are asked to create cloud cost forecasting and budgeting plans, you can indirectly work with them to create the cost plan at the application workload level.

Leverage Cloud Cost Reports to Identify Peak Resource Usage Scenarios

Identifying peak resource utilization patterns for various workloads is one of the stepping stones to a reliable AWS cost budgeting and forecasting. This can be achieved easily through periodic analytics and running reports over the usage data. The best way forward is to leverage other data sources like seasonal customer demand patterns and see whether these patterns have correlations with the peak resource usages.  When these patterns are identified in advance, required base capacity can be taken care by buying pertinent reserved instances (RIs).

However, use of RIs requires proportional amortization allocation that are based on usage hours across specific business units or application workloads. It may also involve upfront capital expenditure (CAPEX) depending on the cloud provider’s business models.

On the other hand, even though RIs take care of predictable usage requirements, they cannot help with spikes. Plus, amortizing RI costs to the appropriate projects or teams (via resource tags) within the organization is a huge challenge too.

Planning Cloud Capacity and RI Amortization

A proportional cloud capacity purchase and amortization allocation based on usage hours requires several measures, like mapping a piece of the upfront RI payment to each hour of usage for the RI, using the workload metadata tags that can be assigned to resources and tieing back to RI usage ( so that proper cost allocation can be done at cost center level or at business unit level).  By doing so, you have visibility into expected increase in cost by various cost centers or business units within the organization.

If you want to know the proportional and hourly amortized costs of RI, few smart third-party tools can throw usage insights that reflect cost incurred by various business units or teams’ RI spend across your organization. The next step is to track overall cloud as well as workload-level consumption against the planned budget so that as a business you can be within the allocated expenditure for cloud usage.

Save more than 50% on your cloud with smart RI planning.

Track Cloud Consumption Against the Plan through Cloud Management Platforms

The organizational level cloud budgeting and cost plan adopted must be regularly tracked at monthly or bi-monthly intervals by comparing the planned usage vs. actual cloud spend so that right financial governance controls can be deployed.

Also a periodic review of planned vs actual usage will help you and your business to understand how to optimize the use of underutilized cloud resources across organization or within specific cost centers to reduce AWS cloud expenditure.

Using tools like Botmetric, you can identify the anomalous usage patterns, detect underused resources and take corrective action in real-time instead of waiting till the end of the quarter period to take stock of overall forecasting.

Another key aspect that cannot be ignored by financial controllers handling the cloud procurement within the organization is to streamline AWS cloud cost budgeting as per organizational level chargeback or internal invoicing based on business units or cost centers. The best way forward to achieve this goal is to plan minimal governance around resource costs at department level or workload level through tagging, allocating extra spend to specific teams and generating chargeback invoices to respective business units.

Applying Predictive Analytics for Projecting Cloud Spend and Forecasting Usage

Even though AWS itself provides a basic AWS Cost Explorer Forecast widget in the AWS console, you will need a tool beyond that, which can help achieveforecasting with ease. You can leverage predictive data analytics tools like Botmetric to keep your cloud spend in control by acting upon smart, intelligent recommendations based on more than 80 AWS best practices.

Also, Botmetric provides predictive Analytics as a way to get forecasting data, deploying financial controls for organizational level governance and meet future demand patterns accurately based on the past usage data.

Adopt Financial Governance Best Practices for Cost Management

While the above four approaches help you at the macro-level-cost-management level within a business and at the cost center level, the best way to optimize your cloud usage and spend is by adopting the best practices advocated by the cloud experts in the ecosystem, such as:

  1. Track storage and data transfer charges at cost center or business unit level
  2. Constantly monitor and remove unused resources at workload level
  3. Track, manage, and fix underused instances at workload level
  4. Look for unattached persistent volumes and old snapshots at organizational level
  5. Look for allocated spend spikes to specific business units or cost centers through chargebacks

There are many more best practices. However, to follow all the best practices, decreed by AWS as well as cloud experts, requires a lot of effort and spending on resources. Nevertheless, there is a way out — using intelligent, AI-powered platforms.

To Wrap-Up

To succeed as a CFO in cloud world, access to consolidated billing report, visibility into cloud data to monitor usage, capability to establish governance policies to control cloud expenses, and successfully forecast future expenses are a must haves. And with access to tools that can provide real-time notifications of exceeded budget limits, budget forecasting, and visibility into your cloud infrastructure costs will be like icing on the cake. Botmetric Cost & Governance provides just that.

With Botmetric you can spend more time determining the best way to optimize your cloud with your team rather than analyzing complex billing reports, following trends on specific infrastructure growth, and continuously monitoring on-going assessments of your cloud spend. Give it a try !

The Biggest Roadblocks for Cloud Practitioners and Why You Should Know

Posted on by Editor

Cloud computing has been increasingly favoured over on-premise computing lately. A majority of IT industry players right from hardware manufacturers, OS, and middleware software developers to independent software vendors (ISVs) have embraced cloud.

A recently published IDG’s Enterprise Cloud Computing Survey (2016) indicates that by 2018 the typical IT department will have at least 60% of their apps and platforms hosted on the cloud.

Future of IT Platform is Cloud

Despite adoption, however, there are a lot of barriers to its adoption and acceleration. Another industry report indicates that 60% of engineering firms are slowing down their cloud adoption plans due to lack of security skills.

Skills gap is considered to be one of the major pet peeve of cloud practitioners across the globe. Apart from this challenge there are other barriers to the cloud adoption too. Just because there are challenges one must not stop there and hinder progress. The buck must not stop there. Why not turn these obstacles into opportunities and problems into possibilities?

As a cloud user, do you want to know the top pet peeves of a cloud practitioner and turn them into possibilities or opportunities? If yes, then you are at the right place. Read on these challenges collected from several cloud experts via an internal survey:

Apprehensions about Losing Control and Visibility over Data

Storing sensitive and proprietary data on external environment carries risks. Despite cloud providers providing successful case studies and guides for best practices, enterprise bureaucrats are still apprehensive about moving their data to the cloud. Because it becomes very difficult to see where the data resides exactly once it is on a public cloud.

The other perspective: If data is stored in the cloud, you can access it from anywhere, anytime, no matter what happens to your machine. Plus, you can have complete control over your data and even remotely erase data if you’re in doubt that it is in the wrong hands. Cloud providers also have fine grained Identity and Access Management (IAM) controls.

Moreover, there are many competitive SaaS platforms that bring data security tools integrated with other DevOps features so that cloud users don’t have to worry about losing control over their sensitive data sets.

Lesser Visibility and Control over Operations Compared to On-Prem IT Infra

A majority of businesses want to track the changes that are made during the IT operations. So, they are worried that there might not be complete visibility into their IT operations such as who is accessing what and when, like in on-prem IT infra.

The other perspective: It is a myth that cloud does not provide complete visibility and control. It provides complete visibility and control to the user, provided you have all authenticated access to it. Further, adopting DevOps platforms and tool chain such as Docker, Ansible etc. can empower enterprise teams to track and manage the entire Application development and deployment lifecycle.

Fear of Bill Shock

Cloud Services are priced entirely different from the simple fixed price models of standard servers in a data center. Budgeting and managing frequent cost changes in the cloud scenario is worrisome for most businesses, because the complex pricing model of the cloud get them overworked or overwhelmed.

The other perspective: Cloud goes with Opex, not Capex. With a well designed cloud architecture, along with a comprehensive cloud management plan, can always keep cloud cost under control and optimized. No bill shock, to be precise!

Good news is that there are several SaaS-based CloudOps solutions that integrate natively with the Core Cloud Platform leveraging the Open APIs. They can dynamically provision and decommission system resources based on dynamic parameters like workloads, user traffic, etc. By optimizing the resource utilization, these CloudOps platforms can bring down the operational costs drastically. Additionally, these platforms feature advanced dashboards that can help companies to establish budgetary controls and track the actual cost accruals against the planned costs. Such tools can help enterprises overcome the fear of costs overshooting.

Acquiring New Skillset for Cloud Management

Cloud platforms have radically altered the application development lifecycle automation and continue to do so with emerging cutting-edge technologies. To that end, businesses have to continually acquire DevOps teams adept with all the new emerging technologies, essentially to manage the cloud servers over the different lifecycle stages such as PoC, test, staging, and production.

The other perspective: Instead of hiring a team of engineers for Ops why not just automate known IT ops using tools and just focus on development? The skills shortage always seems like a problem across the industry. Moreover, cloudOps automation platforms like Botmetric can help the technology complexity underneath the Cloud by automating many of the frequent tasks that are expected to be performed.

The Bottom Line: Many CloudOps Platforms and Tools to the Rescue

Several third party software vendors have ventured to fill the gaps in the core cloud platforms and solve most of the concerns voiced by the cloud users. As a Cloud Expert one should be knowledgeable about these extension tools to bridge the gap between the Cloud platforms capabilities and Enterprise teams’ needs.

Share you feedback in the comment section below or give us a shout out on any of our social media channels. We are all ears.

Share Data-Rich AWS Cloud Reports Instantly with Your Team Directly From Botmetric

Posted on by Tarun G

Once Henry Ford said, “Coming together is a beginning. Keeping together is progress. Working together is success.” This adage holds so true to find success while managing AWS cloud. For the reason that: to achieve complete AWS cloud management is not a one person’s responsibility, but is a shared responsibility and more so a teamwork. And for the teamwork to reap benefits, all the team members need complete visibility with pertinent data in the form of AWS cloud reports in hand. To cater to this need Botmetric has introduced ‘Share Reports’ feature that allows a Botmetric user to share important AWS cost, security or Ops automation reports with multiple AWS users for better collaboration.

If you’re a Botmetric user, you can now:

  • Share the data-rich reports directly from any Botmetric products, thus saving time and effort
  • Educate both Botmetric and non-Botmetric user(s) within your team about various aspects of your AWS infrastructure
  • Highlight items that may need action by other teammates

Why Botmetric Built Share Reports

Currently, Botmetric offers more than 40 reports and 30 graphs and charts. These reports, charts and graphs help for better cloud governance. More so, these data-rich reports offer a great culmination of insights and help keep you updated on your AWS infrastructure.

Earlier, Botmetric empowered its users (those added to your Botmetric account) to download all these reports. However, at times, it’s likely you’ll need to send perpetual reports to other colleagues too that may not be part of your Botmetric Account.

Thus, continuing our mission to provide complete visibility and control for AWS users and your AWS infrastructure, Botmetric now allows you to email/share those reports directly to non-Botmetric user(s) too. By doing so, Botmetric empowers every custodian for cloud in your organization responsible for cloud with pertinent data, even if they are not Botmetric users.

More so, the new share functionality enables you to share specific reports across Cost & Governance, Security & Compliance, and Ops & Automation to custodians who are not Botmetric users in your organization and wish to discover knowledge on certain AWS cloud items.

The new share reports can be used across Botmetric platform in two specific ways:

1. Share Historical Reports

Share all the AWS cloud reports present under reports library on the Botmetric console to other custodians in the team.

Share all the AWS Cloud reports for better cloud management

2. Export and Share Charts and Graphs as CSV Reports

If you find any crucial information in any of the reports under Botmetric Cost & Governance, Security & Compliance or Ops & Automation, you can share using the ‘Share icon’ to any other custodian who isn’t Botmetric user(s) but responsible for cloud.

Share AWS cloud reports on Cost, Security, Ops with the team using Botmetric

For example, you would want to share the list of ports open to public to the person in your team who is responsible for perimeter security. You can do this from Audit Reports section of Security & Compliance.

The Bottom Line:

AWS has more than 70 resources and each resource has multiple family types. With so many variance in AWS’ services, you surely need either holistic information or a particular information at some point for analysis. With Botmetric reports and the new sharability feature, you and your team can together manage and optimize your AWS cloud with minimal effort.    

If you are a current Botmetric user, then Team Botmetric invites you to try this feature and share your feedback. If you’re yet to try Botmetric and want to explore this feature, then take up a 14 day trial . If you have any questions on AWS cloud management, just drop in a line below in the comment section or give us a shout out at @BotmetricHQ.

The Road to Perfect AWS Reserved Instance Planning & Management in a Nutshell

Posted on by Editor

Ninety-eight percent of Google search on ‘AWS reserved instance (RI) benefits’ shows that you can get great discounts and save tremendously compared to on-demand pricing.The fact is, this discounted pricing can be reaped provided you know what RIs are, how to use them, when to buy them, how to optimize them, how to plan them, etc.

Many organizations have successfully put RIs to its best use and have the optimal RI planning and management in place due to the complete knowledge they have.

This overarching, in-depth blog post is a beginner’s guide that helps you leverage RIs completely and correctly, so that you can make that perfect RI planning and management. It also provides information on how to save smartly on AWS cloud.

Upon completely reading this post, you will know the basic 5Ws of AWS RIs, how to bring RIs into practice, types of AWS Reserved Instances, payment attributes associated with instance reservations, attributes to look for while buying/configuring an RI, facts to be taken into account while committing RIs, top RI best practices, top RI governance tactics that help reduce AWS bill shock, and common misconceptions attached to RIs.

The Essence: Get Your RI Basics Right to Reduce AWS Bill Shock

The Backdrop

Today, RIs are one of the most effective cost saving services offered by AWS. Irrespective of whether the reserved instances are used or unused, they will be charged. And AWS offers discounted usage pricing for as long as organizations own the RIs. So, opting for reserved instances over on-demand instances might waste several instances. However, a solid RI planning will provide the requisite ROI, optimal savings, and efficient AWS spend management for a long term.

 

AWS RIs are purchased for several reasons, like savings, capacity reservation, and disaster recovery.

Some of them are listed below here:

1. Savings

Reserved instances provide the highest savings approach in AWS cloud ecosystem. You can lower costs of the resources you are already using with a lower effective rate in comparison to on-demand pricing. Generally, EC2 and RDS RIs are contenders of projecting highest figures in your AWS bill. Hence, it’s advisable to go for EC2 and RDS reservations.

A Case-in-point: Consider an e-commerce website running on AWS on-demand instances.Unexpectedly, it started gaining popularity among customers. As a result, the IT manager sees a huge spike in his AWS bill due to unplanned sporadic activity in the workload. Now, he is under pressure to control both his budget and efficiently run the infrastructure.

A swift solution to this problem is opting for instance reservation against on-demand resources. By reserving instances, he can not only balance capacity distribution and availability according to work demands, but it can also reap substantial savings due to reservation.

P.S: Just reserving the instances will not suffice. Smart RI Planning is the way forward to reap optimal cost savings.

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]Botmetric helps you make wise decisions while reserving AWS instances. It also provides great insights to manage and utilize your RIs that ultimately lead to break-even costs. Get a comprehensive free snapshot of your RI utilization with Botmetric’s free trial of Cost & Governance.[/mk_blockquote]

2. Capacity Reservation

With capacity reservation, there is a guarantee that you will be able to launch an instance at any time during the term of the reservation. Plus, with AWS’ auto-scaling feature, you will be assured that all your workloads are running smoothly irrespective of the spikes. However, with capacity reservation, there will be a lot of underutilized resources, which will be charged irrespective of whether they are used or unused.

A Case-in-Point: Consider you’re running a social network app in your US-West-1a AZ. One day you observe some spike in the workload, as your app goes viral. In such a scenario, reserved capacity and auto-scaling together ensure that the app will work seamlessly. However, during off season, when the demand is less, there will be a lot of underutilized resources that will be charged. A regular health check of the resource utilization and managing them to that end will provide both resource optimization and cost optimization.

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]Botmetric performs regular health check of your usage of reservations and presents them in beautiful graphical representation for you to analyze usage optimally. Further, with the metrics, you can identify underutilization, cost-saving modification recommendations, upcoming RI expirations, and more from a single pane.[/mk_blockquote]

3. Always DR Ready

AWS supports many popular disaster recovery (DR) architectures. They could be smaller environments ideal for small customer workload data center failures or massive environments that enable rapid failover at scale. And with AWS already having data centers in several Regions across the globe, it is well-equipped to provide nimble DR services that enable rapid recovery of your IT infrastructure and data.

The Point-in-Case: Suppose, East coast in the U.S. is hit by a hurricane and everybody lines up to move their infrastructure to US-West regions of AWS. If you have reservation in place beforehand in US-West then your reservation guarantees prevention from exhaustion. Thus, your critical resources will run on US-West without waiting in the queue.

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]Botmetric scans your AWS infra like a pro with DR automation tasks to check if you have configured backup on RDS and EBS properly[/mk_blockquote]

How to Bring RI into Practice

The rationale behind RI is simple: getting AWS customers like you to commit to the usage of specific infrastructure. By doing so, Amazon can better manage their capacity and then pass on their savings onto you.

Here are the basic information on RI types, options, pricing, and terms (1 and 3 year) for you to leverage RIs to the fullest. These will help you bring RI into practice.

Types of AWS Reserved Instances

1. Standard RIs: These can be purchased with one year or three-year commitment. These are the best suited for steady state usage, and when you have good understanding of your long-term requirements. It provides up to 75% in savings compared to on-demand instances.

2. Convertible RIs: These can be purchased only with three year commitment. Unlike Standard RIs, Convertible RIs provide more flexibility and allows you to change the instance family and other parameters associated with a RI at any time. These RIs also provide savings but up to 45% savings compared to on-demand instances. Know more about it in detail.

3. Scheduled RIs: These can be launched within the time-span you have selected to reserve. This allows you to reserve capacity in a stipulated amount of time.

Types of AWS Reserved Instances and their charecteristicsTypes of AWS Reserved Instances and their characteristics

Payment Options

AWS RIs can be bought using any of the three payment options:

1. No-Upfront: The name says it all. You need not pay any upfront amount for the reservation. Plus, you are billed at discounted hourly-rate within the term regardless of the usage. These are only available for one year commitment if you buy Standard RI and for three years commitment if you opt for Convertible RI.

2. Partial Upfront: You pay a partial amount in advance and the remaining amount is paid at discounted hourly rate.

3. All Upfront: You make the full payment at the beginning of the term regardless of the number of hours of utilized. This option provides the maximum percentage of discount.

Attributes to be looked at while buying/configuring RI

Committing RIs

From our experience, a lot of stakeholders take a step back while committing towards reservation, because it’s an important investment that needs lot of deliberation. The fact is: Once you understand the key attributes, then it gives you all the confidence to commit on RIs.

Realize: How to?

  • Identify the instances, which are running constantly or having a higher utilization rate (above 60%)
  • Estimate your future instance usage and identify the usage pattern
  • Spot the instance classes that are the possible contenders for reservation

Evaluate: How to?

Once you know how to realize the RIs, you can identify possibilities and evaluate the alternatives with the following actions:

  • Look for suitable payment plans
  • Monitor On-Demand Vs. Reserved expenditure over time
  • Identify break-even point and payback period
  • Look for requirements of Standard or Convertible RIs

Select: How to?

Once you know how to evaluate, you can analyze and choose the best option that fits your planning, and further empower your infrastructure for greater efficiency with greater savings.

Implement: How to?

Once you know what your requirements are to commit for a RI purchase, implementation is the next stage. It is very crucial you do it right. For the reason that: Discounts might not apply in all cases. For instance, if you happen to choose the incorrect attributes or performs incorrect analysis. At the end of the line, your planned savings might not reflect in your spreadsheets (*.XLS) as calculated.

How to Implement the Chosen RI Like a Pro

The key parameter to reserve an EC2 instance is the instance class. To apply reservation, you can either modify or go for a new RI purchase by selecting platform, region, and instance type to match the reservation.

For Instance:

Consider a company XYZ LLC, where it has an on-demand portfolio of

  • 2*m3.large Linux in AZ us-east-1b
  • 4*c4.large Linux in AZ us-east-1c
  • 2*t2.large Linux in AZ us-east-1a

And XYZ LLC now purchases standards RIs as below:

  • 4*c4.large Linux in AZ us-east-1c
  • 2*t2.large Linux in AZ us-east-1b
  • 4*x1.large Linux in AZ us-east-1a

Based on the above on-demand portfolio and purchases, the following reservations are applied for XYZ LLC:

  • 4*c4.large Linux in AZ us-east-1c. Here’s how: This matches exactly the instance class the reservation was made, so offers discounts
  • 2*t2.large Linux in AZ us-east-1b. Here’s how: The existing instances class is in a different AZ but in the same region, so no discount is applied. However, if you change the scope of RI to region then the reservation will be applied but there is no guarantee of capacity
  • 4*x1.large Linux in AZ us-east-1a. Here’s how: The instance family, region, and OS don’t match. In this case, reservation will not be applied for these instances. However, if XYZ LLC had purchased Convertible RIs, modifying reservation will never be a problem but they have to commit for 3 years with a lesser discount.

Making Sense of the RIs for Payer and Linked Accounts

AWS bills, evidently, includes charges only on payer account for all utilization. However, in larger organizations, where the linked accounts are divided into business units, reservation purchases are made by these individual units. No matter who makes the purchase, the benefits of RI will float across the whole account (payer + its own linked accounts).

For Instance: Let’s assume X is the payer account and Y and Z are its two linked accounts. Then in an ideal situation:

$- Purchase

U-Can be applied

X($) then Y (U) or Z (U)

If Z($) then Y(U) or X(U) or Z(U)

Hence, in a group, reservation can be applied in any instance class available.

How to Govern RIs with Ease

Monitoring just a bunch of RIs are easy when the portfolio is small. However, in case of mid-sized and large sized businesses, RIs generally don’t get proper attention due to the dynamic environment and the plethora of AWS services to manage. This causes a dip in efficiency, unexpected minimal savings, and many more such issues. Nevertheless, this dip in efficiency and bill shock can be assuaged with few tweaks:

Make a regular note of unused and underutilized RIs:

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]Unused and underutilized states of RIs are key issues that lead to inefficiency.[/mk_blockquote]

In case of unused RIs: The reservations were intended to be bought for determined constant utilization but somehow the utility ended just after few months of purchase and the reservation is now in dormant state or unused. If you modify and eliminate them, then they will add to cost savings.

In case of underutilized RIs: Few RIs are bought with the intention to use them for continuous workload but somewhere in the timeline the utility reduced and the reservation is not clocking to its ideal utilization. If you start reusing them, then they will add to cost savings. Read this blog post by Botmetric Director of Product Engineering Amarkant Singh’s post on how to go about unused and underutilized RI modifications and save cloud cost intelligently.

Finding the root cause of unused and underutilized RIs:

1. Incorrect analysis: While performing the analysis to determine RI purchase some miscalculations or lack of understanding of environment can be cause of trouble in management of RIs.

a. Wrong estimation of time (1 year/ 3 years): If you couldn’t understand your projected workload time then purchasing reservation for a longer interval e.g.: 3 years may bring RI into unused state

b. Wrong estimation of count: This could be due to overestimation/underestimation of the number of reservations required. If it’s too many, then you may modify them for DR capability. But if it’s too less, then you may still not satisfy your savings

c. Wrong estimation of projected workload: If you have not understood your workload, then chances are that you could have bought RIs with incorrect attributes like time, number of instances bought, etc. In such cases, RIs either go unused or underutilized

2. Improper Management: RIs, irrespective of the service, can offer optimal savings only when they are modified, tuned, managed, and governed continuously according to your changing infrastructure environment in AWS cloud.

You should never stop at reservation. For instance, if you have bought the recent Convertible RIs, then modifying them for the desired class. And, if you have older RIs, then get them to work for you either by breaking them into smaller instances or by combining them for a larger instance as per the requirement.

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]FACT: Almost 92% of AWS users fail to manage Reserved Instances (RIs) properly, thus failing to optimize their AWS spend.[/mk_blockquote]

If you find all this overwhelming, then try Botmetric Smart RI Planner, which helps with apt RI management, right sizes your RIs and helps save cost with automation.

Top RI Best Practices to Live By

There are few best practices you should follow to ensure your RIs work for you and not the other way around.

Get Unused RIs back to work

If you have bought the recent Convertible RIs, then modifying them for desired class is now a child’s play. However, if you have older RIs, then getting them back to work is not so easy as Convertible RIs. But with just few modifications like breaking them into smaller instances /combining them for a larger instance according to your needs will do the trick.

Keep an eye on expired and near-expiry RIs in your portfolio

Always list your RIs in three ways to keep a constant check on them:

Active: RIs that are either new or will expire in 90 days

Near-expiry: RIs that are nearing to 90 days of expiration. Analyze these RIs and plan accordingly for re-purchase

Expired RIs: RIs that are expired. If there is an opportunity for renewal go ahead with it

Be sure of your workload demands and what suits your profile the best. Standard RIs work like a charm, in regards to cost saving and offering plasticity, only when you have a good understanding of your long-term requirements.

And if you have no idea of your long-term demand, then Convertible RIs are perfect, because you can have a new instance type, or operating system at your disposal in minutes without resetting the term.

[mk_blockquote style=”quote-style” font_family=”none” text_size=”12″ align=”left”]Botmetric has found a smart way to perform the above. It uses the NoOps and AIOps concept to put the RIs back to work. Read this blog to know how.[/mk_blockquote]

Compare on-demand Vs. reserved instances to improve utilization

If you want to improve your utilization of reservation, the game plan is to track on-demand Vs. reserved instances utilization. It is evident from our experience that RI cost over a period of time offers the greatest discounted prices. Read this blog post to know the five tradeoff factors to consider when choosing AWS RIs over on-demand resources.

Compare on-demand Vs. reserved instances to improve utilization

For further benefits, a report on RI utilization that can throw the below insights will help:

1. Future requirement of reservation

2. Unused or underutilized RIs

3. Opportunities to re-use existing RIs

Here is a sample Botmetric RI Utilization Graph for your reference:

AWS Reserved Instance Management Now Made Easy with Botmetric’s Smart RI

Before wrapping-up, here are few common RI misconceptions that you must know.

Common RI Misconceptions You Must Know

  • If you buy one EC2 instance and reserve an RI for that type of EC2, then you don’t own two instances but you own only one
  • RIs are not only available in EC2 and RDS but in five other services as well that can be reserved
  • Purchasing RIs alone and not monitoring and managing them may not give you any savings.
  • Managing and optimizing them is the key
  • Never purchase instance for an instance ID, but for an instance class
  • Buying a lot of RIs will not bring down the AWS bill
  • Managing RIs is very complex. It’s a continuous ongoing process. Few key best practices — if followed — can give desired savings and greater efficiency
  • Older RIs cannot have Region benefit
  • RIs can’t be re-utilized, if you fail to understand your workload distribution RIs can’t be returned, instead
  • AWS RI Marketplace facilitates you to sell your RIs to others

The Wrap-Up

RIs, as quoted earlier, are the highest saving option in your dynamic cloud environment. Buying RIs is not sufficient. A proper road map and management coupled with intelligent insights can get you the desired savings.

AWS is always coming up with new changes. Hence, understanding its services and knowing how to use them for greater benefit will always prove beneficial for your cloud strategy irrespective of your business size, above all for the startup world. And if you find all this overwhelming, then just try Botmetric’s Cost and Governance.

Get Started

Botmetric Augments AWS Cost Allocation & Chargeback with New Undo & Reallocate

Posted on by Tarun G

If you have to manage your AWS cloud budget like a champ, then your AWS cost allocation & chargeback must be perfect.  Thanks to Botmetric Cost & Governance’s Chargeback app, many AWS customers have been able to define, control, allocate, and understand their AWS cost allocation by different cost centers in their organization, while also having an ability to generate internal chargeback invoices.  

The same app is now bolstered with two new additions, namely Undo and Reallocate. These two new features further help AWS users like you manage complexities involved in treating chargebacks, especially when dealing with multiple linked accounts from a central payer account.

Why did Botmetric team build Undo and Reallocate in Chargeback?  

At Botmetric, we know how you’ve always thrived to have better tagging policies to eliminate wastage, allocate cost centers to respective teams, and allocate appropriate cost to responsible items. We also understand the fact that this cannot be achieved ALWAYS, due to those few unallocated resources (the culprit). Botmetric’s Cost & Governance resolves this problem through Chargeback. Using Chargeback,  you can allocate untagged AWS items from within the app.

Moreover, Chargeback enables you with great opportunities in strengthening cost tracking. Often, Botmetric customers allocate a lot of items. Sometimes up to 5000 items at one go. And in midst of allocating hundreds of these items it might so happen that some of the items get wrongly allocated. Human errors, you see. And, this gets its due attention only while verifying the allocated items. There comes the Oops moment!

We know it’s a facepalm moment!

via GIPHY

Have you also faced the same? Wished you had a solution for it? Well, your wish is our command. To that end, team Botmetric has developed a mechanism that helps you to undo these incorrectly allocated items.

Botmetric solves this problem in two ways:

1.Undo

With Undo, you can un-allocate items that you have mistakenly allocated while performing cost allocation. Undo operation will then re-position the item back to its original untagged/unallocated state.

Undo Incorrect Resource Allocation

2. Reallocate

With Reallocate, you can correctly allocate the wrongly allocated item. Plus, you also get an additional capability to reallocate from one cost center to another.

Reallocate incorrect resource allocation

From the above screenshot, you can see that you have the freedom to choose which cost center you want to reallocate from the drop down and hit reallocate. By doing so, you can reallocate the item from an incorrect cost center to the desired one.

Well, there’s one exception to this option. That is: you can only undo or reallocate only those items that are allocated/actioned from Botmetric platform. You cannot modify items for changes done in AWS console. Don’t wait. Explore this feature on Botmetric today. If you have not signed-up with Botmetric Cost & Governance, then we have a 14-day trial for you. If you need any assistance, just give a shout out on Twitter, Facebook, or LinkedIn. We’d be happy to help. 

Top 7 Hacks on How to Get a Handle on AWS S3 Cost

Posted on by Editor

Amazon Simple Storage Service (S3) is one of the most widely deployed AWS Services, next to AWS EC2. It is used for a wide range of use cases such as static HTML websites, blogs, personal media storage, enterprise backup storage. From AWS cost perspective, AWS S3 storage is one of the top preferred resources. For every enterprise looking to optimize AWS Costs, analysing and formulating an effective cost management strategy towards AWS S3 is important. More so, understanding data lifecycle of the applications hosted is the key step towards implementing a good AWS S3 cost management strategy.

Making the most of AWS S3:

With AWS, you pay for the services you use and the storage units you’ve consumed. If AWS S3 service is a significant component of your AWS cost, then implementing AWS S3 management best practices is the way forward. 

For example, if a business has opted for AWS S3 service and has provisioned 100 GB of it but has actually stored only 10 GB of files in it, then AWS would only charge for the 10 GB and not for the entire 100 GB provisioned initially. However, there are various factors that affect the S3 cost too, which many are unaware. Many AWS administrators tend to overlook S3 from cost management perspective because of this aspect.

To this end, we’ve collated few basic checks to get the S3 cost management right as AWS S3 usage grows: 

1. EC2 and S3 buckets should be in the same AWS region because there is a cost involved for data transfers outside of its AWS region.

2. The Naming Schema should be chosen such that access keys generated ensures  files are stored and distributed across multiple drives of the AWS S3 system. If the access keys are distributed evenly, the number of file operations needed to read and write the files will be less. This will lead to less spend costs as there is an additional cost overhead for read-write operations for S3.

3. Only temporary access credentials of AWS S3 should be hardcoded into an application’s code that uses S3. There can be misuse of the S3 resources if access keys are exposed to third party. This can prove very costly, if access credentials are compromised in the future.

4. Monitoring the actual usage of AWS S3 periodically is one of the best practices. By doing so, misuse of the provisioned S3 resources will come to limelight and help in curtailing data compromise.

5. Files form the key object type, and are stored in S3. All files that are no longer relevant should be removed from S3 buckets. All files that are temporary files can be recreated through a computation process. All temporary files generated due to incomplete multi-part uploads should be cleaned up periodically.

6. When using versioning for an S3 bucket, enable “Lifecycle” feature to delete old versions. Here’s why and how: With Lifecycle Management, you can define time-based rules that can trigger ‘Transition’ and ‘Expiration’ (deletion of objects). The Expiration rules give you the ability to delete objects or versions of objects, which are older than a particular age. This ensures that the objects remain available in case of an accidental or planned delete while limiting your storage costs by deleting them after they are older than your preferred rollback window.

7. Try to send the data to S3 in compressible format, because AWS S3 is charged for the amount of units you’ve consumed.

Ultimately, every data stored in the S3 will have its lifecycle stages of creation, usage and then followed by infrequent usage. Just like content creation in a news website. The daily news created along with its images can be stored in AWS S3. Current news items will be accessed most and hence have to be quickly accessible to a reader. At the end of the week, the older daily news content can be moved to the AWS S3 RRS for faster, but slightly infrequent access. At the end of the month, they can be moved to an standard infrequent access storage type. At the end of the quarter, these content can be moved to the low cost rarely accessed archival mode of AWS Glacier.

This data lifecycle is applicable across domains including e-commerce and enterprise computing as well. Hence, leverage data’s inherent lifecycle for AWS S3 cost optimization.

You can also take advantage of Amazon S3 Reduced Redundancy Storage (RRS) as an alternative to S3, because it’s cheaper.

To Conclude:

Once you follow all the above hacks, start observing the bills. And don’t forget to follow other key best practices too. Use RRS where ever you can. Keep your buckets organized. Archive when appropriate. Speed up your data processing with proper access keys names. Use S3, if you are hosting a static website. Architect around data transfer costs. Use consolidated billing.

Finally, AWS provides a simple configuration mechanism to specify the rules of the data lifecycle and transferring of the objects across storage types. So, do take data lifecycle as well as into account when it comes to S3 cost management.

If you are finding it difficult to save on AWS S3 cost, then explore the intelligent Botmetric AWS Cloud Management Platform with a 14-day free trial. It can help you manage your AWS storage resource management and help keep them at optimal pricing levels at all times. For other interesting news on cloud, follow us on Twitter, Facebook, and LinkedIn.

Ace Your AWS Cost Management with New Botmetric’s Chargeback Feature

Posted on by Editor

Editor’s Note: This exclusive feature launch blog post is authored by Botmetric CEO Vijay Rayapati.

How awesome would it be if you, as an AWS user, could define, control and understand the cost allocation by different cost centers in your organization, while providing an ability to generate chargeback invoices. It’s possible with the new Botmetric Cost Management & Governance application’s Chargeback module.

This Chargeback module has the same Cost Allocation and Internal Chargeback features from the earlier version of Botmetric, however, packaged in a new UI with augmented features. These features together will help you manage costs associated with  AWS infra in a smart way. Plus, you can manage budget controls better across projects and departments. More than that, users can manage complexities involved in treating chargebacks when dealing with multiple linked accounts from a central payer account. Additionally, you can gain access to many insightful reports for effective tracking and managing of AWS cloud costs.

Key benefits of the new Chargeback module in Botmetric includes:

  • Define Cost Allocation: We have implemented support for defining cost centers within your company by either department or team or business unit so you can split the overall spend into specific cost centers. This works based on your AWS tags for cost allocation and further defines a grouping within Botmetric for creating a cost centers for your business. You can have multiple centers for cost allocation in Botmetric.

Define Cost Allocation with Botmetric Cost & Governance

 

  • Control Unallocated Spend: From the Chargeback module dashboard, you can quickly identify the cost that is unallocated to any cost center in your business. This will allow you to split or allocate the cost into different cost centers and also inform your cloud team on missing tags for cost management.

 

ontrol Unallocated Spend with Botmetric Cost & Governance

 

  • Spend View within a Cost Center : We launched a drill down view so that you can understand the spend view by various AWS services within a cost center. This will allow you to inform the specific cost center teams if they are about to surpass their allocated budget for the month and let them know what is causing the increase in spend.

 

Spend View within a Cost Center

 

  • Download Chargeback and Cost Allocation Data: You can download the chargeback data for any cost center in CSV format. We also allow you to download the unallocated cost as a CSV file, so that you can share it with your IT team for inputs on how  cost should be allocated for different cost centers.

 

Download Chargeback and Cost Allocation Data

We are excited to release the general availability of enterprise cost allocation and chargeback support in Botmetric for AWS cloud. Now Botmetric customers can better manage their budgeting and internal reporting processes.

Please let us know if we can improve anything in Botmetric Chargeback module that can be more useful for your business. Do take a 14-day test drive, if you have not yet tried Botmetric.  

If you want to know several other new features available in the new release, then read the expert blog post by Botmetric Customer Success Manager, Richa Nath. For other interesting news from us, stay tuned on Twitter, Facebook, and LinkedIn!

November Round-up: Cloud Management Through the Eyes of the Customer

Posted on by Editor

Cloud has indeed been a game changer for many enterprises across verticals. It holds the key to digital disruption, say many. And as the cloud adoption increases by the day, challenges that come along the path have not deterred them from embracing it further, even as they scale. In an effort to provide these cloud-ready and cloud-first companies the mojo to win the cloud game, the GenNext of Botmetric platform was released this November with complete and intelligent cloud management features.

A Milestone Achieved & Many More Extra Miles to Go

Botmetric 2.0 went live recently. The primary goal of building this new unified & intelligent cloud management platform altogether was to provide a great user experience, simplified and consistent design, with intelligent insights & in-context customer engagement. Essentially, Botmetric is a customer-obsessed company, conscientiously nurturing a customer-first culture ever since it was born. For this reason, it went a step ahead, saw cloud management through the eyes of the customer, and rebuilt a whole new platform of three applications offering unified cloud management:

  • Cost Management and Governance: Built for CIO & Business heads
  • Cloud Security and Compliance: Built for CISO & IT Security Engineers
  • Ops & Automation: Built for CTO & DevOps Engineers

During this journey, Botmetric team took a  strategic call to move to micro services architecture to build this appified platform, essentially to speed up its sprint process. And to further nurture  a delightful and seamless customer engagement, Botmetric resorted to Intercom app.

The ‘All-New’ Botmetirc is now ingrained with cutting-edge cloud intelligence. To get further details about the product, read the  exclusive launch blog post by our zealous Customer Success Manager, Richa Nath.

Over the next few months, Botmetric will have few more feathers in its cap. So stay tuned with us on  Twitter,  Facebook, and  LinkedIn.

Knowledge Sharing @ Botmetric

Continuing our new tradition to provide quick bites and snippets on better AWS cloud management, here are few blogs that we covered in the month of November:

5 Surefire AWS Security Best Practices (not Just) for Dummies

5 Salient Amazon DynamoDB Features Every Beginner DevOps Guy Should Know

How Secure is AWS for E-Commerce Businesses? Doubt know more

There are many more insightful blogs populated on cloud and cloud management. Do read them  here.

Finally, continued excitement at AWS re:Invent 2016

If you are at the event, then Botmetric invites you to meet our leaders for a quick chat, and get first hand experience of the ‘All-New’ Botmetric.

If you have not signed-up with Botmetirc yet, then go ahead and  take a 14-day trial. As an AWS user, we are sure you’ll love it. To catch up with what happened last month, read Botmetric’s  October round-up blog. Rain or shine, Botmetric will continue to provide many more AWS cloud management features. Until the next month, keep in touch with us.

Introducing Botmetric 2.0 – A Unified & Intelligent Cloud Management Platform

Posted on by Rajeev Kumar

Editor’s Note: This exclusive launch blog post is penned by Vijay Rayapati, CEO at Botmetric.

Today is a big day, and I have some great news from Botmetric HQ. The all-new ‘Botmetric Platform’ is  out now!

Since Sep 2016, Botmetric 2.0 was in private beta for some of our users, and today we are launching the public availability of our new platform. Based on the feedback and our learning, we decided to turn Botmetric into a platform of applications that are use case targeted, instead of providing a ‘one-size-fits-all’ offering. To this end, Botmetric platform now has three products for unified cloud management:

    • Cost Management and Governance:  Built for CIO & Business heads
    • Cloud Security and Compliance:  Built for CISO & IT Security Engineers
    • Ops & Automation:  Built for CTO & DevOps Engineers

When we decided to work on the new Botmetric platform, the primary goal was to provide a great user experience, simplified and consistent design, with intelligent decisions & in-context customer engagement.

Our objective of taking the platform approach with applications was to reduce the information overload and help users find insights that matter to them in one-click. Above all, users get the information they want when they need without worrying about the rest of the modules.

When we already had a good product, why did we build the new Botmetric platform?

Because, we are on a mission to “simplify your cloud management by leveraging intelligence and automation”

To put this journey in perspective, we have been working with many public cloud users over the past 24 months to  understand their operational challenges in managing cloud; and how Botmetric could help simplify their daily lives. The feedback has been both overwhelming and heartening, like:

  • Botmetric is a good product and adds significant value for cloud management
  • Customers love the  super responsive customer support and design consistency offered by Botmetric
  • There are significant operational challenges that still remain unsolved for them, and Botmetric act as a control point to resolve these issues, etc.

Despite these feedback, as passionate cloud users, we did not stop here. We went ahead, spoke to many of our customers, and unearthed several facts. This sets the stage to answer a key question: Are we addressing the specific and evolving operational needs of our customers? Some of the key lessons we discerned are:

  1. Providing depth in specific areas such as Cost Management and Governance, Automation, Alerts Intelligence, Security and Compliance, instead of generic tools.
  2. Getting to a problem’s resolution, with one click that are further aided by  relevant insights.
  3. Providing great user experience and design across the platform.
  4. Offering deeper user engagement with bolstered, intelligent features and automated decisions.

This is why ‘the new Botmetric platform’ was born.

Five new things that will make you fall in love with the new Botmetric:

  1. Consistent UX across the platform and the apps: Our foremost goal while designing the new platform was to provide a great user experience and significantly improve the utility value for our customers. We have spent  a lot of time talking to many customers & trial users to capture their feedback. The new platform provides a thought through design consistency across interactions of various applications.
  2. Freedom to choose what is necessary with modularized applications: We have unbundled previous Botmetric offerings into three specific applications categorized as Cost & Governance,   Security & Compliance,   Ops & Automation.  This allows users to use what they need without worrying about other things. However, as a company, users can use everything offered by Botmetric to solve their overall operational problems from cost, security, and automation standpoint.
  3. Improved reporting and admin modules: We have completely revamped the reporting modules by Cost & Governance,  Security & Compliance,  Ops & Automation so users can easily discover the relevant reports as they need. The admin module is simplified for easier configuration management.
  4. Brand new features with intelligence: We are rolling out many new features that have been in the work for quite sometime as requested by our users. The Cost & Governance module provides data transfer analysis, chargeback & budgeting capabilities. We have rolled out various compliance policies in the Security & Compliance application to help users understand what they need to fix first.
  5. Highly responsive and modular design – The new platform uses responsive design and AngularJS framework for front end engineering. This makes it easier for our users to use it wherever they want. Our users can try the application across different screen resolutions and devices.

Frequently Asked Questions

Will the old version be available?

The older version will be be available for existing users until 15th of Dec 2016.

Is there a price increase in my subscription?

No, all the existing users will pay as per their previous subscription plans.

Why should I move to new Botmetric?

The new platform is easier to use. It has consistent design, and also simplifies users’ lives with new features.

Will there be any training or webinars on the new Botmetric platform?

Yes. A webinar will be announced very soon that will give a walk through of the product. For further details about this webinar, connect with us on Twitter, Facebook, and LinkedIn.

We are truly excited  about this launch and gratified that  our customers have played a part in this co-creation by participating  in  our private beta, shared feedback and appreciation. We truly appreciate that and also be assured, we are  going to be very focused on improving our new platform further over next few months. We would love to hear your feedback on the new platform and please write to us at  support@botmetric.com.  Do  explore the new  Botmetric today.