Azure Cost Management by Cloudyn provides detailed visibility into Azure consumption, cost and performance.
Azure Cost Management by Cloudyn is now available for free to customers and partners managing Azure spend!
Azure customers can use their Azure subscription to register with "Azure Cost Management" by Cloudyn to access the Cloudyn portal.
(For more details, please see the following Azure Cost Management by Cloudyn documentation.)
"Azure Cost Management by Cloudyn" Documentation and Resources can be found at:
- Documentation: https://docs.microsoft.com/azure/cost-management
- Pricing: https://azure.microsoft.com/en-us/pricing/details/cost-management
- Trial: https://trial.cloudyn.com/
- Forum: https://social.msdn.microsoft.com/Forums/en-US/home?forum=Cloudyn
- Blog: https://azure.microsoft.com/en-us/blog/use-azure-cost-management-for-free
- Videos: https://support.cloudyn.com/hc/en-us/categories/201077165
"Cloudyn" Documentation and Resources can be found at:
Cloudyn provides analysis and insights that enable you to lower your Cloud spend. This guide provides 10 steps to pinpoint areas to address and begin taking action. Many of the items discussed are included in our Optimizer dashboard that summarizes our optimization results.
Steps 1-2: Understand your spend
Steps 3-8: Unused Resources
Steps 9-10: Improve efficiency
1. Where to start?
It is important to gain a perspective of the services that are your main cost drivers. Customers sometimes focus on a particular service only to discover that it accounts for 3% of their total cost! While no cost should be ignored, it makes sense to focus your cost cutting efforts on the most expensive.
The next step is to assess if your spend factors are as expected based on knowledge of your business. For some companies, it’s perfectly normal for network or storage costs to be comparable to those of instances. For others, it would be the sign of a major problem.
To view cost by service, navigate to our Cost Controller dashboard. One of the items shown is Cost by Service. If you click on the report, you are taken to the source report where you can dive deeper by adding additional groupings and filters.
2. Understand your usage trends
Most cost saving analysis starts with trend analysis. Some deployments are very stable while others have peaks and valleys on regular intervals. It is critical that you know your consumption patterns and trends as this will enable you to spot anomalies and unplanned costs.
Navigate to the Actual Cost Over Time report (Cost > Cost Analysis > Actual Cost Over Time). This report shows your cost trends and by applying grouping and filtering to the results (group by Service, Account, or Resource Type) you can identify the account, service, or resource type that incurred the unusual expenses. Try running the report for the last 6 or 12 months with a weekly or monthly resolution. If one account experienced a large month-to-month growth, is it a healthy business related increase or a sign of trouble? Is your snapshot spend stable or increasing due to lack of clean up? You can then use our Asset reports to investigate the asset that led to the charge.
3. Severely underused instance’s
As running instances are the largest expense of most deployments, determining ones that are severely underused is a good place to start. Sometimes instances are left running by accident or with almost no workload. These can be found using the Instances – Lowest CPU report (Optimizer > Sizing Optimization > Instances – Lowest CPU). Select a date range of the Last Month. If you update the Lower Than filter to 5% and the Metric filter to Maximum, the results will show instance with a maximum CPU of 5%. Often these instances will be great candidates to terminate or downsize.
4. Unattached Disks
When instances are terminated, you will continue to pay for their EBS volumes until they are terminated as well. This can lead to waste of thousands of dollars each month. Our Unattached Disks report (Optimizer > Inefficiencies > Unattached Disks) lets you know of your storage that is not attached to active instances.
5. Unused RI’s
It is not uncommon to find that some of your purchased reserved instances are not being used. This is very unfortunate for a number of reasons:
- Reservation purchases are a large investment and it is a waste when they go unused.
- Often you will be paying on-demand rates for running instances that could be using the RI’s if you modified your reservations.
- If your usage has changed and even modifying the RI would not lead to you using them, they can be sold.
Cloudyn’s Currently Unused Reservations report (Optimizer > Inefficiencies > EC2 Currently Unused Reservations) informs you of all of your RI’s that are currently unused. In addition, we analyze your on-demand instance use and provide modification recommendations when relevant.
6. Disks of stopped machines
Another potential area are the attached volumes of stopped instances. While you are not billed for the instance itself, you still pay for the attached EBS volumes. It is recommended to periodically review your stopped instances and if they are no longer needed, to terminate the instance and the volume.
This data can be found in the Instance Explorer (Assets > Compute > Instance Explorer). In the Show filter at the bottom left, select Currently Stopped Instances. Then use the Show/Hide Fields menu above the results to add the Last Running Time and disk related fields. The Last Running Time field shows you the date the instance was last active and the disk fields provide information of the EBS disks allocated to each instance. If you expand the tab of an instance, the attached volume ID’s are listed in the Disks section.
7. Old Snapshots
EBS costs are often significant contributors of an organizations AWS spend. To understand how much of your EBS spend is on volumes vs. snapshots, go to the Actual Cost Analysis report (Cost Allocation > Standard Cost Analysis > Actual Cost Analysis). Add the Service and Resource Type filters and select EBS for Service.
In addition, try checking our Cost Over Time report and if your Snapshot spend has been steadily increasing, it can be due to not properly managing your older snapshots. Go to our Active Snapshots report (Assets > Disks > Active Snapshots) to see a full list of your active snapshots. Sort the results by the Created On column to see your oldest snapshots that are often no longer needed.
If not carefully monitored, S3 storage costs can balloon over time. Often some of the cost is driven by storage that is no longer used or needed. An easy way to investigate is the S3 Cost Over Time report (Assests > Storage (S3) > S3 Cost Over Time). This report shows your historical S3 spending trends per Bucket.
9. EC2 and RDS Sizing Recommendations
When spinning up instances, it is common to provision with more compute power than you actually need. Once workloads are running, it is a good idea to evaluate right sizing the instances and change to instance that fit your needs. The Cost Effective Sizing Recommendation report (Optimizer > Sizing Optimization > Cost Effective Sizing Recommendations) analyses performance metrics of your running EC2 instances and informs you of those that are candidates for downsizing. We even take into account instances that are using Reservations and only recommend sizing those that are on-demand.
For RDS instances, view our RDS Sizing Recommendations report (Optimizer > Sizing Optimization > Cost Effective Sizing Recommendations).
10. EC2 and RDS Buying Recommendations
EC2 and RDS costs can be significantly lowered by purchasing reserved instances. Cloudyn analyses the usage level of each instance type to create RI purchase recommendations and presents you with the savings that can be achieved through their purchase. Of course, you must take into account your deployment plans and not purchase RI’s if they will primarily by used by a project that is winding down in 3 months.
To view our recommendations, go to our EC2 and RDS RI Buying Recommendations reports (Optimizer > Pricing Optimization > EC2 & RDS RI Buying Recommendations). Once you have purchased RI’s, you can use our many tools to manage them and maximize their impact.
- A great way to lower EBS volume costs is to consider switching from IO1 to GP2 volumes. They are cheaper and the provisioned IOPS are included in the price of the volume and not charged separately.
- Always be on guard for a spike in cost or usage by using our Budgeting tools (Cost > Cost Management > Budget to create a budget and Cost > Projection and Budget > Cost vs. Budget Over Time to track spend and create alerts) and Alerts Management (My Tools > Alerts > Alerts Management) to set asserts based on threshold or percentage cost increases.
- Create accountability within your organization by using our Cost Allocation 360 tool for showback or chargeback.
IT chargeback and showback help organizations control how various business units and projects (i.e., internal customers or cost centers) consume IT services (e.g., infrastructure and data transfer) by allocating costs and cross-charging accordingly. Chargeback and showback have been around since the early days of mainframes, and with the introduction of the cloud comes the need for new implementation methods and tools.
Lack of Transparency Due to the virtual and dynamic nature of the cloud, enterprises are challenged by a lack of transparency into their cloud inventories, including their resource usage and costs. In order to implement chargeback, IT costs are distributed among different lines of business (LOB), various divisions and even temporary projects at the end of every month. However, due to a lack of clear visibility into their virtual environments, CIO and CFO teams often find themselves struggling to understand their cloud bills. By trying to manually analyze usage and costs, financial IT managers tend to end up with multiple spreadsheets riddled with inaccuracies. And the situation only worsens as enterprise cloud footprints grow.
Cloud Cost Allocation Cost allocation means calculating the accumulated costs of IT services that specific business units utilize. This can be quite challenging when it comes to medium-sized and large enterprises due to their rapidly changing environments and the need to accurately allocate the costs of every part of their cloud footprints. However, allocating usage to a specific business unit or project is complex, especially in a shared infrastructure. And manually answering questions about how much a specific application’s or region’s underlying infrastructure costs might involve a lot of guesswork. In addition, financial IT managers find it challenging to explain allocated costs to business unit leaders, who therefore can’t hold themselves accountable for their own cloud environments’ footprints and costs. Often, this leads to endless discussions that result in wasted time and effort for both parties.
Chargeback Plan Internal enterprise customers are charged for billable resources that are included in their IT service catalog according to a specific chargeback plan. Cloud chargeback plan can span different usage and cost metrics that are collected for each type of resource. In addition, these can include additional costs for IT maintenance services, for example, (on top of usage costs). IT teams find it difficult to set up fair and objective chargeback plan as well as communicate the plan across their business units due to the various components and hundreds of services involved.
Continuous Optimization One of the main advantages of using the cloud is the ability to adjust your capacity according to your current needs. This includes continuously seeking out cost optimization opportunities. However, the lack of transparency into usage for each business unit as well as across organizations as a whole leaves IT teams incapable of understanding relevant opportunities (e.g., resource consolidation and long term commitments) and taking action in terms of cost optimization without harming system performance.
Define Your N-Level Cloud Start by clearly defining your organization’s structure in terms of resource ownership. You can relate to this hierarchical structure as your n-level cloud. An n-level cloud is a multi-level cloud structure in which the number of levels and complexities is determined by the needs of each business unit or project. In most cases, n-level clouds are highly correlated with an enterprise’s organizational structure:
Implement: Consistent Tagging A hierarchical structure can often times be considered to represent costs in business terms. For example, a business unit may have multiple cloud cost centers, or a cost center may include more than one business unit. From an IT operations’ perspective, the overall cost can be broken down into individual environments, such as dev/test. These singular views of various environments do not overlap, and a customized cost structure can be created for each individual purpose, which is where tags come into play. Cloud vendors such as AWS, MS Azure, Google Cloud Platform and many others, provide you with the ability to categorize and tag each of your resources. They can be tagged with more than one key (category), that reflect the multiple dimensions of visibility that are required. As shown in the picture to the right, this allows you to manage your resources and enable high-level visibility that is not necessarily correlated with the original structure. (e.g., “Stack” = dev/test/production) Make sure that all of your resources are tagged! In order to maintain visibility, a value should be given the same label throughout an entire organization by the relevant resource owner. For example, if the development department assigns the value, “development” to one resource, and “DEV” to another, there is no consistency. Your key and value must be consistent across the pool of resources. Make sure that there is a key defined for all resources that describes the owner, the stack, and the organizational unit.
Define: Custom rules for allocation of untagged resources Even with a consistent tagging policy in place, you can’t be sure that 100% of users will be tagging 100% of resources, 100% of the time. These untagged resources are “invisible” to the cost allocation process and will not be included in chargeback reports, representing a cost gap between the total bill and the chargebacks. Another case of potentially “invisible” costs is with resources that are untaggable (e.g., support fees, AWS reservations). Strict allocation rules should be defined for all untagged (and untaggable) resources. These rules may be defined as a proportion of allocated costs (e.g., if 60%/40% of tagged costs are allocated between business units A/B respectively, then the untagged resources should be allocated with the same proportion), or by strict rules (e.g. regardless of the tagged resource allocation, untagged resources should always be allocated 50%/50% between business units A/B).
Cloud computing deployments require dynamic management. Continuous monitoring and analysis systems for your resource inventory, usage and utilization are required in order to learn variations and patterns. The key to implementing cost allocation, chargeback and showback is to first create total and reliable visibility into your cloud environment at any point in time. The main focus of Cloudyn’s solution is to provide a single view of cost drivers, resource allocation, and utilization across an entire organization. It will then be easy to identify and address the areas that can be optimized.
Key Benefits - The key benefits of Cloudyn’s chargeback solution:
Cloudyn provides a complete picture of all of your cloud environments aggregated into one simple to use dashboard. You are given a single view over multiple business units and multiple cloud platforms and can use it to optimize resource purchases and allocation.
With a comprehensive and flexible report and dashboard system, it is quick and easy for your CIO and CFO teams to create their own specific views. In addition, they can provide your business unit owners with their own usage and costs reports that can hold them accountable for their IT service utilization.
Applying Business Rules to Uncategorized Resources
Despite your company’s best efforts to implement a strict tagging policy, some resources will remain untagged. Other resources are, by definition, “untaggable” (i.e., support costs). Cloudyn offers the capability to set specific parameters for the allocation of these "rogue" resources, either in proportion with the allocation of tagged resources, or by setting strict allocation rules (e.g., 20%/30%/10%/40% among four business units).
Cloudyn’s category manager also aids in overcoming tagging inconsistencies, as mentioned earlier in the challenges section (“DEV” vs. “development”).