In this post, we will see how you can monitor Azure Storage using Netreo.
We will cover the basics of Azure storage focusing on Blob storage and we will see how you can monitor the Blob storage using Netreo.
Azure Storage provides scalable, durable and highly available storage that allows to store from just few files required for powering up a website to huge amount of data required for big data scenarios.
There are four types of storage services:
- Blob storage – contains unstructured objects data that can be any kind of text or binary data like pictures, documents, videos, logs.
- Table storage – a NoSQL key-attribute data store where each value has a property name.
- Queue storage – provides a reliable messaging solution for asynchronous communication between application components
- File storage – a Server Message Block(SMB) file share in the cloud that allow the user to migrate easily applications relying on file shares to Azure
This post is focused on Blob storage so let’s cover some details about it:
- The blob storage account has a 500 TB limit
- Each blob is organized in a container
- There can be as many containers as you want
- A container can contain as many blobs as you want
- There can be three types of blobs:
- Block – optimized for cloud objects like backups, videos, pictures, documents
- Append – optimized for append operations often seen with logs where data is written at the end of the blob
- Page – optimized to be used as IaaS disks
There is a Blob storage account where two containers called container1 and container 2 were created:
And we will use Netreo to monitor this Blob storage account as you can see below:
Once the resources are added in Netreo, there will be another container created in the Blob storage:
This new container(znetreomonitoringcontainer) is used by Netreo as a placeholder for a very small data that is written on the blob storage to verify the integrity of the blob storage. The data is deleted after each check done by Netreo. In this way, you have read and write checks to determine the availability of the blob storage.
From the “Performance Dashboard”, select “Monitoring Settings”, move to “Metrics” tab and expand the “ResponseTime” metric:
Next move to “Alerts” tab to see all the alerts created:
Expand the “Slow Response” alert to see under which circumstances the alert is triggered:
This shows that if for three minutes of monitoring the response time is more than 15000ms(15 seconds), the alert will be triggered.
Let’s see this alert in action. For this, we will change the parameters and the amount of time for which they are monitored before the alert is triggered in a way that will allow us to fire up the alarm quicker.
More exactly, the response time will have to be lower than 50ms and only for one minute:
Save the change and return to the “Dashboards” menu and then select the “Alerts Dashboard”. You should see the alert being triggered like this:
You can also see the alert(like any other alert actually) in the top menu in the “Active Alerts” section:
One other useful alert when you are working with storage is the one that tells you if there are requests(like for instance downloading objects from the storage) that are failing, hence the users cannot retrieve the objects from the storage.
This alert can be built by using other two metrics: the total number of requests and the total number of successful requests. Obviously, if the first metric is higher than the first one, then the difference between them consists of failed requests.
Going back to the monitoring settings in the “Metrics” tab, we can see the two metrics that we are going to use:
Further, on “Alerts” tab, you have this alert that as you can see relies on the Successful Requests and Total Requests metrics:
Again, we modified the amount of time for which the condition has to be active before the alert is being triggered.
To trigger the alarm, we sent multiple requests to the blob storage that we knew they will fail so that the number of failed requests will be added to the total number of requests.
And the alarm was triggered:
Again, the alarm can be seen top drop-down menu of the active alarms:
You can also check the availability of the blob storage and see how much of it is available. The metric looks like this:
And the alert can be configured like this(in this case will be triggered if the availability of the blob storage will be less than 100%):
We reached the end of the post where we discussed the basics of the Azure storage service with a focus on Blob storage.
Then we saw how Netreo can be used to monitor the blob resources and send alerts in case user configurable conditions were met.
Obviously, there are more blob storage metrics that can be set for which alerts can be triggered in case thresholds/conditions set by the user are met, but this is just to give a start and understand how it can be done.