By deciding to use the cloud for storage to provide enterprise applications with the flexibility and scalability they need, a company is faced with the choice of a specific cloud storage system. We suggest using a short instruction that will help you decide which type of cloud storage is best for your projects, and tell you how to combine different types of storage for better system functionality and savings.

Modern cloud storage is represented by three types: block, object, and file. If you look at the features of such storage in the context of real tasks, then they have their advantages and disadvantages.

OPTION 1: BLOCK STORAGE OR CLOUD DISKS

Cloud disks mimic the physical disk media that operating systems typically work with. Such disks are connected to cloud servers and then interact with them as with the disk space of a regular block storage device that serves physical servers. However, the very "virtuality" of these disks additionally provides functionality that is impossible or too expensive to obtain on physical disks.

Features of Cloud Drives

Cloud disks are flexible in management, that is, additional disks can be connected to a running cloud server without turning it off, as well as change the disk type during operation. It is possible to create snapshots (snapshots) and disk images, backups. At the same time, you can work with cloud disks using the usual methods, since they imitate physical devices for storing data.

In addition, if you use cloud disks, you do not have to think about buying and setting up equipment, as well as other technical nuances of storage, as with physical disks - this will be done by the cloud provider. It also guarantees a certain system bandwidth and storage performance: you just need to choose the type of disk for your needs.



The main types of block storage there are several types of cloud drives that differ in a number of characteristics:


HDD - basic disks with the lowest performance and the lowest cost. They are most often used as the basis for file storage and data loading of operating systems.
SSDs are standard drives that are faster than cloud HDDs, but also more expensive. Used for databases, telemetry data, message queues.

 High IOPS SSDs are fast drives with higher performance and cost than standard SSDs. They are used mainly for the same applications (analytics, telemetry, database), but are more loaded and therefore with more serious performance requirements.

 Low Latency NVMe - Ultra-fast and powerful with the lowest latency, but also the highest price tag. They are used for high-performance databases, analytics, and cache applications that require minimal latency.

    The choice of the type of block storage is determined by the type of application and its workloads, we will talk more about the selection algorithm below. The price of such storage depends on the type of disks. From HDDs to High IOPS SSDs, read and write speeds increase, which is essential for application performance. And Low Latency NVMe, in addition, to read and write speed, also has a response speed, which is important for databases and high-load systems with many simultaneous requests and changes. Accordingly, the cost of different drives increases with the growth of performance and response speed: HDD will be cheaper, and Low Latency NVMe is the most expensive.

OPTION 2. OBJECT CLOUD S3 STORAGE

In object storage, S3 data is not stored as files, but as objects with metadata. Metadata are tags that indicate specific information about an object. For example, if you store videos in the repository, then you can specify the title, author, year of release, and other information. In addition, each object has a unique identifier.

To store data in S3, they do not need to be converted to any specific type, so much storage is well suited for unstructured data, media content, archives.

Also, S3 storage is ideal for scenarios when you need to provide access to files for a large number of users at once. For example, video hosting sites or online cinemas can host a huge number of videos, millions of visitors will refer to them, and S3 can withstand such a load.

Backups and copies of infrastructure for disaster recovery are often stored in object storage since it is cheaper than placing such data in regular storage.

Features of cloud S3 storage:

In object storage, the amount of stored data is not limited and it is possible to store any type of object, including unstructured data. If S3 uses many applications at the same time, this does not affect the speed of distribution of objects: users will quickly receive a response to requests. Also, such storage is automatically scaled: you can put any amount of data in it, S3 will adjust to the load, even if this volume suddenly grows multiple times.

The lifecycle of objects stored in S3 can be flexibly managed. For example, you can configure the deletion of archived data in automatic mode, then unnecessary information will not take up space. And for the automatic processing of objects (for example, a photo or video), it is possible to configure the mechanism for notifying about Webhooks events. For example, you can configure automatic video processing after creating an object in the storage.

Access to files in the repository is possible using the object URL, that is, a regular link. Using it, the user can access files in the storage without the ability to change or delete them. A simple example: in an online movie theater on the site, a link to a video is embedded, which is located in the object storage. Any visitor can start the video and watch it, but other actions will be unavailable to him.


Three types of object storage are available on the Mail.ru Cloud Solutions platform:

For data that needs to be accessed frequently (HotBox). This view is suitable for high-load applications that need to store and quickly serve large amounts of content. This includes multimedia services (photo and video hosting, online cinemas, online schools with training videos). Also, such storage is used for the backend of applications and hosting site pages that do not need to be changed often. Another use case: Big Data analytics and storage of unstructured data, from which valuable information is then extracted. For example, the placement of data from scientific experiments or genetic research. By connecting such S3 storage, you will pay more for storing data, but less for distributing it to users.


 For data that is rarely accessed (IceBox). These are archives of companies, including reporting, as well as backups and documents with quick and rare access. Here, data placement will be cheaper, but their distribution to users will be more expensive.
 For data with very rare access (Glacier). Option for heavy data of archives, logs, and backups, which are accessed very rarely, for example, several times a year. Here the cost of storage is the lowest, all traffic is not paid.
S3 can be used alone or can be combined with other cloud storage systems. This is usually done in order to unload the block disks of cloud servers and take out the data from them, which is more profitable to store in object storage.

OPTION 3. FILE STORAGE
Here the data is stored in directories and folders with a strict hierarchy. It is suitable for shared documents, custom files, and heavy files that need to be modified frequently. On the Mail.ru Cloud Solutions platform, you can connect file storage as a service - use it as a remote file storage by connecting to a cloud server.

Features of file storage

File storage is supported by most classic systems, so this type of storage is well suited for Legacy applications to which it is difficult or impossible to connect another storage system in the cloud. You can shrink and enlarge the file storage, but only in manual mode. On the MCS platform, only HDD disks can be used for file storage, so its main performance characteristics will coincide with those of the HDD.

WHAT YOU SHOULD DECIDE BEFORE CHOOSING CLOUD STORAGE FOR A PROJECT

There are several criteria to look out for when choosing cloud storage:

How you will access the data and how often. For example, S3 storage will not work if you want to host large objects and modify them frequently. It is better to choose file or block storage here. S3 is more suitable for storing rarely changed objects; when choosing, you need to choose the right tariff, taking into account the frequency of access to them.
 What system performance do you need. If you need the fastest possible system response and high throughput, then you should connect block storage. In terms of performance, block storages are in the following order: HDD disks, SSD disks, SSD High IOPS disks, Low Latency NVMe disks. Low Latency NVMe will have the lowest latency - only 0.5 ms - these are the fastest disks.

Features of the company's applications. If you are migrating a legacy application, you will have to focus on what protocol it uses to communicate with the repository. Sometimes, due to such technical limitations, only file storage can be used.

 Price. S3 storage will have the lowest price, it will vary depending on the chosen tariff. However, here you only pay for the amount of storage you use. For file and block storage, the cost depends on how much resources you reserve, and the more productive the system, the higher the price. Here it is worth choosing the option that will be the cheapest and at the same time provide the required speed of work. If you suddenly need a faster disk, its type can always be changed in the process, for this you do not need to stop services or restart the cloud server.

HOW TO COMBINE CLOUD STORAGE BETWEEN YOURSELF

Cloud storage can be used separately, but often project requirements assume that different types of data are used and different scenarios for accessing them.

In this case, you can combine cloud storage systems with each other. For example, for an online store, the following combination is suitable:

Connect standard HDDs for the operation of the operating system, since they are inexpensive and will quite cope with this task;

 Faster and more efficient SSD High IOPS are better suited for the operation of the database management system, HDDs may not be able to cope here;
 Low Latency NVMe is ideal for caching since they allow you to access stored data with the lowest latency;

 Media content, such as videos with product reviews and pictures, as well as backups, are more profitable to place in object storage and thus take the load off the disks of cloud servers.

 The combination of different cloud storage systems can reduce costs and increase the overall performance of the IT system.

To make the best choice and choose cloud storage for the needs of the company, it is important to have a good understanding of what data and where will be stored, how it will be accessed. Using the cloud gives you some flexibility: you can combine and replace cloud storage systems, it is easier to scale them and add new resources, which is convenient as the project grows and changes.