Storage (Data Storage System)

SSDs differ from a user hard disk drive by their complex architecture, the ability to unite storages into a data transmission network, separate software for storage system management, advanced backup, compression and virtualization technologies.

Storage systems differ by several parameters, the choice of which determines the use of storage systems.

ITPOD Storage Full Flash FF-300

How a storage system is organized

Storage systems consist of key elements such as arrays of hard disk drives or solid state drives (HDD or SSD), cache memory, a disk controller, an external enclosure, and multiple power supplies to ensure reliability. The main advantage of storage systems is the high speed of data processing, which surpasses the performance of disks inside servers. These systems are categorized into three types: file, block, and object, each of which defines the storage format, access method and manageability, and speed of data interaction.

Data storage tiers

Block storage

. Storage is used like a regular disk that can be formatted, OS can be installed on it, logical disks can be created. Data is stored in blocks rather than files, which speeds up I/O operations. More often used in SAN (Storage Attached Network) type networks. It is suitable for high-performance computing, DBMS, storage of large amounts of data, as a development environment (Dev/Test). The disadvantages are: a) complexity of configuration and maintenance, which require appropriate qualification; b) high cost.

File storage

. Data is stored in the form of files that are placed in directories. Such storage is used to store “cold” information that is not required for operational computing. NAS (Network Attached Storage) is usually built on file storage. Disadvantages: when accumulating large amounts of data, the folder hierarchy becomes more complex, and the speed of storage gradually decreases. It is not suitable for workloads that require high response speed.

Object storage

. A type of storage that is focused on handling large unstructured data up to petabytes in size. Information is stored not as files, but as “objects” with unique identifiers and metadata. Therefore, object storage is similar in structure to a database. It is used in analytics, big data, machine learning, for storing “heavy” media files and backups, development and operation of applications in the cloud, website hosting. It is inferior to block storage in tasks related to transactional loads.

The principle of storage – NAS, SAN and DAS

DAS (Direct Attach Storage)

. A storage system connected directly to a server or workstation via interfaces such as SATA, SAS or USB without using a network. This solution provides high-speed access and reliability by minimizing network components. DAS is ideal for applications that require fast data access on a single device, such as database servers.

Benefits of DAS:

• Ease of deployment and management;
• Fast data transfer;
• Economic in terms of hardware costs.

Disadvantages of DAS:

• Dedicated server required;
• Limited number of connections, supports no more than two servers.

NAS (Network Attache Storage)

A file server that is included in a local network. Access to disk storage is organized through the protocols NFS (in UNIX/Linux systems) or CIFS (Windows). NAS is used to work with file-type data, to which you need collective simultaneous access – for example, to common Word and Excel documents. NAS works “on top” of an existing LAN, via shared switches/routers.

Benefits of NAS:

• Economy and wide availability;
• Easy to set up and manage;
• Flexibility, ability to quickly increase storage capacity;
• Ability to work with devices under any operating system.

Disadvantages of NAS:

• Storage of data exclusively in file format;
• Slower access to data through the network compared to local systems;
• Inability of some applications to work with network disks.

SAN (Storage Area Network)

A network that is suitable for using different types of storage (disks, optical drives, tape arrays), but which are perceived by the operating system as a single logical data storage, or as a network logical disk. Protocols: iSCSI (IP-SAN) and FibreChannel (FC). HBA (Host Bus Adapter) adapters are used to connect computers. SAN uses mostly block-based storage.

The SAN/NAS division is no longer as strict as it was in the early 2000s, because with the advent of the iSCSI protocol, vendors are now producing hybrid solutions.

Benefits of SAN:

• High performance and minimal latency;
• Flexibility and scalability;
• Block storage method;
• Reliability of data exchange and storage;
• Reduces subnetwork load from service traffic.

Disadvantages of SAN:

• Design complexity;
• Significant implementation and operation costs;
• Inability of some applications and systems to work with iSCSI protocol.

How to choose a storage system

First of all, you need to understand what tasks it will solve. Before contacting a vendor (or integrator), you should decide on a few basic parameters.

• Data type

. Different types of data require different access speeds, processing techniques, compression, and so on. For example, a storage system to handle large media files is different from one that is suitable for a transactional DBMS, or a system that will handle unstructured data for a neural network.

• Data volume

. This determines the choice of disk drives. Sometimes you can do without consumer-grade SSDs – if you know that the storage capacity even in the worst case will not exceed 300 GB, and the access speed is not critical.

• Storage resilience

Two metrics – RPO and RTO – are used to evaluate the ability of storage to recover from failures. They help you calculate the cost of data loss over time, and avoid unnecessary backup costs.

RPO (recovery point objective)

The period in which data will be lost is between the time of the disaster and the time when the last backup was created. If the RPO is 12 hours, a storage failure can result in the loss of data accumulated over the last 12 hours. RPO affects the choice of disaster recovery technology and depends on the cost of losing a specific amount of data.

RTO (recovery time objective)

The time it takes to restore access to the storage. The RTO value is important for estimating the cost of system downtime.

• Redundancy

The frequency of backups is chosen based on specific tasks and the required level of protection. The same applies to the location: working data and its backup can be stored in geographically distributed storage (for example, in data centers located in different countries or even continents).

In addition to backups, snapshots – snapshots that are used to roll back to the last working version of the system.

To make backups take up less space, deduplication is used. In this case, only the data that has changed is overwritten into the copy. The difference between backups does not exceed 2% on average, so deduplication helps to save disk space.

• Performance

. If you are purchasing a storage system for a new project (service), and it is difficult to judge the load, it is better to talk to colleagues who have already solved this task. Or contact an experienced vendor who has already launched similar projects. The ideal option is to test the storage system.

• Vendor

Sometimes even for a resource-intensive service, a budget or mid-level solution (StarWind, Huawei, Fujitsu) will do. However, the top manufacturers – NetApp, HPE, Dell EMC – have quite a wide product line, and relatively inexpensive storage systems can also be found here. With regard to domestic solutions, one should pay attention to ITPOD storage systems. The vendor offers flexible and adaptable options for various business needs and focuses on providing productive and cost-effective technologies. In any case, it is advisable not to greatly expand the number of vendors on a single infrastructure.