Product Overview
nCloud is a network-coding based file system that aims at providing fault tolerance and reducing data repair costs when storing files using multiple-cloud storage (or any other kinds of raw storage devices). nCloud mitigates the concerns of a single point of failure and vendor lock-ins. nCloud takes a cloud-native approach to build and run storage management, by harnessing the intrinsic properties of cloud computing, including the seemingly unlimited compute/storage resources, on-demand resource provisioning, and scalable and reliable services for developers.
What is nCloud?
nCloud is a cloud-native hyper-converged infrastructure solution, which leverages network coding to realize large-scale cloud-datacenter-storage with storage efficiency, performance, and fault tolerance.
Benefits:
- High storage efficiency: nCloud leverages network coding to minimize the amount of storage redundancy compared to traditional replication, thereby saving the long-term operational costs.
- High performance: nCloud mitigates the cross-DC traffic to maintain high repair performance. It also mitigates the overhead on other applications that share the cross-DC bandwidth.
- High fault tolerance: nCloud achieves high fault tolerance via network coding in two aspects: (i) availability, which means that any unavailable data remains accessible through data redundancy, and (ii) durability, which means there is no data loss in the face of node or DC failures.
- Security via diversity: nCloud exploits secret sharing to generate secure coded chunks (called shares), such that an adversary cannot recover the stored data if the number of shares is insufficient.
- Software-defined storage management: nCloud provides a configurable, software-defined storage management framework to readily address the heterogeneity of storage resources and application requirements.
Use cases:
nCloud can be used for multiple scenarios.
- Datahub for enterprises, public, governments, schools, universities, etc.
- File server for enterprises, schools, etc.
- Backup solution for enterprises, government, and private/public institutions, etc.
- IoT hub for companies providing IoT services
What technology is used?
Network coding is the key technology used by nCloud. Network coding provides a promising low-cost redundancy mechanism to achieve fault tolerance, while significantly reducing the storage overhead compared to traditional replication and achieving optimal repair performance with theoretical guarantees.
The figure shows the system architecture of nCloud. nCloud manages multiple datacenters (DCs) that are located in different geographical sites. It consists of several entities, as described below.
Client: A client refers to an upper-layer application and sends read/write requests for accessing the data.
Proxy: A proxy acts as the data management layer that serves as the control plane of nCloud. It provides a platform for administrators to configure different storage management policies. It sits between clients and DCs and holds a global view of storage management. It handles read/write requests from clients. Specifically, when a client issues a file write request, the proxy first divides the file into k data chunks and encodes the file into m parity chunks via network coding. It sends the k+m chunks to k+m storage nodes that are located in different DCs. The coding operations can be combined with secret sharing to achieve secure storage against adversaries via diversity. Also, the proxy is responsible for detecting any failure and coordinating repair operations across DCs with cross-DC awareness.
Agent: Each agent manages the storage of a DC, while multiple agents together form the data plane of nCloud. Each agent is responsible for handling storage operations sent by the proxy and managing metadata for each DC. In particular, the agents are the key entities that realize network coding, as they perform encoding of data during repair to mitigate cross-DC traffic