Cloud Computing: Chapter 8 - Data Storage and File Systems, Essays (university) of Computer Science

Download Cloud Computing: Chapter 8 - Data Storage and File Systems and more Essays (university) Computer Science in PDF only on Docsity! Chapter 8 – Storage Systems Contents  Big data.  Evolution of storage systems.  Storage and data models.  Database management systems.  Network File System.  General Parallel File System.  Google File System.  Apache Hadoop.  Chubby.  Online transaction processing.  NoSQL databases.  Bigtable.  Megastore. Cloud Computing: Theory and Practice. Chapter 8 2 Dan C. Marinescu Evolution of storage technology  The capacity to store information in units of 730-MB (1 CD-ROM)  1986 - 2.6 EB  <1, CD-ROM /person.  1993 - 15.8 EB  4 CD-ROM/person.  2000 - 54.5 EB  12 CD-ROM/person.  2007 -295.0 EB 61 CD-ROM/person.  Hard disk drives (HDD) - during the 1980-2003 period:  Storage density of has increased by four orders of magnitude from about 0.01 Gb/in2 to about 100 Gb/in2  Prices have fallen by five orders of magnitude to about 1 cent/MB.  HDD densities are projected to climb to 1,800 Gb/in2 by 2016, up from 744 Gb/in2 in 2011.  Dynamic Random Access Memory (DRAM) - during the period 1990-2003:  The density increased from about 1 Gb/in2 in 1990 to 100 Gb/in2 .  The cost has tumbled from about $80/MB to less than $1/MB. Cloud Computing: Theory and Practice. Chapter 8 5 Dan C. Marinescu Storage and data models  A storage model  describes the layout of a data structure in a physical storage - a local disk, a removable media, or storage accessible via the network.  A data model  captures the most important logical aspects of a data structure in a database.  Two abstract models of storage are used.  Cell storage  assumes that the storage consists of cells of the same size and that each object fits exactly in one cell. This model reflects the physical organization of several storage media; the primary memory of a computer is organized as an array of memory cells and a secondary storage device, e.g., a disk, is organized in sectors or blocks read and written as a unit.  Journal storage  system that keeps track of the changes that will be made in a journal (usually a circular log in a dedicated area of the file system) before committing them to the main file system. In the event of a system crash or power failure, such file systems are quicker to bring back online and less likely to become corrupted. Cloud Computing: Theory and Practice. Chapter 8 6 Dan C. Marinescu Read/write coherence and before-or-after atomicity are two highly desirable properties of any storage model and in particular of cell storage Cloud Computing: Theory and Practice. Chapter 8 7 Dan C. Marinescu M A M A time Read/Write coherence: the result of a Read of memory cell M should be the same as the most recent Write to that cell A A Write item A to memory cell M Read item A from memory cell M Before-or-after atomicity: the result of every Read or Write is the same as if that Read or Write occurred either completely before or completely after any other Read or Write. Current Read/Write Previous Read/Write Next Read/Write time Logical and physical organization of a file  File  a linear array of cells stored on a persistent storage device. Viewed by an application as a collection of logical records; the file is stored on a physical device as a set of physical records, or blocks, of size dictated by the physical media.  File pointer identifies a cell used as a starting point for a read or write operation.  The logical organization of a file  reflects the data model, the view of the data from the perspective of the application.  The physical organization of a file  reflects the storage model and describes the manner the file is stored on a given storage media. Cloud Computing: Theory and Practice. Chapter 8 10 Dan C. Marinescu File systems  File system  collection of directories; each directory provides information about a set of files.  Traditional – Unix File System.  Distributed file systems.  Network File Systems (NFS) - very popular, have been used for some time, but do not scale well and have reliability problems; an NFS server could be a single point of failure.  Storage Area Networks (SAN) - allow cloud servers to deal with non-disruptive changes in the storage configuration. The storage in a SAN can be pooled and then allocated based on the needs of the servers. A SAN-based implementation of a file system can be expensive, as each node must have a Fibre Channel adapter to connect to the network.  Parallel File Systems (PFS) - scalable, capable of distributing files across a large number of nodes, with a global naming space. Several I/O nodes serve data to all computational nodes; it includes also a metadata server which contains information about the data stored in the I/O nodes. The interconnection network of a PFS could be a SAN. Cloud Computing: Theory and Practice. Chapter 8 11 Dan C. Marinescu Unix File System (UFS)  The layered design provides flexibility.  The layered design allows UFS to separate the concerns for the physical file structure from the logical one.  The vnode layer allowed UFS to treat uniformly local and remote file access.  The hierarchical design supports scalability reflected by the file naming convention. It allows grouping of files directories, supports multiple levels of directories, and collections of directories and files, the so-called file systems.  The metadata supports a systematic design philosophy of the file system and device-independence.  Metadata includes: file owner, access rights, creation time, time of the last modification, file size, the structure of the file and the persistent storage device cells where data is stored.  The inodes contain information about individual files and directories. The inodes are kept on persistent media together with the data. Cloud Computing: Theory and Practice. Chapter 8 12 Dan C. Marinescu The NFS client-server interaction. The vnode layer implements file operation in a uniform manner, regardless of whether the file is local or remote. An operation targeting a local file is directed to the local file system, while one for a remote file involves NFS; an NSF client packages the relevant information about the target and the NFS server passes it to the vnode layer on the remote host which, in turn, directs it to the remote file system. Cloud Computing: Theory and Practice. Chapter 8 15 Dan C. Marinescu Local host Application File system API interface Vnode layer NFS client NFS stub Remote host File system API interface Communication network NFS server NFS stub Vnode layer Local file system Remote file system  The API of the UNIX file system and the corresponding RPC issued by an NFS client to the NFS server.  fd  file descriptor.  fh  for file handle.  fname  file name,  dname  directory name.  dfh the directory were the file handle can be found.  count  the number of bytes to be transferred.  buf the buffer to transfer the data to/from.  device  the device where the file system is located. Cloud Computing: Theory and Practice. Chapter 8 16 Dan C. Marinescu Comparison of distributed file systems Cloud Computing: Theory and Practice. Chapter 8 17 Dan C. Marinescu Cloud Computing: Theory and Practice. Chapter 8 20 Dan C. Marinescu SAN disk disk disk disk LAN1 LAN2 LAN3 LAN4 disk disk I/O servers GPFS reliability  To recover from system failures, GPFS records all metadata updates in a write-ahead log file.  Write-ahead  updates are written to persistent storage only after the log records have been written.  The log files are maintained by each I/O node for each file system it mounts; any I/O node can initiate recovery on behalf of a failed node.  Data striping allows concurrent access and improves performance, but can have unpleasant side-effects. When a single disk fails, a large number of files are affected.  The system uses RAID devices with the stripes equal to the block size and dual-attached RAID controllers.  To further improve the fault tolerance of the system, GPFS data files as well as metadata are replicated on two different physical disks. Cloud Computing: Theory and Practice. Chapter 8 21 Dan C. Marinescu GPFS distributed locking  In GPFS, consistency and synchronization are ensured by a distributed locking mechanism. A central lock manager grants lock tokens to local lock managers running in each I/O node. Lock tokens are also used by the cache management system.  Lock granularity has important implications on the performance. GPFS uses a variety of techniques for different types of data.  Byte-range tokens  used for read and write operations to data files as follows: the first node attempting to write to a file acquires a token covering the entire file; this node is allowed to carry out all reads and writes to the file without any need for permission until a second node attempts to write to the same file; then, the range of the token given to the first node is restricted.  Data-shipping an alternative to byte-range locking, allows fine-grain data sharing. In this mode the file blocks are controlled by the I/O nodes in a round-robin manner. A node forwards a read or write operation to the node controlling the target block, the only one allowed to access the file. Cloud Computing: Theory and Practice. Chapter 8 22 Dan C. Marinescu GFS chunks  GFS files are collections of fixed-size segments called chunks.  The chunk size is 64 MB; this choice is motivated by the desire to optimize the performance for large files and to reduce the amount of metadata maintained by the system.  A large chunk size increases the likelihood that multiple operations will be directed to the same chunk thus, it reduces the number of requests to locate the chunk and, at the same time, it allows the application to maintain a persistent network connection with the server where the chunk is located.  A chunk consists of 64 KB blocks and each block has a 32 bit checksum.  Chunks are stored on Linux files systems and are replicated on multiple sites; a user may change the number of the replicas, from the standard value of three, to any desired value.  At the time of file creation each chunk is assigned a unique chunk handle. Cloud Computing: Theory and Practice. Chapter 8 25 Dan C. Marinescu  The architecture of a GFS cluster; the master maintains state information about all system components; it controls a number of chunk servers. A chunk server runs under Linux; it uses metadata provided by the master to communicate directly with the application. The data and the control paths are shown separately, data paths with thick lines and the control paths with thin lines. Arrows show the flow of control between the application, the master and the chunk servers. Cloud Computing: Theory and Practice. Chapter 8 26 Dan C. Marinescu Application Master Meta-information Communication network Linux file system Chunk server Chunk serverChunk server Linux file system Linux file system Chunk data Chunk handle & data count File name & chunk index Chunk handle & chunk location Instructions State information Apache Hadoop  Apache Hadoop  an open source, Java-based software, supports distributed applications handling extremely large volumes of data.  Hadoop is used by many organization from industry, government, and research; major IT companies e.g., Apple, IBM, HP, Microsoft, Yahoo, and Amazon, media companies e.g., New York Times and Fox, social networks including, Twitter, Facebook, and Linkedln, and government agencies such as Federal Reserve.  A Hadoop system has two components, a MapReduce engine and a database. The database could be the Hadoop File System (HDFS), Amazon’s S3, or CloudStore, an implementation of GFS.  HDFS is a distributed file system written in Java; it is portable, but it cannot be directly mounted on an existing operating system. HDFS is not fully POSIX compliant, but it is highly performant. Cloud Computing: Theory and Practice. Chapter 8 27 Dan C. Marinescu The Paxos algorithm  Used to reach consensus on sets of values, e.g., the sequence of entries in a replicated log.  The phases of the algorithm.  Elect a replica to be the master/coordinator. When a master fails, several replicas may decide to assume the role of a master; to ensure that the result of the election is unique each replica generates a sequence number larger than any sequence number it has seen, in the range (1,r) where r is the number of replicas, and broadcasts it in a propose message. The replicas which have not seen a higher sequence number broadcast a promise reply and declare that they will reject proposals from other candidate masters; if the number of respondents represents a majority of replicas, the one who sent the propose message is elected as the master.  The master broadcasts to all replicas an accept message including the value it has selected and waits for replies, either acknowledge or reject.  Consensus is reached when the majority of the replicas send the acknowledge message; then the master broadcasts the commit message. Cloud Computing: Theory and Practice. Chapter 8 30 Dan C. Marinescu Locks  Advisory locks  based on the assumption that all processes play by the rules; do not have any effect on processes that circumvent the locking mechanisms and access the shared objects directly.  Mandatory locks block access to the locked objects to all processes that do not hold the locks, regardless if they use locking primitives or not.  Fine-grained locks  locks that can be held for only a very short time. Allow more application threads to access shared data in any time interval, but generate a larger workload for the lock server. When the lock server fails for a period of time, a larger number of applications are affected.  Coarse-grained locks  locks held for a longer time. Cloud Computing: Theory and Practice. Chapter 8 31 Dan C. Marinescu  A Chubb cell consisting of 5 replicas, one of them elected as a master; n clients use RPCs to communicate with the master. Cloud Computing: Theory and Practice. Chapter 8 32 Dan C. Marinescu C1 C3 C2 C4 Cn-1 Cn Chubby cell Replica Replica Replica Replica Master . . . Transaction processing  Online Transaction Processing (OLTP)  widely used by many cloud applications.  Major requirements:  Short response time.  Scalability.  Vertical scaling  data and workload are distributed to systems that share resources, e.g., cores/processors, disks, and possibly RAM  Horizontal scaling  the systems do not share either primary or secondary storage.  The search for alternate models to store the data on a cloud is motivated by the needs of OLTP applications:  decrease the latency by caching frequently used data in memory.  allow multiple transactions to occur at the same time and decrease the response time by distributing the data on a large number of servers. Cloud Computing: Theory and Practice. Chapter 8 35 Dan C. Marinescu Sources of OLTP overhead  Four sources with equal contribution:  Logging - expensive because traditional databases require transaction durability thus, every write to the database can only be completed after the log has been updated.  Locking - to guarantee atomicity, transactions lock every record and this requires access to a lock table.  Latching – many operations require multi-threading and the access to shared data structures, such as lock tables, demands short-term latches for coordination. A latch is a counter that triggers an event when it reaches zero; for example a master thread initiates a counter with the number of worker threads and waits to be notified when all of them have finished.  Buffer management.  The breakdown of the instruction count for these operations in existing DBMS is: 34.6% for buffer management, 14.2% for latching, 16.2 % for locking, 11.9% for logging, and 16.2 % for manual optimization. Cloud Computing: Theory and Practice. Chapter 8 36 Dan C. Marinescu NoSQL databases  The name NoSQL is misleading. Stonebreaker notes that “blinding performance depends on removing overhead. Such overhead has nothing to do with SQL, it revolves around traditional implementations of ACID transactions, multi-threading, and disk management.”  The soft-state approach allows data to be inconsistent and transfers the task of implementing only the subset of the ACID properties required by a specific application to the application developer.  NoSQL systems ensure that data will be eventually consistent at some future point in time, instead of enforcing consistency at the time when a transaction is committed.  Attributes:  Scale well.  Do not exhibit a single point of failure.  Have built-in support for consensus-based decisions.  Support partitioning and replication as basic primitives. Cloud Computing: Theory and Practice. Chapter 8 37 Dan C. Marinescu Bigtable performance – the number of operations Cloud Computing: Theory and Practice. Chapter 8 40 Dan C. Marinescu Megastore  Scalable storage for online services. Widely used internally at Google, it handles some 23 billion transactions daily, 3 billion write and 20 billion read transactions.  The system, distributed over several data centers, has a very large capacity, 1 PB in 2011, and it is highly available.  Each partition is replicated in data centers in different geographic areas. The system supports full ACID semantics within each partition and provides limited consistency guarantees across partitions.  The Paxos consensus algorithm is used to replicate primary user data, metadata, and system configuration information across data centers and for locking. The version of the Paxos algorithm does not require a single master, instead any node can initiate read and write operations to a write-ahead log replicated to a group of symmetric peers.  The system makes extensive use of Bigtable. Cloud Computing: Theory and Practice. Chapter 8 41 Dan C. Marinescu Megastore’s data model  Reflects a middle ground between traditional and NoSQL databases.  The data model is declared in a schema consisting of a set of tables, composed of entries.  An entry  a collection of named and typed properties; the unique primary key of an entity in a table is created as a composition of entry properties. An entity group consists of the primary entity and all entities that reference it.  A table can be a root or a child table. Cloud Computing: Theory and Practice. Chapter 8 42 Dan C. Marinescu