File Systems - Introduction to Operating Systems - Lecture Slides | CS 1550, Study notes of Computer Science

Download File Systems - Introduction to Operating Systems - Lecture Slides | CS 1550 and more Study notes Computer Science in PDF only on Docsity! 1 Chapter 6: File Systems Chapter 6 2CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) File systems Files Directories & naming File system implementation Example file systems Chapter 6 3CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Long-term information storage Must store large amounts of data Gigabytes -> terabytes -> petabytes Stored information must survive the termination of the process using it Lifetime can be seconds to years Must have some way of finding it! Multiple processes must be able to access the information concurrently Chapter 6 4CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Naming files Important to be able to find files after they’re created Every file has at least one name Name can be Human-accessible: “foo.c”, “my photo”, “Go Panthers!”, “Go Banana Slugs!” Machine-usable: 4502, 33481 Case may or may not matter Depends on the file system Name may include information about the file’s contents Certainly does for the user (the name should make it easy to figure out what’s in it!) Computer may use part of the name to determine the file type Chapter 6 5CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Typical file extensions Chapter 6 6CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) File structures Sequence of bytes Sequence of records 1 byte 1 record 12A 101 111 sab wm cm avg ejw sab elm br S02 F01 W02 Tree 2 Chapter 6 7CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) File types Executable file Archive Chapter 6 8CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Accessing a file Sequential access Read all bytes/records from the beginning Cannot jump around May rewind or back up, however Convenient when medium was magnetic tape Often useful when whole file is needed Random access Bytes (or records) read in any order Essential for database systems Read can be … Move file marker (seek), then read or … Read and then move file marker Chapter 6 9CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) File attributes Chapter 6 10CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) File operations Create: make a new file Delete: remove an existing file Open: prepare a file to be accessed Close: indicate that a file is no longer being accessed Read: get data from a file Write: put data to a file Append: like write, but only at the end of the file Seek: move the “current” pointer elsewhere in the file Get attributes: retrieve attribute information Set attributes: modify attribute information Rename: change a file’s name Chapter 6 11CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Using file system calls Chapter 6 12CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Using file system calls, continued 5 Chapter 6 25CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Linked allocation File is a linked list of disk blocks Blocks may be scattered around the disk drive Block contains both pointer to next block and data Files may be as long as needed New blocks are allocated as needed Linked into list of blocks in file Removed from list (bitmap) of free blocks 0 1 2 3 4 5 6 7 8 9 10 11 Start=9 End=4 Length=2902 Start=3 End=6 Length=1500 0 x 4 6 x Chapter 6 26CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Finding blocks with linked allocation Directory structure is simple Starting address looked up from directory Directory only keeps track of first block (not others) No wasted space - all blocks can be used Random access is difficult: must always start at first block! Logical to physical mapping is done by block = start; offset_in_block = pos % 1020; for (j = 0; j < pos / 1020; j++) { block = block->next; } Assumes that next pointer is stored at end of block May require a long time for seek to random location in file Chapter 6 27CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) A B 40 1 2 -23 -24 5 36 -17 -18 09 -110 -111 -112 -113 -114 -115 Linked allocation using a RAM-based table Links on disk are slow Keep linked list in memory Advantage: faster Disadvantages Have to copy it to disk at some point Have to keep in-memory and on-disk copy consistent -1 -1 -1 Chapter 6 28CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Using a block index for allocation Store file block addresses in an array Array itself is stored in a disk block Directory has a pointer to this disk block Non-existent blocks indicated by -1 Random access easy Limit on file size? 0 1 2 3 4 5 6 7 8 9 10 11 grades 4 4802 Name index size 6 9 7 0 8 Chapter 6 29CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Finding blocks with indexed allocation Need location of index table: look up in directory Random & sequential access both well-supported: look up block number in index table Space utilization is good No wasted disk blocks (allocate individually) Files can grow and shrink easily Overhead of a single disk block per file Logical to physical mapping is done by block = index[block % 1024]; offset_in_block = pos % 1024; Limited file size: 256 pointers per index block, 1 KB per file block -> 256 KB per file limit Chapter 6 30CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Larger files with indexed allocation How can indexed allocation allow files larger than a single index block? Linked index blocks: similar to linked file blocks, but using index blocks instead Logical to physical mapping is done by index= start; blocknum = pos/1024; for(j= 0;j< blocknum /255);j++){ index= index->next; } block= index[blocknum % 255]; offset_in_block= pos% 1024; File size is now unlimited Random access slow, but only for very large files 6 Chapter 6 31CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Two-level indexed allocation Allow larger files by creating an index of index blocks File size still limited, but much larger Limit for 1 KB blocks = 1 KB * 256 * 256 = 226 bytes = 64 MB Logical to physical mapping is done by blocknum = pos / 1024; index = start[blocknum / 256)]; block = index[blocknum % 256] offset_in_block = pos % 1024; Start is the only pointer kept in the directory Overhead is now at least two blocks per file This can be extended to more than two levels if larger files are needed... Chapter 6 32CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Block allocation with extents Reduce space consumed by index pointers Often, consecutive blocks in file are sequential on disk Store <block,count> instead of just <block> in index At each level, keep total count for the index for efficiency Lookup procedure is: Find correct index block by checking the starting file offset for each index block Find correct <block,count> entry by running through index block, keeping track of how far into file the entry is Find correct block in <block,count> pair More efficient if file blocks tend to be consecutive on disk Allocating blocks like this allows faster reads & writes Lookup is somewhat more complex Chapter 6 33CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Managing free space: bit vector Keep a bit vector, with one entry per file block Number bits from 0 through n-1, where n is the number of file blocks on the disk If bit[j] == 0, block j is free If bit[j] == 1, block j is in use by a file (for data or index) If words are 32 bits long, calculate appropriate bit by: wordnum = block/32; bitnum = block% 32; Search for free blocks by looking for words with bits unset (words != 0xffffffff) Easy to find consecutive blocks for a single file Bit map must be stored on disk, and consumes space Assume 4 KB blocks, 8 GB disk => 2M blocks 2M bits = 221 bits = 218 bytes = 256KB overhead Chapter 6 34CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Managing free space: linked list Use a linked list to manage free blocks Similar to linked list for file allocation No wasted space for bitmap No need for random access unless we want to find consecutive blocks for a single file Difficult to know how many blocks are free unless it’s tracked elsewhere in the file system Difficult to group nearby blocks together if they’re freed at different times Less efficient allocation of blocks to files Files read & written more because consecutive blocks not nearby Chapter 6 35CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Issues with free space management OS must protect data structures used for free space management OS must keep in-memory and on-disk structures consistent Update free list when block is removed: change a pointer in the previous block in the free list Update bit map when block is allocated Caution: on-disk map must never indicate that a block is free when it’s part of a file Solution: set bit[j] in free map to 1 on disk before using block[j] in a file and setting bit[j] to 1 in memory New problem: OS crash may leave bit[j] == 1 when block isn’t actually used in a file New solution: OS checks the file system when it boots up… Managing free space is a big source of slowdown in file systems Chapter 6 36CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) What’s in a directory? Two types of information File names File metadata (size, timestamps, etc.) Basic choices for directory information Store all information in directory Fixed size entries Disk addresses and attributes in directory entry Store names & pointers to index nodes (i-nodes) games attributes mail attributes news attributes research attributes games mail news research attributes attributes attributes attributesStoring all information in the directory Using pointers to index nodes 7 Chapter 6 37CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Directory structure Structure Linear list of files (often itself stored in a file) Simple to program Slow to run Increase speed by keeping it sorted (insertions are slower!) Hash table: name hashed and looked up in file Decreases search time: no linear searches! May be difficult to expand Can result in collisions (two files hash to same location) Tree Fast for searching Easy to expand Difficult to do in on-disk directory Name length Fixed: easy to program Variable: more flexible, better for users Chapter 6 38CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Handling long file names in a directory Chapter 6 39CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Sharing files Root directory A foo ? ??? B foo A B C C bar C foo C blah A Papers A Photos A Family A sunset A sunset A os.tex A kids B Photos B lake Chapter 6 40CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Solution: use links A creates a file, and inserts into her directory B shares the file by creating a link to it A unlinks the file B still links to the file Owner is still A (unless B explicitly changes it) a.tex Owner: A Count: 1 a.tex Owner: A Count: 2 b.tex Owner: A Count: 1 b.tex A A B B Chapter 6 41CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Managing disk space Dark line (left hand scale) gives data rate of a disk Dotted line (right hand scale) gives disk space efficiency All files 2KB Block size Chapter 6 42CS 1550, cs.pitt.edu (originaly modified by Ethan L. Miller and Scott A. Brandt) Disk quotas