diff --git a/labs/cow.html b/labs/cow.html deleted file mode 100644 index 2cc18fa..0000000 --- a/labs/cow.html +++ /dev/null @@ -1,109 +0,0 @@ - - -Lab: Copy-on-Write Fork for xv6 - - - - -

Lab: Copy-on-Write Fork for xv6

- -

-Your task is implement copy-on-write fork in the xv6 kernel. You are -done if your modified kernel executes both the cow and usertests -programs successfully. - -

The problem

- -The fork() system call in xv6 copies all of the parent process's -user-space memory into the child. If the parent is large, copying can -take a long time. In addition, the copies often waste memory; in many -cases neither the parent nor the child modifies a page, so that in -principle they could share the same physical memory. The inefficiency -is particularly clear if the child calls exec(), since then most of -the copied pages are thrown away without ever being used. Of course, -sometimes both child and parent modify memory at the same virtual -address after a fork(), so for some pages the copying is truly needed. - -

The solution

- -The goal of copy-on-write (COW) fork() is to defer allocating and -copying physical memory pages for the child until they are actually -needed, in the hope that they may never be needed. - -

-COW fork() creates just a pagetable for the child, with PTEs for user -memory pointing to the parent's physical pages. COW fork() marks all -the user PTEs in both parent and child as read-only. When either -process tries to write one of these COW pages, the CPU will force a -page fault. The kernel page-fault handler detects this case, allocates -a page of physical memory for the faulting process, copies the -original page into the new page, and modifies the relevant PTE in the -faulting process to refer to the new page, this time with the PTE -marked writeable. When the page fault handler returns, the user -process will be able to write its copy of the page. - -

-COW fork() makes freeing of the physical pages that implement user -memory a little trickier. A given physical page may be referred to by -multiple processes' page tables, and should be freed when the last -reference disappears. - -

The cow test program

- -To help you test your implementation, we've provided an xv6 program -called cow (source in user/cow.c). cow runs various tests, but -even the first will fail on unmodified xv6. Thus, initially, you -will see: - -
-$ cow
-simple: fork() failed
-$ 
-
- -The "simple" test allocates more than half of available physical -memory, and then fork()s. The fork fails because there is not enough -free physical memory to give the child a complete copy of the parent. - -

-When you are done, your kernel should be able to run both cow and -usertests. That is: - -

-$ cow
-simple: ok
-simple: ok
-three: zombie!
-ok
-three: zombie!
-ok
-three: zombie!
-ok
-file: ok
-ALL COW TESTS PASSED
-$ usertests
-...
-ALL TESTS PASSED
-$
-
- -

Hints

- -Here's one reasonable plan of attack. Modify uvmcopy() to map the -parent's physical pages into the child, instead of allocating new -pages, and clear PTE_W in the PTEs of both child and parent. -Modify usertrap() to recognize a page fault. When a page fault occurs -on a COW page, allocate a new page with kalloc(), copy the old page to -the new page, and install the new page in the PTE with PTE_W set. -Next, ensure that each physical page is freed when the last PTE -reference to it goes away (but not before!), perhaps by implementing -reference counts in kalloc.c. Finally, modify copyout() to use the -same scheme as page faults when it encounters a COW page. - -

-It may be useful to have a way to record, for each PTE, whether it is -a COW mapping. You can use the RSW (reserved for software) bits in -the RISC-V PTE for this. - - - diff --git a/labs/fs.html b/labs/fs.html deleted file mode 100644 index a21e61f..0000000 --- a/labs/fs.html +++ /dev/null @@ -1,360 +0,0 @@ - - -Lab: file system - - - - -

Lab: file system

- -

In this lab you will add large files and mmap to the xv6 file system. - -

Large files

- -

In this assignment you'll increase the maximum size of an xv6 -file. Currently xv6 files are limited to 268 blocks, or 268*BSIZE -bytes (BSIZE is 1024 in xv6). This limit comes from the fact that an -xv6 inode contains 12 "direct" block numbers and one "singly-indirect" -block number, which refers to a block that holds up to 256 more block -numbers, for a total of 12+256=268. You'll change the xv6 file system -code to support a "doubly-indirect" block in each inode, containing -256 addresses of singly-indirect blocks, each of which can contain up -to 256 addresses of data blocks. The result will be that a file will -be able to consist of up to 256*256+256+11 blocks (11 instead of 12, -because we will sacrifice one of the direct block numbers for the -double-indirect block). - -

Preliminaries

- -

Modify your Makefile's CPUS definition so that it reads: -

-CPUS := 1
-
- -XXX doesn't seem to speedup things -

Add -

-QEMUEXTRA = -snapshot
-
-right before -QEMUOPTS -

-The above two steps speed up qemu tremendously when xv6 -creates large files. - -

mkfs initializes the file system to have fewer -than 1000 free data blocks, too few to show off the changes -you'll make. Modify param.h to -set FSSIZE to: -

-    #define FSSIZE       20000  // size of file system in blocks
-
- -

Download big.c into your xv6 directory, -add it to the UPROGS list, start up xv6, and run big. -It creates as big a file as xv6 will let -it, and reports the resulting size. It should say 140 sectors. - -

What to Look At

- -The format of an on-disk inode is defined by struct dinode -in fs.h. You're particularly interested in NDIRECT, -NINDIRECT, MAXFILE, and the addrs[] element -of struct dinode. Look Figure 7.3 in the xv6 text for a -diagram of the standard xv6 inode. - -

-The code that finds a file's data on disk is in bmap() -in fs.c. Have a look at it and make sure you understand -what it's doing. bmap() is called both when reading and -writing a file. When writing, bmap() allocates new -blocks as needed to hold file content, as well as allocating -an indirect block if needed to hold block addresses. - -

-bmap() deals with two kinds of block numbers. The bn -argument is a "logical block" -- a block number relative to the start -of the file. The block numbers in ip->addrs[], and the -argument to bread(), are disk block numbers. -You can view bmap() as mapping a file's logical -block numbers into disk block numbers. - -

Your Job

- -Modify bmap() so that it implements a doubly-indirect -block, in addition to direct blocks and a singly-indirect block. -You'll have to have only 11 direct blocks, rather than 12, -to make room for your new doubly-indirect block; you're -not allowed to change the size of an on-disk inode. -The first 11 elements of ip->addrs[] should be -direct blocks; the 12th should be a singly-indirect block -(just like the current one); the 13th should be your new -doubly-indirect block. - -

-You don't have to modify xv6 to handle deletion of files with -doubly-indirect blocks. - -

-If all goes well, big will now report that it -can write sectors. It will take big minutes -to finish. - -XXX this runs for a while! - -

Hints

- -

-Make sure you understand bmap(). Write out a diagram of the -relationships between ip->addrs[], the indirect block, the -doubly-indirect block and the singly-indirect blocks it points to, and -data blocks. Make sure you understand why adding a doubly-indirect -block increases the maximum file size by 256*256 blocks (really -1), -since you have to decrease the number of direct blocks by one). - -

-Think about how you'll index the doubly-indirect block, and -the indirect blocks it points to, with the logical block -number. - -

If you change the definition of NDIRECT, you'll -probably have to change the size of addrs[] -in struct inode in file.h. Make sure that -struct inode and struct dinode have the -same number of elements in their addrs[] arrays. - -

If you change the definition of NDIRECT, make sure to create a -new fs.img, since mkfs uses NDIRECT too to build the -initial file systems. If you delete fs.img, make on Unix (not -xv6) will build a new one for you. - -

If your file system gets into a bad state, perhaps by crashing, -delete fs.img (do this from Unix, not xv6). make will build a -new clean file system image for you. - -

Don't forget to brelse() each block that you -bread(). - -

You should allocate indirect blocks and doubly-indirect - blocks only as needed, like the original bmap(). - -

Optional challenge: support triple-indirect blocks. - -

Writing with a Log

- -Insert a print statement in bwrite (in bio.c) so that you get a -print every time a block is written to disk: - -
-  printf("bwrite block %d\n", b->blockno);
-
- -Build and boot a new kernel and run this: -
-  $ rm README
-
- -

You should see a sequence of bwrite prints after the rm.

- -
-
    -
  1. Annotate the bwrite lines with the kind of information that is -being written to the disk (e.g., "README's inode", "allocation -bitmap"). If the log is being written, note both that the log is being -written and also what kind of information is being written to the log. -
  2. Mark with an arrow the first point at which, if a -crash occured, README would be missing after a reboot -(after the call to recover_from_log()). -
-

-
- - -

Crash safety

- -

This assignment explores the xv6 log in two parts. -First, you'll artificially create a crash which illustrates -why logging is needed. Second, you'll remove one -inefficiency in the xv6 logging system. - -

-Submit your solution before the beginning of the next lecture -to the submission -web site. - -

Creating a Problem

- -

-The point of the xv6 log is to cause all the disk updates of a -filesystem operation to be atomic with respect to crashes. -For example, file creation involves both adding a new entry -to a directory and marking the new file's inode as in-use. -A crash that happened after one but before the other would -leave the file system in an incorrect state after a reboot, -if there were no log. - -

-The following steps will break the logging code in a way that -leaves a file partially created. - -

-First, replace commit() in log.c with -this code: -

-#include "kernel/proc.h"
-void
-commit(void)
-{
-  int pid = myproc()->pid;
-  if (log.lh.n > 0) {
-    write_log();
-    write_head();
-    if(pid > 1)            // AAA
-      log.lh.block[0] = 0; // BBB
-    install_trans();
-    if(pid > 1)            // AAA
-      panic("commit mimicking crash"); // CCC
-    log.lh.n = 0; 
-    write_head();
-  }
-}
-
- -

-The BBB line causes the first block in the log to be written to -block zero, rather than wherever it should be written. During file -creation, the first block in the log is the new file's inode updated -to have non-zero type. -Line BBB causes the block -with the updated inode to be written to block 0 (whence -it will never be read), leaving the on-disk inode still marked -unallocated. The CCC line forces a crash. -The AAA lines suppress this buggy behavior for init, -which creates files before the shell starts. - -

-Second, replace recover_from_log() in log.c -with this code: -

-static void
-recover_from_log(void)
-{
-  read_head();      
-  printf("recovery: n=%d but ignoring\n", log.lh.n);
-  // install_trans();
-  log.lh.n = 0;
-  // write_head();
-}
-
- -

-This modification suppresses log recovery (which would repair -the damage caused by your change to commit()). - -

-Finally, remove the -snapshot option from the definition -of QEMUEXTRA in your Makefile so that the disk image will see the -changes. - -

-Now remove fs.img and run xv6: -

-  % rm fs.img ; make qemu
-
-

-Tell the xv6 shell to create a file: -

-  $ echo hi > a
-
- -

-You should see the panic from commit(). So far -it is as if a crash occurred in a non-logging system in the middle -of creating a file. - -

-Now re-start xv6, keeping the same fs.img: -

-  % make qemu
-
- -

-And look at file a: -

-  $ cat a
-
- -

- You should see panic: ilock: no type. Make sure you understand what happened. -Which of the file creation's modifications were written to the disk -before the crash, and which were not? - -

Solving the Problem

- -Now fix recover_from_log(): -
-static void
-recover_from_log(void)
-{
-  read_head();
-  cprintf("recovery: n=%d\n", log.lh.n);
-  install_trans();
-  log.lh.n = 0;
-  write_head();
-}
-
- -

-Run xv6 (keeping the same fs.img) and read a again: -

-  $ cat a
-
- -

-This time there should be no crash. Make sure you understand why -the file system now works. - -

-Why was the file empty, even though you created -it with echo hi > a? - -

-Now remove your modifications to commit() -(the if's and the AAA and BBB lines), so that logging works again, -and remove fs.img. - -

Streamlining Commit

- -

-Suppose the file system code wants to update an inode in block 33. -The file system code will call bp=bread(block 33) and update the -buffer data. write_log() in commit() -will copy the data to a block in the log on disk, for example block 3. -A bit later in commit, install_trans() reads -block 3 from the log (containing block 33), copies its contents into the in-memory -buffer for block 33, and then writes that buffer to block 33 on the disk. - -

-However, in install_trans(), it turns out that the modified -block 33 is guaranteed to be still in the buffer cache, where the -file system code left it. Make sure you understand why it would be a -mistake for the buffer cache to evict block 33 from the buffer cache -before the commit. - -

-Since the modified block 33 is guaranteed to already be in the buffer -cache, there's no need for install_trans() to read block -33 from the log. Your job: modify log.c so that, when -install_trans() is called from commit(), -install_trans() does not perform the needless read from the log. - -

To test your changes, create a file in xv6, restart, and make sure -the file is still there. - -XXX Does this speedup bigfile? - -XXX Maybe support lseek and modify shell to append to a file? - - - - diff --git a/labs/fs1.html b/labs/fs1.html deleted file mode 100644 index 45d3e0c..0000000 --- a/labs/fs1.html +++ /dev/null @@ -1,215 +0,0 @@ - - -Lab: mount/umount - - - - -

Lab: mount/umount

- -

In this lab you will add support for mounting/unmounting of file -systems to xv6. This lab will expose you to many parts of the xv6 -file system, including pathname lookup, inodes, logging/recovery, disk -driver, concurrency, etc. - -

Your job is modify xv6 so that your modified kernel passes the - tests in mounttest. You will have to implement two system - calls: mount(char *source, char *target) - and umount(char *target). Mount attaches the device - referenced by source (e.g., /disk1) at the - location specified by target. For - example, mount("/disk1", "/m") will attach disk1 - at the directory /m. After this mount call, users can use - pathnames such as /m/README to read the - file README stored in the root directory - on disk1. Umount removes the attachment. For - example, umount("/m") unmounts disk1 from /m. - -

There are several major challenges in implementing the mount system -calls: - -

- -

The rest of this assignment provides some hints how you might go -about the above challenges. - -

Adding system calls

- -

Add the stubs for the two systems calls to xv6 so that you can -compile mounttest and add two empty functions for the two system calls -to sysfile.c. Run mounttest and it will fail on the first call -to mount. - - -

Adding a second disk

- -

To be able to mount another disk, you need to extend xv6 to support -at least two disks. Modify virtio_disk.c to support an array of two -disks instead of a single disk. The address of the second disk -is 0x10002000; modify the macro R to take a disk -number (0, 1,..) and read/write to the memory address for that disk. - -

All functions in virtio_disk.c need to take the disk -number as an argument to update the state of the disk that is -read/written to or to receive an interrupt from the disk. -Modify virtio_disk_init to take a disk number as an argument -and update is to that it initializes that disk. Similar, go through -the other functions; make these changes should be most mechanical -(i.e., text substitutions). - -

The second disk interrupts at IRQ 2; modify trap.c to receive that -interrupt and virtio_disk_intr with the number of the disk -that generated the interrupt. - -

Modify the file Makefile to tell qemu to provide a second -disk. Define the variable QEMUEXTRA = -drive -file=fs1.img,if=none,format=raw,id=x1 -device -virtio-blk-device,drive=x1,bus=virtio-mmio-bus.1 and -add $(QEMUEXTRA) to the end of QEMUOPTS. - -

Create a second disk image fs1.img. Easiest thing to do - is just copy the file fs.img. You might want to add rules - to the Makefile to make this image and remove it on make - clean. - -

Add to the user program init a call to create a device for the new - disk. For example, add the line mknod("disk1", DISK, 1); to - init.c. This will create an inode of type device in the root - directory with major number DISK and minor number 1. - -

The first argument of the mount system call ("disk1") will - refer to the device you created using mknod above. In your - implementation of the mount system call, - call virtio_disk_init with the minor number as the argument - to initialize the second disk. (We reserve minor number 0 for the - first disk.) - -

Boot xv6, run mounttest, and make sure virtio_disk_init - gets called (e.g., add print statement). You won't know if your - changes are correct, but your code should compile and invoke the - driver for the second disk. - -

Modify the logging system

- -

After calling virtio_disk_init, you need to also - call loginit to initialize the logging system for the - second disk (and restore the second disk if a power failure happened - while modifying the second disk). Generalize the logging system to - support to two logs, one on disk 0 and one disk 1. These changes - are mostly mechanical (e.g., log. changes - to log[n].), similar to generalizing the disk driver to - support two disks. - -

To make xv6 compile, you need to provide a disk number - to begin_op and end_op. It will be a challenge to - figure out what the right value is; for now just specify the first - disk (i.e., 0). This isn't correct, since modifications to the - second disk should be logged on the second disk, but we have no way - yet to read/write the second disk. Come back to this later when you - have a better idea how things will fit together, but make sure that - xv6 compiles and still runs. - -

Pathname lookup

- -

Modify namex to traverse mount points: when namex - sees an inode to which a file system is attached, it should traverse - to the root inode of that file system. Hint: modify the in-memory - inode in file.h to keep some additional state, and initialize that - state in the mount system call. Note that the inode already has a - field for disk number (i.e., dev), which is initialized and - passed to reads and writes to the driver. dev corresponds - to the minor number for disk devices. - -

Your modified xv6 should be able to pass the first tests in - mounttest (i.e., stat). This is likely to be challenging, - however, because now your kernel will be reading from the second - disk for the first time, and you may run into many issues. - -

Even though stat may return correctly, your code is likely - to be incorrect, because in namex - because iunlockput may modify the second disk (e.g., if - another process removes the file or directory) and those - modifications must be written to the second disk. Your job is to - fix the calls to begin_op and end_op to take the - right device. One challenge is that begin_op is called at - the beginning of a system call but then you don't know the device - that will be involved; you will have to postpone this call until you - know which inode is involved (which tells you will which device is - involved). Another challenge is that you cannot postpone - calling begin_op passed ilock because that - violates lock ordering in xv6; you should not be - calling begin_op while holding locks on inodes. (The log - system allows a few systems calls to run; if a system call that - holds an inode lock isn't admitted and one of the admitted system - calls needs that inode to complete, then xv6 will deadlock.) - -

Once you have implemented a plan for begin_op - and end_op, see if your kernel can pass test0. It - is likely that you will have to modify your implementation of the - mount system call to handle several corner cases. See the tests - in test0. - -

Run usertests to see if you didn't break anything else. Since you - modified namex and begin/end_op, which are at the - core of the xv6 file system, you might have introduced bugs, perhaps - including deadlocks. Deadlocks manifest themselves as no output - being produced because all processes are sleeping (hit ctrl-p a few - times). Your kernel might also suffer kernel panics, because your - changes violate invariants. You may have to iterate a few times to - get a good design and implementation. - -

umount

- -

Once your kernel passes usertests and test0 of mounttest, implement - umount. The main challenge is that umount of a file system should - fail if the file system is still in use; that is, if there is an - inode on the mounted device that has a ref > 0. - Furthermore, this test and unmounting should be an atomic - operation. (Hint: lock the inode cache.) Make sure your kernel - passes test1 of mounttest. - -

Test2 of mounttest stresses namex more; if you have done - everything right above, your kernel should pass it. Test3 tests - concurrent mount/unmounts with file creation. - -

crash safety

- -

One of the main goals of the file system is to provide crash - safety: if there is a power failure during a file system operation, - xv6 should recover correctly. It is difficult to introduce power - failure at the critical steps of logging; instead, we added a system - call that causes a kernel panic after committing an operation but - before installing the operation. Test4 with crashtest tests if your - xv6 recovers the mounted disk correctly. - - - - - -

Optional challenges

- -

Modify xv6 so that init mounts the first disk on the root inode. - This will allow you to remove some code specific for the first disk - from the kernel. - -

Support mounts on top of mounts. diff --git a/labs/lazy.html b/labs/lazy.html deleted file mode 100644 index 9d97cab..0000000 --- a/labs/lazy.html +++ /dev/null @@ -1,132 +0,0 @@ - - -Lab: xv6 lazy page allocation - - - - -

Lab: xv6 lazy page allocation

- -

-One of the many neat tricks an O/S can play with page table hardware -is lazy allocation of heap memory. Xv6 applications ask the kernel for -heap memory using the sbrk() system call. In the kernel we've given -you, sbrk() allocates physical memory and maps it into the process's -virtual address space. There are programs that allocate memory but -never use it, for example to implement large sparse arrays. -Sophisticated kernels delay allocation of each page of memory until -the application tries to use that page -- as signaled by a page fault. -You'll add this lazy allocation feature to xv6 in this lab. - -

Part One: Eliminate allocation from sbrk()

- -Your first task is to delete page allocation from the sbrk(n) system -call implementation, which is the function sys_sbrk() in sysproc.c. The -sbrk(n) system call grows the process's memory size by n bytes, and -then returns the start of the newly allocated region (i.e., the old -size). Your new sbrk(n) should just increment the process's size -(myproc()->sz) by n and return the old size. It should not allocate memory --- so you should delete the call to growproc() (but you still need to -increase the process's size!). - -

-Try to guess what the result of this modification will be: what will -break? - -

-Make this modification, boot xv6, and type echo hi to the shell. -You should see something like this: - -

-init: starting sh
-$ echo hi
-usertrap(): unexpected scause 0x000000000000000f pid=3
-            sepc=0x00000000000011dc stval=0x0000000000004008
-va=0x0000000000004000 pte=0x0000000000000000
-panic: unmappages: not mapped
-
- -The "usertrap(): ..." message is from the user trap handler in trap.c; -it has caught an exception that it does not know how to handle. Make -sure you understand why this page fault occurs. The "stval=0x0..04008" -indicates that the virtual address that caused the page fault is -0x4008. - -

Part Two: Lazy allocation

- -Modify the code in trap.c to respond to a page fault from user space -by mapping a newly-allocated page of physical memory at the faulting -address, and then returning back to user space to let the process -continue executing. You should add your code just before -the printf call that produced the "usertrap(): ..." -message. - -

-Hint: look at the printf arguments to see how to find the virtual -address that caused the page fault. - -

-Hint: steal code from allocuvm() in vm.c, which is what sbrk() -calls (via growproc()). - -

-Hint: use PGROUNDDOWN(va) to round the faulting virtual address -down to a page boundary. - -

-Hint: usertrapret() in order to avoid -the printf and the myproc()->killed = 1. - -

-Hint: you'll need to call mappages(). - -

Hint: you can check whether a fault is a page fault by r_scause() - is 13 or 15 in trap(). - -

Hint: modify unmappages() to not free pages that aren't mapped. - -

Hint: if the kernel crashes, look up sepc in kernel/kernel.asm - -

Hint: if you see the error "imcomplete type proc", include "proc.h" - (and "spinlock.h"). - -

Hint: the first test in sbrk() allocates something large, this - should succeed now. - -

-If all goes well, your lazy allocation code should result in echo -hi working. You should get at least one page fault (and thus lazy -allocation) in the shell, and perhaps two. - -

If you have the basics working, now turn your implementation into - one that handles the corner cases too: - -

- -

Run all tests in usertests() to make sure your solution doesn't -break other tests. - -

-

-

Submit: The code that you added to trap.c in a file named hwN.c where N is the homework number as listed on the schedule. -

- - - - diff --git a/labs/lock.html b/labs/lock.html deleted file mode 100644 index 707d6c4..0000000 --- a/labs/lock.html +++ /dev/null @@ -1,148 +0,0 @@ - - -Lab: locks - - - - -

Lab: locks

- -

In this lab you will try to avoid lock contention for certain -workloads. - -

lock contention

- -

The program user/kalloctest stresses xv6's memory allocator: three - processes grow and shrink there address space, which will results in - many calls to kalloc and kfree, - respectively. kalloc and kfree - obtain kmem.lock. To see if there is lock contention for - kmem.lock replace the call to acquire - in kalloc with the following code: - -

-    while(!tryacquire(&kmem.lock)) {
-      printf("!");
-    }
-  
- -

tryacquire tries to acquire kmem.lock: if the - lock is taking it returns false (0); otherwise, it returns true (1) - and with the lock acquired. Your first job is to - implement tryacquire in kernel/spinlock.c. - -

A few hints: -

- -

Run usertests to see if you didn't break anything. Note that - usertests never prints "!"; there is never contention - for kmem.lock. The caller is always able to immediately - acquire the lock and never has to wait because some other process - has the lock. - -

Now run kalloctest. You should see quite a number of "!" on the - console. kalloctest causes many processes to contend on - the kmem.lock. This lock contention is a bit artificial, - because qemu is simulating 3 processors, but it is likely on real - hardware, there would be contention too. - -

Removing lock contention

- -

The root cause of lock contention in kalloctest is that there is a - single free list, protected by a single lock. To remove lock - contention, you will have to redesign the memory allocator to avoid - a single lock and list. The basic idea is to maintain a free list - per CPU, each list with its own lock. Allocations and frees on each - CPU can run in parallel, because each CPU will operate on a - different list. - -

The main challenge will be to deal with the case that one CPU runs - out of memory, but another CPU has still free memory; in that case, - the one CPU must "steal" part of the other CPU's free list. - Stealing may introduce lock contention, but that may be acceptable - because it may happen infrequently. - -

Your job is to implement per-CPU freelists and stealing when one - CPU is out of memory. Run kalloctest() to see if your - implementation has removed lock contention. - -

Some hints: -

- -

Run usertests to see if you don't break anything. - -

More scalabale bcache lookup

- - -

Several processes reading different files repeatedly will - bottleneck in the buffer cache, bcache, in bio.c. Replace the - acquire in bget with - -

-    while(!tryacquire(&bcache.lock)) {
-      printf("!");
-    }
-  
- - and run test0 from bcachetest and you will see "!"s. - -

Modify bget so that a lookup for a buffer that is in the - bcache doesn't need to acquire bcache.lock. This is more - tricky than the kalloc assignment, because bcache buffers are truly - shared among processes. You must maintain the invariant that a - buffer is only once in memory. - -

There are several races that bcache.lock protects -against, including: -

- -

A challenge is testing whether you code is still correct. One way - to do is to artificially delay certain operations - using sleepticks. test1 trashes the buffer cache - and exercises more code paths. - -

Here are some hints: -

- -

Check that your implementation has less contention - on test0 - -

Make sure your implementation passes bcachetest and usertests. - -

Optional: -

- - - - diff --git a/labs/mmap.html b/labs/mmap.html deleted file mode 100644 index 6f779c4..0000000 --- a/labs/mmap.html +++ /dev/null @@ -1,171 +0,0 @@ - - -Lab: mmap - - - - -

Lab: mmap

- -

In this lab you will use mmap on Linux to demand-page a -very large table and add memory-mapped files to xv6. - -

Using mmap on Linux

- -

This assignment will make you more familiar with how to manage virtual memory -in user programs using the Unix system call interface. You can do this -assignment on any operating system that supports the Unix API (a Linux Athena -machine, your laptop with Linux or MacOS, etc.). - -

Download the mmap homework assignment and look -it over. The program maintains a very large table of square root -values in virtual memory. However, the table is too large to fit in -physical RAM. Instead, the square root values should be computed on -demand in response to page faults that occur in the table's address -range. Your job is to implement the demand faulting mechanism using a -signal handler and UNIX memory mapping system calls. To stay within -the physical RAM limit, we suggest using the simple strategy of -unmapping the last page whenever a new page is faulted in. - -

To compile mmap.c, you need a C compiler, such as gcc. On Athena, -you can type: -

-$ add gnu
-
-Once you have gcc, you can compile mmap.c as follows: -
-$ gcc mmap.c -lm -o mmap
-
-Which produces a mmap file, which you can run: -
-$ ./mmap
-page_size is 4096
-Validating square root table contents...
-oops got SIGSEGV at 0x7f6bf7fd7f18
-
- -

When the process accesses the square root table, the mapping does not exist -and the kernel passes control to the signal handler code in -handle_sigsegv(). Modify the code in handle_sigsegv() to map -in a page at the faulting address, unmap a previous page to stay within the -physical memory limit, and initialize the new page with the correct square root -values. Use the function calculate_sqrts() to compute the values. -The program includes test logic that verifies if the contents of the -square root table are correct. When you have completed your task -successfully, the process will print “All tests passed!”. - -

You may find that the man pages for mmap() and munmap() are helpful references. -

-$ man mmap
-$ man munmap
-
- - -

Implement memory-mapped files in xv6

- -

In this assignment you will implement memory-mapped files in xv6. - The test program mmaptest tells you what should work. - -

Here are some hints about how you might go about this assignment: - -

- -

Run usertests to make sure you didn't break anything. - -

Optional challenges: -

- - - diff --git a/labs/syscall.html b/labs/syscall.html deleted file mode 100644 index 2281f2e..0000000 --- a/labs/syscall.html +++ /dev/null @@ -1,443 +0,0 @@ - - -Lab: Alarm and uthread - - - - -

Lab: Alarm and uthread

- -This lab will familiarize you with the implementation of system calls -and switching between threads of execution. In particular, you will -implement new system calls (sigalarm and sigreturn) -and switching between threads in a user-level thread package. - -

Warmup: RISC-V assembly

- -

For this lab it will be important to understand a bit of RISC-V assembly. - -

Add a file user/call.c with the following content, modify the - Makefile to add the program to the user programs, and compile (make - fs.img). The Makefile also produces a binary and a readable - assembly a version of the program in the file user/call.asm. -

-#include "kernel/param.h"
-#include "kernel/types.h"
-#include "kernel/stat.h"
-#include "user/user.h"
-
-int g(int x) {
-  return x+3;
-}
-
-int f(int x) {
-  return g(x);
-}
-
-void main(void) {
-  printf(1, "%d %d\n", f(8)+1, 13);
-  exit();
-}
-
- -

Read through user/call.asm and understand it. The instruction manual - for RISC-V is in the doc directory (doc/riscv-spec-v2.2.pdf). Here - are some questions that you should answer for yourself: - -

- -

Warmup: system call tracing

- -

In this exercise you will modify the xv6 kernel to print out a line -for each system call invocation. It is enough to print the name of the -system call and the return value; you don't need to print the system -call arguments. - -

-When you're done, you should see output like this when booting -xv6: - -

-...
-fork -> 2
-exec -> 0
-open -> 3
-close -> 0
-$write -> 1
- write -> 1
-
- -

-That's init forking and execing sh, sh making sure only two file descriptors are -open, and sh writing the $ prompt. (Note: the output of the shell and the -system call trace are intermixed, because the shell uses the write syscall to -print its output.) - -

Hint: modify the syscall() function in kernel/syscall.c. - -

Run the xv6 programs you wrote in earlier labs and inspect the system call - trace. Are there many system calls? Which system calls correspond - to code in the applications you wrote? - -

Optional: print the system call arguments. - - -

Alarm

- -

-In this exercise you'll add a feature to xv6 that periodically alerts -a process as it uses CPU time. This might be useful for compute-bound -processes that want to limit how much CPU time they chew up, or for -processes that want to compute but also want to take some periodic -action. More generally, you'll be implementing a primitive form of -user-level interrupt/fault handlers; you could use something similar -to handle page faults in the application, for example. - -

-You should add a new sigalarm(interval, handler) system call. -If an application calls sigalarm(n, fn), then after every -n "ticks" of CPU time that the program consumes, the kernel -should cause application function -fn to be called. When fn returns, the application -should resume where it left off. A tick is a fairly arbitrary unit of -time in xv6, determined by how often a hardware timer generates -interrupts. - -

-You'll find a file user/alarmtest.c in your xv6 -repository. Add it to the Makefile. It won't compile correctly -until you've added sigalarm and sigreturn -system calls (see below). - -

-alarmtest calls sigalarm(2, periodic) in test0 to -ask the kernel to force a call to periodic() every 2 ticks, -and then spins for a while. -You can see the assembly -code for alarmtest in user/alarmtest.asm, which may be handy -for debugging. -When you've finished the lab, -alarmtest should produce output like this: - -

-$ alarmtest
-test0 start
-......................................alarm!
-test0 passed
-test1 start
-..alarm!
-..alarm!
-..alarm!
-.alarm!
-..alarm!
-..alarm!
-..alarm!
-..alarm!
-..alarm!
-..alarm!
-test1 passed
-$
-
- -

The main challenge will be to arrange that the handler is invoked - when the process's alarm interval expires. You'll need to modify - usertrap() in kernel/trap.c so that when a - process's alarm interval expires, the process executes - the handler. How can you do that? You will need to understand - how system calls work (i.e., the code in kernel/trampoline.S - and kernel/trap.c). Which register contains the address to which - system calls return? - -

Your solution will be only a few lines of code, but it may be tricky to - get it right. -We'll test your code with the version of alarmtest.c in the original -repository; if you modify alarmtest.c, make sure your kernel changes -cause the original alarmtest to pass the tests. - -

test0: invoke handler

- -

Get started by modifying the kernel to jump to the alarm handler in -user space, which will cause test0 to print "alarm!". Don't worry yet -what happens after the "alarm!" output; it's OK for now if your -program crashes after printing "alarm!". Here are some hints: - -

- -

test1(): resume interrupted code

- -Chances are that alarmtest crashes at some point after it prints -"alarm!". Depending on how your solution works, that point may be in -test0, or it may be in test1. Crashes are likely caused -by the alarm handler (periodic in alarmtest.c) returning -to the wrong point in the user program. - -

-Your job now is to ensure that, when the alarm handler is done, -control returns to -the instruction at which the user program was originally -interrupted by the timer interrupt. You must also ensure that -the register contents are restored to values they held -at the time of the interrupt, so that the user program -can continue undisturbed after the alarm. - -

Your solution is likely to require you to save and restore - registers---what registers do you need to save and restore to resume - the interrupted code correctly? (Hint: it will be many). - Several approaches are possible; for this lab you should make - the sigreturn system call - restore registers and return to the original - interrupted user instruction. - The user-space alarm handler - calls sigreturn when it is done. - - Some hints: -

- -

Once you pass test0 and test1, run usertests to - make sure you didn't break any other parts of the kernel. - -

Uthread: switching between threads

- -

Download uthread.c and uthread_switch.S into your xv6 directory. -Make sure uthread_switch.S ends with .S, not -.s. Add the -following rule to the xv6 Makefile after the _forktest rule: - -

-$U/_uthread: $U/uthread.o $U/uthread_switch.o
-	$(LD) $(LDFLAGS) -N -e main -Ttext 0 -o $U/_uthread $U/uthread.o $U/uthread_switch.o $(ULIB)
-	$(OBJDUMP) -S $U/_uthread > $U/uthread.asm
-
-Make sure that the blank space at the start of each line is a tab, -not spaces. - -

-Add _uthread in the Makefile to the list of user programs defined by UPROGS. - -

Run xv6, then run uthread from the xv6 shell. The xv6 kernel will print an error message about uthread encountering a page fault. - -

Your job is to complete uthread_switch.S, so that you see output similar to -this (make sure to run with CPUS=1): -

-~/classes/6828/xv6$ make CPUS=1 qemu
-...
-$ uthread
-my thread running
-my thread 0x0000000000002A30
-my thread running
-my thread 0x0000000000004A40
-my thread 0x0000000000002A30
-my thread 0x0000000000004A40
-my thread 0x0000000000002A30
-my thread 0x0000000000004A40
-my thread 0x0000000000002A30
-my thread 0x0000000000004A40
-my thread 0x0000000000002A30
-...
-my thread 0x0000000000002A88
-my thread 0x0000000000004A98
-my thread: exit
-my thread: exit
-thread_schedule: no runnable threads
-$
-
- -

uthread creates two threads and switches back and forth between -them. Each thread prints "my thread ..." and then yields to give the other -thread a chance to run. - -

To observe the above output, you need to complete uthread_switch.S, but before -jumping into uthread_switch.S, first understand how uthread.c -uses uthread_switch. uthread.c has two global variables -current_thread and next_thread. Each is a pointer to a -thread structure. The thread structure has a stack for a thread and a -saved stack pointer (sp, which points into the thread's stack). The -job of uthread_switch is to save the current thread state into the -structure pointed to by current_thread, restore next_thread's -state, and make current_thread point to where next_thread was -pointing to, so that when uthread_switch returns next_thread -is running and is the current_thread. - -

You should study thread_create, which sets up the initial stack for -a new thread. It provides hints about what uthread_switch should do. -Note that thread_create simulates saving all callee-save registers -on a new thread's stack. - -

To write the assembly in thread_switch, you need to know how the C -compiler lays out struct thread in memory, which is as -follows: - -

-    --------------------
-    | 4 bytes for state|
-    --------------------
-    | stack size bytes |
-    | for stack        |
-    --------------------
-    | 8 bytes for sp   |
-    --------------------  <--- current_thread
-         ......
-
-         ......
-    --------------------
-    | 4 bytes for state|
-    --------------------
-    | stack size bytes |
-    | for stack        |
-    --------------------
-    | 8 bytes for sp   |
-    --------------------  <--- next_thread
-
- -The variables &next_thread and ¤t_thread each -contain the address of a pointer to struct thread, and are -passed to thread_switch. The following fragment of assembly -will be useful: - -
-   ld t0, 0(a0)
-   sd sp, 0(t0)
-
- -This saves sp in current_thread->sp. This works because -sp is at -offset 0 in the struct. -You can study the assembly the compiler generates for -uthread.c by looking at uthread.asm. - -

To test your code it might be helpful to single step through your -uthread_switch using riscv64-linux-gnu-gdb. You can get started in this way: - -

-(gdb) file user/_uthread
-Reading symbols from user/_uthread...
-(gdb) b *0x230
-
-
-0x230 is the address of uthread_switch (see uthread.asm). When you -compile it may be at a different address, so check uthread_asm. -You may also be able to type "b uthread_switch". XXX This doesn't work - for me; why? - -

The breakpoint may (or may not) be triggered before you even run -uthread. How could that happen? - -

Once your xv6 shell runs, type "uthread", and gdb will break at -thread_switch. Now you can type commands like the following to inspect -the state of uthread: - -

-  (gdb) p/x *next_thread
-  $1 = {sp = 0x4a28, stack = {0x0 (repeats 8088 times),
-      0x68, 0x1, 0x0 }, state = 0x1}
-
-What address is 0x168, which sits on the bottom of the stack -of next_thread? - -With "x", you can examine the content of a memory location -
-  (gdb) x/x next_thread->sp
-  0x4a28 :      0x00000168
-
-Why does that print 0x168? - -

Optional challenges

- -

The user-level thread package interacts badly with the operating system in -several ways. For example, if one user-level thread blocks in a system call, -another user-level thread won't run, because the user-level threads scheduler -doesn't know that one of its threads has been descheduled by the xv6 scheduler. As -another example, two user-level threads will not run concurrently on different -cores, because the xv6 scheduler isn't aware that there are multiple -threads that could run in parallel. Note that if two user-level threads were to -run truly in parallel, this implementation won't work because of several races -(e.g., two threads on different processors could call thread_schedule -concurrently, select the same runnable thread, and both run it on different -processors.) - -

There are several ways of addressing these problems. One is - using scheduler - activations and another is to use one kernel thread per - user-level thread (as Linux kernels do). Implement one of these ways - in xv6. This is not easy to get right; for example, you will need to - implement TLB shootdown when updating a page table for a - multithreaded user process. - -

Add locks, condition variables, barriers, -etc. to your thread package. - - - - diff --git a/labs/xv6.html b/labs/xv6.html deleted file mode 100644 index 13d581e..0000000 --- a/labs/xv6.html +++ /dev/null @@ -1,238 +0,0 @@ - - -Lab: xv6 - - - - -

Lab: xv6

- -This lab makes you familiar with xv6 and its system calls. - -

Boot xv6

- -

Login to Athena (e.g., ssh -X athena.dialup.mit.edu) and attach the course -locker: (You must run this command every time you log in; or add it to your -~/.environment file.) - -

-$ add -f 6.828
-
- -

Fetch the xv6 source: - -

-$ mkdir 6.828
-$ cd 6.828
-$ git clone git://github.com/mit-pdos/xv6-riscv.git
-Cloning into 'xv6-riscv'...
-...
-$
-
- -

XXX pointer to an update tools page - -

Build xv6 on Athena: -

-$ cd xv6-public
-$ makeriscv64-linux-gnu-gcc    -c -o kernel/entry.o kernel/entry.S
-riscv64-linux-gnu-gcc -Wall -Werror -O -fno-omit-frame-pointer -ggdb -MD -mcmodel=medany -ffreestanding -fno-common -nostdlib -mno-relax -I. -fno-stack-protector -fno-pie -no-pie   -c -o kernel/start.o kernel/start.c
-...
-$ make qemu
-...
-mkfs/mkfs fs.img README user/_cat user/_echo user/_forktest user/_grep user/_init user/_kill user/_ln user/_ls user/_mkdir user/_rm user/_sh user/_stressfs user/_usertests user/_wc user/_zombie user/_cow 
-nmeta 46 (boot, super, log blocks 30 inode blocks 13, bitmap blocks 1) blocks 954 total 1000
-balloc: first 497 blocks have been allocated
-balloc: write bitmap block at sector 45
-qemu-system-riscv64 -machine virt -kernel kernel/kernel -m 3G -smp 3 -nographic -drive file=fs.img,if=none,format=raw,id=x0 -device virtio-blk-device,drive=x0,bus=virtio-mmio-bus.0
-hart 0 starting
-hart 2 starting
-hart 1 starting
-init: starting sh
-$
-
- -

-If you type ls at the prompt, you should output similar to the following: -

-$ ls
-.              1 1 1024
-..             1 1 1024
-README         2 2 2181
-cat            2 3 21024
-echo           2 4 19776
-forktest       2 5 11456
-grep           2 6 24512
-init           2 7 20656
-kill           2 8 19856
-ln             2 9 19832
-ls             2 10 23280
-mkdir          2 11 19952
-rm             2 12 19936
-sh             2 13 38632
-stressfs       2 14 20912
-usertests      2 15 106264
-wc             2 16 22160
-zombie         2 17 19376
-cow            2 18 27152
-console        3 19 0
-
-These are the programs/files that mkfs includes in the -initial file system. You just ran one of them: ls. - -

sleep

- -

Implement the UNIX program sleep for xv6; your sleep should pause - for a user-specified number of ticks. - -

Some hints: -

- -

Run the program from the xv6 shell: -

-      $ make qemu
-      ...
-      init: starting sh
-      $ sleep 10
-      (waits for a little while)
-      $
-    
- -

Optional: write an uptime program that prints the uptime in terms - of ticks using the uptime system call. - -

pingpong

- -

Write a program that uses UNIX system calls to ``ping-pong'' a - byte between two processes over a pair of pipes, one for each - direction. The parent sends by writing a byte to fd[1] and - the child receives it by reading from fd[0]. After - receiving a byte from parent, the child responds with its own byte - by writing to fd[1], which the parent then reads. - -

Some hints: -

- -

primes

- -

Write a concurrent version of prime sieve using pipes. This idea - is due to Doug McIlroy, inventor of Unix pipes. The picture - halfway down the page - and the text surrounding it explain how to do it. - -

Your goal is to use pipe and fork to set up - the pipeline. The first process feeds the numbers 2 through 35 - into the pipeline. For each prime number, you will arrange to - create one process that reads from its left neighbor over a pipe - and writes to its right neighbor over another pipe. Since xv6 has - limited number of file descriptors and processes, the first - process can stop at 35. - -

Some hints: -

- -

find

- -

Write a simple version of the UNIX find program: find all the files - in a directory tree whose name matches a string. For example if the - file system contains a file a/b, then running find as - follows should produce: -

-    $ find . b
-    ./a/b
-    $
-  
- -

Some hints: -

- -

Optional: support regular expressions in name matching. Grep has some - primitive support for regular expressions. - -

xargs

- -

Write a simple version of the UNIX xargs program: read lines from - standard in and run a command for each line, supplying the line as - arguments to the command. The following example illustrates xarg's - behavior: -

-    $ xargs echo bye
-    hello too
-    bye hello too
-    
-    $
-  
- Note that the command here is "echo bye" and the additional - arguments are "hello too", making the command "echo bye hello too", - which outputs "bye hello too". - -

xargs and find combine well: -

-    find . b | xargs grep hello
-  
- will run "grep hello" on each file named b in the directories below ".". - -

Some hints: -

- -

Optional: modify the shell

- -There are endless ways in which the shell could be extended. Here are -some suggestions: - - - - - - -