When you create photos and documents on your phone or computer, you care about two things: You want the device to read and write your files quickly (high performance), and you want to be sure that your files are safe even if your device crashes or loses power (strong reliability). The problem is that techniques that keep files safe usually slow things down. For example, to make reading and writing files faster, file systems often store data in RAM (fast, but temporary memory), but if the computer crashes, everything in RAM is lost. So, the file system must use sophisticated techniques to let applications think that data is safely written to the disk when it actually isn’t yet. What is especially tricky is that saving a single photo involves writing to many different files and locations on the disk. If your computer crashes when only some of those writes have finished, you might end up with a confusing mix of old and new data when you reboot the computer.
People are also reading…
With my students and colleagues, I help design and implement new file system techniques that are both fast and reliable. One new idea we have recently developed is what we call a file system semi-microkernel. Normally, file systems that read and write your files are built into the main operating system (like in MacOS, Windows or Linux). This means if there’s a mistake in the file system, it can crash the entire computer, forcing you to restart everything and possibly lose important files you thought had been saved. With a file system semi-microkernel, we move the file system outside of the operating system and run it separately in user space along with your applications. This way, if the file system has a problem, only the file system crashes and gets restarted, while the rest of the operating system and your other applications keep working normally. Applications that aren’t using the file system at that moment won’t even notice that the file system crashed and restarted.
An interesting research challenge we have solved is how to handle applications that were in the middle of saving data when the file system crashes. We need to make sure these applications don’t lose any data and that everything stays fast, but the straight-forward approaches won’t work. If the file system tries to be fast and keep data only in memory, a crash would lose that data and break applications that are still running. Alternately, if the file system tries to be reliable and save every bit of data to the hard drive, then performance would be much slower — and no one wants to use a slow file system. Our goal is to find a balance where the system is both fast and reliable.
Our new solution is for the file system semi-microkernel to cache data in memory (like traditional file systems) but to also record all the operations it has started or completed whose results have not been written on the disk. Thus, after a crash, the file system restarts using the trusted data on the disk and replays these recorded operations to recreate the data that was lost from memory. We solved many tricky details to make sure the replay happens correctly — even when later actions change data on the disk in ways that affect earlier actions (for example, if an application deletes a file that an earlier action is supposed to write to).
To have an impact on the computer industry, our ideas must be tested in real systems, with real hardware and real workloads. Therefore, we have designed and implemented a new file system semi-microkernel, uFS. In more than 30,000 experiments, we show that applications can continue running correctly even if uFS crashes and restarts, that uFS quickly recovers in just a few hundred milliseconds, and performs as well as or even better than traditional file systems when there are no crashes. We’ve made the uFS source code available to everyone so that anyone can try it out, check our results and even build on our ideas.
—
About the Author
Andrea Arpaci-Dusseau is the Susan Beth Horwitz Professor of Computer Sciences and the Catherine A. Erickson Professor in the School of Computer, Data & Information Sciences. With her husband, Prof. Remzi Arpaci-Dusseau, she has co-advised 30 computer sciences PhD students, focusing on file and storage systems, operating systems, distributed systems, virtualization and scheduling.
