An obscure quirk of the /proc/*/mem pseudofile is its “punch through” semantics. Writes performed through this file will succeed even if the destination virtual memory is marked unwritable. In fact, this behavior is intentional and actively used by projects such as the Julia JIT compiler and rr debugger.
This behavior raises some questions: Is privileged code subject to virtual memory permissions? In general, to what degree can the hardware inhibit kernel memory access?
By exploring these questions1, this article will shed light on the nuanced relationship between an operating system and the hardware it runs on. We’ll examine the constraints the CPU can impose on the kernel, and how the kernel can bypass these constraints.
Most people thought I was crazy for doing this, but I spent the last few months of my gap year working as a short order cook at a family-owned fast-food restaurant. (More on this here.) I’m a programmer by trade, so I enjoyed thinking about the restaurant’s systems from a programmer’s point of view. Here’s some thoughts about two such systems.
This post details my adventures with the Linux virtual memory subsystem, and my discovery of a creative way to taunt the OOM (out of memory) killer by accumulating memory in the kernel, rather than in userspace.
Keep reading and you’ll learn:
Internal details of the Linux kernel’s demand paging implementation
How to exploit virtual memory to implement highly efficient sparse data structures
What page tables are and how to calculate the memory overhead incurred by them
A cute way to get killed by the OOM killer while appearing to consume very little memory (great for parties)
Note: Victor Michel wrote a great follow up to this post here.
Pretty recently I learned about setjmp() and longjmp(). They’re a neat pair of libc functions which allow you to save your program’s current execution context and resume it at an arbitrary point in the future (with some caveats1). If you’re wondering why this is particularly useful, to quote the manpage, one of their main use cases is “…for dealing with errors and interrupts encountered in a low-level subroutine of a program.” These functions can be used for more sophisticated error handling than simple error code return values.
I was curious how these functions worked, so I decided to take a look at musl libc’s implementation for x86. First, I’ll explain their interfaces and show an example usage program. Next, since this post isn’t aimed at the assembly wizard, I’ll cover some basics of x86 and Linux calling convention to provide some required background knowledge. Lastly, I’ll walk through the source, line by line.
Master is when the chef not only knows how to make oatmeal the “right” way, but is able to answer if you ask “What happens if we use half the water? Twice as much butter? No butter at all? 4x the milk?”. Because they’ve tried all these things before.
A master understands where the canonical solution fits in the greater design space of all solutions, and the tradeoffs involved. They can explain, in depth, what happens if you “do it wrong”, and can provide examples of instances where you might want to.
A master understands when the laws of gravity actually break. 1
(This is a random collection of thoughts around my new operating systems project, “gardenOS”.)
What’s up with the “gardening” terminology?
This is a phrase that I feel perfectly describes the ethos of this project. In the same way people have gardens as calm places to express themselves, learn things, and have fun doing work, I want to create an analogous place for exploring my interest in operating systems.
Some key tenants of the approach:
No stress
Have fun
Pursue whatever interests you
I don’t have particularly strong OSdev skills at the moment, so I’m especially focusing on doing small, easy tasks, such as cleaning up the build system. I do this in the same way you might spend an afternoon picking weeds in your little garden. It’s easy, well understood, not too complicated, no large decisions to be made — and it concretely improves the project. It’s a concrete win you can lock in, in a fixed amount of time, with fairly little work.
Disclaimer: I’m almost apprehensive to even give this project a name
I’m worried that even naming this project (“gardenOS”) will put too much pressure on me. I am deeply aware that OS’s take monumental amounts of time and energy to even get to basic states. And at my current rate (~2 hours a week), it’s unlikely we will get to even basic levels soon.
I wasn’t expecting things to get this philosophical this quickly. My focus above all, is to have fun, learn things, and make some small progress each week in the stream. Each week where I do a stream or do some work, any any of that happens, is a win.
I explicitly hold very few expectations around a future “goal” of the project or where I want it to end up. I just want to have fun and learn things about operating systems.
The loose vision I have is to create a minimal OS for play and experimentation. It should be a high quality codebase, and I should work on it as if I wanted to present my best self as an aspiring pro systems programmer.
The ethos of the project
Even though the project is in a maximally nascent stage, I already feel a certain ethos evolving. In the project, we emphasize:
Relaxed, casual, kind attitude
Learning mindset
Ambition to use programming best practices, and aspiring to become pro systems programmers
Portray an accurate “slice of life” of systems programming. Meaning: Show the day to day, which is not always thrilling tasks (like adding a syscall), but simply just working on the build system or refactoring some code.
My experience after doing four streams
I’ve done four streams so far (2 live, 2 recorded). My big worries are that I’ll run out of stuff to do and have to scramble to find something to do, live.
This has never happened — I’m always surprised at the adventures “we” find ourselves going on
The use of “we”
The project is just me at the moment, but there’s something about using “we” to describe it that makes me feel good.
It’s not just to make myself seem more impressive, or feel less alone. When I use it, I think of the handful of enthusiastic people that have joined me in the live streams, or left comments with questions or encouragement.
Even in these earliest of days, there is some tiny, microscopic community feel forming. So when I say “we”, I speak for this community.
Work update: I reduced to working 4 days/week and the cost/benefit asymmetry is astonishing.
Just one extra free day might not sound like much, but I feel like I gain >100% more high quality free time (Friday off is even better than Saturday; Sunday is not high quality free time for me – too much adulting to do).
And I lose only 20% of my productive work capacity (Well a bit more; Friday would be a more productive day than average for me b/c it’s quieter & less meetings).
I’ve constantly felt squashed the last few years, but always convinced myself it was a me problem, rather than a possibility that even “normal” working hours didn’t actually leave me with enough free time for everything I had to do*. (Maybe some of both)
But my energy and mood are way better than in a long time, so maybe it goes to show that the latter was the case, and one extra day can go a long way. (But that’s less surprising when you frame it as 100% more time).
I do need to be a bit more conscious of how I use my work week, and I have noticed a tendency to try to fit 5 days of work into 4… but overall it’s going well. I’m curious if I end up filling up the extra personal capacity and end up just as stressed, but I somehow doubt that will happen.
—
*You might ask, well why are you so busy anyway? Are you just piling on voluntary responsibilities?
I’ve thought about this at length and I think my answer is glibly, “expat life”.
Tip: When you’re learning a new programming language, look up prominent open source projects and copy their style.
Aside from the core language, there are many conventions & little details to learn: naming (variables, classes, files), file structuring, literal code formatting
These are things few blogs talk about because it’s highly opinionated. But nevertheless when you’re learning, you’ll benefit from at least some reference for these.
Find a few “professional” open source projects and browse to see what various interpretations of “professional style” are. Then pick one you like most.
Be careful of picking projects that are too old — they might use older style for consistency with legacy, even though they might ideally wish to modernize it.
And ideally pick projects whose contributors are experienced engineers who work on it full-time. Since they “live in” the codebase, they’re less likely to tolerate sloppiness – or are at least more invested in cleaning it up.
The last idea is influenced by @awesomekling, who talks about similar things in his classic “My top advice for programmers looking to improve” car-talk video =]
This is something my former colleague, Trent Brunson, told me on Twitter once that I’ll never forget. I tweeted earnestly how there were many things I wanted to share, but was concerned that everyone else knows these things already.
Trent’s (now-deleted) reply was:
I can’t think of a single time where I haven’t learned something from you. Keep sharing! Even if people already know the plot, they appreciate the way you tell the story.
So go forth, share freely, and don’t worry if the topic has already been covered, or if people are already familiar with what you’d like to share.
I deal a lot with decision paralysis and this has been a helpful mental tool. Decisions I might get stuck on:
Which product do I buy of these options
Which apartment do I move into
Which travel itinerary do I book
Decisions are effectively constraints on your life path. At one point, you have N options open to you, but after the (permanent) decision, you now have only one, the one you chose.
The paralysis comes from worrying about making the wrong decision and suffering because of it. But at a certain point, the decision process saturates. You run out of time and energy, and spending more of them doesn’t create a better decision. (It might worsen it in fact as you second-guess a previously good decision.) But even at this saturation point, you still might be unsure what to do.
Here, I find that it helps to just:
Recognize that moving forward is more important than making the “best” decision
Pick a promising option and accept that I’ll deal with whatever future consequences
Creative potential is to potential energy as creative output is to kinetic energy.
I feel strongly that everyone has potential in them to be creative in some form — “creative potential”. When someone creates something, the potential is released into some kind of output — “creative output”. (This applies even to ephemeral creativity like a live performance.)
Just like how electric potential energy (voltage) is converted to tremendous kinetic energy (current) with lightning, or via a short in a circuit, creative potential can be violently converted into creative output when the conditions are right (a “channel”). Another name for this process is “creative flow”.
The trick is finding the right conditions for a person to unleash their creative potential. The space of environmental conditions is vast with infinite parameters, including:
Art form
Instruments available
Time & place
Solo or with others
While I firmly believe people can discover their medium at any age, I think it’s important to focus on finding at least one pre-adulthood. Unfortunately after that point I think people lose confidence in themselves and resign themselves as “uncreative”, which can be a tough hurdle to get over.
I’m happy to say I’ve personally, finally found a channel for my writing practice, right here on this blog. More on that here.
I think a good reason to publish art sooner than later, and to default to creating in public, is so your art can start accruing interest as soon as possible. (“Art” used loosely; think content & projects also).
If you make art but only ever keep it private, there is a zero chance of it ever being discovered or enjoyed. In that sense, there is some wasted potential, like cash in the mattress.
But if the art is published (not necessarily promoted), there’s a non-zero chance of it being discovered and enjoyed by other people (potentially many others) with no work on your part. That’s what I mean by interest — some potential upside, at no cost to you.
I’ve been practicing this with this very blog. I write a lot and tell no one because that feels good at the moment. However, I already did spot some interest accrued: The “Nixers” newsletter #221 included a link to a recent post where I did zero promotion. Thank you!
P.S. Another advantage is that the work is readily available to be shared if you ever choose to at any point in the future — no need to dig it up and upload it.