Brandon's Bytes

Project: libmem [2]: Let's turn it into a library!

10 Feb 2018

This post is a copy from my old website and blog. I have kept its original post date.

In the previous post I showed you a really basic, one-off read from memory. It’s alright for demonstartion purposes, but to make it actually useful we need to accomplish some goals:

  1. Make the code reusable
  2. Make the code efficient

For goal 1, we could take this chunk of code and toss it into a function. However, efficiency is going to take a hit when we constantly open and close the device file (the performance hit is especially relevant with the KPTI patch introduced in Linux to mitigate Meltdown, which adds overheads to system calls).

In addition, for goal 2, seeking and reading may not be the most effective way to read/write to physical memory. A more effective way, in the case where our accesses are restricted to a reasonable range of addresses (i.e. not the entire virtual address space) would be to memory map the file using mmap(), which will cause the OS to provide our application direct translations from the process’s virtual addresses to the desired physical addresses. This allows us to simply dereference addresses and not require lseek() and read() calls.

In order to fulfill these two goals, logically we’d keep some sort of data structure to do some book keeping for us. If we restricted ourselves to just the lseek() and read() implementation, we would really only need to keep track of the file descriptor. However, in order to implement the memory mapped implementation, there will be additional book keeping data (such as the pointer returned from and the associated address ranges of mmap()).

To fulfill these two goals, I decided to turn this into a library: “libmem”, which provides a header for the necessary C struct and enums. Let’s take a look at how I implemented them at the time of this post: GitHub link to the commit

First, let’s look at inc/mem/mem.h where I have the header for this library.

It’s always good to define the API from a functional standpoint first, before delving into the implementation details.

mem_status mem_ctor(mem_context * mem, mem_mode mode, int write,
                    void * start_addr, void * end_addr);

This is the function for constructing a memory context struct. mode is a parameter that signifies if this context should be memory mapped or seek/read mode: the type is defined in the enum higher up in the file. write signifies if this memory context is writable: sometimes we just want to read and it’s always nice to have a safeguard against killing our system. start_addr and end_addr are the beginning and end addresses, inclusive, of the accessible memory if mode is for memory mapped mode; otherwise these parameters are unused.

void mem_dtor(mem_context * mem);

Of course, we need a destructor function as well to clean up when we’re done.

ssize_t mem_read (mem_context * mem, void * addr, void * buf, size_t count);
ssize_t mem_write(mem_context * mem, void * addr, const void * buf, size_t count);

Here we have analogues of the *nix read/write interface for files. What’s different is that instead of providing file descriptors, we provide a pointer to our memory context struct, and instead of having to seek before hand, we provide the base address (or “offset” in the *nix read/write/seek terms) to begin reading from/writing to.

typedef struct _mem_context
    int fd;     // memory device handle
    mem_mode mode;
    int write;  // write permission

    // mmap mode only data
    void * map; // mmap memory pointer
    void * s_addr;  // start
    void * e_addr;  // end
    size_t map_range;   // range between s_addr and e_addr
    size_t map_len;     // needed for munmap()
} mem_context;

This is the book keeping struct that I use. It contains the file descriptor from opening /dev/mem. We can see that we keep a hold of the mode, which is used to figure out which way to read/write to memory in our read/write functions. The more interesting bits are the memory mapped mode only fields, which provide vital information for the mem_read() and mem_write(). I’ll discuss these fields in more detail when we get to to the implementation part in the next post.