C Programming | Working with memory

Index

malloc, calloc, realloc & free
Getting the memory
Shared memory

We need memory to perform processes and store values. C programs usually get their memory by calling the function malloc() and release that used memory calling the function free() when they're done.

Some programming languages have garbage collectors to take care of memory. C doesn't. It may be seen as a negative aspect, but it's in fact a really good one since we as programmers can have more specific control on how memory is managed.

— There are three basic ways to store memory. If we know the memory needed, it can be stored as static or as automatic, and if we don't know how much memory we are going to need, then it can be stored as dynamic.

static memory allocation applies to global variables and variables marked with static. It is handled when the program starts and has a fixed size when the program is created.

int horsePower = 120;
static float pressure = 34.0f;

Automatic memory allocation applies to variables defined inside functions that aren't marked as static.

function GetTorque(EngineType engine) {
  float defaultTorque = 2.4f;
  ...
}

dynamic memory allocation is performed at run-time to allocate an arbitrary amount of memory at an arbitrary point in the program. This operation is handled by the operating system the program is running on, and the memory itself it's allocated on the heap.

int *speed = malloc(sizeof(int);

— The memory assigned to a program in a common architecture can be divided in four blocks:

+-------------+------------+-----------+---------------------------+
|    Code     |   Static   |   Stack   |           Heap            |
+-------------+------------+-----------+---------------------------+

Code stores the instructions to execute, the program code.
Static/Global stores the variables declared outside functions (and the ones marked as static) that consequently are accessible anywhere while the program is running. This block is available until the program closes.
Stack stores the information from function calls and local variables. If we exceed the amount reserved for this block, the program will crash. The point about the information inside the stack block is that once a process ends, it's automatically removed from the memory block (until it's needed again).
Heap/Global stores large amounts of memory and maintains the variables in memory. Unlike the other blocks, the size of the Heap block is not fixed and we can control how much memory we want to use, and for how long we want to maintain data in the memory.

The way a heap block is implemented can vary between operating systems or compilers. When we work with dynamic memory allocation, we're always working with the heap memory block.

The only limit for the heap block is the available amount of memory that the system running the program has.

malloc, calloc, realloc & free

The C standard library includes functions that deal with dynamic memory allocation. These (malloc(3), calloc(3), realloc(3), free(3)) are the four functions that generally deal with dynamic memory allocation in C. They are included in stdlib.h.

malloc

void *malloc(size_t size)

When we call malloc(3), we are asking for a block of memory of a certain size in the heap memory block. malloc(3) returns a pointer to a block.

If there's not more memory available, malloc(3) returns NULL.
malloc(3) doesn't initialize the allocated memory. We can use memset() to do that.

int *speed = malloc(sizeof(int));

speed = 180;

calloc

void *calloc( size_t num, size_t size)

If we know the number of elements that we want to store and the size of each element, we can use calloc(3).

calloc(3) also initializes the bytes in the block to zeroes, which avoids random garbage. This is useful when debugging.

int *checkpoints;
checkpoints = calloc(sizeof(int), 2);

checkpoints[0] = 1;
checkpoints[1] = 2;

realloc

void *realloc(void* pointer, size_t size)

If we allocated a block of memory but at a certain point of our program we need to change its size, we can call realloc(3).

We need to pass the memory block we want to change, and the new size (that can be bigger or smaller).

The address that realloc(3) returns can be different from the one of the original memory block. Once we reallocate a block we need to point to the new address, otherwise the program would crash.

checkpoints = realloc(checkpoints, sizeof(int)*200);

free

void free(void *memory)

When we are done using the memory, we can call free(3) to tell the program's memory that the specified block can be back to the operating system.

free(speed);
free(checkpoints);

If we don't call free(3) after using a particular memory block we will be making an unnecessary memory usage.

Getting the memory

We know that calling malloc(3) gets us memory to store dynamic values in the heap block, but what happens there is like a black box. Where does the memory come from?

malloc(3) calls a function named mmap() where the magic happens. mmap(2) requests memory from the kernel.

void mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

*addr indicates where we want to allocate the memory. It can be NULL if we don't care where to store the values, otherwise we can specify a memory address (void*)0xFEEDB0000 and the system will try to satisfy the allocation.
length determines the size of the memory block that we want to map. We can request sizes that aren't multiples of 4k, but the system is going to return 4k multiples either way, so we can set it to 4096 as a base and we are good to go. If we want to allocate more blocks we can multiply the initial value.

PAGESIZE 4096

prot stands for protection and indicates which use do we want with the mapped memory. Common uses are PROT_READ and PROT_WRITE.
flags tell the kernel how we want the memory to be managed. We can make memory available only for the ongoing process with MAP_PRIVATE, we can share the memory with external processes with MAP_SHARED, or use MAP_ANONYMOUS as common cases.
fd stands for file descriptor and is used to access i/o resources. File descriptors are non-negative integers. If we don't want to store any file, we can pass a negative value -1.
offset indicates where we want to start allocating memory so we can map only the parts we want.

We can un-map memory calling munmap(2):

int munmap(void *addr, size_t length);

— mmap(2) is useful when working with files since it allows us to handle them as memory buffers. A dedicated article on files will cover it.

Shared memory

Since mmap(2) allows having memory buffers, we can use them as shared memory in scenarios where we don't want to use pipes or signals and we want different processes to communicate each other.

When a program starts running, it becomes a process. A program may have multiple processes. We can identify each process by the id the system creates to differentiate them using the function getpid(2).

If we use the command-line with a tool like top, we can see all running processes and each one's ID.

*nix systems create processes using fork(2), which clones a process creating a parent process and a child process. Let's see how it works.

int main() {
  printf("A single process. ID: %d\n", getpid());
}

The example above will print the statement once.

int main() {
  fork();
  printf("A single process. ID: %d\n", getpid());
}

If we call fork(2) in the main function we are cloning the process and we'll print twice the printf call. We should see different values for each process ID.

— Usually we want to make multiple processes so we can have each one doing different things. Right now we have two processes but apart from the ID, it's not clear which one is the parent and which one is the child.

Luckily we know the returning values of each one:

The parent returns the ID of the child.
The child returns 0;
On error, the parent returns -1 and no child process is created.

— At this point, changes made in either the parent or the child process are made locally so they can't see each other changes.

int non_shared = 4;
int main() {
  //check if the process is the child
  if (fork() == 0)
    non_shared = 0;
  else
    //parent waits for child to complete before the next instruction
    wait(NULL); 

  printf("Parent not shared value: %d\n", non_shared);
  return 0;
}

If we want the parent and child processes to be able to communicate with each other, we can make use of mmap(2) to create a memory buffer that shares the information.

int non_shared = 4;
int main() {
  uint8_t *shared_mem = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0);
  
  int pid = fork();

  //check if the process is the child
  if (pid == 0)
    *shared_mem = 1;
    non_shared = 0;
  else
    //parent waits for child to complete before the next instruction
    wait(NULL); 

  printf("not shared value: %d\n", non_shared);
  printf("shared value: %d\n", *shared_mem);
  return 0;
}

Now we can take some advantage on this and perform different operations for each process with shared values.

#include <stdio.h>
#include <unistd.h>   //fork()
#include <sys/mman.h> //mmap() 

int non_shared = 4;
int main() {
  int *shared_mem = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0);

  int pid = fork();

  //check if the process is the child
  if (pid == 0)
    *shared_mem = non_shared + 2;
  else
    //parent waits for child to complete before the next instruction
    wait(NULL); 

  int result = *shared_mem / 2;
  if(pid != 0)
    printf("\nOperation result is: %d", result);
  return 0;
}

vertex farm

n0mad coder's blog