- general purpose volatile memory allocation library
$ LD_PRELOAD=libvmmalloc.so.1 command [ args... ]
cc [ flag... ] file... -lvmmalloc [ library... ]
void *malloc(size_t size);
void free(void *ptr);
void *calloc(size_t number, size_t size);
void *realloc(void *ptr, size_t size);
int posix_memalign(void **memptr, size_t alignment, size_t size);
void *aligned_alloc(size_t alignment, size_t size);
void *memalign(size_t alignment, size_t size);
void *valloc(size_t size);
void *pvalloc(size_t size);
size_t malloc_usable_size(const void *ptr);
void cfree(void *ptr);
transparently converts all dynamic memory allocations into
Persistent Memory allocations.
The typical usage of libvmmalloc
does not require any modification of the
target program. It is enough to load libvmmalloc
before all other
libraries by setting the environment variable LD_PRELOAD
. When used in
that way, libvmmalloc
interposes the standard system memory allocation
routines, as defined in malloc
(3), and provides that all dynamic memory allocations
are made from a memory pool
built on a memory-mapped file,
instead of the system heap. The memory managed by libvmmalloc
different attributes, depending on the file system containing the
memory-mapped file. In particular, libvmmalloc
is part of the
Persistent Memory Development Kit
because it is sometimes useful
to use non-volatile memory as a volatile memory pool, leveraging its capacity,
cost, or performance characteristics.
may be also linked to the program, by providing the
**-lvmmalloc* argument to the linker. Then it becomes the default memory
allocator for the program.
NOTE: Due to the fact the library operates on a memory-mapped file, it may
not work properly with programs that perform fork(2) not followed by
There are two variants of experimental fork
available in libvmmalloc. The desired library behavior may be selected by
setting the VMMALLOC_FORK
environment variable. By default variant #1
is enabled. See ENVIRONMENT
for more details.
uses the mmap
(2) system call to create a pool of
volatile memory. The library is most useful when used with Direct
storage (DAX), which is memory-addressable persistent storage that
supports load/store access without being paged via the system page cache. A
Persistent Memory-aware file system is typically used to provide this type of
access. Memory-mapping a file from a Persistent Memory-aware file system
provides the raw memory pools, and this library supplies the traditional
interfaces on top of those pools.
The memory pool acting as a system heap replacement is created automatically at
library initialization time. The user may control its location and size by
setting the environment variables described in ENVIRONMENT
, below. The
allocated file space is reclaimed when the process terminates or in case of
Under normal usage, libvmmalloc
will never print messages or
intentionally cause the process to exit. The library uses pthreads
to be fully MT-safe, but never creates or destroys threads itself. The library
does not make use of any signals, networking, and never calls select
be set for libvmmalloc
to work properly. If either of them
is not specified, or if their values are not valid, the library prints an
appropriate error message and terminates the process. Any other environment
variables are optional.
Specifies a path to the directory where the memory pool file should be created.
The directory must exist and be writable.
Defines the desired size (in bytes) of the memory pool file. It must be not less
than the minimum allowed size VMMALLOC_MIN_POOL
as defined in
NOTE: Due to the fact the library adds some metadata to the memory pool, the
amount of actual usable space is typically less than the size of the memory
- VMMALLOC_FORK=val (EXPERIMENTAL)
controls the behavior of libvmmalloc
in case of
(3), and can be set to the following values:
- 0 - fork(2) support is disabled. The behavior of
fork(2) is undefined in this case, but most likely results in
memory pool corruption and a program crash due to segmentation fault.
- 1 - The memory pool file is remapped with the MAP_PRIVATE
flag before the fork completes. From this moment, any access to memory
that modifies the heap pages, both in the parent and in the child process,
will trigger creation of a copy of those pages in RAM (copy-on-write). The
benefit of this approach is that it does not significantly increase the
time of the initial fork operation, and does not require additional space
on the file system. However, all subsequent memory allocations, and
modifications of any memory allocated before fork, will consume system
memory resources instead of the memory pool.
This is the default option if VMMALLOC_FORK
is not set.
- 2 - A copy of the entire memory pool file is created for the use of
the child process. This requires additional space on the file system, but
both the parent and the child process may still operate on their memory
pools, not consuming system memory resources.
NOTE: In case of large memory pools, creating a copy of the pool file may stall
the fork operation for a quite long time.
- 3 - The library first attempts to create a copy of the memory pool
(as for option #2), but if it fails (i.e. because of insufficient
free space on the file system), it will fall back to option #1.
NOTE: Options 2
are not currently supported on
Environment variables used for debugging are described in DEBUGGING
relies on the library destructor being called from the main
thread. For this reason, all functions that might trigger destruction (e.g.
(3)) should be called in the main thread. Otherwise some of the
resources associated with that thread might not be cleaned up properly.
Two versions of libvmmalloc
are typically available on a development
system. The normal version is optimized for performance. That version skips
checks that impact performance and never logs any trace information or
performs any run-time assertions. A second version, accessed when using
libraries from /usr/lib/pmdk_debug
, contains run-time assertions and
trace points. The typical way to access the debug version is to set the
environment variable to /usr/lib/pmdk_debug
, as appropriate. Debugging output is controlled
using the following environment variables. These variables have no effect on
the non-debug version of the library.
The value of VMMALLOC_LOG_LEVEL
enables trace points in the debug version
of the library, as follows:
- 0 - Tracing is disabled. This is the default level when
VMMALLOC_LOG_LEVEL is not set.
- 1 - Additional details on any errors detected are logged, in
addition to returning the errno-based errors as usual.
- 2 - A trace of basic operations is logged.
- 3 - Enables a very verbose amount of function call tracing in the
- 4 - Enables voluminous tracing information about all memory
allocations and deallocations.
is set, debugging output is written to
Specifies the name of a file where all logging information should be written. If
the last character in the name is “-”, the PID
current process will be appended to the file name when the log file is
created. If VMMALLOC_LOG_FILE
is not set, output is written to
to 1 enables logging human-readable summary
statistics at program termination.
Unlike the normal malloc
(3), which asks the system for additional memory
when it runs out, libvmmalloc
allocates the size it is told to and
never attempts to grow or shrink that memory pool.
may not work properly with programs that perform
(2) and do not call exec
(3) immediately afterwards. See
for more details about experimental fork
If logging is enabled in the debug version of the library and the process
(2), no new log file is created for the child process,
even if the configured log file name ends with “-”. All logging
information from the child process will be written to the log file owned by
the parent process, which may lead to corruption or partial loss of log data.
Malloc hooks (see malloc_hook
(3)), are not supported when using
depends on jemalloc, written by Jason Evans, to do the heavy
lifting of managing dynamic memory allocation. See:
(7) and <http://pmem.io>