Memory Configuration For Redis 3.0

Premise

Redis on Windows 3.0 introduces a new memory management architecture that:

• improves performance
• reduces the system requirements
• simplifies configuration

On top of the new architecture, the switch from dlmalloc to jemalloc improves the heap management reducing heap fragmentation.

Memory Configuration Settings Changes

In Redis on Windows 2.8 the memory allocation architecture was based on a memory mapped file that was created at startup. The size and location of the memory mapped file were configured using two flags:

• maxheap
• heapdir

Since Redis on Windows 3.0 doesn't use a memory mapped file anymore, those two flags are treated as any other invalid flag, therefore if they are present in the configuration file or passed as a command line argument, Redis will fail to start.

System Paging File Size Requirements

The new memory management architecture uses the system paging file to back the Redis heap, therefore there are some system requirements to make sure that Redis doesn't run out of heap space.

Redis on Windows 2.8 was allocating the heap space at once at startup, if more heap space was needed later on, it was going to fail with an out of memory error. Redis on Windows 3.0, on the contrary, allocates the heap memory on demand. At startup it only allocated the minimum amount of heap required to start and it keeps allocating more memory when needed.

Given that the heap is subject to fragmentation and given some other internal requirements (i.e. copy on write mechanism to simulate the Unix fork API), a safe minimum requirement for the system paging file is 2 times the size of physical memory.

By default a freshly installed Windows machine allows the system paging file to grow up to 3.5 times the size of physical memory (provided that there is enough disk space for it), therefore the default system paging file configuration is already sufficient to run Redis on Windows 3.0.

Preventing Redis from running out-of-memory

If other programs, beside Redis, are running on the same machine and they also use the system paging file or if the system paging file reaches a high degree of fragmentation, there is still the possibility of an out-of-memory error. A machine reboot will defragment the system paging file and reduce the possibility of such an event. Increasing the size setting and setting the Initial size equal to the Maximum size also helps.

Note

Since Redis uses the system paging file to allocate the heap memory, the Working Set memory usage showed by the Windows Task Manager or by other tools such as ProcessExplorer will not always be accurate. For example, right after a background save of the RDB or the AOF files, the working set value may drop significantly. In order to check the correct amount of memory used by the redis-server to store the data, use the INFO client command. The INFO command shows only the memory used to store the redis data, not the extra memory used by the Windows process for its own requirements. That extra amount of memory not reported by the INFO command can be calculated subtracting the Peak Working Set reported by the Windows Task Manager and the used_memory_peak reported by the INFO command.