One of the FX trading desks kept a reference data structure in memory: all live position limits and risk parameters for every currency pair, updated in real time from a risk management system. The structure held about 40GB of data and was read by the trading engine on every price update.

Putting 40GB on the Java heap was not an option. GC pauses on a 40GB heap are seconds, not milliseconds. The solution was off-heap allocation — memory that exists outside the GC’s visibility, managed explicitly by the application.

What “Off-Heap” Means

The JVM manages two kinds of memory:

On-heap: objects created with new. Managed by the garbage collector. Automatic allocation and reclamation. Visible to GC.

Off-heap: memory allocated outside the GC heap. Not tracked by the garbage collector. Must be explicitly freed (or managed via try-with-resources). Invisible to GC — no impact on GC pause time, regardless of size.

JVM Process Memory
┌─────────────────────────────────────────────┐
│  Java Heap (GC-managed)                     │
│  -Xmx controls size                         │
│  Everything in here → GC sees it            │
├─────────────────────────────────────────────┤
│  Off-heap / Native Memory                   │
│  No JVM size limit (OS limit only)          │
│  GC doesn't see it → no pause impact        │
│  Must be freed explicitly                   │
└─────────────────────────────────────────────┘

sun.misc.Unsafe: The Low-Level API

sun.misc.Unsafe (renamed to jdk.internal.misc.Unsafe in Java 9, but still accessible) exposes raw memory operations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

// Get the Unsafe instance (not via constructor — it's private):
Field f = Unsafe.class.getDeclaredField("theUnsafe");
f.setAccessible(true);
Unsafe unsafe = (Unsafe) f.get(null);

// Allocate 1GB of off-heap memory:
long address = unsafe.allocateMemory(1024L * 1024 * 1024);

// Write/read primitives at raw addresses:
unsafe.putLong(address, 42L);
long value = unsafe.getLong(address);

// Write/read with volatile semantics (memory barrier):
unsafe.putLongVolatile(null, address, 42L);
long value2 = unsafe.getLongVolatile(null, address);

// Free when done (no GC to do this):
unsafe.freeMemory(address);

This is essentially malloc/free from C, accessible from Java. There are no bounds checks, no null checks, no GC. Write past the end of your allocation and you corrupt memory. Forget to call freeMemory and you leak. Unsafe is aptly named.

A Simple Off-Heap Structure

A fixed-size record store for currency pair limits — 8 bytes per field, 6 fields, packed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

public class CurrencyLimitStore {
    private static final int RECORD_SIZE = 48;  // 6 × 8 bytes
    // Field offsets:
    private static final int MAX_NOTIONAL  = 0;
    private static final int MAX_POSITION  = 8;
    private static final int BID_LIMIT     = 16;
    private static final int ASK_LIMIT     = 24;
    private static final int LAST_UPDATED  = 32;
    private static final int FLAGS         = 40;

    private final Unsafe unsafe;
    private final long baseAddress;
    private final int capacity;

    public CurrencyLimitStore(int capacity) {
        this.unsafe = getUnsafe();
        this.capacity = capacity;
        this.baseAddress = unsafe.allocateMemory((long) capacity * RECORD_SIZE);
        unsafe.setMemory(baseAddress, (long) capacity * RECORD_SIZE, (byte) 0);
    }

    public double getMaxNotional(int pairId) {
        return Double.longBitsToDouble(
            unsafe.getLong(baseAddress + (long) pairId * RECORD_SIZE + MAX_NOTIONAL)
        );
    }

    public void setMaxNotional(int pairId, double value) {
        unsafe.putLong(
            baseAddress + (long) pairId * RECORD_SIZE + MAX_NOTIONAL,
            Double.doubleToLongBits(value)
        );
    }

    public void close() {
        unsafe.freeMemory(baseAddress);
    }
}

For a 5,000-pair structure, this allocates 240KB — tiny. The 40GB case is the same pattern but with more complex fields and a larger capacity.

Reading this structure in a tight loop is significantly faster than reading equivalent on-heap objects because:

  1. No object headers (a Java object has a 16-byte header before the first field)
  2. Records are packed contiguously — sequential access is cache-friendly
  3. No GC interference, even under heavy allocation pressure

Chronicle Map: Off-Heap Hash Map

Writing raw Unsafe code for production use is risky. Chronicle Map wraps off-heap allocation in a safe, usable API — an off-heap Map<K, V> with persistence to a memory-mapped file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

ChronicleMap<String, LimitData> limits = ChronicleMap
    .of(String.class, LimitData.class)
    .name("currency-limits")
    .entries(10_000)
    .averageKey("EUR/USD")
    .averageValue(new LimitData())
    .createPersistedTo(new File("/dev/shm/limits.dat"));  // tmpfs = in memory

// Write:
limits.put("EUR/USD", new LimitData(maxNotional, maxPosition));

// Read (zero-copy with try-with-value):
try (ExternalMapQueryContext<String, LimitData, ?> ctx =
         limits.queryContext("EUR/USD")) {
    if (ctx.entry() != null) {
        LimitData data = ctx.entry().value().get();
        return data.maxNotional;
    }
}

Chronicle Map stores keys and values in off-heap memory. The LimitData value is serialised to a contiguous byte region — no GC overhead for reads or writes.

The try-with-value pattern for reads is important: it’s a zero-copy path where data is a flyweight pointing directly into the off-heap memory, not a copied object. The lock on the map entry is held until the try block exits.

Memory-Mapped Files for Persistence

Both Chronicle Map and Chronicle Queue use memory-mapped files for persistence. The OS maps a file on disk into the process’s virtual address space. Writes to the mapped region eventually flush to the file; reads from the region may or may not hit disk depending on the OS page cache.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

// Memory-mapped file, 1GB:
RandomAccessFile raf = new RandomAccessFile("/data/prices.dat", "rw");
FileChannel channel = raf.getChannel();
MappedByteBuffer buffer = channel.map(
    FileChannel.MapMode.READ_WRITE, 0, 1024 * 1024 * 1024L);

// Write to mapped region (goes to OS page cache, eventually flushed to disk):
buffer.putDouble(offset, price);

// Force flush to disk immediately:
buffer.force();

For the risk parameter store, we used a memory-mapped file on a tmpfs (RAM-backed filesystem). This gave us:

  • Persistence (the file exists between process restarts)
  • Off-heap storage (not visible to GC)
  • mmap semantics (OS manages the virtual memory mapping)
  • RAM performance (no actual disk I/O)

On process restart, the data was immediately available from the file without re-loading from the source system.

The Caution You Must Internalize

Off-heap code is memory-unsafe code. The JVM cannot protect you from:

1
2
3
4
5
6
7
8

// Corrupt another process's memory:
unsafe.putLong(addressThatHappensToBeAnOSDataStructure, 0xDEADBEEFL);

// Read garbage — another thread freed this memory:
long value = unsafe.getLong(freedAddress);  // undefined behaviour

// Crash the JVM — write past allocation:
unsafe.putLong(baseAddress + capacity * RECORD_SIZE + 1000, 0);  // SIGSEGV

The discipline required:

  • Every allocateMemory must have a corresponding freeMemory path
  • Bounds checking in any code that takes external indices
  • Thread safety — putLong is not atomic unless you use putLongVolatile
  • Valgrind or similar tools for testing memory correctness

For most applications, the GC overhead of on-heap storage is acceptable, and the safety is worth it. Off-heap is the right choice specifically when the data structure is large enough that GC visibility would cause unacceptable pause times, and when you have the engineering discipline to manage it correctly.