Java local cache (high performance settings)

2024-07-12

In the Java world, caching technology is widely used to improve application performance. It is mainly divided into two categories: remote caching and local caching. Remote caching, with its excellent performance and flexibility, is often implemented through popular solutions such as Redis and Memcached. Local caching, with its lightweight and fast access characteristics, is represented by technologies such as HashMap, Guava Cache, Caffeine, and Ehcache.

In future blog posts, we will delve into the secrets of remote caching, but today, let's focus on local caching. This article will first take you to appreciate the charm of local caching technology and provide you with a comprehensive overview. Then, we will delve into the caching technology known as the king of performance, explore the principles and implementation methods behind it. Finally, through a series of practical cases, we will show how to effectively use these high-performance local caching technologies in daily work to improve your development efficiency and application performance.

1. Java local cache

1.1 HashMap

Using the underlying implementation of Map, we can store the objects to be cached directly in memory, which is a direct and efficient method.

Advantage：This method is simple and direct, does not rely on external libraries, and is very suitable for applications with simple cache requirements and relatively simple scenarios.

Disadvantages: However, this approach lacks an automatic cache elimination mechanism, and if more advanced caching strategies need to be implemented, it may require higher customized development costs.


public class LRUCache extends LinkedHashMap {
 
    /**
     * 可重入读写锁，保证并发读写安全性
     */
    private ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
    private Lock readLock = readWriteLock.readLock();
    private Lock writeLock = readWriteLock.writeLock();
 
    /**
     * 缓存大小限制
     */
    private int maxSize;
 
    public LRUCache(int maxSize) {
        super(maxSize + 1, 1.0f, true);
        this.maxSize = maxSize;
    }
 
    @Override
    public Object get(Object key) {
        readLock.lock();
        try {
            return super.get(key);
        } finally {
            readLock.unlock();
        }
    }
 
    @Override
    public Object put(Object key, Object value) {
        writeLock.lock();
        try {
            return super.put(key, value);
        } finally {
            writeLock.unlock();
        }
    }
 
    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
        return this.size() > maxSize;
    }
}

1.2 Guava Cache

Guava Cache is a cache technology developed by Google, which is based on the LRU (least recently used) replacement algorithm. However, with the rise of Caffeine, Guava Cache has gradually faded out of people's sight. Caffeine not only inherits the advantages of Guava Cache, but also surpasses it in many aspects. Although no specific sample code is provided here, readers who are interested in Guava Cache can visit its official website for more information.

Advantage: Guava Cache supports setting maximum capacity limits and provides two expiration strategies: based on insertion time and access time, and also supports some basic statistical functions.

Disadvantages: With the release of Spring Boot 2 and Spring 5, they both no longer recommend the use of Guava Cache.

1.3 Caffeine

Caffeine is an open source cache technology that uses the W-TinyLFU algorithm, which is a cache eviction strategy that combines the advantages of LRU and LFU (least frequently used). Caffeine's cache performance is close to the theoretical optimal solution and can be regarded as an upgraded version of Guava Cache.


public class CaffeineCacheTest {
 
    public static void main(String[] args) throws Exception {
        //创建guava cache
        Cache<String, String> loadingCache = Caffeine.newBuilder()
                //cache的初始容量
                .initialCapacity(5)
                //cache最大缓存数
                .maximumSize(10)
                //设置写缓存后n秒钟过期
                .expireAfterWrite(17, TimeUnit.SECONDS)
                //设置读写缓存后n秒钟过期,实际很少用到,类似于expireAfterWrite
                //.expireAfterAccess(17, TimeUnit.SECONDS)
                .build();
        String key = "key";
        // 往缓存写数据
        loadingCache.put(key, "v");
 
        // 获取value的值，如果key不存在，获取value后再返回
        String value = loadingCache.get(key, CaffeineCacheTest::getValueFromDB);
 
        // 删除key
        loadingCache.invalidate(key);
    }
 
    private static String getValueFromDB(String key) {
        return "v";
    }
}

1.4 Ehcache

Ehcache is a pure Java in-process caching framework known for its fast and efficient performance. It is widely recognized and used as the default cache provider for Hibernate.

Advantage：Ehcache provides a wide range of cache eviction algorithms, including LFU (least frequently used), LRU (least recently used), and FIFO (first in, first out). It supports different types of cache storage, such as in-heap cache, off-heap cache, and disk cache, to meet different storage requirements. In addition, Ehcache also supports a variety of cluster solutions, which effectively solves the problem of data sharing.

Disadvantages: Although Ehcache excels in many aspects, it falls slightly short when it comes to performance compared to Caffeine.


 
public class EncacheTest {
 
    public static void main(String[] args) throws Exception {
        // 声明一个cacheBuilder
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache("encacheInstance", CacheConfigurationBuilder
                        //声明一个容量为20的堆内缓存
                        .newCacheConfigurationBuilder(String.class,String.class, ResourcePoolsBuilder.heap(20)))
                .build(true);
        // 获取Cache实例
        Cache<String,String> myCache =  cacheManager.getCache("encacheInstance", String.class, String.class);
        // 写缓存
        myCache.put("key","v");
        // 读缓存
        String value = myCache.get("key");
        // 移除换粗
        cacheManager.removeCache("myCache");
        cacheManager.close();
    }
}

2. High-performance cache Caffeine

2.1 Cache Types

2.1.1 Cache

Cache<Key, Graph> cache = Caffeine.newBuilder()    .expireAfterWrite(10, TimeUnit.MINUTES)    .maximumSize(10_000)    .build();
// 查找一个缓存元素， 没有查找到的时候返回nullGraph graph = cache.getIfPresent(key);// 查找缓存，如果缓存不存在则生成缓存元素,  如果无法生成则返回nullgraph = cache.get(key, k -> createExpensiveGraph(key));// 添加或者更新一个缓存元素cache.put(key, graph);// 移除一个缓存元素cache.invalidate(key);

The Cache interface provides the ability to explicitly search, update, and remove cached elements. cache.get may return null when the cached element cannot be generated or an exception is thrown during the generation process, causing the generation of the element to fail.

2.1.2 Loading Cache

LoadingCache<Key, Graph> cache = Caffeine.newBuilder()    .maximumSize(10_000)    .expireAfterWrite(10, TimeUnit.MINUTES)    .build(key -> createExpensiveGraph(key));
// 查找缓存，如果缓存不存在则生成缓存元素,  如果无法生成则返回nullGraph graph = cache.get(key);// 批量查找缓存，如果缓存不存在则生成缓存元素Map<Key, Graph> graphs = cache.getAll(keys);

A LoadingCache is a Cache implementation with CacheLoader capabilities attached.
If the cache does not exist, the corresponding cache element will be generated through CacheLoader.load.

2.1.3 Async Cache

AsyncCache<Key, Graph> cache = Caffeine.newBuilder()    .expireAfterWrite(10, TimeUnit.MINUTES)    .maximumSize(10_000)    .buildAsync();
// 查找一个缓存元素， 没有查找到的时候返回nullCompletableFuture<Graph> graph = cache.getIfPresent(key);// 查找缓存元素，如果不存在，则异步生成graph = cache.get(key, k -> createExpensiveGraph(key));// 添加或者更新一个缓存元素cache.put(key, graph);// 移除一个缓存元素cache.synchronous().invalidate(key);

AsyncCache is an asynchronous form of Cache, which provides the ability of Executor to generate cache elements and return CompletableFuture. The default thread pool implementation is ForkJoinPool.commonPool(). Of course, you can also customize your thread pool selection by overriding and implementing the Caffeine.executor(Executor) method.

2.1.4 Async Loading Cache

AsyncLoadingCache<Key, Graph> cache = Caffeine.newBuilder()    .maximumSize(10_000)    .expireAfterWrite(10, TimeUnit.MINUTES)    // 你可以选择: 去异步的封装一段同步操作来生成缓存元素    .buildAsync(key -> createExpensiveGraph(key));    // 你也可以选择: 构建一个异步缓存元素操作并返回一个future    .buildAsync((key, executor) -> createExpensiveGraphAsync(key, executor));
// 查找缓存元素，如果其不存在，将会异步进行生成CompletableFuture<Graph> graph = cache.get(key);// 批量查找缓存元素，如果其不存在，将会异步进行生成CompletableFuture<Map<Key, Graph>> graphs = cache.getAll(keys);

AsyncLoadingCache is the asynchronous form of LoadingCache, which provides the function of asynchronously loading and generating cache elements.

2.2 Eviction Strategy

Based on capacity

// 基于缓存内的元素个数进行驱逐LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .maximumSize(10_000)    .build(key -> createExpensiveGraph(key));
// 基于缓存内元素权重进行驱逐LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .maximumWeight(10_000)    .weigher((Key key, Graph graph) -> graph.vertices().size())    .build(key -> createExpensiveGraph(key));

Time-based

// 基于固定的过期时间驱逐策略LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .expireAfterAccess(5, TimeUnit.MINUTES)    .build(key -> createExpensiveGraph(key));LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .expireAfterWrite(10, TimeUnit.MINUTES)    .build(key -> createExpensiveGraph(key));
// 基于不同的过期驱逐策略LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .expireAfter(new Expiry<Key, Graph>() {      public long expireAfterCreate(Key key, Graph graph, long currentTime) {        // Use wall clock time, rather than nanotime, if from an external resource        long seconds = graph.creationDate().plusHours(5)            .minus(System.currentTimeMillis(), MILLIS)            .toEpochSecond();        return TimeUnit.SECONDS.toNanos(seconds);      }      public long expireAfterUpdate(Key key, Graph graph,           long currentTime, long currentDuration) {        return currentDuration;      }      public long expireAfterRead(Key key, Graph graph,          long currentTime, long currentDuration) {        return currentDuration;      }    })    .build(key -> createExpensiveGraph(key));

Based on references

// 当key和缓存元素都不再存在其他强引用的时候驱逐LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .weakKeys()    .weakValues()    .build(key -> createExpensiveGraph(key));
// 当进行GC的时候进行驱逐LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .softValues()    .build(key -> createExpensiveGraph(key));

2.3 Refresh Mechanism

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()    .maximumSize(10_000)    .refreshAfterWrite(1, TimeUnit.MINUTES)    .build(key -> createExpensiveGraph(key));

The refresh strategy can only be used in LoadingCache. Unlike eviction, if a cache element is queried during refresh, its old value will still be returned until the refresh of the element is completed. The new value after the refresh will be returned.

2.4 Statistics

Cache<Key, Graph> graphs = Caffeine.newBuilder() .maximumSize(10_000) .recordStats() .build();

Data collection can be enabled by using the Caffeine.recordStats() method. The Cache.stats() method will return a CacheStats object that contains some statistical indicators, such as:

hitRate(): query cache hit rate
evictionCount(): The number of caches evicted
averageLoadPenalty(): The average time it takes for new values to be loaded

With the RESTful Controller provided by SpringBoot, it is very convenient to query the usage of Cache.

Technology Sharing