Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Description of Chat Memory

        Chat MemoryIt is called chat memory, which stores historical conversations between users and big models, so that big models can use these historical conversations to understand what users have said recently and what they mean.

However, if historical conversations are stored in Chat Memory all the time, the required storage space will become larger and larger. Therefore, Chat Memory also supports extended functions such as window limitation, elimination mechanism, persistence mechanism, etc.

---

ChatMemory is an interface that provides two implementation classes by default:

• MessageWindowChatMemory

According to the size of the collection, old messages are eliminated. As a sliding window, keep N The newest message is pushed in first, and old messages that no longer fit are evicted. However, since each message can contain a different number of tokens,MessageWindowChatMemory Very useful for rapid prototyping.

• TokenWindowChatMemory

Based on the size of the token, old messages are eliminated. It also operates as a sliding window, but focuses on retaining N The latest token, and evicts older messages as needed. Messages are indivisible. If a message does not fit, it will be evicted entirely. TokenWindowChatMemory requires aTokenizer To calculate eachChatMessage Token in .

---

Difference between TokenWindowChatMemory and MessageWindowChatMemory

TokenWindowChatMemory is similar to MessageWindowChatMemory, but the difference is that the capacity is calculated in different ways. MessageWindowChatMemory directly takes List<ChatMessage> The size of the TokenWindowChatMemory will use the specified Tokenizer to<ChatMessage> The corresponding number of Tokens is estimated and then compared with the set maxTokens. If it exceeds maxTokens, it will be eliminated, and the oldest ChatMessage will also be eliminated.

Tokenizer is an interface. The default OpenAiTokenizer implementation class is used to estimate how many tokens a ChatMessage corresponds to. Many large model APIs charge based on the number of tokens used. Therefore, when you are sensitive to costs, it is recommended to use TokenWindowChatMemory to control the total number of tokens used in a session.

Both implementation classes have a ChatMemoryStore property inside. ChatMemoryStore is also an interface, and there is an InMemoryChatMemoryStore implementation class by default. Over time, out-of-the-box implementations will be added for popular storages such as SQL databases, document storage, etc. In the meantime, you can implement this interface to connect to any storage of your choice

---

Case practice

Case 1: Building a popular naming master based on ChatMemory

public class NameDemo {
 
    interface NamingMaster {
        String talk(String desc);
    }
    public static void main(String[] args) {
        ChatLanguageModel chatModel = ZhipuAiChatModel.builder()
                .apiKey("智普apikey")
                .build();
        ChatMemory chatMemory = MessageWindowChatMemory.withMaxMessages(10);
 
        NamingMaster namingMaster = AiServices.builder(NamingMaster.class)
                .chatLanguageModel(chatModel)
                .chatMemory(chatMemory)
                .build();
 
        System.out.println(namingMaster.talk("我姓李，帮我取一个好听的女孩名字，就一个你觉得最好的"));
        System.out.println("---");
        System.out.println(namingMaster.talk("换一个"));
    }
}

Case 2: Customize ChatMemoryStore to persist ChatMessage to disk

Introducing Maven dependencies

<dependency>
	<groupId>org.mapdb</groupId>
	<artifactId>mapdb</artifactId>
	<version>3.0.9</version>
	<exclusions>
		<exclusion>
			<groupId>org.jetbrains.kotlin</groupId>
			<artifactId>kotlin-stdlib</artifactId>
		</exclusion>
	</exclusions>
</dependency>

 Customize ChatMemoryStore to implement persistent storage

public class PersistentChatMemoryStore implements ChatMemoryStore {
 
	private final DB db = DBMaker.fileDB("chat-memory.db").transactionEnable().make();
	private final Map<String, String> map = db.hashMap("messages", Serializer.STRING, Serializer.STRING).createOrOpen();
 
	@Override
	public List<ChatMessage> getMessages(Object memoryId) {
		String json = map.get((String) memoryId);
		return ChatMessageDeserializer.messagesFromJson(json);
	}
 
	@Override
	public void updateMessages(Object memoryId, List<ChatMessage> messages) {
		String json = ChatMessageSerializer.messagesToJson(messages);
		map.put((String) memoryId, json);
		db.commit();
	}
 
	@Override
	public void deleteMessages(Object memoryId) {
		map.remove((String) memoryId);
		db.commit();
	}
}

  Code Testing

public class PersistentDemo {
 
    interface NamingMaster {
        String talk(String desc);
    }
    public static void main(String[] args) {
         ChatLanguageModel chatModel = ZhipuAiChatModel.builder()
                .apiKey("智普apikey")
                .build();
 
        ChatMemory chatMemory = MessageWindowChatMemory.builder()
                .chatMemoryStore(new PersistentChatMemoryStore())
                .maxMessages(10)
                .build();
 
       NamingMaster namingMaster = AiServices.builder(NamingMaster.class)
                .chatLanguageModel(chatModel)
                .chatMemory(chatMemory)
                .build();
 
        System.out.println(namingMaster.talk("我姓李，帮我取一个好听的女孩名字，就一个你觉得最好的"));
        System.out.println("---");
        System.out.println(namingMaster.talk("换一个"));
    }
}

Case 3: Storing chat history for each user separately

public class NameDemo {
 
    interface NamingMaster {
        String talk(@MemoryId Integer userId, @UserMessage String desc);
    }
 
    public static void main(String[] args) {
        ChatLanguageModel chatModel = ZhipuAiChatModel.builder()
                .apiKey("智普apikey")
                .build();
 
        NamingMaster namingMaster = AiServices.builder(NamingMaster.class)
                .chatLanguageModel(chatModel)
                .chatMemoryProvider(userId ->  MessageWindowChatMemory.withMaxMessages(10))
                .build();
 
        System.out.println(namingMaster.talk(1, "我姓李，帮我取一个好听的女孩名字，就一个你觉得最好的"));
        System.out.println("---");
        System.out.println(namingMaster.talk(2, "我姓赵，帮我取一个好听的男孩名字，就一个你觉得最好的"));
        System.out.println("---");
        System.out.println(namingMaster.talk(1, "换一个"));
    }
}