How can Short Term and Long Term Cache help in building seamless AI solutions?
In the previous posts, we talked about building powerful and self-learning Data Machines effortlessly. In this post, we will discuss how important memory implemented using caching, can be for building intelligent automations and how Data Machines support it.
Memory for AI automations can be very similar to the concept of session based, cookie based or local storage-based information storage in software applications. Unlike software applications where the underlying development frameworks provide the caching mechanisms, AI models do not provide the same functionality.
Rather, the system that implements the AI model will need to take the assistance of the development framework it’s being hosted on to leverage the caching features. This can work well when using one or two models but can get incrementally cumbersome if a solution needs to be built using multiple AI models.
But why is memory important? In a stateless implementation, with a single processing step, the input is received, processed and delivered without retaining the information anywhere. In a multi-step process however, information may need to be retained for its usage in a later step.
There are two specific types of memory implementations supported by Data Machines – Short Term Cache and Long Term Cache
Short Term Cache
Let’s take a simple example, applicable to any modern web app, a user might enter a question in English and receive the answer from the website in English, where the source language of the website is in English.
Now, let’s assume that the user might not be familiar with English to frame a question properly. To properly serve the user, when they ask a question in a language other than English, the language needs to be detected and saved, the question needs to be translated to English to retrieve the answer and then the answer needs to be translated back to the language that was originally identified.
Caching plays an important role here in maintaining the information of the detected language until the last step is executed as seen in the illustration below. This type of memory is automatically enabled on all Data Machines and users do not need to do anything additional while building the Data Machines.
Long Term Cache
This type of cache needs an explicit action by the user to extend the default caching offered by Data Machines and utilize it between multiple executions of the Machines. This can be particularly useful when building a Data Machine where the response from the previous execution needs to be retained in memory in order to carry on a contextual interaction with a Data Machine, for example a Q&A engine.
To leverage the long term cache of a Data Machine, the sessionid output parameter needs to be used by the developer while making repeat executions of a Data Machine, so all the executions are performed in the same session context. The sessionid parameter is provided as an output for every Data Machine step inference. If the parameter is not provided as an input while executing the Data Machine, a new session id is generated, otherwise the id provided by the input will be used to maintain the session context.
By allowing the developer to provide the session id, the scope of Data Machine operation is vastly expanded by allowing the Data Machine to interact with the developer’s app by using the same session id for a user in the app as the session context within the Data Machine. This allows all of the output from the Data Machine to be saved inside the app’s own session management or temporary storage system.
Both Short Term and Long Term caching provide an advantage where Data Machines built using complex AI models, can be seamlessly integrated into any software app without the need to maintain a dedicated session management system, especially for the AI Models.
In subsequent posts, we will dive deeper into the Interoperability and Security mechanisms offered by Data Machines.
Free Trial
To learn more about Data Machines and get started with building your own AI solutions, sign up for a Free trial and start building today.
Sign Up as a Partner
If you’re a Software Development firm looking to rapidly build AI solutions for your customers, Data Machines offer you the best possibilities. Reach out to us here.
