Since April 2024, OVHcloud offers in alpha mode its AI endpoints. In this article, I'm going to see how to work with them using the langchain4j library that makes it super easy to discuss with LLM while coding in Java. (NB : langchain offers libraries for others langages if you are not a Java developer)
Retrieve a token to try
To start, we need to get a token. To do so, just connect to https://endpoints.ai.cloud.ovh.net/ (NB : you'll need to create a free account on OVH) and retrieve your free token. It'll be usable for 1 week.
Discover the available endpoints
We can see that by the date of this article (May, 1st 2024), OVHcloud offers different kinds of models
In this article, I'll focus on the Assistant ones (ChatGPT-like ones)
Implement with langchain4j
In a new gradle project, add the dependencies:
// In build.gradle.kts
dependencies {
implementation("dev.langchain4j:langchain4j:0.30.0")
implementation("dev.langchain4j:langchain4j-open-ai:0.30.0")
implementation("dev.langchain4j:langchain4j-embeddings:0.30.0")
}
As it's a demo, I'll implement a single class with a Main method to interact with the LLMs. To easily switch from one model to another, let's create an enum that will list all models available
enum Models {
private String baseUrl;
private String model;
Models(String url, String modelName) {
baseUrl = url;
model = modelName;
}
}
When writing this article (May 2024), 4 models are available;
enum Models {
Mixtral8x7b("https://mixtral-8x7b-instruct-v01.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1", "Mixtral-8x7B-Instruct-v0.1"),
Llama213b("https://llama-2-13b-chat-hf.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1", "Llama-2-13b-chat-hf"),
Mistral7b("https://mistral-7b-instruct-v02.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1","Mistral-7B-Instruct-v0.2"),
CodeLLama13b("https://codellama-13b-instruct-hf.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1", "CodeLlama-13b-Instruct-hf");
...
}
In our Main method, we build the model. Available endpoints are compliant with the OpenAI specification. So we can use this kind of model:
OpenAiChatModel model = OpenAiChatModel.builder()
.baseUrl(modelToUse.baseUrl)
.apiKey(API_KEY)
.modelName(modelToUse.model)
.maxTokens(NB_TOKENS)
.build();
Then we create an assistant, with the option chatMemory to enable history support in our interaction with the LLM
Assistant assistant = AiServices.builder(Assistant.class)
.chatLanguageModel(model)
.chatMemory(chatMemory)
.build();
Now, we can start using the AI endpoints easily with a little program to interact with the console:
System.out.println("Ask a question : ");
in = new Scanner(System.in);
String question = in.nextLine();
String answer = assistant.chat(question);
System.out.println(answer);
A little while loop to act like a discussion
while (inChatMode) {
System.out.println("Another question about this topic ? Y or N");
in = new Scanner(System.in);
String continueChat = in.nextLine();
switch (continueChat) {
case "Y":
System.out.println("Ask a question : ");
in = new Scanner(System.in);
question = in.nextLine();
answer = assistant.chat(question);
System.out.println(answer);
case "N": inChatMode = false;
default:
System.out.println("Unknown option, retry");
}
}
The outcome looks like this
Which model you want to use ?
1. Mixtral-8x7B-Instruct-v0.1
2. Llama-2-13b-chat-hf
3. Mistral-7B-Instruct-v0.2
4. CodeLlama-13b-Instruct-hf
Pick one : 1
Using model Mixtral-8x7B-Instruct-v0.1 ...
> Ask a question :
Who is Yoda
Yoda is a fictional character in the Star Wars franchise, who appears as a small, green, humanoid alien with pointed ears, oversized eyes, and a long, thin tongue. He is a legendary Jedi Master known for his wisdom, powerful mastery of the Force, and his unique, unconventional way of speaking English, such as "Hmmm, yes. A Jedi's strength flows from the Force."
Yoda is one of the most popular and enduring characters in the Star Wars saga, and he has played a significant role in several films, including "The Empire Strikes Back," "Return of the Jedi," and "The Phantom Menace." He is also a central character in the Star Wars prequel trilogy, where he serves as a mentor to Anakin Skywalker and Obi-Wan Kenobi.
In addition to his appearances in the Star Wars films, Yoda has appeared in numerous books, comics, and animated series, and he remains a beloved and iconic figure in popular culture.
-------------------------
> Another question about this topic ? Y or N
Y
> Ask a question :
Who are is friends
Yoda has many friends and allies throughout the Star Wars franchise. Here are some of the most notable ones:
1. Obi-Wan Kenobi: Obi-Wan is a Jedi Knight who was trained by Yoda as a youngling. The two of them have a deep and respectful relationship, and Yoda often serves as a mentor and advisor to Obi-Wan.
2. Luke Skywalker: Luke is a Jedi Knight who was trained by Yoda in the ways of the Force. Yoda becomes a surrogate grandfather figure to Luke, and he plays a crucial role in helping Luke develop his Jedi abilities.
3. Mace Windu: Mace Windu is a Jedi Master who serves as a member of the Jedi Council. He and Yoda have a close and respectful relationship, and they often work together to address the challenges facing the Jedi Order.
4. Qui-Gon Jinn: Qui-Gon is a Jedi Master who is known for his unorthodox methods and his strong connection to the Force. Yoda has a deep respect for Qui-Gon and his abilities, and he often serves as a mentor to him.
5. Yaddle: Yaddle is a female member of Yoda species who serves as a member of the Jedi Council. She is wise and powerful in the ways of the Force, and she and Yoda have a close and respectful relationship.
These are just a few of the many friends and allies that Yoda has throughout the Star Wars franchise. He is a revered and beloved figure among the Jedi, and he has earned the respect and admiration of countless beings throughout the galaxy.
> Another question about this topic ? Y or N
Y
> Ask a question :
In which movies can we see him
Yoda appears in several movies in the Star Wars franchise. Here is a list of the movies in which he appears:
1. The Empire Strikes Back (1980): In this movie, Yoda trains Luke Skywalker in the ways of the Force on the planet Dagobah.
2. Return of the Jedi (1983): Yoda appears in several scenes in this movie, including a memorable scene in which he confirms to Luke that Darth Vader is his father.
3. The Phantom Menace (1999): Yoda appears as a member of the Jedi Council in this movie, and he plays a key role in the battle against the droid army on Naboo.
4. Attack of the Clones (2002): Yoda appears as a member of the Jedi Council in this movie, and he leads the clone army into battle against the Separatist droid army on Geonosis.
5. Revenge of the Sith (2005): Yoda plays a significant role in this movie, as he engages in a lightsaber duel with Emperor Palpatine and goes into exile on Dagobah.
In addition to these movies, Yoda appears in several other Star Wars media, including animated TV shows, video games, and books. He is a beloved and iconic character in the franchise, and he remains a popular and enduring figure in popular culture.
> Another question about this topic ? Y or N
N
Conclusion
OVHcloud makes it easy to use LLMs (for free for now 😉).
With the combination of Langchain4J, you easily code with your favorite language to enable AI in your application.
By the time of this article (May 8th 2024):
AI endpoint at OVHcloud are free
Langchain4j is in version 0.30.0
The little demo project is available in my GitLab experimentations group: https://gitlab.com/fun_with/fun-with-ai/fun-with-langchain4j-and-ovh