Last Updated on July 21, 2024 by Sakshi Singh
Elon Musk’s xAI has unveiled Grok-1.5, which comes with increased reasoning skills and a context length of 128,000 tokens. It will be available on the š¯•¸ platform for early testers and existing Grok users in the coming days. The startup stated that this most recent iteration would significantly improve coding and math-related operations.
In LLMs, tokens are the smallest units of data that may be processed. In text, a token can be a word or a portion of a word. According to reports, the improved version of Grok can handle more complex instructions as well. Grok effectively allows the model to have 16 times more short-term memory and, as a result, is capable of processing significantly lengthier documents.
To demonstrate Grok-1.5’s problem-solving ability, xAI benchmarked it against popular tests. Grok-1.5 outperformed Mistral Large and Claude 3 Sonnet in the MMLU test, scoring 81.3% (5-shot). In the MATH test, it achieved 50.6% (4 shots), defeating Claude 3 Sonnet once more. It earned a remarkable 90% on the next GSM8K test, but only after eight shots of urging. Finally, on the HumanEval test, the Grok-1.5 model received 74.1% with a 0-shot.
The company conducted the NIAH test (Needle in a Haystack) to assess its retrieval capability, and it received excellent results. xAI has not released the parameter size for this incremental model. To give you an insight, Grok-1 is trained on 314 billion parameters, making it one of the largest open-source models available. It is likewise built on the Mixture of Experts (MoE) architecture. xAI also released the model weights and architecture under Apache 2.0, which is fantastic.
Grok-1.5 is developed on a proprietary distributed training architecture that uses JAX, Rust, and Kubernetes. A customized training coordinator guarantees that faulty nodes are automatically identified and removed from the training job. In summary, they have optimized their training procedure.
Features to expect in Grok 1.5
Improved user experience
Grok-1.5 promises to provide a more intuitive and interesting user experience because of its increased reasoning capabilities and expanded contextual length. By comprehending larger contexts and offering more insightful responses, the chatbot can better respond to its customers’ various demands.
Possible application
Grok-1.5 has far-reaching ramifications beyond casual conversations. Its enhanced problem-solving abilities make it an invaluable tool for a wide range of jobs, from educational support to professional aid. Furthermore, its longer context duration enables more sophisticated interactions across multiple domains.
Continuously innovating
Grok-1.5 exemplifies xAI’s commitment to pushing the limits of AI technology. They seek to stay at the forefront of the field by constantly refining and upgrading their models, which drives innovation and shapes the future of conversational AI.
Musk officially formed xAI in July to compete with OpenAI. The initial version of Grok was unveiled in November, and Musk announced a day later that his AI startup would be merged with social media site X. The chatbot was initially only available to X Premium+ subscribers who paid a $16 monthly subscription fee, but Musk stated in early March that it would be open-source to all users.
In terms of Grok-1.5 availability, the platform states that early testers, existing Grok users, and X Premium+ customers will have the opportunity soon. It is intended to gradually spread to a larger audience.
Leave a Reply
You must be logged in to post a comment.