Introducing Narrow AI: Taking The Engineer Out Of Prompt Engineering

In the rapidly evolving landscape of AI, LLMs have become indispensable tools for businesses across industries. As the technology advances and new models emerge, many organizations find themselves grappling with the challenges of model migration and escalating costs.

Today, we're announcing the launch of Narrow AI, which we've designed to optimize LLM spend by seamlessly transitioning between models.

The Challenge: Building in a multi-model world

The world of LLMs is moving at a breakneck pace. Only in April did we see the launch of the Llama 3 class, and now we already have a new class of Llama 3.1 models which require evaluating and integrating in applications. For teams heavily invested in building on top of LLMs, this presents significant challenges in keeping systems up to date:

1. The task of evaluating how your existing prompts perform on new models as they come out
2. The time-consuming process of patching and adapting your prompts to each new model and achieving confidence in its reliability before deployment
3. Balancing performance requirements with cost considerations
4. Navigating the landscape of closed and open-weight models

Every time a new model is released, teams face up to a week of manual work including:

• Thoroughly evaluating the new model's capabilities
• Rewriting and optimizing prompts for the new model
• Extensive testing to ensure consistent performance
• Documenting changes internally

As LLM usage scales, costs can quickly escalate. Many organizations find themselves paying premium prices for higher-end larger models when more affordable alternatives could suffice for certain tasks, but they lack the resources to quickly evaluate every model as they are released.

Our goal is to provide a neutral tool you can use to automatically evaluate which provider and model would be the best to serve each individual task in your prompt chains or systems. In doing so, we hope to help you avoid vendor lock in, maintain privacy and security arrangements by allowing you to self-host with open-weights models, and avoid having to utilize higher cost models than required to power your tasks in production.

Enter Narrow AI: Your LLM Task Efficiency Partner

Narrow AI is built on a straightforward premise: LLM workflows should be flexible, cost-effective, and continuously optimized. Our platform offers a suite of tools designed to monitor, write, and optimize your production prompts:

1. Automated Model Migration

We help teams automatically test and adapt existing prompts to work with new models. Our system analyzes your current prompts, understanding their intent and structure, then generates optimized versions for your target model. This process, which traditionally could take a week of manual engineering, is reduced to minutes.

2. Intelligent Cost and Performance Optimization

Narrow AI goes beyond simple cost reduction. Our platform:

• Analyzes your LLM usage patterns
• Allows you to create specialised prompts to route requests to the appropriate model based on complexity and requirements
• Helps transition from closed-weight to open-weight models, allowing you access to run your task on existing cloud platform credit balances, or on platforms which help with better performance

By enabling the use of open-weight models, Narrow AI opens up new possibilities for performance and cost optimization:

• Leverage high-speed providers like Groq to significantly accelerate your applications
• Run models on your own hardware for greater control and security
• Benefit from being able to run your workload against an existing credit balance

3. Continuous Performance Monitoring

Narrow AI provides ongoing monitoring of your LLM operations, tracking key metrics such as:

• Response quality
• Latency
• Cost per request
• Model-specific performance variations

This data-driven approach ensures that your LLM strategy remains efficient and effective over time, allowing you to respond to new production use cases by either re-training existing prompts, or automatically creating new prompts to serve an unexpected new category of user input data.

Real World Results: Our Tool In Action

We're still in the early stages of rolling out our automated prompt engineering technology, but so far here are some of the wins both internally and from our earliest customers:

• We've reduced some of our most used LLM task costs by 95% while maintaining output quality by migrating them from OpenAI models to more efficient open-weight models like Llama3 8b.
• Our platform has helped reduce a weeks worth of manual evaluation and migration for building a report on a new model into an automated process that is finished in a day.
• Our API is also used inside products of AI startups to help their users create much more reliable and cost efficient prompts in a matter of seconds rather than having to do any manual prompt writing.

The Future of LLM Operations

At Narrow AI, we believe that the future of AI systems lies not just in using more powerful models, but in the proper use of the right model for each task. Our platform empowers organizations to:

1. Stay adaptable in a rapidly changing model landscape
2. Maximize the cost efficiency
3. Seamlessly transition between closed and open-weight models as needed
4. Leverage cutting-edge providers and hardware solutions for optimal performance

Join the Narrow AI Platform

If you're looking to reduce costs, improve performance, efficiently manage model transitions, or explore the benefits of open-weight models, we'd like to be your partner in doing so.

Sign up for our beta program today and experience a new approach to AI efficiency.

‍