Orca’s online, memory-controlled deep learning platform

Model-attached AI memory

Orca builds on leading open-source models to create predictive AIs capable of mimicking the distribution of external, human-readable datasets at inference time. Orca converts these datasets into memorysets - specialized data structures directly connected into the model’s archetypes. 

By connecting the model’s reasoning with knowledge stored in the memorysets, Orca allows you to build, steer, and correct models by simply editing data.

Building this memory starts with uploading a dataset that meets a few simple criteria:

• The dataset is structurally similar to the inputs and outputs the model will see in production. You can opt to use existing training data, real examples you’ve collected or synthetically created datasets.
• You have a small set of high-quality labelled examples for each class of data. Orca has tools that can help you label the remaining data, help you identify and correct incorrect labels, and identify areas that benefit from additional datapoints.

At each inference, an Orca memory-augmented model mimics the distribution of its attached memoryset to produce its output. To correct or steer the model, the memoryset can be edited to make an immediate change in model behavior without iterative retraining. In addition, memorysets can be swapped for every inference in milliseconds. 

Taken together these capabilities enable customization on two levels. First, a single model can cover many use cases by simply swapping memory sets. Second, context specific to an individual user/customer/subset of the distribution can be added as appropriate for each inference. 

For example, you can easily increase the model’s accuracy for geography-specific regulations, customer preferences or any other reason your business demands. For generative use cases Orca enables you to tailor agentic/LLM evaluation specific to each prompt/source. For predictive classification/regression/ranking use cases Orca enables efficient customization at large scale without requiring model proliferation or complex overlay filtering & post processing.

Memory-augmented deep-learning models

Memory-augmentation creates a significant difference from traditional deep-learning models. Instead of relying solely on memorized training data to make predictions, Orca’s memory-augmented models respond immediately to memoryset changes.

Orca unlocks greater accuracy, speed, and customization for critical use cases, including:

• Customizable, low-latency online LLM/Agentic eval
• Continuously re-trained classification & recommendation
• Per inference customization
• Real-time adaptive threat detection
• Semi-supervised anomaly detection
• Predictive AI explainability & audit

Agentic data management co-pilot

Building AI/ML models frequently devolves into a frustrating form of guessing and checking to improve your training data, retraining the model then testing the results. Orca’s immediate updating with changes in memorysets unlocks automated & semi-automated workflows that you can use to optimize your datasets. 

Orca’s co-pilot allows you to scan your dataset while building models for:

• Unnecessary datapoints that do not have any influence on the model’s prediction and could complicate steering your models.
• Incorrect labeling on specific memories that could cause errors or inconsistent behavior.
• Opportunities to improve data density for specific classes of data, especially when your model is less confident making predictions due to limited datapoints.

Ultimately, these tools eliminate the pain of searching for relevant datapoints and allow you to confidently and quickly optimize your AI. With these simplifications, current AI teams can go faster and more team members (software engineers, product managers and many others) can help build reliable, responsive deep-learning models.

Workflow & audit inspector

Deep-learning models are traditionally black boxes, making it difficult to determine why a model generates an output. This black box nature creates significant challenges troubleshooting a model when it misbehaves. Orca’s memory-augmented models can trace the specific data that mattered for each output. This property powers instrumentation, so you can audit a model’s reasoning, identify intelligent corrective actions, and improve model performance.

For each inference, the system records the specific data in the memoryset used in model reasoning along with the quantitative weighting of importance in the model’s conclusion. Based on this transparent tracing, Orca provides an Inspector tool that allows you easily see and interpret this telemetry data.  Orca’s SmartExplain capability translates the specifics of each data reference during reasoning into a human readable explanation of “why” the model reasoned to its conclusion for a specific inference. By providing these written explanations, both seasoned ML engineers and non-expert users can confidently and quickly edit a memoryset so the model makes predictions with greater accuracy and confidence. Taken together with instant model steering without retraining, this capability enables rapid troubleshooting and correction. In addition, auditability of model reasoning becomes a simple reporting function instead of “black box” guesswork. 

When a steering or correction edit is made to a memoryset, Orca proactively recommends possible additional edits to the memorysets to produce more accurate predictions.

By leveraging the edits you make and recommending next actions, Orca allows you to efficiently steer your AI and have comprehensive control over the outputs of your AI systems, even if you’ve only observed a small number of problematic behaviors.

You can also extend these steering tools to incorporate feedback mechanisms from user data and human-in-the-loop signals to automatically update memorysets. For example, an Orca-powered content moderation system may flag a new social media post as potentially harmful, but suppose a human reviewer decides it’s safe. This review immediately becomes a new memory, so the rate of false positives can continue to decline over time. While exact feedback systems will vary based on your specific application, the Inspector tool can automatically highlight changes for your review or automatically update the memoryset.