Mayada Gonimah - CTO at Thread AI
Colin Bell - EVP Cloud at BRINC Drones

November 7, 2024

Today we’re excited to delve deep into what it means to serve critical infrastructure in the field. In the first installment of our customer series, Powered by Lemma, we’re thrilled to share one of the many workflows our customer BRINC Drones uses in service of their mission to revolutionize public safety.

We hope this use case inspires and pushes competition to drive home the value of intentional AI. Oftentimes teams may find it challenging to project themselves and their workflows onto horizontal infrastructure. We aim to demonstrate the power and flexibility of Lemma's composable platform by showcasing the innovative work of customers like BRINC Drones.

One of the first workflows BRINC Drones has built with Lemma Workers is one to improve the process of After Action Reporting.

Public safety agencies face a constant challenge: how to efficiently and accurately document incidents after they occur. After action reports (AARs) are essential for analyzing past performance and enhancing future actions. However, creating AARs can be a time-consuming and tedious process, often relying on manual data entry from first responders who are already stretched thin. These reports contain a detailed review of a specific incident or operation, such as a crime scene investigation, arrest, or tactical response. Its primary goal is to identify areas for improvement, learn from mistakes, and enhance future performance. It involves the review of written assets, hours of audio and sometimes video assets. While reviews are often used for internal purposes, there are cases when they may be required for submission as supplemental evidence in court.

BRINC was looking for ways to improve and automate some of the manual processing involved in this end to end workflow in a way that was secure, observable, durable and allowed for various experimentation techniques with clear human in the loop access points. Given the sensitive nature of the work, adherence to certain compliance standards was also required for portions of the underlying infrastructure powering this workflow, e.g. CJIS, HIPAA compliance.

Rich interactions with the output and the behavior of this workflow would feed into BRINC’s LiveOps offering and into their data sinks, where end users such as law enforcement officers can interact directly with this data.

Brinc’s Live Ops

BRINC LiveOps is a unified drone operations software platform designed to enhance coordination, efficiency, and effectiveness in public safety drone operations. It provides a comprehensive suite of features for managing, deploying, and monitoring drones across various missions.

In addition to the ability to natively scale and allow for secure tenant segregation across workflows, there were a number of considerations BRINC evaluated when considering the embedding of AI into operational workflows. Some of these key features included:

But most importantly, the ability to construct a complex multi-chain compound system behind simple, elegant APIs was key. With Lemma’s infrastructure, BRINC can continue to focus on their core intellectual property rather than on the plumbing that is required to chain together disparate systems (AI or otherwise) that cut across different protocols, authentications, and data flow modalities.

The Lemma Functions Registry comes with a powerful set of pre optimized AI native building blocks as well as the ability to integrate with arbitrary third party calls, across REST, gRPC and soon GraphQL.

Lemma’s Rich Functions Registry

Below is a version of the AAR Worker composed by BRINC on the Lemma Platform. A Worker is the fundamental unit of a workflow. It represents a complete process defined as a sequence of interconnected States. Workers are immutable to provide safety measures; changes can be made to a Worker by creating and publishing a new version. It is also immediately production ready and accessible via a number of modalities (e.g. REST, gRPC), allowing it to be easily embedded into live operations. All data that flows through every State of this Worker is observable and retriable. Lemma’s dedicated multimodal data plane ejects the data upon workflow completion where data retention is configurable by the end user. BRINC has the ability to create different variations of these workers per public agency or department with no overhead.

Each trigger of the Lemma Worker consists of a new Run, which is a scoped invocation of the Worker with dedicated access to a limited Context. The Context is the product facing primitive that natively combines data from State inputs and outputs as well as runtime parameters passed at invocation, allowing for richer transformations at the middle or edges of a workflow.

Brinc’s Lemma Worker

The AAR Lemma Worker is triggered via a REST endpoint or via a Connector on the onset of new assets, in this case, audio files (in formats including but not limited to MP3, MP4, FLAC and WAV) from BRINC’s external AWS S3 bucket. Some optional runtime parameterization may be passed to the Lemma Worker via LiveOps to improve the quality of the model inferences (e.g. certain geographic information captured in LiveOps may inform which languages are expected to be spoken. Other runtime parameters allow for dynamic model selection, allowing the most suitable model to be used based on real-time conditions, such as the specific language being spoken, or background noise levels.

In the event where language translation needs to happen, data is filtered and passed onto a different set of models. Another key component of the AAR is a summary for which only specific language models may be used (with certain security and data exemption guarantees pre configured with model providers). Sometimes key words are extracted and passed with supplemental data for the end users to engage within the LiveOps platform. The Lemma Worker also orchestrates calls to proprietary business logic (which may be externally hosted on products like Amazon Lambda or Google Cloud Functions). Finally the draft of the report is published onto the BRINC data sink, which in this particular Worker is another S3 bucket. BRINC webhooks are invoked on both success and failures, allowing for downstream triaging to easily take place without the need to poll on Lemma Workers.

Since the rollout of the AAR worker, BRINC has built a number of other workers on the platform and continues to find ways to revolutionize the way that public safety agencies create after action reports. By automating this process, public safety agencies can save time, improve accuracy, and create a more consistent reporting process, allowing them to enhance decision-making, free up limited resources, and ultimately improve public safety outcomes.