Using AI engineering to help navigate complex operational data

We are increasingly seeing AI moving from proof of concepts and standalone experiments into the connective tissue of engineering delivery. As engineers, we can design faster, prototype faster, code with smarter assistants, and embed autonomous agents where they shave off real toil. And the most exciting part for me, is doing this where it matters; on important, complex client challenges.

The real-world challenge

Imagine you’re a call centre agent for a major roadside assistance provider. You’ve been taking back-to-back calls from frustrated customers. One customer’s car broke down on the motorway. It was towed to a dealership, the part to fix it was out of stock, a hire car was arranged in the meantime whilst the vehicle went into storage, and now the part is finally available, but no one yet knows when the dealership will fix the car.

Throw in a few reschedules and cancellations, and you’ve got a typical call for roadside assistant agents.

On a recent client engagement in the motoring services industry, we had the opportunity to use Azure's latest OpenAI offering to crunch through a timeline of complex data points from just such a customer journey. In just a few hundred milliseconds, we were able to provide a summary of the key findings - and then productionise it in less than a week - pretty cool. Here’s how...

From scattered data to a slick UI

We (a delivery team of product, UX, engineering, and QA) were brought in to build a greenfield system to help roadside assistant agents manage the lifecycle of a vehicle repair more effectively. The data was scattered across multiple internal and external systems, so we worked through some gnarly backend integrations to consolidate the disparate data. As part of the new UI, we also designed a beautiful timeline view of the vehicle repair and related events, with grouped data points, highlighted events, and advanced filtering.

After just six sprints, we were testing our minimal viable product with real customer service agents. The timeline worked well, but for complex cases, it still took up to 30 seconds for agents to identify the key details. And with a customer waiting on the line, they really needed the insight in under 10 seconds - any longer, and the delay starts to impact the customer experience.

Here comes AI

Our product director approached me a bit later with that look - the one that means “I’ve got an idea.”

“Let's use AI to summarise the timeline.”

After the initial feeling of “how the heck are we going to do this?” with such a tightly scoped roadmap; the technical cogs of the mind began to turn. Our system was deployed on Azure, and one of the client engineers had done a bit of experimentation with Azure Foundry. After a quick discussion with them about their approach, and some targeted testing, all the pieces of a practical solution started to come together.

Building lightweight and lean AI integration

We used Azure's semantic kernel to integrate with the OpenAI API, which sounds complicated, but it’s just a software development kit (SDK). While it offers a bunch of powerful features to build your own AI agents, all we needed was to input context, plus data, plus prompt, and to output our shiny summary.

With the help of the client’s platform engineering team, we created specific resource groups to keep a close eye on cost. We built a lightweight web API as a facade (we used .NET 8, but you can also use Java/Python), which registers the Azure semantic kernel on startup.

We chose GPT4 as our model; it’s good at deductive reasoning, affordable, and better at following strict instructions, which was perfect for our use case. Plus, it’s easily changed. We used the chat completion API, which supports full conversation history and response streaming – but our first iteration didn’t need this – so we kept it simple.

Keeping things tight with structured prompts

For the inputs, we needed just three parameters:

1. A ‘system message’ that sets the context and guardrails for the model to abide by. For example: Your response must be strictly factual, based solely on the information provided. Do not make assumptions or include speculation.
2. A ‘user message’ that contains the prompt and our chronological timeline data, this controls what we want the model to do with the data. For example: Do not use generative AI. You are analysing a chronological series of notes and events. Use the vehicle's VRN when referring to it, if available. Format any locations and highlight key info, booking references, and symptoms with double asterisks. Present only the key facts. The data is
3. A ‘json schema’ that allows us to force the model to reply with a given format, ensuring a consistent output.

Caching, cost control, and speedy loads

We read the response synchronously and displayed it above the timeline in the UI, preserving the model’s formatting.

To optimise performance and cost, we cached the result to Redis, and only re-generated the summary if the timeline data changed. This enabled repeat renders of the component to load in a sub-30 millisecond response time and saved approximately 1500 tokens against our quota. We also stored the prompt and context in source control for governance and traceability, and feature-flagged the summary so we can toggle it off instantly if needed.

From idea to impact in less than a week

After we deployed to production, we ran into hallucinations early on - GPT occasionally invented details not in the data. Prompt design became key, and getting it right took iteration. Asking non-tech users to test prompts in Foundry gave mixed results, so we built a simple, feature flagged input box that let product owners tweak prompts directly in the UI. This gave us a fast, controlled way to refine the output in real time.

In less than a week, we went from an idea to production with tangible, real-world impact. We had already built solid data foundations, defined a clear use case, and utilised cloud-native AI tools that are evolving fast.

With the current pace of change, this is just a sprinkle of AI – but with minimal change and process impact. It demonstrates how API-driven AI - available out of the box - can save time, enhance user experience, and reduce cognitive load for frontline staff.

What I also love is this experience shows how our team has the autonomy to pick the right tech for the right problem; and take it to production, fast. We get to learn at the cutting edge of engineering delivery and AI every week, with space to experiment and the support to land it safely.