GenAI Applications Reference Architecture – Part 1

It’s been quite the year in an ML engineer’s life. I used to think my post “Riding the fear cycle into the future” was dramatic but GenAI has delivered and then some.

We developed prompt libraries, redesigned workflows, built GenAI backed features in existing products, and built GenAI apps as well. As we worked through the gamut my teams kept coming back with similar observations, recurring problems and repeatable solutions — all signs alluding to the need for a reference architecture for GenAI apps. We found a ton of interesting papers, articles and blog posts but very few were bold enough to take on the entire stack/landscape. Two that we would like to acknowledge here — Generative AI Lifecycle Patterns by Ali Arsanjani and Emerging Architectures for LLM Applications by Matt Bornstein and Rajko Radovanovic.

Both of these proved to be a great resources for us to ground our discussions with a common vocabulary, framework and an overall solution stack. These articles helped our teams arrive at a solution architecture hypothesis, communicate it clearly and identify tools/products/services that they wanted to incorporate either in the solution design or delivery.

We have created our own version of the GenAI applications reference architecture, inspired by the two mentioned before, but with a few tweaks grounded in our experiments and implementations. A few notes on what and why —

1. We have re-arranged the blocks to better reflect how we see the GenAI applications landscape
2. Though extremely useful, the a16z depiction doesn’t work for explaining a prompt’s logic flow or a GenAI apps workflow. On the other hand we found Ali Arsanjani’s depiction quite apt for conversations on solution architecture — we have used his diagram as our starting point while we layer on top some of the concepts and tool/product specific details from the a16z LLM Applications Stack.
3. The diagram can get a little overwhelming — our apologies. But, putting it all together in one place has really helped us make sense of it. It has also helped us craft a variety of interesting GenAI based solutions. We hope you will see the merits of doing so.
4. There are components on this diagram which may not have industry traction yet but we still have them on with the intent of piloting the idea or at the least asking a question. e.g. Prompt Validation (pre-submission) — we haven’t come across any tools or projects on GitHub that claim to do this but it seems like a meaningful addition to the responsible AI and RBAC tool kits.
5. The products/tools/platforms depicted in dotted rectangles inside most of the concept rectangles are not meant to be comprehensive by a long shot — they are the ones that we have either used or experimented with. We will keep augmenting and tweaking the dotted rectangles as we get familiar with more of what’s out there.
6. The GenAI landscape is rapidly evolving. Our nifty diagram was probably outdated the day it was crystallised. Also, our understanding might be limited and different from your’s. We would love to hear from you on any difference of opinion, corrections, or tweaks in depiction that will make it better.
7. We have no pride of authorship — this work heavily borrows from the two great articles mentioned earlier and from a long list of others directly or indirectly. Please feel free to use our work as we have used their’s.

GenAI Applications Reference Architecture

Part 2 of this post will cover a few solution archetypes and how they can be depicted using the reference architecture.

Author: Ani Kannal

This article was originally authored and published on Medium and can be accessed here.