Kurt Shintaku's Blog

We’ve documented “best practices” by which creators of Generative AI solutions using Azure OpenAI Services can improve the response time of end user’s requests, along with GitHub repos for code samples

“New LLM models require massive amounts of compute to run, and unoptimized applications can run quite slowly, leading users to become frustrated. Creating a positive user experience is critical to the adoption of these tools, so minimising the response time of your LLM API calls is a must. The techniques shared in this article demonstrate how applications can be sped up by up to 100x their original speed* through clever prompt engineering and a small amount of code!”

Read the guide here:

• The LLM Latency Guidebook: Optimizing Response Times for GenAI Applications
  https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/the-llm-latency-guidebook-optimizing-response-times-for-genai/ba-p/4131994