This is the sixth and final in a series of blog posts about How to build with GenAI- From strategy to implementation. In this series, we will explore the following questions:
- Is GenAI the right strategy for your product roadmap?
- Should you build or buy your GenAI model?
- How do you navigate the complexity of data to deliver clear results?
- Should you take a Human-in-the-loop approach?
- How do you manage costs while developing with GenAI?
***
Building Gen AI systems isn’t just a technical challenge—it’s a journey filled with surprises, moments of trial and error, and plenty of lessons along the way. At Yotascale, we’ve learned that while AI can feel like magic, it’s still grounded in very real complexities.
Jeff Harris, our Director of Strategy and Operations, reflects: “When we started out, we didn’t have a roadmap—we were figuring it out as we went. Some things worked right away, others took a lot more effort, and there were plenty of surprises along the way.”
In this final post of our series, we’re sharing the highs, the lows, and the lessons we’ve learned while building and scaling Gen AI solutions. Whether you’re just getting started or refining an existing system, we hope these insights will help you navigate your own AI journey.
Lesson 1: Quick Wins Aren’t Always Quick Fixes
One of the biggest surprises for our team was how quickly we could get a basic Gen AI feature up and running—and how much work it took to refine it.
Jeff Harris, our Director of Strategy and Operations, shares: “When we first rolled out our AI assistant, we thought we might be able to launch in a month. But once we started refining, we realized there was so much more to do to get it right. It was quick to prototype, but tuning it to meet user expectations took time.”
The takeaway? Early success is exciting, but don’t underestimate the effort required to fine-tune AI systems for real-world scenarios.
Lesson 2: Embrace Iteration as a Core Practice
Gen AI isn’t a “set it and forget it” tool—it’s a system that thrives on iteration and feedback.
Jim Meyer, our VP of Engineering, explains: “Every interaction with the AI is an opportunity to improve it. Users teach the system what works and what doesn’t, and that feedback loop makes the AI better over time.”
By starting with a narrow use case and expanding gradually, Yotascale was able to build trust and refine its Gen AI assistant to deliver smarter, more tailored results.
Lesson 3: Collaboration Drives Smarter Solutions
Keeping humans in the loop isn’t just about ensuring accuracy—it’s about building a partnership between people and AI.
“Our AI is a guide,” Jeff notes. “It suggests tag groupings or answers cloud cost questions, but it’s the user’s input that makes the results truly valuable. Collaboration is what turns a good system into a great one.”
By framing AI as an assistive partner, you not only empower users but also build trust that strengthens adoption and long-term success.
Lesson 4: Manage Cloud Costs with Eyes Wide Open
AI can feel magical, but the costs of implementation are very real—and they can escalate quickly if you’re not careful.
Jeff Harris, our Director of Strategy and Operations, recalls a case of unexpected sticker shock: “We activated a new feature—the live voice conversation preview—without fully checking on the cost. The functionality was exciting, but the price tag was a real wake-up call. It taught us to approach every new AI feature with caution.”
The lesson here is clear: test responsibly. Before trying out the latest AI bells and whistles, make sure you understand how they’re priced and how those costs might scale in production.
At Yotascale, we’ve focused on tracking and optimizing cloud costs early to avoid surprises. Simple steps like monitoring token usage, choosing cost-effective models, and refining prompts have made a big difference in keeping cloud costs manageable while delivering real value to users.
Lesson 5: Plan for the Unexpected
No matter how carefully you plan, surprises are inevitable. Whether it’s the discovery of an unforeseen cost driver or realizing a feature isn’t performing the way you expected, Gen AI projects are full of learning moments.
At Yotascale, we encountered surprises at nearly every stage of development. Jim Meyer, our VP of Engineering, reflects: “Early on, we realized that AI models don’t handle numbers as well as you might expect. It wasn’t just a matter of teaching the system what to do—it was about refining our prompts and data structure to work around those limitations.”
One unexpected challenge was balancing speed with quality during feature rollouts. Jeff Harris recalls: “We wanted to move fast, but some early iterations missed the mark because we hadn’t accounted for edge cases. We learned to launch with clear guardrails—starting small, getting feedback, and iterating quickly.”
Here’s how to stay adaptable when surprises arise:
- Expect the Unexpected: Build extra time into your roadmap for troubleshooting and refinement.
- Start with Small Experiments: Instead of launching a full-scale feature, test with a narrow use case to uncover potential issues early.
- Lean on Feedback Loops: Treat every user interaction as an opportunity to learn and improve. User feedback is your best guide for identifying and resolving unexpected challenges.
As Jim puts it: “AI is an evolving field. The best models we have today will feel outdated a year from now. The key is to stay flexible, keep learning, and adapt as the technology improves.”
Planning for the unexpected doesn’t mean avoiding risk—it means being ready to embrace it with an open mind and a problem-solving approach.
Looking Ahead: The Future of Gen AI
The lessons we’ve shared here aren’t the end of the story—they’re just the beginning. Gen AI continues to evolve, offering new opportunities to drive innovation, enhance collaboration, and deliver value to users.
As Jeff sums it up: “Gen AI isn’t just a tool—it’s a journey. The organizations that succeed with it will be the ones that embrace its challenges and keep finding new ways to make it work for their people.”
At Yotascale, we’re excited to be part of that journey and can’t wait to see where it leads next.
