Alberto Medina

Large Language Models (#LLMs) are evolving rapidly, and the demand for running them at scale is skyrocketing. #Ollama is stepping up to meet this need with its official container image on Docker Hub, excelling in local #deployments. However, there’s a catch for containerized environments: downloading #models requires the ollama serve command to be running within the #container. This extra step complicates automated deployments via #CI #CD #pipelines, making seamless model access challenging. To take your #model to production, it needs to be #container-ready—a feature Ollama has yet to fully support. The gap in #AIOps for smooth deployments gets blurred with the inference services available as an easy way to take it to production. Goal is to overcome obstacles and deliver solutions 😉 so here is to get you started on having your (O)#llama #production ready in a private #deployment from your #local to #cloud all in #oneshot . Enjoy Medium: https://lnkd.in/g9jqBQ8d dev.to: https://lnkd.in/gi_gGwam #AI #LLM #Ollama #llama3 #phi3 #mistral #S6overlay #AIOps #CI #CD #oneshot #container #docker #podmanEnjoy

11

sharpening writing skill is one of killer skill!

2

This is the initial infrastructure Tradescoop was built on. A few things have changed, some additions and removals; for example, we’ve gone from MongoDB on Digital Ocean to Mongo Atlas, Redis cloud to Memorystore, but the general appraoch remains. Simplicity. Shouts to Karl Obinna Amalu, CSM for cleaning it up and adding color ✨ #gcp #infrastructure #devops

19

1 Comment

So much fun talking about the power of LLMs with Amazon Bedrock and Anthropic Claude. In summary, the ever-evolving landscape of GenAI, it's becoming clear that there won't be a single LLM (Large Language Model) to rule them all. The rise of application-specific LLMs heralds a new era, where various models will carve out niches to excel in. This diversity opens up a wide playing field for innovation and tailored solutions. When it comes to deploying AI models, striking the right balance in evaluation is paramount. We must weigh factors like cost, speed, and precision to ensure optimal performance and resource utilization. Finding this equilibrium ensures that our GenAI solutions are not just efficient but also effective in real-world scenarios. Anthropic Claude emerges as a frontrunner in the realm of LLMs, distinguished by its exceptional capabilities and groundbreaking advancements. Boasting unparalleled scale, efficiency, and adaptability, Anthropic Claude represents a pinnacle of GenAI innovation. Its ability to seamlessly integrate with Amazon Bedrock amplifies its prowess, providing users with an unparalleled ecosystem for knowledge integration and generation. With state-of-the-art architecture and a commitment to pushing the boundaries of what's possible, Anthropic Claude stands as a testament to the transformative potential of GenAI in shaping the future of technology and beyond. Prompt engineering holds the key to maximizing the potential of applications. Crafting prompts that consider persona, context, tags, and positioning can significantly enhance the model's ability to comprehend and generate relevant responses. This strategic approach empowers AI systems to better serve their intended purposes. In the realm of GenAI architectures, a robust knowledge base is essential. Amazon Bedrock stands out as a key component for architectures like RAG (Retrieval-Augmented Generation), providing a solid foundation to fuel intelligent decision-making and content generation. As we march towards AI integration, bridging the gap with legacy applications is crucial. Agents equipped with open APIs offer a seamless way to facilitate communication and interoperability, ensuring a smooth transition into the AI-driven future. Let's continue to explore, innovate, and collaborate as we navigate the exciting frontier of GenAI together! #aws #LanguageModels #Innovation #TechTrends #AIIntegration #webdevcon #claude #bedrock

68

Considering the rapid advancements in Large Language Models (LLMs), it's becoming clear that for 95% of companies and domains, the underlying LLMs will soon be a commodity provided by major players. The real play for these companies will be at a higher level, focusing on building neural agents and advanced neural planning. Imagine this: planning what needs to be done by neural agents will itself leverage LLMs, becoming your unique edge. For most, fine-tuning an LLM might be necessary, but the true focus should be on higher layers. Think about this: If ChatGPT, Gemini, and Claude are all trained on relatively the same data from the internet, why do they understand and respond to your queries differently? It's their planning skills that give them superpowers. Neural planning will be the research focus for years to come, until it too becomes a commodity offered by the big players.

34

2 Comments

In the era of LLMs, "sexy" might not be the term, but bag-of-words solutions are still kicking! When the position of a word matters, consider using Positional Vectorizer over TFIDF. Check it out: https://lnkd.in/dDRYFrAX

14

1 Comment

Want to figure out the best way to leverage your data with LLMs? Check out this guide to get started!

2

Signal boosting this post from my colleague Arun Shankar. My snarky summary "It's like caching, but semantic" :P Oddly, I think it's a reasonable summary... semantically similar inputs hit the same cache, otherwise dissimilar input miss.... ... but how do you do that on non-exact matches and how to you manage such a system? I guess you'll have to read: https://lnkd.in/gPgtM8T5

13

1 Comment

One of the challenges of incorporating AI into systems is that in some respects it's very similar to traditional software development, and in other respects it is not. Our software supply chain teams have been working for some time on how to translate software supply chain principles and practices to fit the needs of the AI supply chain. I'm very proud of the work our team and partners have done here. This report describes the key insights they have made. #ai #hardening

20

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

14

3 Comments

This overview of Generative AI and Retrieval-Augmented Generation by Gwen (Chen) Shapira of Nile Database is fantastic. Highly recommend it for other infrastructure nerds like me who want to know what's going on in the AI world. #generativeai #rag https://lnkd.in/gJum7_qa

9

Meta is facing BIG restructuring toward innovation and growth. CTO, Andrew “Boz” Bosworth, has unveiled the most extensive restructuring of Reality Labs to date, splitting our hardware division into two focused groups: Metaverse and Wearables. 🔷 Metaverse: This team now includes our cutting-edge Quest headset line and is dedicated to pushing the boundaries of immersive experiences. 🔷 Wearables: Encompassing all other hardware efforts, including the innovative smart glasses collaboration with Ray-Ban. This strategic move aims to streamline and sharpen our focus on delivering groundbreaking products in these two domains.  #Meta #Innovation #Wearables #Metaverse #TechTransformation https://lnkd.in/gQ5RdydW

5

How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out. https://lnkd.in/dwczVutW

43

5 Comments

Truth. This blog articulates the shortcomings of most tiered storage solutions in delivering predictable performance at scale. Without some rebuild that harnesses cloud primitives like AWS S3 and GCP GCS, mere incremental adjustments to existing products won't adequately serve users. From a product manager's perspective, sometimes incremental adjustments isn't a path to sustainable product adoption and success.

9

How do you #SRE? take this year’s Catchpoint survey and contribute to better tools and practices for reliability engineers everywhere

1

I wrote some thoughts and observations about Google laying off their Python support team. You can read them if you are so inclined. Blog post here: https://lnkd.in/gMFhuZ76

1

If Google decided to part with the core Go team, what would that mean for its future adoption? The other day, I read on HN that Google took the decision to lay off (or restructure/demote, you name it) its internal Python team. This got me to speculate what would happen, if the company theoretically decided to focus on something other than being Go's core custodian. Of course, Google has a much bigger stake in Go (and respectively, a lot more to lose from such an event), plus Go is open-source, so the community will certainly keep the momentum going for a while. With Google being Google, and usually making decisions in favor of its core businesses, how would you feel if it decided to shift the Go team to doing something else? Would you still trust the Go team's long-term promise? With a strong upper hand, how would one prevent a myriad of features making it into the language without control? https://lnkd.in/e2CxbXH4 #google #golang #python #programming