Ram Cherukuri

AWS continues to add capabilities to Amazon Bedrock that make it the easiest way to build high quality, low latency, and cost-effective GenAI applications. Today the team announced:   - Custom Model Import, which lets customers easily bring their own models (from SageMaker and S3) to Bedrock to leverage its leading GenAI features, such as RAG, agents, Knowledge Bases (guardrails), inference deployment, and advanced customization to keep teaching and refining models.   - Model Evaluation, which lets customers quickly select predefined evaluation criteria (like accuracy and robustness) and run reports that help them understand which is the best model for them.   - Meta’s new Llama 3 models, now available on Bedrock and can support a wide range of use cases including text summarization and classification, sentiment analysis, language translation, and code generation.   - New Cohere Models, Command R and R+, are state-of-the-art FMs customers can use to build enterprise-grade generative AI applications with advanced RAG capabilities, in 10 languages, to support their global business operations. These will arrive on Bedrock in the coming weeks.   - New Amazon Titan models with Amazon Titan Image Generator (GA today) helping customers produce studio-quality images, at low cost, using natural language prompts (and applies an invisible watermark to help reduce the spread of disinformation)… along with Amazon Titan Text Embeddings V2 model, which reduces storage and compute costs, while out-performing other leading models on text embeddings.   Tens of thousands of customers are already building real-world production-ready GenAI apps and we're excited for even more customers to get started. https://lnkd.in/g4rkak47

2,851

53 Comments

When we started AWS, we looked at the undesirable situation of developers spending 70% of their time on the undifferentiated muck of managing infrastructure (vs only 30% on what differentiated their customer experiences) and aimed to flip that equation on its head. It’s made a pretty big difference in innovation and developer productivity.   Roughly 18 years later, we’re trying to solve another 70-30-like misalignment for developers, where developers are telling us that they’re spending roughly 70% of their time on repetitive and tedious tasks and code.   Today, we’re launching the general availability of Amazon Q, the world’s most capable GenAI-powered assistant for accelerating software development and leveraging companies’ internal data—and that aims to remove much of the muck of repetitive and tedious coding and data tasks for developers and employees at large.   On the software development side, Q doesn’t just generate code, it also tests code, debugs coding conflicts, and transforms code from one form to another (today, developers can save months using Q to move from older versions of Java to newer, more secure and capable ones; in the near future, Q will help developers transform their .net code as well). Q Agents does multi-step planning and reasoning to allow developers to string together multiple requests and have Q implement them.   On the internal data side, most companies have voluminous troves of internally relevant data that resides in wikis, intranet pages, Salesforce, storage repositories like Amazon S3, and a bevy of other data stores and SaaS apps that are hard to access. It makes answering straightforward questions about company policies, products, business results, code, people, and many other topics hard and frustrating. Q makes this much simpler. You can point Q at all of your enterprise data repositories, and it’ll search all this data, summarize logically, analyze trends, and engage in dialog with customers about it.   We also introduced today a powerful new capability called Q Apps, which lets employees describe, in natural language, what apps they want to build on top of this internal data, and Q Apps will quickly generate that app. This is going to make it *so* much easier for internal teams to build useful apps from their own data.   Customers are gravitating to Q, and we already see companies like Brightcove, British Telecom, Datadog, GitLab, GoDaddy, National Australia Bank, NCS, Netsmart, Slalom, Smartsheet, Sun Life, Tata Consultancy Services, Toyota, and Wiz using Q. And, we’ve only been in beta til today.   Very excited about how Q will change what’s possible for our customers, and being a part of helping them innovate more quickly.

9,803

173 Comments

A couple of recent college graduates from India against AI finetuning giants like Anyscale and Run:ai? I met Arko C at a NASSCOM event last October. He was building Xylem AI (YC S24) , a platform that allows you to train and deploy LLMs in production. So naturally most Indian VCs rejected him. Because how could a team of three recent college grads compete with the likes SV startups like Anyscale that have raised hundreds of millions of dollars, have teams of experienced AI researchers and a stash of H100 GPUs? Arko hustled and closed two Fortune 500 companies for paid contracts. I personally know unicorns that haven't managed to do this. He then got into YC in the last round in May. There's a lot to still be proved out, but Arko exemplifies how valuable grit is in building startups. I’m doing a webinar with him tomorrow on challenges of using LLMs in production. Link to register in the comments. Disclaimer: I'm a small investor in Xylem.

153

5 Comments

Exciting New Capabilities for Amazon Bedrock! 🚀 Amazon Bedrock is growing rapidly, with tens of thousands of customers across industries choosing it as their generative AI foundation. Key updates: ✨ Custom Model Import lets you easily bring your own proprietary models to Bedrock and leverage its powerful capabilities. ✨ Model Evaluation helps you quickly analyze and compare models to find the best fit for your use case. ✨ Guardrails provides best-in-class technology to implement safeguards aligned with your responsible AI policies. ✨ Plus, new models like Amazon Titan Text Embeddings V2 for efficient retrieval augmented generation, and latest models from Cohere (Command R) and Meta (Llama 3) Exciting time for generative AI innovation on AWS! Also, not mentioned here but equally important: plenty of silicon to choose from to run those models: from new Nvidia chips to Amazon's own 2nd gen high perfomance low cost Inferentia and Trainium chips!! (Oh, and it's great to see our Travel and Hospitality customers mentioned in this PR: Choice Hotels International, Delta Air Lines, United Airlines, TUI ) Check out the details at: https://lnkd.in/gJHGQRYV Patricia Yates Jason Bennett Troy Barker Vasile Balan Davor Golac Steven M. Elinson Greg Land Clare Ward Massimo Morin Yogesh Bendre Abhijeet Patil Keith Papsin Ravi Kumar

36

Data marts, warehouses, lakes, whatever the name, the pattern of consolidating data across an enterprise for comprehensive analytics is a familiar one. Every enterprise has many data sources, some operational, some analytics, some telemetry, and bringing them together has always been a challenge. Cleaning the data, curating for consistency, exploring, doing data science, building reports, monitoring, is another series of challenges in bringing a range of tools to bear on the data, thanks to diverse data formats and tooling. With Gen AI, the requirements (e.g., vector indexing for RAG) and potential value of the data have both gone up. And the pressure to ensure right use of the data, to comply to a broad range of regulations, to secure the data has never been higher. We believe the time has come to come together around open data standards to ensure that data is easy to consolidate and easy to plug into a wide variety of tools, all in a properly governed way. Josh and I describe the approach we are taking at Microsoft with Fabric and Purview. We are building on open Parquet-based updatable table formats such as Delta, Iceberg and Hudi, and interop across them with XTable. We are making it easy to consolidate data with mechanisms like mirroring and shortcuts. Ultimately, users should be able to use their tool of choice for a given task with the least friction.

246

11 Comments

From where I sit in the SF Bay Area, cradle of innovation for at least the last 30 years, I'm a true believer in the promise of the #AI revolution. That said, I find it fascinating to consider views on the opposing side. I'm sharing below two opinion pieces from the last two weeks suggesting that the AI revolution is fundamentally overhyped and a bubble ripe for bursting, and that AI tools are actually not useful or transformative. Contrast this with venture capitalist Kai-Fu Lee's prognostication that 50% of white collar jobs will be displaced by #AI within three years, and that OpenAI and the other cutting edge movers in the space are headed to a trillion+ dollar valuation. Which side will win out? Where will we be by 2030 in the AI race? I'd love to hear your thoughts in the comments.

7

3 Comments

“Google was built for this moment. We’ve been pioneering GPUs for more than a decade,” Google CEO Sundar Pichai Introducing Trillium, Google's sixth-generation Tensor Processing Unit (TPU). Trillium was specifically designed to accelerate the development and deployment of advanced AI models. With significant improvements in compute power, memory, and networking capabilities Key advancements in Trillium include: ➡ Enhanced compute performance: 4.7X increase per chip, enabling faster model training and serving. ➡Increased memory and bandwidth: Doubled High Bandwidth Memory (HBM) capacity and bandwidth, along with Interchip Interconnect (ICI) bandwidth, for handling larger models and datasets. ➡SparseCore accelerator: third-generation SparseCore, Specialized for processing ultra-large embeddings common in advanced AI workloads. ➡Sustainability: 67% more energy-efficient than the previous generation. 👉 👉👉👉 https://lnkd.in/dyASZGWy #AI #ML

2

1 Comment

So much fun talking about the power of LLMs with Amazon Bedrock and Anthropic Claude. In summary, the ever-evolving landscape of GenAI, it's becoming clear that there won't be a single LLM (Large Language Model) to rule them all. The rise of application-specific LLMs heralds a new era, where various models will carve out niches to excel in. This diversity opens up a wide playing field for innovation and tailored solutions. When it comes to deploying AI models, striking the right balance in evaluation is paramount. We must weigh factors like cost, speed, and precision to ensure optimal performance and resource utilization. Finding this equilibrium ensures that our GenAI solutions are not just efficient but also effective in real-world scenarios. Anthropic Claude emerges as a frontrunner in the realm of LLMs, distinguished by its exceptional capabilities and groundbreaking advancements. Boasting unparalleled scale, efficiency, and adaptability, Anthropic Claude represents a pinnacle of GenAI innovation. Its ability to seamlessly integrate with Amazon Bedrock amplifies its prowess, providing users with an unparalleled ecosystem for knowledge integration and generation. With state-of-the-art architecture and a commitment to pushing the boundaries of what's possible, Anthropic Claude stands as a testament to the transformative potential of GenAI in shaping the future of technology and beyond. Prompt engineering holds the key to maximizing the potential of applications. Crafting prompts that consider persona, context, tags, and positioning can significantly enhance the model's ability to comprehend and generate relevant responses. This strategic approach empowers AI systems to better serve their intended purposes. In the realm of GenAI architectures, a robust knowledge base is essential. Amazon Bedrock stands out as a key component for architectures like RAG (Retrieval-Augmented Generation), providing a solid foundation to fuel intelligent decision-making and content generation. As we march towards AI integration, bridging the gap with legacy applications is crucial. Agents equipped with open APIs offer a seamless way to facilitate communication and interoperability, ensuring a smooth transition into the AI-driven future. Let's continue to explore, innovate, and collaborate as we navigate the exciting frontier of GenAI together! #aws #LanguageModels #Innovation #TechTrends #AIIntegration #webdevcon #claude #bedrock

68

Summary of keynote from Google Cloud Next 24. Gives a glimpse of enhancements centered around AI as a theme in Google Cloud, Vertex AI and Gemini..... https://lnkd.in/grbfrf9H #google #googlecloud #gemini #ai #vertexai

3

Graviton is a super efficient (and green) way to manage a lot of the heavy lifting around GenAI and AI/ML... both in prepping the training data and doing the inference.

13

Google engineering levels fun tidbit I read: To become a Google Fellow (Level 10) is to win an honor that will follow you for life. That said, there are 2 Senior Fellows (Level 11, equivalent to an SVP): Sanjay Ghemawat and Jeff Dean. Google’s engineers exist in a Great Chain of Being that begins at Level 1. At the bottom are the I.T. support staff. Level 2s are fresh out of college. Level 3s often have master’s degrees. Getting to Level 4 takes several years, or a Ph.D. Most progression stops at Level 5. Level 6 engineers (top 10%) are so capable that they could be said to be the reason a project succeeds. Level 7s are Level 6s with a long track record. Principal Engineers, the Level 8s, are associated with a major product or piece of infrastructure. Distinguished Engineers, the Level 9s, are spoken of with reverence. To become a Google Fellow, a Level 10, is to win an honor that will follow you for life. Google Fellows are usually the world’s leading experts in their fields. Jeff and Sanjay are Google Senior Fellows—the company’s first and only Level 11s.

165

22 Comments

Weekend update: TrialSight AI is now Zeeq AI (rhymes with "seek")! Zeeq (https://zeeq.ai) is an #AI agent inspired by the Clinical Trials Transformation Initiative (CTTI) AACT project that watches FDA ClinicalTrials.gov for the latest updates and emails you a personalized newsletter that answers your questions for trials that match your criteria. Whether you are a patient, researcher, pharma executive, or industry analyst, Zeeq helps you stay on top of the latest developments any time a change is added to ClinicalTrials.gov. The agent is now even faster (see for yourself) and still free to sign up (no usernames or login required). I'm getting close to go-live. The monitoring logic is ready and testing the email generation this morning. Still need to add a passwordless/accountless way to manage the subscriptions.

10

1 Comment

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

14

3 Comments

🌐 🤖 More choices for enterprises: We have recently integrated Anthropic's Claude 3.5 and Sonnet on Vertex AI, providing enterprises with more choices and flexibility in deploying advanced AI models. These tools offer improved performance and scalability, making them ideal for various applications. By leveraging Claude 3.5 and Sonnet, businesses can enhance their AI capabilities and drive innovation. This update supports more robust and efficient AI workflows. https://ow.ly/iQHt50SCfpx #VertexAI #AI #MachineLearning #EnterpriseTech #Innovation

1

Please join me on June 25th San Francisco, CA, USA for a panel titled "A BLUEPRINT FOR SCALABLE & RELIABLE ENTERPRISE AI/ML SYSTEMS" w/ moderation by Hira Dangol from Bank of America and my fellow panelists Rama Akkiraju from Nvidia and Nitin Aggarwal from Google. Recent breakthroughs in AI technology have captured the world's attention, and now it's time to address the challenge of creating truly robust and reliable AI systems. AIQCon's mission is to harness the power of our community to develop new GOLD STANDARDS for AI quality. The global movement is bringing together renowned leaders and professionals from across the industry to establish reliable AI quality solutions and standards. If you're working on AI systems, you won't want to miss this one! Use the discount code "Community" for a 15% discount on registration. Register at https://lnkd.in/gqRE48fC. #AI #EnterpriseAI #AIQuality #AIQCon https://lnkd.in/gqRE48fC

58

3 Comments

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

We had a full house in a fireside chat with Arvind Jain, Glean's CEO and co-founder, at Stanford University Graduate School of Business! It was an honor to talk with him about building successful companies in the dynamic #AI era. He shared invaluable lessons from his remarkable journey: building Google search infra, founding Rubrik (which went public last month!), and founding Glean, recently nominated by Forbes as one of the Top 50 AI companies! 🎯 Start with the problem, not the solution Arvind is very bullish on Glean's mission to solve app fragmentation and workplace knowledge. He saw his customers at Rubrik struggle with it and took the time to validate it. People tend to undermine the importance of a problem and get too excited about the solution. However, if the solution solves the wrong problem, users will not adopt it. 🤝 Validate hypotheses with people you don't know Friends want to support and might avoid critical feedback. Arvind and the team spent the first year of Glean sending cold LinkedIn reach-outs to potential customers to validate the problem and the solution. While he could easily leverage his network, he wanted unbiased feedback from people who didn't know him. 🎧 When it's noisy outside - stay focused on your customers. That's your moat. With every week's announcements of AI innovations, Glean's Northstar is to be customer-driven and leverage new technologies where there's a fit. Even though they are busy with the growth struggles of a Series D company, Glean teams are working on 70 % of core work : 30% of experimental projects, to stay ahead of the curve. 📈 Companies should plan how to embrace AI internally As a 2nd-time founder, Arvind is looking for new opportunities to use AI at Glean and improve #productivity, beyond using Glean Assistant. With so much innovation, customers are very excited about AI demos but get disappointed when they try to use new tools for their specific use cases. It will take a (short) while before we see a significant impact on productivity.

187

10 Comments

Learn how to secure and govern your LLMs with UAPIM so that you can safely implement these powerful AI and LLM models into your business.

1

In his letter to shareholders on April 11, 2024, Andy Jassy documented the company’s “primitive services” 🧩 approach to #innovation. Instead of full-form solutions, #AWS builds “discrete, foundational building blocks that builders can weave together in whatever combination they desire.” Every piece of innovation can contribute to several solutions across multiple permutations. 🧠 This is AWS' modular product philosophy, which ultimately determines your capability to partner with them. While helping them innovate faster, it also creates a flywheel 🎡 of sustained effort: ��️ Decentralized innovations are continuously stacked over existing solutions. This creates continuous #integration challenges. 💼 Partnering solutions must, therefore, keep up with changing compliance norms and guidance to find the best value out of their relationship with AWS 💪 Developer teams need constant re-education or capability growth to maintain integrations 🟰 Partnering with AWS is an equal relationship. AWS commits to providing the best facilities to all involved stakeholders and effecting small changes throughout the process. In turn, partners and builders are expected to stay ahead of the innovation curve (even in the marketplace). This is also how the most successful ISVs on the AWS #marketplace operate. 🏪 Recognize that listing is a journey Instead of treating their listing as a milestone, a successful Cloud GTM recognizes it as their public storefront inside an environment co-owned by the cloud provider. They understand the effort required to constantly optimize their presence and update their listing in line with changes in their public pricing, packaging, or business goals. 💻 Obsess with maintenance for backend integrity To stay ahead of the market as a software supplier and keep up with the dynamicity of AWS as a platform, businesses need to constantly map changing guidance to their internal systems of truth and third-party GTM tools for visibility across the revenue pipeline. This requires either sharp vendor choices or onerous dev/SalesOps bandwidth to maintain. 💬 Invest sustained effort in maintaining relationships with hyperscaler reps High performers invest in strategy, but true #CloudGTM champions also invest in relationships. By working with AWS reps, businesses can discover new alignments, co-marketing, and co-selling opportunities. This isn’t as easy as just identifying the right AWS sales rep for every opportunity. Co-selling success requires an ISV to build its own ‘right to win’ as a solution that multiplies in value when positioned ‘alongside’ AWS. All of it must ultimately influence your innovation, operational, and tooling choices. These choices must help you innovate quickly and capture the impact of said innovation programmatically. This is how you leverage AWS’s trajectory as a constantly growing cloud solution. What's helped you remain agile? I’d love to know.

21