Alberto Barrera Jimenez

During my three years as a Software Developer at Amazon, I experienced firsthand how deeply ingrained the Operational Excellence (OE) culture is in our daily operations. This culture isn't just about maintaining high standards—it's about continuous improvement and proactive problem-solving. Here are some intriguing insights from my time at Amazon: 🔍 Weekly OE Meetings: Every Monday morning, our team gathered to review key performance metrics, including service availability and response times. These meetings were crucial for identifying trends and swiftly addressing any issues from the previous week. It was fascinating to see how structured these discussions were, ensuring that everyone was aligned and focused on improvement. ⏲️ On-Call Responsibilities: One unique aspect of Amazon’s OE culture is the rotating on-call duty. Each week, a different software engineer would be the primary responder for high-severity issues, using tools like PagerDuty to manage alerts. This rotation not only ensured that issues were addressed promptly but also fostered a sense of shared responsibility among the team. 📊 Key Metrics: We closely monitored critical metrics such as the 99th and 99.9th percentiles of service response times and aimed for zero error rates. Tracking both system errors and dependency errors helped us pinpoint problems accurately and resolve them efficiently. The emphasis on these metrics underscored Amazon's commitment to reliability and performance. 🛠️ Root Cause Analysis (RCA): Conducting RCAs for every significant issue was a game-changer. It was not just about fixing the problem at hand but understanding the underlying cause to prevent future occurrences. This process was essential for maintaining long-term stability and continuous improvement. 💡 Continuous Improvement: One of the most inspiring aspects of Amazon’s OE culture is the constant drive for improvement. Feedback was encouraged from all team members, and processes were regularly updated based on lessons learned. This iterative approach ensured that we were always moving forward, adapting to new challenges, and enhancing our operations. I've detailed these insights and more in an article I recently wrote on Substack. If you're interested in learning more about Amazon’s OE practices and how you can implement similar strategies in your own organisation, check out the full article(first link in the comments). Connect on LinkedIn: Dipak A.

28

1 Comment

Samsara's latest report highlights how industry leaders are leveraging AI to boost efficiency and safety.  Dive into the full coverage to see how AI is transforming the way we operate:

2

Amazon Bedrock Studio 😎. "With Bedrock Studio, you - and your teams - can sow a million different generative AI ideas, care and feed them, and watch the beautiful results bloom in-front of your eyes. " #aws #artificialintelligence #genai

1

Simy log in to the Amazon Bedrock Studio web experience using company credentials (SSO) and start experimenting with Bedrock FMs and tools to build applications. With Bedrock Studio, developers get a secure environment outside of the AWS Management Console to leverage Bedrock tools such as Agents, Knowledge Bases, and Guardrails.

8

Our customers can now create Route53 Alias records for their Amazon OpenSearch custom endpoints and gain faster DNS resolutions while saving on additional CNAME costs. Good job on the launch Sachet Alva and team - https://lnkd.in/g_aaUp7S. #aws #opensearch #route53

48

Our SIGMOD '24 paper explores MemoryDB's architecture, which delivers in-memory performance and strong consistency at scale. A key innovation is the decoupling of durability and replication from the in-memory database engine, which allows scaling compute and storage independently based on customer workload needs. Customer data integrity, security, and availability are paramount for MemoryDB. The paper dives deep into innovations that the team built to make that possible (a personal favorite is off-box snapshotting which allows us to create consistent database snapshots on separate clusters to avoid impacting live customer traffic and preventing resource contention). We also share hardened operational experiences from running a distributed database at cloud scale. #aws #memorydb #inmemory #databases #durability #sigmod

91

1 Comment

ICYMI: Thoughtworks was at Product-Led Summit NYC! Attendees heard from Joel Zien, Head of Strategic Growth at Thoughtworks as a speaker and the chair, and Carl Nygard, Chief Technology Officer at Thoughtworks, wrapped up the conference with a Generative AI panel. In Joel’s fireside chat with Matt Skavenski, SVP Engineering at Peloton Interactive, they explored why 'hardware is so hard', and drew from years of experience at the crossroads of hardware and software. Curious about some of the universal lessons in product innovation when it comes to software and hardware? Keep reading! Product Marketing Alliance #ProductLedSummit #ProductThinking #BuildBetterProducts

30

Exciting recap from our time at Product-Led Summit NYC! Check out some of the fireside chat takeaways below. ⬇ #ProductLedSummit #ProductThinking

2

<rant> Yesterday morning I wrote a post about the phrase "scaling independently" while waiting for my KIND cluster to spin up, and I promised that I would give an example of where you really DO want things to "scale independently" in a separate post. Well, that post is here. Let's talk about #apachekafka and its new Raft-based metadata solution, known as KRaft (yes, it evokes Mac-and-Cheese, which is partially why I like it so much). KRaft is a replacement for ZooKeeper's former role in Kafka. In a KRaft-based cluster, the Metadata Quorum is a group of Kafka Servers that have the "controller" role. The Metadata Quorum stores information such as "what topics exist in my cluster?" and "which Broker is the leader for which Partition?" and "which follower replicas live on which Broker?". This information is CRITICAL for the availability and consistency of a Kafka cluster. In order to ensure consistency of the Metadata Quorum, one specific Controller is chosen as the leader. All write requests go through that leader. In order to ensure availability of the Metadata Quorum, there can be "follower" Controllers in the Metadata Quorum which also store the metadata updates (synchronous replication). In case of failure of the leader, one of the followers can become the leader. In this specific case, it DOES make sense to scale the Brokers and Controllers independently. Why? First, the Controllers don't actually scale. One can be a leader at a time, so the others are just providing backup (note: for reasons beyond the scope of this post, it's best to have an odd number). Additionally, before KIP-853 is implemented, it's actually really hard to add or remove Controllers to/from the Metadata Quorum without downtime. Secondly, it's possible for a misbehaving client to take down a Broker, for example by sending way too much data to a specific partition. If it's just a single Broker that's lost, most of the cluster will continue on and live to fight another day. However, losing a Controller is a Very Bad Thing. Thus, by separating the Controllers from the Brokers, we can improve the availability of the cluster. It is indeed possible to run Kafka with some servers that share the responsibility of Controller and Broker. This is great for development (especially local dev) and also in *SOME* highly resource-constrained production environments. However, I would suggest as a rule of thumb that you should probably: - Separate out your Controllers on their own isolated machines - Start with a Metadata Quorum of size 3, which allows losing one Controller and continuing on alive - Put your Brokers on another set of nodes. Now, there's another good example of how Kafka scales X and Y independently: Compute and Storage. Watch out for the next "Colt Rant" about this, coming later this week! </rant> PS—here's the link to the previous post: https://lnkd.in/gXU77rKW

12

6 Comments

We’d like to reintroduce ourselves: We’re Amazon Web Services (AWS) Amplify. Build a cloud-powered backend in TypeScript Connect it to your frontend using your favorite framework Deploy on every `git push`. Amplify Gen 2 is now ready for production! https://go.aws/3Qudgx6 Amplify is a fullstack TypeScript experience for building cloud-connected apps. AWS Amplify helps you accomplish two jobs: - Host your web app - Build and connect to a cloud backend With Amplify Gen 2, every part of your app’s cloud backend is defined in TypeScript. Need an Auth backend? TypeScript. Data backend? TypeScript. Storage backend? TypeScript. Everything is defined in TypeScript. What’s not changing? Amplify is built by and on AWS, giving you the ability to add any of the 200+ AWS services when you need to. Including generative AI services such as Amazon Bedrock? You guessed it: TypeScript.

47

3 Comments

🚀 Exciting News Db2 Users! 🚀 ⌚ Amazon Web Services (AWS) has introduced on-demand Db2 licenses through AWS Marketplace! 🕺 This innovative solution enables Db2 users to procure database licenses for RDS Db2 workloads with ease. #AWS #AmazonRDS #DB2 #DatabaseManagement #Innovation #CloudComputing #TechNews #AWSUpdates

9

🚀 Amazon Q, the most capable generative AI-powered assistant designed for accelerating software development and leveraging company knowledge and data is now generally available on AWS. With Amazon Q, employees can get answers to questions across their business, such as company policies, product information, business results, code base, employees, and many other topics by connecting to enterprise data repositories to summarize the data logically, analyze trends, and engage in dialog about the data. https://lnkd.in/ghtA_rWQ .

3

We just completed another Metaverse project (webGL + Web3), which incorporates AI-based terraforming (as far as I know, first of it’s kind). Users will be able to create whatever they want on their land (based on amount of tokens they stake) by simply using text prompts. A few tech highlights: * WebGL platform that is mobile friendly (as long as your device has a browser, you should be able to join) * Web3 wallet system with support for both non-custodial (Metamask, WalletConnect) and custodial wallets (email, phone number, Google) * NFT minting contract/process for land ownership * AI-based system for generating 3D models * A very intuitive UI (built into the whole experience) for placing/rotating/scaling generated models * Text and voice chat support https://lnkd.in/eqK_7Rji

56

2 Comments

🧑‍💻🤖 We're rolling out our next-generation custom chips, designed to supercharge AI across Facebook and Instagram. These enhancements mean faster and more accurate ranking and recommendations, improving your experience every day. Dive into how our latest AI innovations are powering better, smarter interactions. #Meta #AI

5

Now we have two first class data catalogs integrated with MS Fabric now available in Azure. Of course DataBricks is the heavy heavy favorite in the enterprise lakehouses as well as AI (as of the conference that just finished 🤓)

6

Retool's newest State of AI report is here! See what 700+ developers and tech leaders said about their AI stacks, use cases, challenges, and more as AI adoption accelerates:

5