[9:15] So many years ago we developed our own switch gear control system. We call it AMCOP. ... Today we use dozens of different makes and models of switchgear from several partners but they’re all controlled by AMCOP. And this means that we can operate our global data center exactly the same way everywhere.
[13:00] DeSantis: Rather than using a big third-party UPS, we now use small battery packs and custom power supplies that we integrate into every rack. You can think about this as a micro-UPS, but it’s far less complicated. And because we designed it ourselves, we know everything about it and control all the pieces of the software. And as we discussed earlier, this allows us to eliminate complexity from features we don’t need and we can iterate at Amazon speed to improve the design.
[35:10] DeSantis: Now we use that Nitro chip I just talked about to create specialized hardware which we call the Nitro controller. We use the Nitro controller to turn any server into an EC2 instance. The Nitro controller runs all the code that we use to manage and secure the EC2 instance and virtualize and secure our network and storage. And by running on the Nitro controller rather than on the server, we’re able to improve customer instance performance, increase security, and innovate more quickly.
[39:35] We’ve been delighted about results customers have achieved migrating their large scale inference workloads to Inferentia. Amazon Alexa recently moved their inference workload from Nvidia GPU-based hardware to Inferentia-based EC2 instances and reduced costs by 30% while achieving 25% improvement in their end to end latency.
[46:55] There are also security concerns with SMT. SMT is the main vector that researchers have focused on for so-called side channel attacks. These attacks try to use SMT to inappropriately share and access information from one thread to another. Now we don’t share threads from the same processor core across multiple customers with EC2 to ensure customers are never exposed to these potential SMT side channel attacks.
