This week at a glance
The AI SRE market moved from wrapping general models to training its own. Datadog previewed Scout, a model post-trained with reinforcement learning on simulated incidents, and says it matches leading frontier models on its own benchmark at three to eight times lower cost. The tests are Datadog’s, and Scout only proposes fixes for now. The same week, Dynatrace and Datadog each announced a Leader placement in Forrester’s AIOps Platforms Wave, and Serval raised $150 million at a $3 billion valuation to take on ServiceNow with agent-built automations. Agents are also supervising each other. Virtana lets other vendors’ agents join an incident over MCP or A2A, under token and approval limits, and Monte Carlo’s Bayes finds deployed agents, diagnoses failures and fixes them.
The control question is unresolved. In a theCUBE Research study, none of the 19 leaders with agents in production let them take high-risk actions without a person, but only four of 25 had tested an agent incident. Google’s Agent Compute Engine puts each agent in a microVM with no keys inside, and Akamai launched a WebAssembly agent sandbox. Agent volume is now a capacity problem: GitHub saw 7.4 billion commits in September, five times a year earlier, and has added four million CPU cores this year after outages. On the infrastructure side, Netflix showed how it keeps GPU job success above 99.5%, AWS let ECS tasks share a GPU, and Gartner warned that more than $3.5 trillion of AI infrastructure spending could face write-downs by 2030, with most neoclouds losing independence.
On our watch list
- An outside test of Scout. Every Scout result so far comes from Datadog, and AlertEval is Datadog’s own benchmark. Watch for third-party runs on SREGym or Grafana’s o11y-bench, a general availability date, and whether Bits is allowed to apply fixes rather than only propose them.
- More vendors training their own ops models. Scout is built on an open model, GLM-5.3 Flash, with a training reward for evidence as well as answers. Watch whether other observability vendors follow with small specialist models, and whether they publish prices against frontier-model agents.
- How buyers use the Forrester Wave. Dynatrace and Datadog both claim Leader in the same ten-vendor report. Watch which criteria each pushes in renewals, especially autonomous AI SRE and least-agency agent controls.
- Cross-vendor agents in real incidents. Virtana now accepts outside agents over MCP and A2A with token caps and logging. Watch for the first customer accounts of a mixed-vendor agent team, and for who is blamed when one makes a bad change.
- ACE in customers’ hands. Google says a customer preview is a few months away and will not price it until general availability. Watch whether AWS and Azure change their sandboxes, which by default let agents use credentials inside them.
- GitHub’s Azure migration and reliability. GitHub expects to finish moving to multi-region Azure early next year, with a compute-storage split it says gives up to 35 times the write throughput. Watch the status page for a quiet quarter before relying on it alone.
- Neocloud consolidation. Gartner expects more than 80% of neocloud providers to stop operating independently by 2030. Watch for acquisitions and capacity-contract renegotiations, and for whether newly funded inference clouds such as General Compute fill their planned capacity.
- GPU utilisation figures. 79% of VentureBeat survey respondents report GPU utilisation at or below 50%, while one enterprise VP told theCUBE that breaking even takes about 80%. Watch whether GPU-sharing features such as ECS MPS move those figures.
- Serval against ServiceNow. Serval says ARR has doubled every quarter since December and names ServiceNow as the platform customers are replacing. Watch for named switch-overs beyond IT, and for how the 100-engineer ProServal team scales.
- IBM’s automation consolidation. Infragraph is due to reach general availability in December, with shared billing and identity across IBM’s automation products in early 2027. Watch whether the “time travel” rollback ships after the Q1 release.
This week’s topic map. AI SRE and AIOps platforms sit at the upper right with Datadog, its Scout model, Dynatrace, the Forrester Wave, Serval and ServiceNow. Agents watching agents sit at the top with Virtana, MCP and A2A, and Monte Carlo, and human approval and theCUBE Research sit just above the AI Ops hub. GPU fleet reliability fills the left with Netflix, Kubernetes, AWS and Nvidia. Agent isolation and the AI infrastructure bill share the lower centre with Google, Agent Compute Engine, Akamai, Oracle, Gartner and neoclouds, and GitHub and New Relic hang off agents as traffic on the right.
View interactive topic map →
Article index
30 articles, grouped by sub-theme. Twenty-one are from this week’s coverage window (October 25 to November 1). Nine are longer foundational reads on the beat. Sponsored coverage, vendor blogs and contributed pieces are labelled within each group.
AI SRE and AIOps platforms
Observability vendors now train their own investigation models and compete on analyst rankings, while service-management challengers raise money to replace incumbents. The Dynatrace item is its own press release, and the Grafana item is a vendor blog. The Pragmatic Engineer deep dive is paywalled after the introduction, and its author discloses an investment in incident.io.
Agents that watch agents
Tools whose job is to supervise other agents: pull them into incidents, check their output, and score them against human judgment. Customer results quoted in these pieces come from the vendors.
Running agents safely: isolation, oversight, recovery
Where agents run, what they can reach, and who can stop them. The SiliconANGLE resilience piece is theCUBE coverage of a segment sponsored by Dell. The InfoWorld analysis is contributed, and the CNCF post is by two CNCF ambassadors.
When agents become the traffic
Agent volume is now a capacity problem for the platforms they use, and some providers are designing for agents as customers. The New Relic item is a vendor blog.
GPU fleet operations
Keeping GPUs healthy and shared, and measuring the service rather than the hardware. The New Stack inference piece is sponsored by HPE. The AWS and NVIDIA items are vendor-published.
The AI infrastructure bill
How much is being spent, how much sits idle, and who is betting on what. Utilisation figures in the VentureBeat piece are self-reported survey answers. theCUBE’s Breaking Analysis discloses that many companies it cites are its sponsors or clients.
Models, providers and portability
Longer reads on keeping agent systems independent of any one model provider. Both are vendor-affiliated: the InfoWorld author leads developer relations at Dremio, and the VentureBeat guest post is by a Confluent product executive.
Detailed write-ups
1. Datadog trains its own AI SRE model, and two vendors claim the AIOps Wave
InfoQ / Dynatrace / Grafana Labs · October 28–30, 2026
Datadog has released Scout, a model trained to investigate production problems, inside Bits Investigation, its AI SRE product. It is in closed preview with Datadog engineers and a small number of customers, with no general availability date. Scout starts from the open GLM-5.3 Flash model and was post-trained with reinforcement learning on incident simulations built from patterns in past cases. Datadog says the simulations contain no customer data. Early training rewarded only the final answer, and Scout learned to make fewer claims and stop investigating early, so Datadog changed the reward to credit correct answers, exploration and supporting evidence. “Its training rewards finding the cause and supporting its conclusions with evidence,” the Datadog team wrote.
On AlertEval, Datadog’s own benchmark, Scout at medium effort scored about the same as GPT-6.1 Sol and Claude Opus 5.5 at high effort, at an estimated three to eight times lower inference cost. On SREGym, a public set of 90 faults, it resolved 71 at medium, 79 at high and 80 at its highest setting, against 82 for GPT-6.1 Sol at its highest. Datadog ran all of these tests, and no prices are published. For now Scout investigates, diagnoses and proposes recovery steps; an engineer makes the change. Grafana Labs offers one way to check claims like these: its open o11y-bench reached version 1.1 with 64 tasks across PromQL, LogQL, TraceQL, multi-step investigations and dashboard work, runs each task three times, and verifies cited traces against Tempo. A no-tools control failed all 64.
The same week, Dynatrace and Datadog each announced a Leader placement in The Forrester Wave: AIOps Platforms, Q4 2026, which assessed ten providers on 24 criteria. Dynatrace says it had the highest Current Offering score and top marks in 16 criteria, including causal root cause analysis, autonomous AI SRE and orchestration, and algorithmic trust and explainability. Datadog says it scored top marks in 11, including Innovation, Roadmap and least-agency controls for agents. “Customers want to put AI to work across their operations, but they need answers they can trust before they act,” said Dynatrace chief product officer Steve Tack.
Sources: InfoQ (Datadog Scout) — https://www.infoq.com/news/2026/10/datadog-scout-sre-model/ · Dynatrace (Forrester Wave AIOps, press release) — https://ir.dynatrace.com/news-events/press-releases/detail/443/independent-research-firm-names-dynatrace-a-leader-in-aiops · Grafana Labs (o11y-bench v1.1, vendor blog) — https://grafana.com/blog/evaluate-ai-models-for-observability-o11y-bench-joins-fireworks-specialized-intelligence-index-and-reaches-v1-1/
2. Agents that supervise other agents
SiliconANGLE · October 28–29, 2026
Virtana now lets its AI agents bring other vendors’ agents into an IT incident, as long as they support the Model Context Protocol or Agent2Agent. Outside agents have been able to query more than 25,000 real-time metrics through Virtana’s MCP server since February, and a shared dependency graph shows every agent in the incident how applications map to infrastructure. In Virtana’s example, a third-party agent flags a slowdown and calls Virtana’s root-cause agent, which traces it to a database bottleneck after a deployment. The limits are set by the customer: autonomy applies only to incident types defined in advance, engineers can be required to carry out or approve each fix, policy decides which outside agents may join and how many tokens they can spend, and every action is logged.
Monte Carlo launched Bayes, an agent that finds the agents a company has deployed and the data that feeds them, watches for unreliable behaviour, works out whether the agent or its input data is at fault, applies a fix and checks the result. Customers choose how far it goes, from monitoring to background fixes. Snowflake is using it for customers’ Cortex agents, and Monte Carlo says a South American financial institution saw 200% more incidents detected and 80% fewer false positives, figures that come from the vendor. “There aren’t enough engineers to keep those agents trustworthy, but Bayes changes the math,” said chief executive Barr Moses. Crux Labs, which launched with more than $6 million, uses evaluator agents to apply expert judgment to agent traces, noting that many teams review only the top 1% or fewer by hand.
Sources: SiliconANGLE (Virtana cross-vendor agents) — https://siliconangle.com/2026/10/28/virtana-lets-its-ai-agents-pull-in-agents-from-other-vendors-to-work-it-incidents/ · SiliconANGLE (Monte Carlo Bayes) — https://siliconangle.com/2026/10/28/monte-carlo-data-launches-bayes-an-ai-agent-for-autonomously-monitoring-and-fixing-autonomous-agents/ · SiliconANGLE (Crux Labs) — https://siliconangle.com/2026/10/29/crux-labs-raises-over-6m-to-improve-enterprise-agents-with-human-judgment/
3. Netflix runs GPU reliability like a distributed system
InfoQ / AWS / The New Stack / NVIDIA · October 27–30, 2026
Netflix runs its GPU platform on Kubernetes and treats hardware failure as normal. It reports more than 99.95% GPU-node availability, and job success rose from about 98% to more than 99.5%. Device checks use NVIDIA DCGM for GPU errors, memory, interconnect and heat, and network checks count and inspect AWS Elastic Fabric Adapter interfaces. A node agent turns the results into Kubernetes node conditions that controllers act on, and the checks fail closed: a diagnostic that hangs counts as unhealthy. Repairs escalate from blocking new work to draining the node, killing the job, resetting the GPU, rebooting and finally replacing the machine, through Karpenter’s node auto-repair under Netflix’s own disruption limits. Training jobs use checkpoints and elastic workers, and application failures are classified separately from hardware ones. InfoQ notes that one slow worker can hold back a synchronous job without ever failing.
AWS added GPU sharing to Amazon ECS using NVIDIA’s Multi-Process Service. Teams set the GPU memory each container needs and can cap its share of compute, without code changes, on ECS Managed Instances or EC2. On Managed Instances, ECS replaces the instance if the sharing daemon fails. A New Stack piece sponsored by HPE warns that a busy GPU says nothing about service quality: track time to first token and time per output token separately, profile the service with realistic prompts before launch, and test answer quality after every model or prompt change. NVIDIA published a validated Kubernetes recipe for its Dynamo inference stack, version 1.4.2, which bundles the GPU Operator, scheduler, Prometheus and a gateway and makes the NATS messaging component optional.
Sources: InfoQ (Netflix GPU reliability) — https://www.infoq.com/news/2026/10/netflix-gpu-distributed-systems/ · AWS What’s New (ECS GPU sharing, vendor announcement) — https://aws.amazon.com/about-aws/whats-new/2026/10/amazon-ecs-gpu-sharing-nvidia-mps/ · The New Stack (busy GPUs, sponsored by HPE) — https://thenewstack.io/ai-inference-kubernetes-observability/ · NVIDIA Technical Blog (Dynamo runtime, vendor blog) — https://developer.nvidia.com/blog/build-a-validated-kubernetes-runtime-for-nvidia-dynamo
4. The AI infrastructure bill, and who might not pay it
Network World / VentureBeat / SiliconANGLE / The Register · October 27–31, 2026
Gartner estimates that more than $3.5 trillion of AI infrastructure spending could face write-downs by the end of 2030 if providers keep investing at the current pace. Providers are expected to spend nearly $1 trillion in 2026 and about $6 trillion more from 2027 to 2030. To earn that back, data centre owners would need about $8.5 trillion in revenue through 2030, against about $3.6 trillion projected from applications. Gartner also predicts that more than 80% of neocloud providers will stop operating independently by 2030. “The scenario described in our research is the most likely scenario given current trends, but not the only possible scenario,” said Gartner senior director analyst Will Sommer.
Utilisation is the operator’s side of the same problem. In VentureBeat’s monthly surveys from July to September, 79% of 218 respondents reported GPU utilisation at 50% or below, and 70% of that group still plan platform changes; the figures are self-reported. VentureBeat covered Nvidia and Oracle making Vera Rubin NVL72 systems generally available on Oracle Cloud Infrastructure. Oracle will also host Google’s TPU 8t and 8i chips from 2027, the first time Google TPUs will be offered through another cloud, with no prices or configurations yet. “A lower infrastructure price is not much of a bargain if the development and operational costs outweigh the savings,” said HyperFrame Research’s Stephanie Walter.
Money is still arriving for specialised inference. General Compute raised $100 million at an $800 million valuation to run only inference chips from Cerebras, SambaNova and Positron, sold as bare metal or under a guaranteed tokens-per-second agreement. Neocloud Quake AI, which runs 22,000 GPUs, told The Register it is building APIs so that autonomous agents, not people, can find and consume its compute. In theCUBE’s Breaking Analysis, one enterprise executive said GPUs need to run at about 80% utilisation just to break even.
Sources: Network World (Gartner $3.5T forecast) — https://www.networkworld.com/article/4242389/gartner-warns-3-5t-in-ai-infrastructure-spending-could-be-at-risk-by-2030.html · VentureBeat (Nvidia and Oracle, idle GPUs) — https://venturebeat.com/infrastructure/nvidia-and-oracle-expand-vera-rubin-cloud-infrastructure-pitching-better-utilization-as-enterprises-report-idle-gpus · Network World (Google TPUs on OCI) — https://www.networkworld.com/article/4241566/oracle-adds-google-tpus-to-oci-to-expand-enterprise-ai-compute-options.html · SiliconANGLE (General Compute) — https://siliconangle.com/2026/10/28/inference-cloud-startup-general-compute-reels-in-100m-despite-broader-ai-funding-fears/ · The Register (Quake AI neocloud) — https://www.theregister.com/ai-and-ml/2026/10/28/neocloud-thinks-ai-agents-not-meatbags-will-be-its-main-customers/5305496 · SiliconANGLE (Breaking Analysis) — https://siliconangle.com/2026/10/31/can-enterprise-ai-pay-for-the-boom/
5. Agent traffic forces GitHub to rebuild
The Register / New Relic · October 28, 2026
GitHub recorded 7.4 billion commits in September 2026, five times as many as a year earlier, driven largely by developers running AI agents. The growth caught it unprepared. Outages followed in February, April and August, and reconstructed status-page data put its 90-day availability at about 84% on May 7. At its Universe conference, GitHub said it has added four million CPU cores, 160 petabytes of high-speed storage and 50 terabits per second of network capacity this year, plus more capacity for Actions. The Register’s headline reads as if the cores are still to come; GitHub describes them as already added. The longer fix is architectural: GitHub is separating compute from storage, which its benchmarks say allows up to 35 times more write throughput, and moving to a multi-region Azure deployment. “We expect to complete the migration early next year,” said Microsoft’s EVP of CoreAI, Jay Parikh.
New Relic used the same event to tie production monitoring to agent-written code. Change Tracking for GitHub Copilot reads a pull request, writes a monitoring plan, and grades the deployment at six points from ten minutes to 72 hours. If it finds a regression it opens a GitHub issue, and Copilot can draft the fix. An experimental tool shows token use, latency and spend in the developer’s IDE. New Relic cites its own survey: 78% of engineering leaders report more production incidents after deployment, while 94% rate AI-generated code as higher quality during review.
Sources: The Register (GitHub capacity) — https://www.theregister.com/ai-and-ml/2026/10/28/github-powering-up-four-million-extra-cores-to-stop-its-service-going-down/5305863 · New Relic (GitHub Universe integrations, vendor blog) — https://newrelic.com/blog/news/github-universe-2026
6. Sandboxes that keep the keys outside
TechTarget / SiliconANGLE / CNCF Blog · October 27–30, 2026
Google’s Agent Compute Engine (ACE) runs each agent in its own hardware-isolated microVM that wakes when a model or user responds and sleeps when idle. The sandboxes hold no passwords or keys. A Google gateway checks each request against the agent’s temporary identity and adds credentials on the way out, and the sandboxes have no direct internet or corporate network access. Agents can be snapshotted, branched into parallel copies and rolled back. ACE already runs Google’s Gemini agents; a customer preview is expected in a few months, with pricing at general availability. Agent Substrate is an open-source version for customers’ own Kubernetes clusters. TechTarget notes that AWS and Azure sandboxes by default let agents use credentials inside them. Analysts warned about lock-in. “Ownership needs sustained, useful demand, not just busy hardware,” said Hyperframe Research’s Stephanie Walter.
Akamai added Akamai Inference, managed access to 12 open-weight models through OpenAI-compatible APIs, and Agent Sandbox, which runs agents in WebAssembly isolation on its Functions service using the open-source Spin framework. Administrators can cap each agent’s compute, memory, system API access and outbound network access. A CNCF ambassador post explains why the platform has to enforce these limits. An agent’s reach grows as tools, leftover credentials and temporary delegations pile up, and a prompt only states intent. The authors’ key rule: an agent’s identity must not be able to change the objects that define its own permissions. “An autonomous agent will do exactly what its tools let it do, and no more,” they write.
Sources: TechTarget (Google ACE) — https://www.techtarget.com/news/googles-ace-offers-cloud-alternative-for-agentic-isolation/873359/ · SiliconANGLE (Akamai Inference and Agent Sandbox) — https://siliconangle.com/2026/10/27/akamai-expands-its-ai-cloud-platform-with-dedicated-inference-and-agent-execution-environments/ · CNCF Blog (agent boundary problem) — https://www.cncf.io/blog/2026/10/29/the-boundary-problem-in-agentic-ai-is-a-platform-problem/
7. Humans still approve, but few have tested recovery
SiliconANGLE / TechTarget / The Pragmatic Engineer · October 27–29, 2026
A theCUBE Research study of 25 enterprise technology and security leaders, discussed in a segment sponsored by Dell, found that none of the 19 with agents in production let them carry out high-risk actions without a person involved. Preparation lags. Only four of the 25 had tested an agent incident, 11 admitted gaps in reconstructing what an agent did across systems, and 12 had no plan to detect abnormal agent behaviour. Restoring an agent does not undo the records and transactions it already changed. “We are actually leaning on human approval to secure and govern these AI actions,” said Krista Case of theCUBE Research.
TechTarget’s CIO reporting shows how teams are redesigning. Meta’s agent for onboarding data columns applied one subsystem’s rules to another in production; Meta rebuilt it as 14 specialised agents under a small deterministic orchestrator and cut setup from three or four days to under an hour. Trust Bank Singapore found its automated runbooks went stale, so it now uses fixed runbooks to gather evidence, AI to analyse it and engineers to decide. In an OutSystems survey of 858 executives, 82% felt confident they could govern agents but only 26% felt well prepared to resolve conflicts between them. “Authority shouldn’t sit with the agents themselves,” said OutSystems CIO Tiago Azevedo.
The Pragmatic Engineer’s deep dive explains the pressure on on-call teams. AI coding agents have roughly doubled pull-request frequency at many companies, and Rootly’s data shows off-hours pages have nearly doubled. Stripe’s in-house agent averages about three minutes for a simple investigation and 14 for a deep one, and ResolveAI rebuilt its agent three times into separate investigator, verifier and communicator roles. Fully autonomous AI SRE is not here yet. The author discloses an investment in incident.io, and most of the piece is paywalled.
Sources: SiliconANGLE (agent resilience, sponsored segment) — https://siliconangle.com/2026/10/28/human-in-the-loop-ai-agent-recovery-plans-aicybersecurityandresiliencesummit/ · TechTarget (CIOs and agent workflows) — https://www.techtarget.com/news/what-cios-are-learning-as-ai-agents-take-on-real-workflows/873135/ · The Pragmatic Engineer (AI SRE, paywalled) — https://newsletter.pragmaticengineer.com/p/ai-sre
8. Serval’s $3 billion bet against ServiceNow, and IBM’s consolidation
SiliconANGLE / TechTarget · October 29, 2026
Serval raised $150 million at a $3 billion valuation, led by Sequoia, its third round in just over a year and $277 million in total. Its service-management platform lets staff describe a workflow in plain language while agents build and run the automation. It began on the IT help desk and has spread to HR, finance and legal. Its Catalyst agent scans tickets for repetitive work and drafts automations for an administrator to review. Serval says ARR has doubled every quarter since December but has not disclosed the figure. It names ServiceNow as its main competitor and says one carmaker switched in 45 days. A new services unit will hire more than 100 automation engineers, about half of them recent graduates. “Nobody has 10 years of experience doing this, because two years ago the tools didn’t exist,” said chief executive Jake Stauch.
IBM is going the other way, folding its automation products together. From early 2027 they will share billing, metering and identity on IBM’s platform, with a single console to follow under the name IBM Automation 2.0. The Infragraph knowledge graph is due for general availability in December, and a “time travel” feature to roll infrastructure back using Terraform topology data is expected after a release in the first quarter. “Where we want to get to is more converged products,” said Tyler Lynch, IBM’s outbound CTO for IT automation.
Sources: SiliconANGLE (Serval Series C) — https://siliconangle.com/2026/10/29/serval-raises-150m-at-3b-valuation-to-bring-its-automation-agents-beyond-it/ · TechTarget (IBM Automation 2.0) — https://www.techtarget.com/news/ibm-automation-20-will-tighten-integrations-unify-ui/873250/
Calls to action
- Benchmark any AI SRE on your own incidents. Replay a set of past incidents through each candidate, including the agent you already run, and score both the answer and the evidence it cites. Do not rely on a vendor’s own benchmark.
- Set limits before letting outside agents into incidents. List which agents may join, what they can read and change, and how many tokens they can spend, and log every action they take.
- Move credentials out of agent sandboxes. Check whether your agents hold keys inside their runtime. Where possible, inject credentials at a gateway that checks each request, and stop agents from editing their own permissions.
- Run an agent incident drill. Practise stopping an agent, blocking its tools and reconstructing what it changed across systems. Most teams in theCUBE’s study had never done this.
- Make GPU health checks fail closed. Treat a hung diagnostic as unhealthy, separate hardware failures from application failures, and set an escalation path from draining a node to replacing it.
- Measure inference by latency, not GPU load. Track time to first token and time per output token separately and compare production against a pre-launch baseline.
- Check GPU utilisation before buying more capacity. If utilisation is below half, try sharing GPUs between smaller jobs before signing for more.
- Keep a second path if you depend on GitHub. Until its migration finishes, make sure releases and urgent fixes can still ship during a GitHub or Actions outage.
|