Traditional database architecture assumes you have one or two big database clusters per application, usually accompanied by a connection pooling solution. AI-assisted development and vibe coding platforms turn this model inside out and are driving applications toward many small databases: one per user, application, preview environment, or tenant.
That model fits D1 particularly well. D1 is Cloudflare's managed, serverless database based on SQLite. It allows you to create millions of databases and only pay for what you use.
The question becomes: how do you scale the control plane for millions of databases without making every query slower by sending it through centralized bottlenecks?
We made a lot of architectural changes to scale out D1 to support platforms with millions of created databases, hundreds of thousands of which are active at any point.
This post is a behind-the-scenes deep dive into how the D1 control plane works. Many of the architectural patterns we will describe can be used by anyone building on top of Cloudflare’s Developer Platform. The post also gives a glimpse into how we build our own control planes.
There was already a steady growth in D1 usage when we started planning these changes, but we Continue reading
More than 20% of the web sits behind Cloudflare’s network, and our team, the Data team, owns the data pipeline responsible for plumbing that data (and more) from our global network to applicable central data centers. The Cloudflare network is a globally distributed system supporting tens of thousands of servers. Data that power Cloudflare Logs, SQL API, observability, internal analytics, and more flow through this network and use our data pipeline to serve customers.
In May of this year, we were providing data delivery at roughly two billion events per second. This includes customer traffic as well as data sent from internal teams. Year after year, we tackle the same problem — exponential traffic growth against linear hardware growth. We have to react to this increase in order to keep resource costs under control. Over the past five years, we’ve made various improvements to compensate for the data volume increase in comparison to hardware allocation, equivalent to deferring two and a half years of hardware expansion. This post details those changes: breaking up our monolithic data schema in 2022, adjusting compression in 2024, rearchitecting pipeline replication in spring 2026, decentralizing sampling in summer 2026, and introducing aggregations into the Continue reading
In short, automating the AI agent lifecycle is this year’s pitch: Google Cloud’s walkthrough of 30th July 2026 turns the whole thing into a conversation with a coding agent. On Kubernetes the deploy half of that lifecycle is already a pipeline and a set of manifests, and 2026 has shown what happens when the pipeline is the soft spot: in March a compromised GitHub Action dumped runner memory and hunted for Kubernetes tokens, and in February a GitHub issue title talked an AI triage bot into exposing a project’s publishing credentials. The pipeline that ships an agent holds more authority than the agent does, and it is the one place that can ship the agent’s permissions in the same commit as its code. This post walks the path from a merged pull request to a governed pod stage by stage, says what each stage must enforce when the workload is an agent, then says what a governance layer like Tigera Lynx can and cannot do about it. It is written for the platform and security engineers who own that pipeline, and the leads who sign off on what it deploys.
On 19th March 2026, from about 17:43 UTC, an attacker Continue reading
I decided to make Netmiko the default configuration deployment method netlab uses for Cisco IOS XR. It works great, and I don’t have to worry about whether the user runs IOS XRd control-plane (in which case scripts work) or vRouter (in which case there’s a VM inside the container, so no luck).
After fixing the initial glitches (more on that another time), all integration tests passed except the OSPFv2 default origination test. It turns out the default waiting time (30 seconds) is way too low; Cisco IOS XR needs almost 90 seconds (after its reboot completes) to originate the OSPFv2 default route.
Logpush delivers logs in batches as quickly as possible, enabling Cloudflare to provide information almost in real time. During our recent Birthday Week, we made Logpush available on all self-serve plans and Transformers generally available.
Those changes were among eight major updates to Cloudflare Observability and reflect our commitment to make every Cloudflare feature available to everyone. This post takes a deeper look at how Transformers work inside Logpush's export pipeline. Observability teams use Logpush to export Cloudflare logs to security information and event management systems (SIEMs), analytics platforms, and object storage. Before those logs reach their destination, teams may need to remove health checks, omit sensitive fields, or reshape records for dashboards and investigations. Today, this often requires a separate processing service that teams must deploy, scale, monitor, and maintain.
For example, a team that collects logs from multiple CDNs can add a static provider field to normalize records before they reach its analytics platform. Teams also pay to export, ingest, and store records they discard as soon as they arrive, so filtering them before delivery can reduce downstream data volume and cost.
With Transformers, that processing moves into the Logpush export workflow.Teams define a SQL query Continue reading
With the growing number of SRv6 implementations that netlab supports, it made sense to create a set of topologies to help you get started exploring the SRv6 world. The sample topologies include:
Read the getting started document and have fun exploring the scenarios 😎
Following last week’s release of Clef and Clef-flash, Cloudflare’s open-weight decision models, we decided to bring forth more gifts. Today, we’re releasing Clef-omni, which takes in audio and video input alongside text and image. We also cut the price of Clef-flash so it is now cheaper than Jev, and we made Clef faster.
Although the model game is still early for Cloudflare, innovation and iteration is in our DNA, and we apply these principles to everything we do. In fact, the story of Clef came together over the course of less than a week. We decided we wanted to do something in the decision model space on a Friday evening, trained the model over the weekend, and launched it on Thursday. Even with such a short timeline, we were able to ship performant, high-quality, open-weight models for the community — imagine what more we can do in the future.
For today, we’re excited to keep up the momentum with new additions and improvements to our Clef family of models. This is just the beginning, and we’ll continue to get better, faster, cheaper, and more innovative.
Our new Clef-omni model is able Continue reading
Understanding why an application is using more resources than expected, whether that is CPU or memory, can be challenging. Logs and aggregate metrics can only get you so far. Thankfully there is a better way: CPU or memory profiling can show you the exact function where your application is using CPU or allocating memory.
Today we are happy to announce support for CPU and memory profiling of Workers and Durable Objects. From the Workers Observability page, you can now request an on-demand CPU or memory profile of an active Worker, inspect it as an interactive flamegraph, and download the profile file for further analysis.
This method of profiling gives you a useful perspective into what your code is doing and how you can improve it in a real-world setting. That’s because the best way to understand your application is to profile it in production.
To try this on one of your Workers, you have a choice of using the CLI or the Cloudflare Dashboard.
To use the CLI, make sure you have the cf package installed, then simply run:
To use the dashboard, head to the Cloudflare dashboard and then access the list of Workers on your account Continue reading
The Deno team is joining Cloudflare to radically simplify self-hosting Workers and Durable Objects so developers can use the same primitives in more places.
For what this means and how it came about, here’s the story from Ryan Dahl, Senior Principal Engineer, and Kenton Varda, Distinguished Engineer.
We launched device configuration with Netmiko in the September 2026 netlab release and made it easier to use in the October 2026 release. It will become the default configuration method for several popular platforms (Arista EOS, Cisco IOS, Cisco Nexus OS, Cisco IOS XR, FRRouting virtual machines) in a few months, but before doing that, I wanted to check whether the change matters.
TL&DR: Yes, it does.
You might have noticed that it’s been quiet here. I know that at least two people noticed, as they contacted me to ask, “Are you OK?” (Thank you to those who reached out.) I am OK. It’s been quiet here because, for what is probably the first time since I starting blogging on this site over 21 years ago, I have lost my voice.
What does that mean? Every writer has their own voice, their own style of writing that reflects them. It evolves naturally from who they are. It’s not just their literary style, although literary style is part of it. A writer’s voice encompasses why they write and how they write. While one writer’s voice might be like another’s, I believe each writer’s voice is uniquely their own. It’s like their appearance—two people can be similar, but each person’s appearance is uniquely their own.
From the beginning, this site was a way for me to document what I was learning and share that learning with others. That has always been its primary purpose. I didn’t write to “get noticed,” and I didn’t write to become an “influencer” or a “content creator.” I wrote to Continue reading
Just about everyone is watching the financial markets, either wondering how much better they can get, or when the AI financing bubble is going to crash. Dough Dawson, from POTs and PANs, joins Tom and Russ to talk about the logic of a coming crash, and consider where engineers should be looking for potential future career fields to weather the storm.
download