·12 min read·Empyrean Engineering

Release Log Vol. 2: Swarm-Class Sensor Fusion, EO/IR Computer Vision & Local Counter-UAS

release-logsensor-fusioncounter-UASdrone-swarmmulti-hypothesis-trackingEO/IRcomputer-visionPTZUTMDronetagDDIL

Introduction

In our first Release Log Blog, we kicked off saying things ship fast at Empyrean Defense. That still holds true, but when you work on the wrong things, that certainly makes things ship much slower. For some of us here who have run software shops in security, both at SaaS vendors and internal teams, you often do not know you are building the wrong thing until it is far too late.

In our case, as a fledgling startup, even a few weeks spent cooking on something is far too long. However, not all is doom and gloom as we have shipped (and are shipping) some exciting things. Since we last put up a blog of any sort, we were accepted into the Summer 2026 OregonUAS Cohort, where we get to work with amazing partners and other folks building towards American UAS dominance and the systems that support them.

Our core IP, the Fusion Engine, has seen remarkable improvements across both our live and wargaming simulation platform. We have vastly improved how electro-optical and infrared (EO/IR) camera systems integrate with the platform, and we added a few new integrations and improved some old ones.

Building the Wrong Thing

The Decision Dominance Engine (DDE), as you can probably tell from the website, is a large software platform. When you are supporting multi-intelligence fusion and production, as well as sensor fusion, data normalization, and making it work at scale, there’s no other way for it to be but large. One of the important pieces for Unmanned Traffic Management (UTM) and Counter-UAS (CUAS) is our UAS Operations module.

The UAS Operations module supports a 19-part flight risk management risk calculation engine for pre-flight, as well as live conformance monitoring against important aspects that are mandated by FAA Part 107 and future mandates under the forthcoming Part 108. In plain terms: before a drone takes off, we tell you how risky the flight is, and while it’s in the air, we tell you if it’s breaking the rules. That part is solid and is arguably a very important piece of tech that we will continue to improve and add more risk factors and weighting to.

Where we went wrong was trying to spin that off into its own product. The Decision Dominance Engine, while it can support cloud deployments, is really built towards hyper-localized deployments onto a single node or a cluster. That said, it is explicitly a “single tenant” experience where everything from identity management, API access, encryption, down to every single module and workspace is self-contained. We support limited federation with ATAK Servers (currently one-way outbound) and data export via API, but we wanted to spin out the risk scoring and conformance monitoring into a multi-tenant, pay-as-you-go, multi-subscription-tier SaaS product. Put simply, we tried to pull one feature out of a big, self-contained system and turn it into its own subscription web app.

Big mistake.

The premise was solid, but the market for that sort of tool is not established, and the total addressable market is not very big. More to the point, until Parts 108 and 146 ADSPs are up and running, and folks are forced to comply, there is not much other than better insurance rates and internal governance that can benefit from enhancement to pre-flight risk management.

We wasted far too many cycles on building a totally new identity management layer, infrastructure, Development & Operations (DevOps) tooling, and porting code and styling from the Decision Dominance Engine to what we were going to call SkyFiler. Perhaps it is something we will revisit as regulatory and market demand want, but all told we burnt weeks that could have been spent on much more important systems engineering and integrations, or even ML engineering pipelines, and it is definitely an important lesson.

As engineers, it’s important to not chase the new shiny. In the Army, we had the concept of the “Good Idea Fairy” which was usually synonymous with the mythical creature: the Blue Falcon. We “Blue Falcon’ed” ourselves with this idea thinking that UAS Operations would just be easy to carve out and dump onto AWS or Azure auto-scaling compute and call it a day.

Don’t do that.

Fusion for the Swarm Era

Drone swarms are one thing that seems to occupy the collective consciousness of this industry. Either folks are talking about building them, controlling them, integrating them with piloted airframes (the “Collaborative Combat Aircraft” program), countering them, or just scared shitless overall. We will save black-pilling you about the logistics of drone swarms on either side of the offensive-defensive continuum and focus on the countering part.

There is a lot that needs to happen before a missile is fired, and in a time where new Executive Orders and legislation such as the Safer Skies Act are being discussed, a lot more needs to be taken into account when it comes to CUAS within our own borders (that is to say, CONUS). It is why so much of our effort in CUAS was spent on the UTM side of the house, because knowing what to target relies so much on also knowing what not to target, too.

The core tech we have patent pending is our Fusion Engine. It is a multi-hypothesis capable fusion system that can work on-demand, in specific geographical zones, and supports multiple well-known algorithms for multi-hypothesis tracking (MHT), covariance calculations, association/disassociation, and more. Obviously, we’re never going to write in-depth about it, but there is a big difference in taking highly accurate military-class sensors (radar, gimbaled EO/IR, multi-microphone acoustic arrays) that track a handful of spread out slow moving Shahed-136 OWAs and tracking 30, 40, 50, 100, or more of them or mixed target classes. Simply put: tracking a few drones is easy and tracking a hundred at once, mixed in with everything else in the sky, is not and is where most systems fall apart.

The hardest part of sensor agnosticism is not in data normalization, as the Empyrean Data Fabric was built to handle that, and reconciles myriads of ways that the most banal things such as geocoordinates or labeling work - but dealing with the latency and accuracy differences. Even when pulling it off the wire in binary, you will always have some latency. You add API-driven sensors, sensors coming 7 hops across a C-Band mesh network, satellite-based data, or just plain malfunctioning sensors, and you already have a huge latency shit sandwich to deal with.

Then you add the fact that sensors are not all created equal even in the same class. One radar canted up may bring you returns on birds and call it a drone, another will pull wave clutter in and tell you it is a car. Acoustics may mistake a chainsaw or grinder in the distance for a propjet OWA, you get the point. Managing identity provenance across multiple sensor types is just as important as the kinematics of the object for defeating spoofing, and Making Sense of the battlefield or airspace.

We introduced several new algorithms to help us with this, leaving the end result user configurable, such that you can assign trust weighting of your own to your different sensors. Different operators will know exactly about this, we spoke with CCT folks, EW folks, SHORAD guys in the Army, and everyone had anecdotes to share about certain sensors or specific manufacturers or models that misbehaved when it mattered the most.

Secondly, we introduced better algorithms for track initiation and MHT. If you take every single ground deflection, chainsaw, or mislabeled subject from a cued & slewed camera and promoted it through sensor fusion, you are already losing the battle. Operator fatigue from cognitive overload and inaccurate systems is just as dire in CUAS as it is in cybersecurity. Some of us lived both of those lives. At the end of the day, it’s hard to “fluke” the physics or kinematics, so we introduced ways to dynamically choose when to initiate tracks based on sensor accuracy (refresh rates, CEPs, etc.) and latency.

Lastly, we introduced two new algorithms for picking the best of multiple tracks when the sensors disagree due to latency or target misidentification. This lives purely in the physics-backed and kinematics realm. As tracks are bracketed, adjudicated, initiated, and go through multiple coasting periods where they drop off, we keep running tallies as part of MHT but also picking the highest likelihood for association. Furthermore, we tested the algorithms in our wargaming engine which creates digital twins and found clear winners in the case we have very accurate sensors in extreme density swarms.

We tested against UAS of all classes and other targets of note, including gnarly scenarios that think tanks have nightmares about, such as decoys mixed with kinetic effectors that cross over each other and maneuver around one another in density that would make a SHORAD battery commander soil their combat pants. While RMSE, track purity, and false positives do exist against the sensors we tested with (and simulated), we found a clear winner that does rely on vendor data we unfortunately will not always have access to (or be told, even under Bro-NDA).

So, the last innovation is that, of these two “swarm class” fusion algorithms we will seamlessly fallback to the second-best algorithm that still offered 90% recall rate and an average 102m RMSE with just a single class of sensor (radars, with track handover between sectors). In multi-sensor fusion tests, this continued to improve greatly and led to great pKill numbers in simulation. Bottom line is that when we don’t have the best data, the system automatically switches to a backup method that still catches about 9 out of 10 targets with radar alone.

We’ve already said too much, but sensor fusion is never really done. We continue to hit the lab and grind in the field and have several more papers to read and algorithms to implement to further increase efficacy in the worst scenarios.

Let’s Get Visual

One of the first enhancements we made to the product after we exited our alpha stage was to add support for EO/IR systems in the tool. Within ATAK, you have the Camera Tool which allows you to add just about any Real Time Streaming Protocol (RTSP) network-attached camera. We largely mirrored that functionality, complete with dedicated iconography on the COP, bearing lines, and a rough bearing & geolocation solver.

We went further with support for a CPU-bound version of an open-source Computer Vision (CV) container that allows you to examine live-streams across many cameras and place icons on the map. This predated our policy engine, our Flow builder (that allows for more complex, multi-step automation), and a lot of our physics engine solvers. Which is to say: it was not very accurate.

We finished a complete retool of the entire feature and rebuilt it from the ground up. We trained our own CV model and extended YOLOv11 to run on NVIDIA CUDA cores (our target infrastructure is GB10-class nodes), completely overhauled the data model, and made a much more accurate onboarding workflow. In our unceasing effort to support sensor agnosticism, we also made the system plug-and-play beyond cheap, fixed RTSP cameras to support everything from the Orion SDK (used primarily by Trillium) as well as full ONVIF support for Pan-Tilt-Zoom (PTZ) cameras.

What this means is that we have much more accurate machine learning assisted labeling and alerting. We built a new geolocation and optical calibration wizard that will take a latitude, longitude, and height above ground along with a Circular Error Probability (CEP) and match pixels from a live still onto the same location in satellite imagery. In our testing, we’re getting ~2m positional accuracy as well as 1.05-1.7 degrees of accuracy at 1X zoom with really terrible cameras in heavy urbanized environments. It’s much harder to pick an accurate reference point in a residential neighborhood filled with sheds, fences, shrubs, and cars than it is at an airfield, port, or other critical facility. In the simplest terms: you can now point a cheap camera at a cluttered neighborhood and the system can still tell you where something is to within a couple of meters.

Likewise, the onboard wizard extends to PTZ control verification as well as zoom calibration, allowing us to calculate Horizontal Field of View (HFOV) at different zoom levels and correct for calibration from there. This all feeds into the ability to command cameras with humans in the loop, or to automate detection as well as cue-to-slew capabilities for achieving Positive Identification (PID) of targets of note: drones, cars, people, animals, foreign objects, and anything else important to you.

To top it all off, we’re extending this feature into integration with our Visual Positioning Service (VPS) to eventually allow for using geotechnical datasets to find position and orientation around important objects instead of relying on traditional GNSS. This matters more for DDIL (denied, degraded, intermittent, or limited) environments where GPS (even M-Code) cannot be fully relied on or trusted due to novel electronic warfare TTPs employed by opposition forces. In other words: we’re building a way for systems to figure out where they are by recognizing their surroundings, so they keep working when GPS gets jammed. VPS is in its early days, we’re not claiming to be Niantic (nor will we develop a mobile game to capture the dataset), but bringing low-cost, high sophistication services to those who need it is well within our wheelhouse.

More work is yet to come on this feature, but we are confident to call it V2, and you will start to see more EO/IR related content as well as a website refresh showcasing this capability coming soon.

Supporting Local Counter-UAS and Unmanned Traffic Management (UTM)

The last feature we have shipped in this iteration is an integration with the Dronetag Scout running as an “on premises” node. We already support Dronetag’s Cloud Platform, which takes drone data from Scout and RIDER sensors and pushes it to their platform. This integration has the side benefit of being able to get any public sources for free. What that means is anyone who sets up a Scout or RIDER as a public sensor can feed ODID and ADS-B data into the cloud platform, and you get it for free. As campuses, critical infrastructure, public venues, police departments, and other agencies set up their sensors as public, you get the benefit within DDE of gathering data on overlapping and/or remote jurisdictions.

However, not everyone can nor wants to rely on a cloud SaaS platform for critical Counter-UAS and UTM use cases. The cloud platform as an aggregation surface is great, but has at minimum a 10 second latency as well as relies on a stable internet connection. If you are running a DDE node stack in an air gap or DDIL environment, or just lack remote satellite, cellular, and/or tactical mesh network capabilities back to your WAN; using Scout is the answer. Translation: no internet? No problem. Plug the drone detector straight into your network and get data in real time, instead of 10+ seconds late from the cloud.

The Scout integration is considered a “Push” based integration in our parlance, and uses our receiver-based service to stream, batch, normalize, and ontologize one or more LAN-connected Scout nodes. This works like our Skywatch product, which is a Rust-based ADS-B and UAT decoder that uses commodity SDR hardware for airspace awareness. You can connect one or more Scouts into whatever network interface and VLAN that DDE is wired into and receive data from them. Using mesh networking devices such as Silvus StreamCasters or similar data radios can allow you to connect disparate teams and mobile platforms.

The Scout has LTE capabilities as an option, but mesh networking paired with the onboard GNSS receiver and HTTPS certificate capabilities allows you to create secure and private C-UAS and UTM coverage in your area of operations. With the proper mesh networking backbone, you can have a truck or tactical vehicle with Echodyne radars and Dronetag Scouts feeding information to DDE over a mesh. At your command center, you can have another radar, EO/IR, and/or Scout installation to provide better situational awareness.

Otherwise, it’s a simple Power-over-Ethernet (PoE) connection to power and receive data from the Scout. Simply plug it into a PoE-capable switch, configure the integration, and you are detecting cooperative drones. Like any other integration that produces live tracks with kinematics, this works with our Fusion Engine. If you have a 4-pack of Echodyne Echoshield radars, some acoustic sensors, a few PTZ EO/IR cameras, and a Dronetag Scout you have an end-to-end Counter-UAS and UTM capability all within the same platform. Bottom line: a handful of off-the-shelf sensors plus DDE gives you complete drone detection and airspace tracking in one system.

We are in early discussions with other C-UAS and UTM sensor providers to expand our offering and build Dronetag-like integrations with their sensors. If you are a program manager or developer with a C-UAS or UTM sensor provider, or are interested in DDE and require us to integrate your own tech, please get in touch to expand Decision Dominance to your customers and use cases.

Looking Forward

The burns of hard lessons learned are mostly healing and much progress has been made even between the time this release blog was started (this was supposed to be for last month. oops.) and now. Some of that progress we can’t talk about just yet, but we will soon.

In the next release, we’ll cover new partnerships and our expansion into electromagnetic spectrum operations (EMSO). That includes a new Empyrean Defense app that lets smaller end-user devices connect to software-defined radios (SDRs) and communicate back to central servers. We’re also going bigger on 3D, with full 3D replay and more 3D scene rendering for both planning and live operations.

Plainly: more partners, more radios, and a better way to see what happened (and plan what happens next) in three dimensions.

Till then.

Empyrean Defense

Want to discuss this topic?

We're always happy to talk about the problem space, the platform, or how we might work together.