Introducing etw-rs: an ETW parsing library built for flexibility and performance

This blog is not my usual Event Tracing for Windows (ETW) security research fare. Instead, it is a tale of engineering and why I felt the need to create another Rust ETW parsing library.

This is my third ETW library side quest, but my first "solo" adventure.

My first love library was Microsoft's krabsetw. During my early phase of security research with ETW, I accidentally became the largest non-Microsoft contributor as I contributed support upstream for each of the various ETW features and edge cases that I was investigating at the time. This gave me an appreciation for when abstractions are powerful and for when they inadvertently cause friction for non-standard use cases. With krabs, the Win32 API was deliberately nowhere in sight.

My second library was used internally by Elastic's EDR agent. I conceptualized and delivered their provenance-enriched ETW Threat-Intelligence call stack capability, and this included a refactor of the ETW development ergonomics and parsing performance. This was actually a bittersweet moment as the code that I replaced was the most beautifully commented code that I have ever read. This was also deliberately an in-place refactor, so some of my performance ideas were non-starters at that time.

Origin is a Rust-first software company with a strong requirement for ETW data. The most obvious library choice was ferrisetw - the Rust port of krabsetw.

However, I wanted modern flexibility around system logger sessions and escape hatches for my security research. These are significant changes to the shape of the API, not features I could reasonably wedge into a krabsetw clone. I also knew that more performance was possible, and for ETW this matters. ETW delivers each session on a single thread, so per-event parsing cost is the ceiling on throughput. You can spread providers or keywords across sessions, but then events from different sessions arrive out of order, and correlating them isn't fun. ETW is deliberately asynchronous and high latency so that its impact on the operating system stays low, which is the right trade-off, but it makes the out-of-order problem worse. So every nanosecond per event would count if I wanted to support the full firehose of kernel ETW events.

To help explain why a port wasn't enough, let's first revisit the evolution of ETW parsing.

A brief history of ETW parsing

1. Windows 2000

In the original ETW release, events were defined by Managed Object Format (MOF) schemas that lived in the Windows Management Instrumentation (WMI) repository. Access to the WMI data was facilitated via Component Object Model (COM) interfaces. This meant robust MOF parsing involved COM boilerplate to access WMI, then WMI boilerplate to walk the class structure and extract the property definitions, and finally ETW boilerplate to handle the parsing for all of the types.

Microsoft's reference example of this triple boilerplate horror is here - Retrieving Event Data Using MOF - Win32 apps | Microsoft Learn

The ergonomics and performance were both poor so in practice most ETW consumers at the time just hardcoded the schema details for the known schemas. This was a slight improvement - until new event versions arrived and broke the code.

2. Windows Vista

Vista brought us modern manifest providers and the Trace Data Helper (TDH) library which contained the TdhGetProperty helper. Parsing was now much more ergonomic…once you had written some helpers for the helper library.

This looked roughly like this -

template <typename T>
static bool GetIntegerProperty(EVENT_RECORD* event, PCWSTR name, T& value) {
  PROPERTY_DATA_DESCRIPTOR pdd{};
  pdd.ArrayIndex = ULONG_MAX;
  pdd.PropertyName = (ULONGLONG)name;

  ULONG status = TdhGetProperty(event, 0, NULL, 1, &pdd, sizeof(T), (PBYTE)(&value));
  return (ERROR_SUCCESS == status);
}

static bool GetUnicodeStringProperty(EVENT_RECORD* event, PCWSTR name, std::wstring& value) {
  PROPERTY_DATA_DESCRIPTOR pdd{};
  pdd.ArrayIndex = ULONG_MAX;
  pdd.PropertyName = (ULONGLONG)name;

  ULONG size = 0;
  if (ERROR_SUCCESS != TdhGetPropertySize(event, 0, NULL, 1, &pdd, &size))
    return false;
  if ((size % sizeof(wchar_t)) != 0)
    return false;

  value.resize(size / sizeof(wchar_t));
  if (ERROR_SUCCESS != TdhGetProperty(event, 0, NULL, 1, &pdd, size, (PBYTE)value.data()))
    return false;

  if (!value.empty() && value.back() == L'\0')
    value.pop_back();

  return true;
}

VOID WINAPI EventRecordCallback(PEVENT_RECORD event) {
  if (event->EventHeader.ProviderId != MicrosoftWindowsKernelProcess)
    return;
  if (event->EventHeader.EventDescriptor.Id != ProcessStart)
    return;

  ULONG processId = 0;
  if (!GetIntegerProperty(event, L"ProcessID", processId))
    return;

  std::wstring imageName{};
  if(!GetUnicodeStringProperty(event, L"ImageName", imageName))
    return;

  printf("Process Start: PID=%u Image=%ls\n", processId, imageName.c_str());
}

Microsoft provides a more complete example here - Using TdhGetProperty to Consume Event Data - Win32 apps | Microsoft Learn

Unfortunately the API required scanning all of the properties for every single invocation. This resulted in such bad performance that Microsoft now warns against it for new code.

A useful overview of the Vista improvements to ETW can be found in this article - Event Tracing: Improve Debugging And Performance Tuning With ETW | Microsoft Learn

3. Windows 7

Windows 7 brought us a new API: TdhFormatProperty. It was now possible to do a schema-aware single pass of an event's properties - giving us robustness and hopefully performance. The TDH boilerplate was still necessary though, and it was also very easy to write non-performant code.

A (non-performant) example looks roughly like this -

bool GetEventInfo(EVENT_RECORD* event, std::unique_ptr<BYTE[]>& buffer, PTRACE_EVENT_INFO& info) {
  ULONG size = 0;
  ULONG status = TdhGetEventInformation(event, 0, nullptr, nullptr, &size);
  if (ERROR_INSUFFICIENT_BUFFER != status)
    return false;

  buffer = std::make_unique<BYTE[]>(size);
  info = reinterpret_cast<PTRACE_EVENT_INFO>(buffer.get());

  status = TdhGetEventInformation(event, 0, nullptr, info, &size);
  return (ERROR_SUCCESS == status);
}

PEVENT_PROPERTY_INFO FindPropertyInfo(PTRACE_EVENT_INFO info, PCWSTR name, USHORT& index) {
  for (USHORT i = 0; i < info->TopLevelPropertyCount; ++i) {
    auto prop = &info->EventPropertyInfoArray[i];
    auto propName = (PCWSTR)((PBYTE)info + prop->NameOffset);

    if (_wcsicmp(propName, name) == 0) {
      index = i;
      return prop;
    }
  }

  return nullptr;
}

bool GetProperty(EVENT_RECORD* event, PCWSTR name, std::wstring& value) {
    std::unique_ptr<BYTE[]> infoBuffer;
    PTRACE_EVENT_INFO info = nullptr;
    if (!GetEventInfo(event, infoBuffer, info))
        return false;

    USHORT targetIndex = 0;
    PEVENT_PROPERTY_INFO targetProp = FindPropertyInfo(info, name, targetIndex);
    if (!targetProp)
        return false;

    USHORT pointerSize = (event->EventHeader.Flags & EVENT_HEADER_FLAG_32_BIT_HEADER) ? 4 : 8;

    PBYTE userData = reinterpret_cast<PBYTE>(event->UserData);
    ULONG userDataLength = event->UserDataLength;

    for (USHORT i = 0; i <= targetIndex; ++i) {
        auto prop = &info->EventPropertyInfoArray[i];

        if ((prop->Flags & PropertyStruct) != 0)
            return false;
        if ((prop->Flags & PropertyParamCount) != 0 || prop->count > 1)
            return false;

        ULONG consumed = 0;
        USHORT outType = prop->nonStructType.OutType;
        USHORT inType = prop->nonStructType.InType;

        ULONG formattedSize = 0;
        ULONG status = TdhFormatProperty(info, nullptr, pointerSize, inType, outType, (USHORT)userDataLength, userData, &formattedSize, nullptr, &consumed);
        if (ERROR_INSUFFICIENT_BUFFER != status)
            return false;

        std::wstring formatted(formattedSize / sizeof(wchar_t), L'\0');
        status = TdhFormatProperty(info, nullptr, pointerSize, inType, outType, (USHORT)userDataLength, userData, &formattedSize, formatted.data(), &consumed);
        if (ERROR_SUCCESS != status)
            return false;

        if (!formatted.empty() && formatted.back() == L'\0')
            formatted.pop_back();

        if (i == targetIndex) {
            value = std::move(formatted);
            return true;
        }

        if (consumed > userDataLength)
            return false;

        userData += consumed;
        userDataLength -= consumed;
    }

    return false;
}

VOID WINAPI EventRecordCallback(PEVENT_RECORD event) {
    if (event->EventHeader.ProviderId != MicrosoftWindowsKernelProcess)
        return;

    if (event->EventHeader.EventDescriptor.Id != PROCESS_START)
        return;

    std::wstring processIdStr;
    if (!GetProperty(event, L"ProcessID", processIdStr))
        return;

    std::wstring imageName{};
    if (!GetProperty(event, L"ImageName", imageName))
        return;

   printf("Process Start: PID=%ls ImageName=%ls\n", processIdStr.c_str(), imageName.c_str());
}

A more complete example is here - Using TdhFormatProperty to Consume Event Data - Win32 apps | Microsoft Learn

The major API performance footgun is that TdhFormatProperty implicitly converts all data into wide strings which is not suitable for all use cases. Further, the example only shows consuming all properties sequentially from a single event. In practice, we process streams of events and often only want a subset of the properties.

To actually use TDH performantly there was still a lot of boilerplate to write.

4. krabsetw

Microsoft's own development teams encountered this and they helpfully open-sourced their library for us.

Our parsing example is now delightfully simple.

void ProcessEventCallback(const EVENT_RECORD &event, const krabs::trace_context &trace_context) {
  if (event.EventHeader.EventDescriptor.Id != PROCESS_START)
    return;

  krabs::schema schema(event, trace_context.schema_locator);
  krabs::parser parser(schema);

  ULONG processId = parser.parse<ULONG>(L"ProcessID");
  std::wstring imageName = parser.parse<std::wstring>(L"ImageName");

  printf("Process Start: PID=%u ImageName=%ls\n", processId, imageName.c_str());
}

A more complete example is here - krabsetw NativeExamples/user_trace_001.cpp | GitHub

krabsetw gave us a lot -

• Abstractions that simplified ETW trace session management.
• Extensive pre-parsing event filtering support.
• A cache for 10x faster TDH schema lookups.
• A strongly typed property parsing API.
• Windows 7+ support.

So why didn't we use the Rust port?

What's missing?

1. Helpers, not abstractions

In hindsight, abstraction was not a perfect fit for ETW.

We use abstractions to create stable interfaces that hide implementation details that might change over time. This is particularly suitable when you own the implementation.

krabsetw does not own ETW and it does not own the ETW provider implementations.

Most of my contributions to krabsetw have been to extend (or bend) the abstraction to cover ETW edge cases - either modern capabilities, or historical curiosities.

I'm primarily a security researcher though, so my scenarios are sometimes obtuse. In most cases, krabsetw provides a very sensible and intuitive interface.

Origin's use cases required both ETW features not yet supported by ferrisetw and ETW mechanisms that would further stress the integrity of the krabsetw abstraction.

I wanted a flexible helper library - not abstractions that don't always fit.

With etw-rs, I deliberately closely mirrored the Win32 API surface and provided (unsafe) escape hatches to directly control Win32 API parameters that were not yet supported by the Rust API. This is primarily to support research and prototyping - use in production at your own risk.

// SecurityTrace: undocumented bit 14 of WMI_LOGGER_CONTEXT.Flags.
//
// EVENT_TRACE_PROPERTIES.LogBuffersLost is a union with
// WMI_LOGGER_INFORMATION.Flags in the kernel. Setting bit 14 here causes
// StartTraceW to propagate it into WMI_LOGGER_CONTEXT.Flags.SecurityTrace.
//
// When set, only Antimalware-PPL processes can query or modify the session
// via ControlTrace(). Non-PPL admin processes get ERROR_ACCESS_DENIED on
// query — though stop is NOT gated (known gap).
//
// Reference: https://www.originhq.com/research/securitytrace-etw-ppl
const SECURITY_TRACE_FLAG: u32 = 0x4000; // bit 14

// SAFETY: We are setting LogBuffersLost, which is an output field that
// StartTraceW overwrites. The kernel reads it as Flags before that happens.
// No other invariants of the properties struct are violated.
unsafe {
  trace.properties_mut().LogBuffersLost = SECURITY_TRACE_FLAG;
}

Another more common example is that krabsetw assumes there is a 1:1 mapping between enabled provider GUIDs and received provider GUIDs. While exceptions are rare, they do exist. So sometimes I was "enabling" providers, just so that I could process the already expected events.

I also found myself enabling the providers in order of expected volumes to optimize the linear provider scan dispatch performance - which is on the hot path.

// krabsetw/krabs/krabs/ut.hpp
for (auto& provider : trace.providers_) {
  if (schema->ProviderGuid == provider.get().guid_) {
    provider.get().on_event(record, trace.context_);
    return;
  }
}

The set of expected GUIDs is known at compile time, so a small amount of boilerplate buys us compiler-optimized dispatch on the hot path.

fn event_callback(event: &RawEvent, cache: &mut ProcessingContext) -> Result<()> {
  // Route to appropriate handler based on provider
  let guid = event.provider_guid();
  match guid {
    process::GUID => handle_process_event(event, cache),
    file::GUID => handle_file_event(event, cache),
    tcpip::GUID => handle_tcpip_event(event, cache),
    // This provider was not explicitly enabled, but events still arrive for it.
    // This is one example of where an approach that abstracted to a callback
    // function per enabled provider isn't ideal.
    etw_subsystem::GUID => handle_stack_walk_event(event),
    _ => {
      debug_assert!(false, "unexpected provider: {guid:?}");
      Ok(())
    }
  }
}

2. Less backwards compatibility

Windows 7 is end-of-life but some krabsetw features are still pinned at Windows 7 support.

For example, my least favorite abstraction in krabsetw is the distinction between UserTrace and KernelTrace.

The system logger in Windows 11 is the evolution of the Windows 2000 kernel logger. For a library that supported Windows 7, the KernelTrace name mostly made sense. Mostly.

In Windows 2000, there was a special ETW session called "NT Kernel Logger" and this was the only mechanism to consume from the (classic) Windows Kernel Trace provider - which was the only source of kernel events at the time. This session still exists for backwards compatibility and the Windows 11 SDK still includes the KERNEL_LOGGER_NAME macro. So the "Kernel" prefix was historically strongly associated only with these classic kernel events.

In fact, Microsoft documentation still inaccurately claims today that -

- NT Kernel Logger Constants - Win32 apps | Microsoft Learn

Microsoft's PerfView project also includes a file called kerneltraceeventparser.mof from the Vista source tree which hints at the internal naming conventions of the era.

But this also started to change with Vista which introduced modern provider equivalents for many of the classic Windows Kernel Trace events. There were now kernel providers other than the classic provider.

The next major change landed in Windows 8 where the API documentation updated the nomenclature to "System Trace" or "System Logger," and that single special session was no longer special. It was no longer the sole consumer of the classic kernel provider events, and the classic kernel provider was no longer the sole source of kernel ETW events.

- Configuring and Starting a SystemTraceProvider Session - Win32 apps | Microsoft Learn

To access one of these eight system logger sessions, there was a new logger mode -

- Logging Mode Constants - Win32 apps | Microsoft Learn

Most recently, the Windows 10 SDK build 20348 introduced System Providers - the ability to configure system logger tracing using the usual modern APIs.

In summary, in krabsetw:

A UserTrace is a trace session that can only subscribe to providers via GUID and keyword including:

• User-mode providers such as Microsoft-Windows-DNS-Client.
• Kernel-mode providers such as Microsoft-Windows-Kernel-Process.
• User-mode global logger (UMGL) providers such as Heap Trace Provider.
• Kernel-mode system trace providers via their System Provider aliases if you also set EVENT_TRACE_SYSTEM_LOGGER_MODE.

A KernelTrace is a trace session that can only subscribe to providers via EnableFlags or extended PERFINFO_MASK, specifically:

• Kernel-mode system trace providers.

At no point did Microsoft documentation state that a system logger session could not also consume events from other providers. This was simply the de facto default on Windows 7 and was thus the necessary approach taken by krabsetw to maintain full compatibility.

All of which is to say that the KernelTrace abstraction is a Windows 7 fossil, and that Windows 8+ (and etw-rs) will happily let you consume any combination of providers in a single trace session which provides certain ordering guarantees for event correlation.

// Full example: https://github.com/originsec/etw-rs/blob/main/examples/15_combined_trace.rs

let mut trace = Trace::new("combined_trace")?;
trace
  .set_events(&[system_process::PERF_PROCESS, PERF_NO_SYSCONFIG])
  .enable_provider(kernel_process::GUID, kernel_process::keyword::PROCESS)?;

trace.set_event_callback(|event, context| {
  if event.provider_guid() == system_process::CLASS_GUID
    && event.opcode() == system_process::opcode::START
  {
    let mut parser = context.parser(event)?;
    let pid: u32 = parser.get("ProcessId")?;
    // Only the classic event includes the command line and user SID
    let sid: Sid = parser.get("UserSID")?;
    let cmdline: String = parser.as_ref::<WideStrRef>("CommandLine")?.to_string_lossy();
    println!("[+] SystemLogger   pid:{pid} sid:{sid} {cmdline}");
  }

  if event.provider_guid() == kernel_process::GUID
    && event.id() == kernel_process::event::PROCESS_START
  {
    let mut parser = context.parser(event)?;
    let pid: u32 = parser.get("ProcessID")?;
    let image_name: String = parser.as_ref::<WideStrRef>("ImageName")?.to_string_lossy();
    // Only the modern event includes the sequence number and integrity
    // Note: EVENT_HEADER_EXT_TYPE_SID logs the creator thread's SID
    let sequence: u64 = parser.get("ProcessSequenceNumber")?;
    let integrity: String = parser.as_ref::<SidRef>("MandatoryLabel")?.to_string();
    println!("[+] Kernel-Process pid:{pid} sequence:{sequence} integrity:{integrity} {image_name}");
  }

  Ok(())
});

3. Performance

I borrowed the concept of a schema cache from krabsetw as this was a clear performance gain.

The next step was to investigate the parsing performance itself.

Our basic etw-rs example has two subtle differences from krabsetw.

fn handle_process_event(event: &RawEvent, cache: &mut ProcessingContext) -> Result<()> {
  if event.id() != PROCESS_START {
    return Ok(());
  }

  let mut parser = cache.parser(event)?;

  let process_id: u32 = parser.get("ProcessID")?;
  let image_name: String = parser.get("ImageName")?;

  println!("Process Start: PID={process_id} ImageName={image_name}");
  Ok(())
}

First, because we are in Rust, our strings are UTF-8 whereas the TDH schema uses native Windows wide strings which are UTF-16LE. To avoid incurring string conversion costs during parsing, we pre-convert the schema property names to Rust strings and pre-populate a map of names to indices.

Second, we pre-populate our parser with the offsets of all fixed-location properties. The key insight was that only events with two or more variable-length properties have variable offsets for some properties. This meant that we could pre-calculate the offsets for most properties once per schema instead of for every event. In fact, most Windows events follow this pattern - fixed-width properties first, and variable-length properties at the end.

I benchmarked whole-event parsing performance for five typical events - covering a variety of property counts and types.

1. Microsoft-Windows-Kernel-Process ProcessStart
   • UInt32 ProcessID, UInt64 ProcessSequenceNumber, FILETIME CreateTime, UInt32 ParentProcessID, UInt64 ParentProcessSequenceNumber, UInt32 SessionID, UInt32 Flags, UInt32 ProcessTokenElevationType, UInt32 ProcessTokenIsElevated, SID MandatoryLabel, UnicodeString ImageName, UInt32 ImageChecksum, UInt32 TimeDateStamp, UnicodeString PackageFullName, UnicodeString PackageRelativeAppId, UInt32 SecurityMitigations
2. Microsoft-Windows-RPC ClientCallStart
   • GUID InterfaceUuid, UInt32 ProcNum, UInt32 Protocol, UnicodeString NetworkAddress, UnicodeString Endpoint, UnicodeString Options, UInt32 AuthenticationLevel, UInt32 AuthenticationService, UInt32 ImpersonationLevel
3. Microsoft-Windows-Kernel-Registry CreateKey
   • Pointer BaseObject, Pointer KeyObject, UInt32 Status, UInt32 Disposition, UnicodeString BaseName, UnicodeString RelativeName
4. Microsoft-Windows-Kernel-File Create
   • Pointer Irp, Pointer FileObject, UInt32 IssuingThreadId, UInt32 CreateOptions, UInt32 CreateAttributes, UInt32 ShareAccess, UnicodeString FileName
5. Microsoft-Windows-Kernel-Process ThreadStart
   • UInt32 ProcessID, UInt32 ThreadID, Pointer StackBase, Pointer StackLimit, Pointer UserStackBase, Pointer UserStackLimit, Pointer StartAddr, Pointer Win32StartAddr, Pointer TebBase, UInt32 SubProcessTag

The benchmarks were run on an Intel Core Ultra 7 268V with power throttling disabled and the processing thread pinned to a Performance Core. A snapshot of live events was captured once and then used for all future runs. The reported value is the median per-event parsing duration from 1,000 runs. The Rust implementations do additional work - converting the raw Windows-native UTF-16LE strings to Rust-native UTF-8 strings.

We successfully retained the elegant usability of krabsetw while improving performance.

4. More performance…less ergonomics

Our get() property accessor is backed by an optimized O(1) name-to-index map, but we actually know the expected property index in advance. Passing this as an explicit hint lets us further optimize the fast path and avoid the map lookup when the hint is correct.

I was also tired of manually copying definitions from manifest XML and reformatting them into comments. So I wrote a helper for this.

// BEGIN Generated by etw-rs generate-provider (comment-only).
// generate-provider Microsoft-Windows-Kernel-Process --event-comment 1 --query-release --drop-old-versions
//
// Event: ProcessStart (id 1)
// Keywords: WINEVENT_KEYWORD_PROCESS
//
// Version 3: // Since Windows 10 1903
//   [ 0] ProcessID                    UInt32
//   [ 1] ProcessSequenceNumber        UInt64
//   [ 2] CreateTime                   FileTime
//   [ 3] ParentProcessID              UInt32
//   [ 4] ParentProcessSequenceNumber  UInt64
//   [ 5] SessionID                    UInt32
//   [ 6] Flags                        UInt32
//   [ 7] ProcessTokenElevationType    UInt32
//   [ 8] ProcessTokenIsElevated       UInt32
//   [ 9] MandatoryLabel               SID
//   [10] ImageName                    UnicodeString
//   [11] ImageChecksum                UInt32
//   [12] TimeDateStamp                UInt32
//   [13] PackageFullName              UnicodeString
//   [14] PackageRelativeAppId         UnicodeString
//
// Version 4 (extends v3): // Since Windows 11 24H2
//   [15] SecurityMitigations  UInt32
//
// END Generated by etw-rs generate-provider (comment-only).

fn handle_process_event(event: &RawEvent, cache: &mut ProcessingContext) -> Result<()> {
  if event.id() != PROCESS_START {
    return Ok(());
  }

  let mut parser = cache.parser(event)?;

  let process_id: u32 = parser.get_hinted("ProcessID", 0)?;
  let image_name: String = parser.get_hinted("ImageName", 10)?;

  println!("Process Start: PID={process_id} ImageName={image_name}");
  Ok(())
}

Performance was noticeably better, but the API was now less convenient to use. One improvement was for get() to learn the hint order at runtime - as each event typically only has one parsing path.

5. Too much performance…too little ergonomics

The other observation was this: why transcribe just the index as a hint when we already had the entire event layout from the schema? This meant that we could pre-generate a parser with pre-calculated offsets for most known versions at compile time.

Our generated block was now -

// BEGIN Auto-generated by etw-rs generate-provider.
// generate-provider Microsoft-Windows-Kernel-Process --event-only 1 --query-release --drop-old-versions
etw::provider_schema! {
  guid: Guid::from_u128(0x22fb2cd6_0e7b_422b_a0c7_2fad1fd0e716),
  events: {
    event { // ProcessStart [WINEVENT_KEYWORD_PROCESS]
      id: 1,
      name: "ProcessStart",
      version { 3, fields: { // Since Windows 10 1903
        ProcessID: u32,                   // [0] UInt32
        ProcessSequenceNumber: u64,       // [1] UInt64
        CreateTime: u64,                  // [2] FileTime
        ParentProcessID: u32,             // [3] UInt32
        ParentProcessSequenceNumber: u64, // [4] UInt64
        SessionID: u32,                   // [5] UInt32
        Flags: u32,                       // [6] UInt32
        ProcessTokenElevationType: u32,   // [7] UInt32
        ProcessTokenIsElevated: u32,      // [8] UInt32
        MandatoryLabel: SidRef,           // [9] SID
        ImageName: WideStrRef,            // [10] UnicodeString
        ImageChecksum: u32,               // [11] UInt32
        TimeDateStamp: u32,               // [12] UInt32
        PackageFullName: WideStrRef,      // [13] UnicodeString
        PackageRelativeAppId: WideStrRef, // [14] UnicodeString
      }}
      version { 4, extends_version: 3, fields: { // Since Windows 11 24H2
        SecurityMitigations: u32, // [15] UInt32
      }}
    }
  }
}
// END Auto-generated by etw-rs generate-provider.

Our properties were now typed accessors with generated names, and our callback was now -

fn handle_process_event(event: &RawEvent, cache: &mut ProcessingContext) -> Result<()> {
  if event.id() != PROCESS_START {
    return Ok(());
  }

  let (process_id, image_name) = match event.version() {
    // Static versioned parsers give maximum speed for known version(s)
    3 => {
      let mut parser = ProcessStartV3Parser::new(event);
      (parser.ProcessID()?, parser.ImageName()?)
    }
    4 => {
      let mut parser = ProcessStartV4Parser::new(event);
      (parser.ProcessID()?, parser.ImageName()?)
    }

    // Fallback safely to dynamic schema lookup for new version best effort
    _ => {
      debug_assert!(false, "dynamic schema fallback used");  // version canary
      let mut parser = cache.parser(event)?;
      let process_id: u32 = parser.get_hinted("ProcessID", 0)?;
      let image_name: String = parser.get_hinted("ImageName", 10)?;
      (process_id, image_name)
    }
  };

  println!("Process Start: PID={process_id} ImageName={image_name}");
  Ok(())
}

This introduced a lot of boilerplate maintenance to handle all known versions and fall back to the hinted dynamic parser otherwise. I wanted performance and convenience.

6. Best of all worlds

So that boilerplate was further wrapped in a generated parser class. Our static parser with hinted dynamic fallback now offered a very similar interface to the pure dynamic approach, but with compile-time property name correctness and almost twice the performance.

fn handle_process_event(event: &RawEvent, cache: &mut ProcessingContext) -> Result<()> {
  if event.id() != PROCESS_START {
    return Ok(());
  }

  let mut parser = ProcessStartParser::new(event, cache)?;
  let process_id = parser.ProcessID()?;
  let image_name = parser.ImageName()?;

  println!("Process Start: PID={process_id} ImageName={image_name}");
  Ok(())
}

One minor issue was that each property accessor internally incurred an identical version dispatch cost. I investigated per-event monomorphized handler functions with dispatch macros, but this impacted ergonomics without always providing a clear performance benefit. I decided that this was taking the hunt for performance too far and left it out.

We now had the ergonomics that we wanted, and our parsing performance was exceptional.

At this point, the get_hinted() API variant was hidden as it was less ergonomic and less performant than the static parser. The get() API remains as it is more convenient for hand-written code, and also supports reflection use cases.

I also opened PRs in both krabsetw and ferrisetw to backport as much of the performance as was practical. For krabsetw I was able to land an initial 2-3x parsing performance improvement in v4.4.9.

For most events, parsing performance was 10x better than the current Rust alternatives. For simple events like Thread Start, which only have fixed-width properties, the performance was 100x better.

Conclusion

Three libraries in, my conclusion is that ETW doesn't want to be abstracted - it just wants ergonomic helpers. To this end etw-rs closely mirrors the Win32 API surface, keeps the escape hatches open, and moves the parsing cost to compile time where the schema allows it. The result kept krabsetw's beautiful ergonomics and significantly improved on the parsing performance.

The initial 0.1.0 release is on crates.io with source, examples, and the benchmark harness at https://github.com/originsec/etw-rs (Apache-2.0, Windows 10 1703+).

Issues, PRs, and obscure provider curiosities are all welcome - the edge cases were the point.