Building a Visual Studio Add-In To Leverage WinDBG and SOS

 

Hi Community,

I am currently building an Visual Studio add-in to leverage SOS and make it more user friendly, thus I have called it Visual SOS. On this blog post I will describe the main debugging interfaces that are required as well as an artefact called SOS Wrapper which encapsulates the functionality as a native DLL that is invoked from the add-in which is written in C# effectively.

Windows has a built-in debugger engine, and like with every other feature in the  operating system, it also provides means for the developers to build code around it. The debugger engine API is COM based, so every interface required has IUnknown as the base interface.

In Visual SOS, we require to use the following interfaces:

  • IDebugOutputCallbacks:  This interface is called by the engine and it allows to send output from the client to the IDebugOutputCallbacks object that is registered with the client.
  • IDebugEventCallbacks: This interface is called by the engine and it allows to subscribe to events fired by the debugging engine.

The way Visual SOS works is as follows:

  • The entire core functionality is encapsulated in C++ library that exposes a few functions as “extern”
  • The add-in code is written in its entirety in C#/.NET but it makes calls to the native library, this is the same approach I took when building Memory Inspector a few years back.
  • Notifications are done using a message sink in the form of a callback or delegate that is passed from managed code to native code, similar to the technique I used and described here

The native functionality lives in SOSWrapper, below its Initialize method

 1 void SOSWrapper::Initialize() {
 2     if (!m_bIsInitialized) {
 3         CoInitialize(NULL);
 4 
 5         if (SUCCEEDED(DebugCreate(__uuidof(IDebugClient), (void**)&m_pDbgClient))) {
 6             if (SUCCEEDED(m_pDbgClient->QueryInterface(__uuidof(IDebugControl), (void**)&m_pDbgControl))) {
 7                 
 8                 // Load extensions
 9                 std::for_each(m_configReader.Extensions_get().begin(),
10                         m_configReader.Extensions_get().end(), 
11                         [&, this](ExtInformation& item) {    m_pDbgControl->AddExtension(item.Path.data(), NULL, &item.pHandle);    });
12 
13                 m_pDbgClient->SetOutputCallbacks(&m_pOutputCallback);
14                 m_pDbgClient->SetEventCallbacks(&m_pEventCallback);
15 
16                 if (SUCCEEDED(LoadSos()))
17                     m_bIsInitialized = TRUE;
18             }
19         }
20         CoUninitialize();
21     }
22 }

Initialize method

 

Visual SOS  implements a XML-based config file which allows to specify what debugger extensions to load. The way it parses this config file is described here. The implementation of the required debugger engine interfaces are:

 1 #include  "../../dbghelp/inc/dbgeng.h"
 2 
 3 class EventCallback : public IDebugEventCallbacks {
 4 
 5 private:
 6     long m_ref;
 7 
 8 public:
 9     EventCallback();
10 
11     ~EventCallback();
12 
13     // IUnknown
14     STDMETHOD_(ULONG, AddRef)();
15     STDMETHOD_(ULONG, Release)();
16     STDMETHOD(QueryInterface)(__in REFIID InterfaceId, __out PVOID* Interface);
17 
18     // IDebugEventCallbacks
19     STDMETHOD(ExitThread)(__in ULONG ExitCode);
20     STDMETHOD(SessionStatus)(__in ULONG Status);
21     STDMETHOD(ExitProcess)(__in ULONG ExitCode);
22     STDMETHOD(GetInterestMask)(__out PULONG Mask);
23     STDMETHOD(Breakpoint)(__in PDEBUG_BREAKPOINT Bp);
24     STDMETHOD(SystemError)(__in ULONG Error, __in ULONG Level);
25     STDMETHOD(ChangeEngineState)(__in ULONG Flags, __in ULONG64 Argument);
26     STDMETHOD(ChangeSymbolState)(__in ULONG Flags, __in ULONG64 Argument);
27     STDMETHOD(ChangeDebuggeeState)(__in ULONG Flags, __in ULONG64 Argument);
28     STDMETHOD(UnloadModule)(__in_opt PCSTR ImageBaseName, __in ULONG64 BaseOffset);
29     STDMETHOD(Exception)(__in PEXCEPTION_RECORD64 Exception, __in ULONG FirstChance);
30     STDMETHOD(CreateThread)(__in ULONG64 Handle, __in ULONG64 DataOffset, __in ULONG64 StartOffset);
31     STDMETHOD(LoadModule)(__in ULONG64 ImageFileHandle, __in ULONG64 BaseOffset, __in ULONG ModuleSize,
32         __in_opt PCSTR ModuleName, __in_opt PCSTR ImageName, __in ULONG CheckSum, __in ULONG TimeDateStamp);
33     STDMETHOD(CreateProcess)(__in ULONG64 ImageFileHandle, __in ULONG64 Handle, __in ULONG64 BaseOffset,
34         __in ULONG  ModuleSize, __in_opt  PCSTR ModuleName, __in_opt  PCSTR ImageName, __in ULONG CheckSum,
35         __in ULONG TimeDateStamp, __in ULONG64 InitialThreadHandle, __in ULONG64 ThreadDataOffset,
36         __in ULONG64 StartOffset);
37 };

EventCallback.h

  1 #include "stdafx.h"
  2 #include "EventCallback.h"
  3 
  4 EventCallback::EventCallback() {
  5     m_ref = 1;
  6 
  7 }
  8 
  9 EventCallback::~EventCallback() {
 10 
 11 }
 12 
 13 STDMETHODIMP EventCallback::QueryInterface(__in REFIID InterfaceId, __out PVOID* Interface) {
 14     *Interface = NULL;
 15     if (IsEqualIID(InterfaceId, __uuidof(IUnknown)) || IsEqualIID(InterfaceId, __uuidof(IDebugEventCallbacks))) {
 16         *Interface = (IDebugEventCallbacks *)this;
 17         InterlockedIncrement(&m_ref);
 18         return S_OK;
 19     } else {
 20         return E_NOINTERFACE;
 21     }
 22 }
 23 
 24 STDMETHODIMP_(ULONG) EventCallback::AddRef() {
 25     return InterlockedIncrement(&m_ref);
 26 }
 27 
 28 STDMETHODIMP_(ULONG) EventCallback::Release() {
 29     if (InterlockedDecrement(&m_ref) == 0) {
 30         delete this;
 31         return 0;
 32     }
 33     return m_ref;
 34 }
 35 
 36 
 37 
 38 STDMETHODIMP EventCallback::ExitThread(__in ULONG ExitCode) {
 39     return S_OK;
 40 }
 41 
 42 STDMETHODIMP EventCallback::SessionStatus(__in ULONG Status) {
 43     return S_OK;
 44 }
 45 
 46 STDMETHODIMP EventCallback::ExitProcess(__in ULONG ExitCode) {
 47     return S_OK;
 48 }
 49 
 50 STDMETHODIMP EventCallback::GetInterestMask(__out PULONG Mask) {
 51     auto retval = S_OK;
 52 
 53     if (Mask != nullptr)
 54         *Mask = DEBUG_EVENT_BREAKPOINT;
 55 
 56     return retval;
 57 }
 58 
 59 
 60 STDMETHODIMP EventCallback::Breakpoint(__in PDEBUG_BREAKPOINT Bp) {
 61     return DEBUG_STATUS_BREAK;
 62 }
 63 
 64 STDMETHODIMP EventCallback::SystemError(__in ULONG Error, __in ULONG Level) {
 65     return S_OK;
 66 }
 67 
 68 
 69 STDMETHODIMP EventCallback::ChangeEngineState(__in ULONG Flags, __in ULONG64 Argument) {
 70     return S_OK;
 71 }
 72 
 73 STDMETHODIMP EventCallback::ChangeSymbolState(__in ULONG Flags, __in ULONG64 Argument) {
 74     return S_OK;
 75 }
 76 
 77 STDMETHODIMP EventCallback::ChangeDebuggeeState(__in ULONG Flags, __in ULONG64 Argument) {
 78     return S_OK;
 79 }
 80 
 81 STDMETHODIMP EventCallback::UnloadModule(__in_opt PCSTR ImageBaseName, __in ULONG64 BaseOffset) {
 82     return S_OK;
 83 }
 84 
 85 STDMETHODIMP EventCallback::Exception(__in PEXCEPTION_RECORD64 Exception, __in ULONG FirstChance) {
 86     return S_OK;
 87 }
 88 
 89 
 90 STDMETHODIMP EventCallback::CreateThread(__in ULONG64 Handle, __in ULONG64 DataOffset, __in ULONG64 StartOffset) {
 91     return S_OK;
 92 }
 93 
 94 STDMETHODIMP EventCallback::LoadModule(__in ULONG64 ImageFileHandle, __in ULONG64 BaseOffset, __in ULONG ModuleSize,
 95                                         __in_opt PCSTR ModuleName, __in_opt PCSTR ImageName, __in ULONG CheckSum,
 96                                         __in ULONG TimeDateStamp) {
 97 
 98     return S_OK;
 99 }
100 
101 
102 STDMETHODIMP EventCallback::CreateProcess(__in ULONG64 ImageFileHandle, __in ULONG64 Handle, __in ULONG64 BaseOffset,
103     __in ULONG  ModuleSize, __in_opt  PCSTR ModuleName, __in_opt  PCSTR ImageName, __in ULONG CheckSum,
104     __in ULONG TimeDateStamp, __in ULONG64 InitialThreadHandle, __in ULONG64 ThreadDataOffset,
105     __in ULONG64 StartOffset) {
106 
107     return S_OK;
108 }

EventCallback.cpp

 

In summary, the debugger engine is available and it allows building cool stuff around it. I’ll keep you posted on the development/progress of this add-in, so stay tuned Smile

Angel

Leave a Reply

Your email address will not be published. Required fields are marked *

Filed under: Microsoft - @ February 6, 2015 5:46 am

Tags: , ,