Nisdom. As opposed to Wisdom.

The problem with the future is that it keeps turning into the present.

Embedding a 64-bit Binary in a 32-bit Windows C++ App

| Comments

Some time ago I bumped into a nasty little problem on Windows 7. I had this small Windows tray utility written in C++ whose purpose was, among other things, to detect if Microsoft Word was running and whether it was a top-level app. One of the APIs I was using for that purpose was EnumProcesses. Here’s how that looked like:

1
2
3
4
5
6
7
8
9
10
if (::EnumProcesses(aProcesses, sizeof(aProcesses), &cbNeeded)) {
  cProcesses = cbNeeded / sizeof(DWORD);
  for (i = 0; i < cProcesses; i++) {
    if (aProcesses[i] != 0) {
      HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, aProcesses[i]);
      if (NULL != hProcess) {
        HMODULE hMod;
        DWORD cbNeeded;
        if (::EnumProcessModules(hProcess, &hMod, sizeof(hMod), &cbNeeded)) {
          ::GetModuleFileNameEx(hProcess, hMod, processName, sizeof(processName));

Everything was peachy for me until one day the app could not detect Word anymore. Surprise, surprise, it misbehaved when I ran it for the first time on Windows 7 64-bit and with 64-bit Word 2010. It turned out that from 32-bit virtualised process (running under the WOW64 subsystem) I wasn’t able to enumerate 64-bit processes!

There were two obvious solutions to the problem a) create a 64-bit version of my tray app and distribute it separately and b) use WMI for the job. I won’t go into too many details why I discarded WMI. It suffices to say that it’s much slower compared to using WIN32 APIs directly and it requires you to use COM. More importantly, I had the existing code base already running and I didn’t want to turn everything upside-down when all I needed was to create another project file and to select all correct check-boxes for the 64-bit compilation.

So I went with the 64-bit compilation solution, but there was still something bothering me. How can I avoid putting two separate executable images in the installation package? I remembered I always saw something interesting while using extremely popular Process Explorer utility on 64-bit Windows, but didn’t pay too much attention to it before. Here’s what I saw:

It turns out that Process Explorer creates 64-bit version of itself and, even more interestingly, 32-bit version packs its 64-bit counterpart as a binary resource. Once the procexp.exe is started, it unpacks 64-bit binary from its resources to a temp folder and runs it as a child process. Well, exactly the thing I was hoping to achieve. So let me show you how I did it so you too can create your own, self-unpacking, 64-bit version of your 32-bit app!

  1. First things first, you will need to create an additional 64-bit Visual Studio configuration of your project. This baby needs to compile your code to run natively in 64-bit Windows.
  2. Go to your 32-bit project and add to its Resources->General properties under Additional Include Directories something like $(SolutionDir)x64\$(Configuration). This will enable you to reference your 64-bit binary from the .rc file.
  3. Embedding a 64-bit binary in a 32-bit binary’s resources is dead simple. Just add this somewhere inside your .rc file:
    1
    2
    3
    
    #if !defined (_WIN64)
    IDR_MYTRAYAPP64 RCDATA "MyTrayApp64.exe"
    #endif
    
  4. Also make sure you have following in the Resource.h file:
    1
    2
    3
    
    #ifndef WIN64
    #define IDR_MYTRAYAPP64              400
    #endif
    
    Replace 400 with anything you like better. Also, don’t forget to add WIN64 preprocessor constant to your 64-bit project file. If you now try to build 32-bit binary, you will notice that its size has grown by the amount of the 64-bit binary. Enough with the configurations, lets now head for some coding excerpts that will enable extracting binary from resources and all the good stuff.
  5. I’ll start with the good old entry point function WinMain:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    
    int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE /hPrevInstance/,
                         LPSTR lpCmdLine, int nCmdShow) {
    
    #if defined _WIN64
      if (!getCmdOption(argv, argv + __argc, "-run64", out)) {
        displayMessage(hInstance, IDS_MISSING_ARG_RUN64);
        return FALSE;
      }
    #endif
    
    
    Here I’m reading various command line options. Since code for the 32-bit and 64-bit version is identical, I am using preprocessor guards for the platform specifics. I am merely ensuring that the 64-bit binary can not be run by itself (merely warning the accidental double-clickers).
  6. Further down the road we have something like this:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    
    #if !defined (WIN64)
      if (is64BitWindows()) {
        HRSRC res = ::FindResource(hInstance, MAKEINTRESOURCE(IDR_MYTRAYAPP64), RT_RCDATA);
        if (!res) {
          LOG(ERROR) << "unable to find embedded resource MyTrayApp64.exe";
          return false;
        }
        HGLOBAL resHandle = ::LoadResource(NULL, res);
        if (!resHandle) {
          LOG(ERROR) << "unable to load resource MyTrayApp64.exe";
          return false;
        }
        char resData = (char)::LockResource(resHandle);
        DWORD resSize = ::SizeofResource(NULL, res);
    
        char tempPath[MAX_PATH + 1];
        DWORD tempPathSize = ::GetTempPath(MAX_PATH, tempPath);
        if (tempPathSize == 0 || tempPathSize > MAX_PATH) {
          LOG(ERROR) << "unable to get path to temporary folder";
          return false;
        }
    
        string targetPath(tempPath);
        targetPath.append("MyTrayApp64.exe");
        ofstream outputFile(targetPath, std::ios::binary);
        outputFile.write((const char *)resData, resSize);
        outputFile.close();
    
    Is64BitWindows is a small utility function that returns true if our normal 32-bit process is being run from the WOW 64-bit subsystem i.e. 64-bit Windows.
    1
    2
    3
    4
    5
    6
    
    #if !defined (WIN64)
      bool is64BitWindows() {
        BOOL f64 = FALSE;
        return ::IsWow64Process(::GetCurrentProcess(), &f64) && f64;
      }
    #endif
    
    What happens next is loading and locking specific resource and writing it to a file in a temporary folder. That was easy! But before running that image, I want to do one more small thing.
  7. I’ll create a new job object and assign my currently running process to it:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    
    HANDLE job = NULL;
    if (processInJob == 0) {
      job = ::CreateJobObject(NULL, NULL);
      if (NULL == job) {
        LOG(ERROR) << "unable to create job object";
        return false;
      }
    
      JOBOBJECT_EXTENDED_LIMIT_INFORMATION jeli;
      ::ZeroMemory(&jeli, sizeof(jeli));
      jeli.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE |
        JOB_OBJECT_LIMIT_BREAKAWAY_OK;
      if (0 == ::SetInformationJobObject(job, JobObjectExtendedLimitInformation,
                                         &jeli, sizeof(jeli))) {
        LOG(ERROR) << "unable to setup job object";
        ::CloseHandle(job);
        return false;
      }
    
      if (0 == ::AssignProcessToJobObject(job, ::GetCurrentProcess())) {
        LOG(ERROR) << "could not assign process to job object, error: " << ::GetLastError();
        ::CloseHandle(job);
        return false;
      }
    
      LOG(INFO) << "assigned process pid " << ::GetCurrentProcessId() << " with a job";
    }
    
    If process is part of the job, all other child processes I might create after would also be part of the same job. The interesting flag here is JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE which essentially means that if a parent process dies, all its children will die too. In this way I don’t have to worry about 64-bit process staying alive after its counterpart is gone.
  8. Lastly, I can create a 64-bit process:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    
    STARTUPINFO si;
    PROCESS_INFORMATION pi;
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);
    ZeroMemory(&pi, sizeof(pi));
    
    string params(lpCmdLine);
    params.append(" -run64 &quot;&quot;");
    
    LOG(INFO) << "starting 64 bit app";
    if (::CreateProcess(targetPath.c_str(), const_cast<char*>(params.c_str()),
                        NULL, NULL, TRUE, 0, NULL, NULL, &si, &pi)) {
      LOG(INFO) << "waiting for 64 bit app to exit";
      ::WaitForSingleObject(pi.hProcess, INFINITE);
      ::CloseHandle(pi.hThread);
      ::CloseHandle(pi.hProcess);
      if (NULL != job)
        ::CloseHandle(job);
    }
    
    Strange thing to notice here is that after I’ve created 64-bit process I immediately started waiting for it to exit. This might seem pretty lame, but actually creates a great placeholder for the idea of running both versions of app simultaneously (hence the job). How they will communicate is something you’ll have to figure out according to your needs.

c++, programming, windows

« neo4j tweaks A simple Ruby sitemap.xml generator »

Comments