Finding MSVC with CMake • Izzy Muerte

It would appear that my concept of time continues to be an ephemeral or completely non-existent experience on this plane of reality. I apologize for the long delay, given that I last wrote in June that this very post was Coming Soon™. Today, I’ll be discussing in more detail one of the ways you can write a CMake Toolchain File so that you no longer have to use vcvarsall.bat, devenv.cmd, or other “shortcut” when you want to target MSVC. This is a more technical post than what I usually write, so apologies if you’re on mobile. Some of the code snippets might be a pain to read. I’m aware my site’s design has bitrotted to a degree. I’ll hopefully spend some time redesigning some parts of it in the coming months, as I would like to better organize my technical posts, articles, and one-offs.

In addition to relieving yourself of having to run “yet another shell” to extract information, this will let us cache the INCLUDE and LIBPATH directories under CMake, which means that you won’t have to worry about passing flags in or having the correct environment variables on every run.

To kick this off, we need to discuss the meat of where we’ll be acquiring our information: vswhere. The link will take you to the github project, which will explain the details of vswhere, however to quickly sum it up:

It is installed with all Visual Studio versions
It can find all Visual Studio installations
It can provide said information to us in JSON.

This is ideal, as CMake now has JSON parsing support via string(JSON). However, this only gets us the information to select from the possible set of toolchains. The primary work comes from selecting the correct set of options. After all, not everyone wants to select “latest” when they’re building (I sure as shit can’t at work given our target specifications, but I digress), but luckily CMake does provide us some Visual Studio specific variables that MSBuild users will be using anyhow. Because of this, we can effectively hijack them for the Makefile or Ninja based generators:

CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE
CMAKE_VS_PLATFORM_NAME

These variables specifically permit us to set the host and target system architectures, which do matter when we’re selecting a cross compiler of sorts.

We’ll also need a way to specify the edition of CMake (e.g., Community, Professional, BuildTools), as well as the product year (e.g., 2022, 2019, 2017). Technically speaking, these variables can be anything. They are not standard. I would suggest selecting a useful name. For the purpose of this post, however, we’ll be using IZ_MSVS_VERSION, IZ_MSVS_EDITION, and IZ_MSVS_TOOLSET. And if I see anyone using these exact variables out in the wild (and I’ll be able to tell because GitHub’s search is decent now!), I’m gonna post a single issue on your repository with a newspaper (🗞) emoji to digitally bop you one, because you shouldn’t be copying these variable names. At all. Please pick something different.

Searching, Searching, Searching

With all of that out of the way, we’re free to get started writing out toolchain file. The very first thing we want is to make sure our custom variables get copied through to any try_compile calls. This matters a great deal, as our toolchain info would be lost when isolated otherwise:

include_guard(GLOBAL)

set(CMAKE_TRY_COMPILE_PLATFORM_VARIABLES
  IZ_MSVS_VERSION
  IZ_MSVS_EDITION
  IZ_MSVS_TOOLSET)

Next, we need to shim in the value detection for the target and host for Visual Studio. If the VS variables are not set, we can fall back onto the CMAKE_HOST_SYSTEM_PROCESSOR and CMAKE_SYSTEM_PROCESSOR variables.

if (NOT CMAKE_GENERATOR MATCHES "^Visual Studio")
  if (NOT DEFINED CMAKE_SYSTEM_PROCESSOR)
    set(CMAKE_SYSTEM_PROCESSOR "${CMAKE_HOST_SYSTEM_PROCESSOR}")
  endif()
  if (NOT DEFINED CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE)
    set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "x86")
    if (CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "AMD64")
      set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "x64")
    elseif (CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "ARM64")
      set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "arm64")
    endif()
  endif()
  if (NOT DEFINED CMAKE_VS_PLATFORM_NAME)
    set(CMAKE_VS_PLATFORM_NAME "x86")
    if (CMAKE_SYSTEM_PROCESSOR STREQUAL "AMD64")
      set(CMAKE_VS_PLATFORM_NAME "x64")
    elseif (CMAKE_SYSTEM_PROCESSOR STREQUAL "ARM64")
      set(CMAKE_VS_PLATFORM_NAME "arm64")
  endif()
endif()

Next, we need to do 5 things in quick succession:

Find vswhere
Get the Visual Studio installation path
Get the Root Windows SDK Directory
Find the MSVC Tools Version directory
Find the Windows SDK Version Directory

Where is VSWhere?

The reason we want to find vswhere is so that it can be overridden by users at configure time. Letting them set -DVSWHERE_EXECUTABLE=<path> gives a lot more customization to developers, as you can then override the information with whatever you want, allowing for a user-controlled hook over their environment.

# Block was added in CMake 3.25, and lets us create variable scopes without
# using a function.
block(SCOPE_FOR VARIABLES)
  cmake_path(
    CONVERT "$ENV{ProgramFiles\(x86\)}/Microsoft Visual Studio Installer"
    TO_CMAKE_PATH_LIST vswhere.dir
    NORMALIZE)
  # This only temporarily affects the variable since we're inside a block.
  list(APPEND CMAKE_SYSTEM_PROGRAM_PATH "${vswhere.dir}")
  find_program(VSWHERE_EXECUTABLE NAMES vswhere DOC "Visual Studio Locator" REQUIRED)
endblock()

Visual Studio Installation Search

Next we need to find the installation path for the Visual Studio edition that we want to use. If you’re on CI, I would personally set your equivalent IZ_MSVS_EDITION to BuildTools, as this is a smaller install and doesn’t install the Visual Studio directory itself. This is a bit of a doozy, but it can be simplified if you’re willing to always select the latest installation (which is whatever version was most recently updated by the installer not the latest version itself)

  if (DEFINED IZ_MSVS_EDITION)
    set(product "Microsoft.VisualStudio.Product.${IZ_MSVS_EDITION}")
  else()
    set(product "*")
  endif()
  message(CHECK_START "Searching for Visual Studio ${IZ_MSVS_EDITION}")
  execute_process(COMMAND "${VSWHERE_EXECUTABLE}" -nologo -nocolor
      -format json
      -products "${product}"
      -utf8
      -sort
    ENCODING UTF-8
    OUTPUT_VARIABLE candidates
    OUTPUT_STRIP_TRAILING_WHITESPACE)
  string(JSON candidates.length LENGTH "${candidates}")
  string(JOIN " " error "Could not find Visual Studio"
  "${IZ_MSVS_VERSION}"
  "${IZ_MSVS_EDITION}")
  if (candidates.length EQUAL 0)
    message(CHECK_FAIL "no products")
    # You can choose to either hard fail here, or continue
    message(FATAL_ERROR "${error}")
  endif()

  if (NOT IZ_MSVS_VERSION)
    string(JSON candidate.install.path GET "${candidates}" 0 "installationPath")
  else()
    # Unfortunately, range operations are inclusive in CMake for god knows why
    math(EXPR stop "${candidates.length} - 1")
    foreach (idx RANGE 0 ${stop})
      string(JSON version GET "${candidates}" ${idx} "catalog" "productLineVersion")
      if (version VERSION_EQUAL IZ_MSVS_VERSION)
        string(JSON candidate.install.path 
          GET "${candidates}" ${idx} "installationPath")
        break()
      endif()
    endforeach()
  endif()
  if (NOT candidate.install.path)
    message(CHECK_FAIL "no install path found")
    message(FATAL_ERROR "${error}")
  endif()
  cmake_path(
    CONVERT "${candidate.install.path}"
    TO_CMAKE_PATH_LIST candidate.install.path
    NORMALIZE)
  message(CHECK_PASS "found : ${candidate.install.path}")
  set(IZ_MSVS_INSTALL_PATH "${candidate.install.path}"
    CACHE PATH "Visual Studio Installation Path")
endblock()

Windows SDK Root Discovery

Next, we need to find the Windows SDK Root directory. This involves using some of the newer features of the CMake API regarding registry values under windows.

message(CHECK_START "Searching for Windows SDK Root Directory")
cmake_host_system_information(RESULT IZ_MSVS_WINDOWS_SDK_ROOT QUERY
  WINDOWS_REGISTRY "HKLM/SOFTWARE/Microsoft/Windows Kits/Installed Roots"
  VALUE "KitsRoot10"
  VIEW BOTH
  ERROR_VARIABLE error
  PATH)
  if (error)
    message(CHECK_FAIL "not found : ${error}")
  else()
    cmake_path(CONVERT "${IZ_MSVS_WINDOWS_SDK_ROOT}"
      TO_CMAKE_PATH_LIST IZ_MSVS_WINDOWS_SDK_ROOT
      NORMALIZE)
    message(CHECK_PASS "found : ${IZ_MSVS_WINDOWS_SDK_ROOT}")
  endif()

Version Directories

With that out of the way (phew!), we can now move on to the next few steps. Luckily, these two steps are more or less the same. We just want to compare them to a specific value for each case. For this reason, I’ll be writing this as a function, but you can choose to do it inline:

function (msvs::directory out-var)
  if (${out-var})
    return()
  endif()
  cmake_parse_arguments(ARG "" "VARIABLE;PATH;DOC" "" ${ARGN})
  message(CHECK_START "Searching for ${ARG_DOC}")
  # We want to get the list of options, but *not* the full path string, hence
  # the use of `RELATIVE`
  file(GLOB candidates
    LIST_DIRECTORIES YES
    RELATIVE "${ARG_PATH}"
    "${ARG_PATH}/*")
  list(SORT candidates COMPARE NATURAL ORDER DESCENDING)
  if (NOT DEFINED ARG_VARIABLE)
    list(GET candidates 0 ${out-var})
  else()
    foreach (candidate IN LISTS candidates)
      if ("${ARG_VARIABLE}" VERSION_EQUAL candidate)
        set(${out-var} "${candidate}")
        break()
      endif()
    endforeach()
  endif()
  if (NOT ${out-var})
    message(CHECK_FAIL "not found")
  else()
    message(CHECK_PASS "found : ${${out-var}}")
    set(${out-var} "${out-var}" CACHE INTERNAL "${out-var} value")
  endif()
endfunction()

With this function, it’s now as simple as selecting the paths we want:

cmake_language(CALL msvs::directory msvc.tools.version
  IZ_MSVS_TOOLS_VERSION
  DIRECTORY "${IZ_MSVS_INSTALL_PATH}"
  VARIABLE IZ_MSVS_TOOLSET
  DOC "MSVC Toolset")

# Your CMAKE_SYSTEM_VERSION should line up with the minimum SDK version you're
# targeting exactly.
cmake_language(CALL msvs::directory IZ_MSVS_WINDOWS_SDK_VERSION
  DIRECTORY "${IZ_MSVS_WINDOWS_SDK_ROOT}/Include"
  VARIABLE CMAKE_SYSTEM_VERSION
  DOC "Windows SDK")

Finding the Actual Tools

Now that we’ve found all the initial paths we need, we can now extrapolate a ton of additional paths that are needed for finding tools, include paths, and libraries. Hang tight, you just gotta trust the process here:

set(windows.sdk.host "Host${CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE}")
set(windows.sdk.target "${CMAKE_VS_PLATFORM_NAME}")
set(msvc.tools.dir "${IZ_MSVS_INSTALL_PATH}/VS/Tools/MSVC/${IZ_MSVS_TOOLS_VERSION}")

block(SCOPE_FOR VARIABLES)
  list(PREPEND CMAKE_SYSTEM_PROGRAM_PATH
    "${msvc.tools.dir}/bin/${windows.sdk.host}/${windows.sdk.target}"
    "${IZ_MSVS_WINDOWS_SDK_ROOT}/bin/${IZ_MSVS_WINDOWS_SDK_VERSION}/${windows.sdk.target}"
    "${IZ_MSVS_WINDOWS_SDK_ROOT}/bin")
  find_program(CMAKE_MASM_ASM_COMPILER NAMES ml64 ml DOC "MSVC ASM Compiler")
  find_program(CMAKE_CXX_COMPILER NAMES cl REQUIRED DOC "MSVC C++ Compiler")
  find_program(CMAKE_RC_COMPILER NAMES rc REQUIRED DOC "MSVC Resource Compiler")
  find_program(CMAKE_C_COMPILER NAMES cl REQUIRED DOC "MSVC C Compiler")
  find_program(CMAKE_LINKER NAMES link REQUIRED DOC "MSVC Linker")
  find_program(CMAKE_AR NAMES lib REQUIRED DOC "MSVC Archiver")
  find_program(CMAKE_MT NAMES mt REQUIRED DOC "MSVC Manifest Tool")
endblock()

As long as these calls succeed, you’ll know you’ve got a proper toolchain discovered. But! We’re not quite done yet. The last bits of setup needed include setting the correct linker directories and include directories.

Finalizing the ✨Experience✨

This is effectively the time where you could start calling find_library and importing specific operating system libraries you’d like to link to without listing their names simply. It’s up to you how you would like to handle it, but for brevity, I’ll just be calling include_directories and link_directories.

Some of this could also be done entirely with generator expressions, but I’ll leave that as an exercise for the reader.

set(includes ucrt shared um winrt cppwinrt)
set(libs ucrt um)

list(TRANSFORM includes PREPEND "${IZ_MSVS_WINDOWS_SDK_ROOT}/Include/${IZ_MSVS_WINDOWS_SDK_VERSION}/")
list(TRANSFORM lib PREPEND "${IZ_MSVS_WINDOWS_SDK_ROOT}/Lib/${IZ_MSVS_WINDOWS_SDK_VERSION}/")
list(TRANSFORM lib APPEND "/${windows.sdk.target}")

# We could technically set `CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES` and others,
# but not for the library paths.
include_directories(BEFORE SYSTEM "${msvc.tools.dir}/include" ${includes})
link_directories(BEFORE "${msvc.tools.dir}/lib/${windows.sdk.target}" ${lib})

Wrapping Up

And just like that, we’ve now got a custom toolchain file for automatically finding MSVC. It’s quite a bit of work involved, and as you will most undoubtedly discover cannot cover every edge case, which is most likely why autodetection isn’t setup under CMake if you pass -DCMAKE_CXX_COMPILER=cl. It also cannot resolve the main issue that I have with MSVC: it’s absolute dogwater. I have never seen a single instance where MSVC outshines the competition. Any point where it does the right thing, there are 3-4 places where it does the wrong thing, and then causes a myriad of compatibility issues (e.g., [[no_unique_address]])

Frankly, if I had my way, I’d recommend you stick to using clang or clang-likes (such as zig cc). It’ll work with the Windows SDK, it has better code gen, and your life will be easier.

CMake Build Systems C++