Little Oddity in MASM

During Windows port of vmlatency I had to write a bit of assembly using MASM. I needed to execute few instructions not covert by available intrinsics. While doing the port I found a minor oddity in MASM.

Operand Size for LGDT

I needed a function that simply calls LGDT instruction thus loading Global Descriptor Table register from a given memory location. This is done as a part of host state restoring routine performed by vmlatency.

The tool runs in 64-bit mode only at the moment therefore I only had to write x64 assembly wrappers. LGDT instruction has the following format in 64-bit mode: LGDT m16&64 thus having a pointer to 10-byte (referred as TBYTE by MASM) memory location. Based on the information I expected this assembly code to encode LGDT instruction:

lgdt tbyte ptr [rcx]

But this code doesn’t compile with MASM and produces this error:

error A2024:invalid operand size for instruction

With a bit of experimenting, I found that MASM expects a 6-byte (referred as FWORD) memory location for LGDT instruction. This is valid for 32-bit mode where LGDT has another format: LGDT m16&32.

This is just a minor oddity that is easy to find and get around. Despite this oddity right binary sequence is generated as shown by manual inspect of driver binary and error-free execution:

12006:       0f 01 11                lgdt   (%rcx)

Other Similar Instructions

SGDT has precisely the same operand kind as LGDT, but it stores GDT value to the given memory location instead. MASM ignores operand size specifier for SGDT while only accepts fword ptr for LGDT.

The oddity repeats exactly for LIDT and SIDT instructions that do similar operations with Interrupt Descriptor Table register.