[IP] Opteron Exposed: Reverse Engineering AMD K8 Microcode Updates
Begin forwarded message:
From:
Date: July 28, 2004 8:24:27 AM PDT
If you decide to forward this to others please take my name off the
mail – I am not sure if this is urban legend or real.
Topic: Opteron Exposed: Reverse Engineering AMD K8 Microcode Updates
Name: Anonymous (not@xxxxxxx) 7/22/04
Opteron Exposed: Reverse Engineering AMD K8 Microcode Updates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Summary
~~~~~~~
This document details the procedure for performing microcode updates
on the AMD K8 processors. It also gives background information on the
K8 microcode design and provides information on altering the microcode
and loading the altered update for those who are interested in
microcode hacking.
Source code is included for a simple Linux microcode update driver for
those who want to update their K8's microcode without waiting for the
motherboard vendor to add it to the BIOS. The latest microcode update
blocks are included in the driver.
Background
~~~~~~~~~~
Modern x86 microprocessors from Intel and AMD contain a feature known
as "microcode update", or as the vendors prefer to call it, "BIOS
update". Essentially the processor can reconfigure parts of its own
hardware to fix bugs ("errata") in the silicon that would normally
require a recall.
This is done by loading a block of "patch data" created by the CPU
vendor into the processor using special control registers. Microcode
updates essentially override hardware features with sequences of the
internal RISC-like micro-ops (uops) actually executed by the processor.
They can also replace the implementations of microcoded instructions
already handled by hard-wired sequences in an on-die microcode ROM.
AMD's U.S. Patent 6438664 ("Microcode patch device and method for
patching microcode using match registers and patch routines") goes into
substantial detail on this.
Typically microcode update blocks are stored in the BIOS flash ROM and
loaded into the processor as the system boots. They can also be loaded
by the operating system; for instance, Linux contains a microcode
device driver for Intel chips.
AMD recently released a "BIOS fix" to motherboard makers to address
Errata 109, in which REP MOVS instructions caused subsequent
instructions to be skipped under specific pipeline conditions.
Previously it was not clear if and how AMD even supported microcode
updates in the K8 family until this announcement. After analyzing a
number of BIOS images, it appears that AMD has secretly used the
microcode update facility on several occasions over the past few years,
but obviously avoided publicly disclosing that it actually had bugs
patchable in this manner.
Early K7 (Athlon) cores initially supported microcode updates as well,
until ironically the microcode update mechanism itself was found to be
broken and subsequently listed as an errata!
The following sections describe the microcode update procedure,
obtained by clean room reverse engineering various vendors' BIOS code.
The actual microcode update blocks are embedded in the BIOS image; the
most recent updates (created June 2004) have been included in the Linux
driver source code attached to this description.
Microcode Update Procedure
~~~~~~~~~~~~~~~~~~~~~~~~~~
The update procedure expects the 64-bit virtual address of the update
data, including the 64 byte header, to be in edx:eax:
edx = high 32 bits of 64-bit virtual address
eax = low 32 bits of 64-bit virtual address
ecx = 0xc0010020 (MSR to trigger update)
Execute wrmsr with these register values. If the address and update
block data are valid, wrmsr completes successfully. Otherwise, a GP
fault is taken.
The microcode does not appear to update MSR 0x8B with the new update
signature as it does on Intel processors, despite the fact that some
BIOS code I have analyzed does seem to check this field. It is possible
the MSR is only updated under certain conditions, for instance when
microcode is loaded before initializing the cache controller.
Nonetheless, as we shall see below, the processor is clearly doing
something internally when it claims to accept an update in this manner.
The update generally takes around 5500 clock cycles. This was tested
on an Athlon 64 with CPUID 0x0F48. It was not tested on any other K8
cores, although the driver source code includes updates for CPUIDs
0x0F4A and 0x0F50.
Microcode Block Format
~~~~~~~~~~~~~~~~~~~~~~
The microcode block consists of a 64 byte header and an 896 byte data
area. The processor consumes both the header and data area during an
update. The header consists of various fields. The most important ones
are:
- Offset 0: 32-bit word for update creation date (e.g. 0x20040602)
- Offset 12: 32-bit checksum: sum of all 32-bit words in the data area
- Offset 24: Low 8 bits of the cpuid (e.g. 0xf48 -> 0x48).
- Offset 28: 4 bytes: 0x01 0xaa 0xaa 0xaa (evidently a reference to
(A)MD.)
The microcode blocks are typically padded out to 2048 bytes, just as
the Intel format blocks are.
Microcode Format
~~~~~~~~~~~~~~~~
Surprisingly, the microcode itself is in no way encrypted as it is in
Intel microcode updates; the raw data loaded into the microcode patch
array is directly exposed. The repetitive structure of the data, bit
patterns and fields characteristic of microcode indicate that
apparently no encryption was performed.
U.S. Patent 6438664 describes the most probable structure of this
data; the bit patterns in the update blocks show the outline of the uop
triads and control fields known to exist in K8 microcode. Further
analysis of the microcode format is in progress.
Even more surprising is the total lack of strong authentication that
the update block has not been damaged or altered. The processor's sole
means of validating an update is to take the sum of all 32-bit words in
the 896 byte update block and compare it to the 32-bit checksum at
offset 12; this verification is done by microcode already stored in the
microcode ROM.
I have tested modifying random bits within the update block,
regenerating a correct checksum, and loading the block into the
processor. In many cases the processor accepts the block with no
visible effects; other cases cause a spontaneous reboot.
Most alarming is the way in which certain bit modifications cause the
processor to perform very bizarrely, for instance raising segfaults and
performing incorrect computations on certain microcoded instructions.
The processor also apparently does not check the header to see if the
loaded update matches its exact model and stepping; it is possible to
load updates intended for an Opteron onto an Athlon 64 CPU, although
this will crash the machine or cause bizarre behavior.
Depending on which data block bits are modified, loading an invalid
update apparently causes an internal fault and the CPU spontaneously
reboots.
Implications
~~~~~~~~~~~~
The ability to fundamentally alter instruction decoding and execution
on AMD K8 processors is sure to interest hardware hackers everywhere.
Unfortunately, it is not clear if this has much practical use. The
updates are structured to patch specific microcode lines, and there are
a very limited number of patch slots available (around 64 if the
patented technique was actually implemented as described). Adding
useful new instructions to the ISA is therefore unlikely; at best we
could enable a previously undefined opcode to execute a few lines of
uops and return. The primary purpose of microcode patching is to modify
or disable defective functionality, rather than add new features.
Interestingly, this does have serious implications for system
security. If one is able to get root access on a machine even once, it
is hypothetically possible to install a microcode update specifically
to help compromise security from userspace at a later time. Such an
update could be flashed into the BIOS to make it persistent across
reboots.
For instance, by patching the appropriate microcode lines, it may be
possible to catch an opcode that would normally be illegal, and instead
handle it by tricking the TLB into thinking we're in kernel mode when
in fact the attacker has only compromised a userspace process. From
there, the attacker could control the entire machine, all without
altering a single bit of "software".
Imagine the fiasco that would ensue if a system were compromised by
altering the CPU itself. This would be the hardware equivalent of Ken
Thompson's legendary self-replicating compiler
(http://www.acm.org/classics/sep95). A few years ago, Intel had to
answer to public scrutiny over the exploitability of their own
microcode update feature; their solution was security through obscurity
and layers of encryption and authentication (see
http://www.eetimes.com/news/97/963news/hole.html). Evidently AMD was
not as wise by assuming their microcode was uncrackable.
There may also be a hidden danger to altering K8 microcode without
complete information. It is possible (though very unlikely) that the
microcode could electrically reconfigure signal routing in a fashion
similar to FPGAs, for instance to cut off defective logic and reroute
signals to redundant arrays. This approach has been used in the past
and the AMD patents even suggest it.
If this were the case, there is a very remote chance the CPU itself
could be permanently damaged, for instance, by tri-stating pass
transistors into a high current draw state or adjusting the K8's
voltage and frequency scaling controls out of spec. This is not meant
to discourage potential hackers; I have just seen programmable logic
literally destroyed by buggy "software" bitstreams.
Nonetheless, I suspect that with sufficient analysis or maybe a bit of
inside information, one could do some interesting things with microcode
hacking.
At the very least, the information here should be useful for adding
AMD support to the Linux microcode update driver, which already
supports Intel's update mechanism.
-------------------------------------
You are subscribed as roessler@xxxxxxxxxxxxxxxxxx
To manage your subscription, go to
http://v2.listbox.com/member/?listname=ip
Archives at: http://www.interesting-people.org/archives/interesting-people/