pyvex — Binary Translator

PyVEX provides an interface that translates binary code into the VEX intermediate represenation (IR). For an introduction to VEX, take a look here: https://docs.angr.io/docs/ir.html

Translation Interface

class pyvex.block.IRSB(data, mem_addr, arch, max_inst=None, max_bytes=None, bytes_offset=0, traceflags=0, opt_level=1, num_inst=None, num_bytes=None)

The IRSB is the primary interface to pyvex. Constructing one of these will make a call into LibVEX to perform a translation.

IRSB stands for Intermediate Representation Super-Block. An IRSB in VEX is a single-entry, multiple-exit code block.

Variables:
  • arch (archinfo.Arch) – The architecture this block is lifted under
  • statements (list of IRStmt) – The statements in this block
  • next (IRExpr) – The expression for the default exit target of this block
  • offsIP (int) – The offset of the instruction pointer in the VEX guest state
  • stmts_used (int) – The number of statements in this IRSB
  • jumpkind (str) – The type of this block’s default jump (call, boring, syscall, etc) as a VEX enum string
  • direct_next (bool) – Whether this block ends with a direct (not indirect) jump or branch
  • size (int) – The size of this block in bytes
  • addr (int) – The address of this basic block, i.e. the address in the first IMark
Parameters:
  • data (str or bytes or cffi.FFI.CData or None) – The bytes to lift. Can be either a string of bytes or a cffi buffer object. You may also pass None to initialize an empty IRSB.
  • mem_addr (int) – The address to lift the data at.
  • arch (archinfo.Arch) – The architecture to lift the data as.
  • max_inst – The maximum number of instructions to lift. (See note below)
  • max_bytes – The maximum number of bytes to use.
  • num_inst – Replaces max_inst if max_inst is None. If set to None as well, no instruction limit is used.
  • num_bytes – Replaces max_bytes if max_bytes is None. If set to None as well, no byte limit is used.
  • bytes_offset – The offset into data to start lifting at.
  • traceflags – The libVEX traceflags, controlling VEX debug prints.
  • opt_level – The level of optimization to apply to the IR, 0-2. 2 is highest, 0 is no optimization.

Note

Explicitly specifying the number of instructions to lift (max_inst) may not always work exactly as expected. For example, on MIPS, it is meaningless to lift a branch or jump instruction without its delay slot. VEX attempts to Do The Right Thing by possibly decoding fewer instructions than requested. Specifically, this means that lifting a branch or jump on MIPS as a single instruction (max_inst=1) will result in an empty IRSB, and subsequent attempts to run this block will raise SimIRSBError(‘Empty IRSB passed to SimIRSB.’).

Note

If no instruction and byte limit is used, pyvex will continue lifting the block until the block ends properly or until it runs out of data to lift.

extend(extendwith)

Appends an irsb to the current irsb. The irsb that is appended is invalidated. The appended irsb’s jumpkind and default exit are used.

Parameters:extendwith – The IRSB to append to this IRSB
pp()

Pretty-print the IRSB to stdout.

expressions

A list of all expressions contained in the IRSB.

instructions

The number of instructions in this block

size

The size of this block, in bytes

operations

A list of all operations done by the IRSB, as libVEX enum names

all_constants

Returns all constants in the block (including incrementing of the program counter) as pyvex.const.IRConst.

constants

The constants (excluding updates of the program counter) in the IRSB as pyvex.const.IRConst.

constant_jump_targets

A set of the static jump targets of the basic block.

constant_jump_targets_and_jumpkinds

A dict of the static jump targets of the basic block to their jumpkind.

class pyvex.block.IRTypeEnv(arch, types=None)

An IR type environment.

Variables:types (list of str) – A list of the types of all the temporaries in this block as VEX enum strings. types[3] is the type of t3.
lookup(tmp)

Return the type of temporary variable tmp as an enum string

add(ty)

Add a new tmp of type ty to the environment. Returns the number of the new tmp.

IR Components

class pyvex.stmt.IRStmt

IR statements in VEX represents operations with side-effects.

class pyvex.stmt.NoOp

A no-operation statement. It is usually the result of an IR optimization.

class pyvex.stmt.IMark(addr, length, delta)

An instruction mark. It marks the start of the statements that represent a single machine instruction (the end of those statements is marked by the next IMark or the end of the IRSB). Contains the address and length of the instruction.

class pyvex.stmt.AbiHint(base, length, nia)

An ABI hint, provides specific information about this platform’s ABI.

class pyvex.stmt.Put(data, offset)

Write to a guest register, at a fixed offset in the guest state.

class pyvex.stmt.PutI(descr, ix, data, bias)

Write to a guest register, at a non-fixed offset in the guest state.

class pyvex.stmt.WrTmp(tmp, data)

Assign a value to a temporary. Note that SSA rules require each tmp is only assigned to once. IR sanity checking will reject any block containing a temporary which is not assigned to exactly once.

class pyvex.stmt.Store(addr, data, end)

Write a value to memory..

class pyvex.stmt.CAS(addr, dataLo, dataHi, expdLo, expdHi, oldLo, oldHi, end)

an atomic compare-and-swap operation.

class pyvex.stmt.LLSC(addr, storedata, result, end)

Either Load-Linked or Store-Conditional, depending on STOREDATA. If STOREDATA is NULL then this is a Load-Linked, else it is a Store-Conditional.

class pyvex.stmt.MBE(event)
class pyvex.stmt.Dirty(cee, guard, args, tmp, mFx, mAddr, mSize, nFxState)
class pyvex.stmt.Exit(guard, dst, jk, offsIP)

A conditional exit from the middle of an IRSB.

class pyvex.stmt.LoadG(end, cvt, dst, addr, alt, guard)

A guarded load.

class pyvex.stmt.StoreG(end, addr, data, guard)

A guarded store.

class pyvex.expr.IRExpr

IR expressions in VEX represent operations without side effects.

child_expressions

A list of all of the expressions that this expression ends up evaluating.

constants

A list of all of the constants that this expression ends up using.

class pyvex.expr.Binder(binder)

Used only in pattern matching within Vex. Should not be seen outside of Vex.

class pyvex.expr.VECRET
class pyvex.expr.GSPTR
class pyvex.expr.GetI(descr, ix, bias)

Read a guest register at a non-fixed offset in the guest state.

class pyvex.expr.RdTmp(tmp)

Read the value held by a temporary.

class pyvex.expr.Get(offset, ty)

Read a guest register, at a fixed offset in the guest state.

class pyvex.expr.Qop(op, args)

A quaternary operation (4 arguments).

class pyvex.expr.Triop(op, args)

A ternary operation (3 arguments)

class pyvex.expr.Binop(op, args)

A binary operation (2 arguments).

class pyvex.expr.Unop(op, args)

A unary operation (1 argument).

class pyvex.expr.Load(end, ty, addr)

A load from memory.

class pyvex.expr.Const(con)

A constant expression.

class pyvex.expr.ITE(cond, iffalse, iftrue)

An if-then-else expression.

class pyvex.expr.CCall(retty, cee, args)

A call to a pure (no side-effects) helper C function.

exception pyvex.expr.PyvexOpMatchException
exception pyvex.expr.PyvexTypeErrorException
class pyvex.const.IRConst
class pyvex.const.U1(value)
class pyvex.const.U8(value)
class pyvex.const.U16(value)
class pyvex.const.U32(value)
class pyvex.const.U64(value)
class pyvex.const.F32(value)
class pyvex.const.F32i(value)
class pyvex.const.F64(value)
class pyvex.const.F64i(value)
class pyvex.const.V128(value)
class pyvex.const.V256(value)
pyvex.const.get_type_size(ty)

Returns the size, in BITS, of a VEX type specifier e.g., Ity_I16 -> 16

Parameters:ty
Returns:
pyvex.const.get_type_spec_size(ty)

Get the width of a “type specifier” like I16U or F16 or just 16 (Yes, this really just takes the int out. If we must special-case, do it here. :param tyspec: :return:

class pyvex.enums.VEXObject

The base class for Vex types.

class pyvex.enums.IRCallee(regparms, name, addr, mcx_mask)

Describes a helper function to call.

class pyvex.enums.IRRegArray(base, elemTy, nElems)

A section of the guest state that we want te be able to index at run time, so as to be able to describe indexed or rotating register files on the guest.

Variables:
  • base (int) – The offset into the state that this array starts
  • elemTy (str) – The types of the elements in this array, as VEX enum strings
  • nElems (int) – The number of elements in this array

Lifting System

exception pyvex.lift.LiftingException
class pyvex.lift.Lifter(arch, addr)

A lifter is a class of methods for processing a block.

Variables:
  • data – The bytes to lift as either a python string of bytes or a cffi buffer object.
  • bytes_offset – The offset into data to start lifting at.
  • max_bytes – The maximum number of bytes to lift. If set to None, no byte limit is used.
  • max_inst – The maximum number of instructions to lift. If set to None, no instruction limit is used.
  • opt_level – The level of optimization to apply to the IR, 0-2. Most likely will be ignored in any lifter other then LibVEX.
  • traceflags – The libVEX traceflags, controlling VEX debug prints. Most likely will be ignored in any lifter other than LibVEX.
  • allow_lookback – Should the LibVEX arm-thumb lifter be allowed to look before the current instruction pointer. Most likely will be ignored in any lifter other than LibVEX.
lift()

Lifts the data using the information passed into _lift. Should be overridden in child classes.

Should set the lifted IRSB to self.irsb. If a lifter raises a LiftingException on the data, this signals that the lifter cannot lift this data and arch and the lifter is skipped. If a lifter can lift any amount of data, it should lift it and return the lifted block with a jumpkind of Ijk_NoDecode, signalling to pyvex that other lifters should be used on the undecodable data.

pyvex.lift.lift(irsb, arch, addr, data, max_bytes=None, max_inst=None, bytes_offset=None, opt_level=1, traceflags=False)

Recursively lifts blocks using the registered lifters and postprocessors. Tries each lifter in the order in which they are registered on the data to lift.

If a lifter raises a LiftingException on the data, it is skipped. If it succeeds and returns a block with a jumpkind of Ijk_NoDecode, all of the lifters are tried on the rest of the data and if they work, their output is appended to the first block.

Parameters:
  • irsb (IRSB) – The IRSB to set to the lifted block (overriden by the lifted block)
  • arch (archinfo.Arch) – The arch to lift the data as.
  • addr – The starting address of the block. Effects the IMarks.
  • data – The bytes to lift as either a python string of bytes or a cffi buffer object.
  • max_bytes – The maximum number of bytes to lift. If set to None, no byte limit is used.
  • max_inst – The maximum number of instructions to lift. If set to None, no instruction limit is used.
  • bytes_offset – The offset into data to start lifting at.
  • opt_level – The level of optimization to apply to the IR, 0-2. 2 is maximum optimization, 0 is no optimization.
  • traceflags – The libVEX traceflags, controlling VEX debug prints.

Note

Explicitly specifying the number of instructions to lift (max_inst) may not always work exactly as expected. For example, on MIPS, it is meaningless to lift a branch or jump instruction without its delay slot. VEX attempts to Do The Right Thing by possibly decoding fewer instructions than requested. Specifically, this means that lifting a branch or jump on MIPS as a single instruction (max_inst=1) will result in an empty IRSB, and subsequent attempts to run this block will raise SimIRSBError(‘Empty IRSB passed to SimIRSB.’).

Note

If no instruction and byte limit is used, pyvex will continue lifting the block until the block ends properly or until it runs out of data to lift.

pyvex.lift.register(lifter, arch_name)

Registers a Lifter or Postprocessor to be used by pyvex. Lifters are are given priority based on the order in which they are registered. Postprocessors will be run in registration order.

Parameters:lifter – The Lifter or Postprocessor to register
class pyvex.lift.libvex.LibVEXLifter(arch, addr)
lift()

Lifts the data using the information passed into _lift. Should be overridden in child classes.

Should set the lifted IRSB to self.irsb. If a lifter raises a LiftingException on the data, this signals that the lifter cannot lift this data and arch and the lifter is skipped. If a lifter can lift any amount of data, it should lift it and return the lifted block with a jumpkind of Ijk_NoDecode, signalling to pyvex that other lifters should be used on the undecodable data.

pyvex.lift.util.irsb_postprocess.irsb_postproc_flatten(irsb_old, irsb_new=None)
Parameters:
  • irsb_old (IRSB) – The IRSB to be flattened
  • irsb_new (IRSB) – the IRSB to rewrite the instructions of irsb_old to. If it is None a new empty IRSB will be created
Returns:

the flattened IRSB

Return type:

IRSB

pyvex.lift.util.vex_helper.make_format_op_generator(fmt_string)

Return a function which generates an op format (just a string of the vex instruction)

Functions by formatting the fmt_string with the types of the arguments

exception pyvex.lift.util.lifter_helper.ParseError
class pyvex.lift.util.lifter_helper.GymratLifter(arch, addr)

This is a base class for lifters that use Gymrat. For most architectures, all you need to do is subclass this, and set the property “instructions” to be a list of classes that define each instruction. By default, a lifter will decode instructions by attempting to instantiate every class until one works. This will use an IRSBCustomizer, which will, if it succeeds, add the appropriate VEX instructions to a pyvex IRSB. pyvex, when lifting a block of code for this architecture, will call the method “lift”, which will produce the IRSB of the lifted code.

lift(disassemble=False, dump_irsb=False)

Lifts the data using the information passed into _lift. Should be overridden in child classes.

Should set the lifted IRSB to self.irsb. If a lifter raises a LiftingException on the data, this signals that the lifter cannot lift this data and arch and the lifter is skipped. If a lifter can lift any amount of data, it should lift it and return the lifted block with a jumpkind of Ijk_NoDecode, signalling to pyvex that other lifters should be used on the undecodable data.

class pyvex.lift.util.instr_helper.Instruction(bitstrm, arch, addr)

Base class for an Instruction. You should make a subclass of this for each instruction you want to lift. These classes will contain the “semantics” of the instruction, that is, what it _does_, in terms of the VEX IR.

You may want to subclass this for your architecture, and add arch-specific handling for parsing, argument resolution, etc, and have instructions subclass that instead.

The core parsing functionality is done via a “bit format”. Each instruction should be a subclass of Instruction, and will be parsed by comparing bits in the provided bitstream to symbols in the bit_format member of the class. Bit formats are strings of symbols, like those you’d find in an ISA document, such as “0010rrrrddddffmm” 0 or 1 specify hard-coded bits that must match for an instruction to match. Any letters specify arguments, grouped by letter, which will be parsed and provided as bitstrings in the “data” member of the class as a dictionary. So, in our example, the bits 0010110101101001, applied to format string 0010rrrrddddffmm will result in the following in self.data: {‘r’: ‘1101’,

‘d’: ‘0110’, ‘f’: ‘10’, ‘m’: ‘01’}
Implement compute_result to provide the “meat” of what your instruction does.
You can also implement it in your arch-specific subclass of Instruction, to handle things common to all instructions, and provide instruction implementations elsewhere..

We provide the VexValue syntax wrapper to make expressing instruction semantics easy. You first convert the bitstring arguments into VexValues using the provided convenience methods (self.get/put/load) store/etc. This loads the register from the actual registers into a temporary value we can work with. You can then write it back to a register when you’re done. For example, if you have the register in ‘r’, as above, you can make a VexValue like this: r_vv = self.get(int(self.data[‘r’], 2), Type.int_32) If you then had an instruction to increment r, you could simply: return r_vv += 1 You could then write it back to the register like this: self.put(r_vv, int(self.data[‘r’, 2))

Note that most architectures have special flags that get set differently for each instruction, make sure to implement those as well. (override set_flags() )

Override parse() to extend parsing; for example, in MSP430, this allows us to grab extra words from the bitstream when extra immediate words are present.

All architectures are different enough that there’s no magic recipe for how to write a lifter; See the examples provided by gymrat for ideas of how to use this to build your own lifters quickly and easily.

Create an instance of the instruction :param irsb_c: The IRSBCustomizer to put VEX instructions into :param bitstrm: The bitstream to decode instructions from :param addr: The address of the instruction to be lifted, used only for jumps and branches

fetch_operands()

Get the operands out of memory or registers Return a tuple of operands for the instruction :return:

lift(irsb_c, past_instructions, future_instructions)

This is the main body of the “lifting” for the instruction. This can/should be overriden to provide the general flow of how instructions in your arch work. For example, in MSP430, this is:

  1. Figure out what your operands are by parsing the addressing, and load them into temporary registers
  2. Do the actual operation, and commit the result, if needed.
  3. Compute the flags
Returns:
commit_result(res)

This where the result of the operation is written to a destination. This happens only if compute_result does not return None, and happens before compute_flags is called. Override this to specify how to write out the result. The results of fetch_operands can be used to resolve various addressing modes for the write outward. A common pattern is to return a function from fetch_operands which will be called here to perform the write.

Parameters:args – A tuple of the results of fetch_operands and compute_result

:return n/a

compute_result(*args)

This is where the actual operation performed by your instruction, excluding the calculation of flags, should be performed. Return the VexValue of the “result” of the instruction, which may be used to calculate the flags later. For example, for a simple add, with arguments src and dst, you can simply write:

return src + dst:
Parameters:args
Returns:A VexValue containing the “result” of the operation.
compute_flags(*args)

Most CPU architectures have “flags” that should be computed for many instructions. Override this to specify how that happens. One common pattern is to define this method to call specifi methods to update each flag, which can then be overriden in the actual classes for each instruction. :return: n/a

match_instruction(data, bitstrm)

Override this to extend the parsing functionality. This is great for if your arch has instruction “formats” that have an opcode that has to match. :param data: :param bitstrm: :return: data

disassemble()

Return the disassembly of this instruction, as a string. Override this in subclasses. :return: The address (self.addr), the instruction’s name, and a list of its operands, as strings

load(addr, ty)

Load a value from memory into a VEX temporary register. :param addr: The VexValue containing the addr to load from. :param ty: The Type of the resulting data :return: a VexValue

constant(val, ty)

Creates a constant as a VexValue :param val: The value, as an integer :param ty: The type of the resulting VexValue :return: a VexValue

get(reg, ty)

Load a value from a machine register into a VEX temporary register. All values must be loaded out of registers before they can be used with operations, etc and stored back into them when the instruction is over. See Put().

Parameters:
  • reg – Register number as an integer, or register string name
  • ty – The Type to use.
Returns:

A VexValue of the gotten value.

put(val, reg)

Puts a value from a VEX temporary register into a machine register. This is how the results of operations done to registers get committed to the machine’s state. :param val: The VexValue to store (Want to store a constant? See Constant() first) :param reg: The integer register number to store into, or register name :return: None

store(val, addr)

Store a VexValue in memory at the specified loaction. :param val: The VexValue of the value to store :param addr: The VexValue of the address to store into :return: None

jump(condition, to_addr, jumpkind='Ijk_Boring', ip_offset=None)

Jump to a specified destination, under the specified condition. Used for branches, jumps, calls, returns, etc. :param condition: The VexValue representing the expression for the guard, or None for an unconditional jump :param to_addr: The address to jump to. :param jumpkind: The JumpKind to use. See the VEX docs for what these are; you only need them for things aren’t normal jumps (e.g., calls, interrupts, program exits, etc etc) :return: None

ccall(ret_type, func_obj, args)

Creates a CCall operation. A CCall is a procedure that calculates a value at runtime, not at lift-time. You can use these for flags, unresolvable jump targets, etc. We caution you to avoid using them when at all possible though.

For an example of how to write and use a CCall, see gymrat/bf/lift_bf.py :param ret_type: The return type of the CCall :param func_obj: The function object to eventually call. :param args: List of arguments to the function :return: A VexValue of the result.

Errors

exception pyvex.errors.PyVEXError