Working with Code Objects¶
Code type is the foundational abstraction in
codetransformer. It provides high-level APIs for working with
logically-grouped sets of instructions and for converting to and from CPython’s
Constructing Code Objects¶
The most common way constructing a Code object is to use the
from_pycode() classmethod, which accepts a
There are two common ways of building raw code objects:
- CPython functions have a
__code__attribute, which contains the bytecode executed by the function.
compile()builtin can compile a string of Python source code into a code object.
from_pycode(), we can build a Code
object and inspect its contents:
>>> from codetransformer import Code >>> def add2(x): ... return x + 2 ... >>> co = Code.from_pycode(add.__code__) >>> co.instrs (LOAD_FAST('x'), LOAD_CONST(2), BINARY_ADD, RETURN_VALUE) >>> co.argnames ('x',) >>> c.consts (2,)
We can convert our Code object back into its raw form via the
>>> co.to_pycode() <code object add2 at 0x7f6ba05f2030, file "<stdin>", line 1>
Once we have the ability to convert to and from an abstract code representation, we gain the ability to perform transformations on that abtract representation.
Let’s say that we want to replace the addition operation in our
function with a multiplication. We could try to mutate our
Code object directly before converting back to
Python bytecode, but there are many subtle invariants  between the
instructions and the other pieces of metadata that must be maintained to ensure
that the generated output can be executed correctly.
Rather than encourage users to mutate Code objects in place,
codetransformer provides the
class, which allows users to declaratively describe operations to perform on
sequences of instructions.
Implemented as a
CodeTransformer, our “replace
additions with multiplications” operation looks like this:
from codetransformer import CodeTransformer, pattern from codetransformer.instructions import BINARY_ADD, BINARY_MULTIPLY class add2mul(CodeTransformer): @pattern(BINARY_ADD) def _add2mul(self, add_instr): yield BINARY_MULTIPLY().steal(add_instr)
The important piece here is the
_add2mul method, which has been decorated
pattern. Patterns provide an API for
describing sequences of instructions to match against for replacement and/or
CodeTransformer base class
looks at methods with registered patterns and compares them against the
instructions of the Code object under transformation. For each matching
sequence of instructions, the decorated method is called with all matching
instructions *-unpacked into the method. The method’s job is to take the
input instructions and return an iterable of new instructions to serve as
replacements. It is often convenient to implement transformer methods as
generator functions, as we’ve done here.
In this example, we’ve supplied the simplest possible pattern: a single
instruction type to match.  Our transformer method will be called on
BINARY_ADD instruction in the target code object, and it will yield a
BINARY_MULTIPLY as replacement each time.
To apply a
CodeTransformer to a function, we
construct an instance of the transformer and call it on the function we want to
modify. The result is a new function whose instructions have been rewritten
applying our transformer’s methods to matched sequences of the input function’s
instructions. The original function is not mutated in place.
>>> transformer = add2mul() >>> mul2 = transformer(add2) # mult2 is a brand-new function >>> mul2(5) 10
When we don’t care about having access to the pre-transformed version of a function, it’s convenient and idiomatic to apply transformers as decorators:
>>> @add2mul() ... def mul2(x): ... return x + 2 ... >>> mul2(5) 10
|||For example, if we add a new constant, we have to ensure that we
correctly maintain the indices of existing constants in the generated
|||Many more complex patterns are possible. See the docs for