Python Internals (P3) - Setting free

Sumeet Sumeet Follow Feb 26, 2020 · 3 mins read
Python Internals (P3) - Setting free
Share this

Frames, scope, objects and set free


Above image represents the Python environment and essentially is one of the high level ways to represent how the code works internally. While programming in Python we define functions. Every function has it’s own scope which corresponds to frames. Every frame maintain it’s value stack. Value stack is used by bytecode to reference and perform various operations on these values. Once the relevant operation is performed, the values are popped off the stack. This is also one way for Python to do the garbage collections also known as reference counting garbage collector. Let us write another program involving functions in Python as below.

Program 2

Following is the bytecode for the same.

Function Bytecode

When functions are defined and compiled, new frame (scope) is created. Every new function is put on something called as ‘Frame Stack’. When execution is completed i.e. if RETURN is called, the objects on frame stack disappear. While compiling functions in python source code, it doesn’t bind the code blob to the function name given (foo and bar). Function is not bound untill runtime. MAKE_FUNCTION does this job. You can even see this in the bytecode. A single dis command for has identified the functions within the files and has already being represented by individual bytecode for the main function/frame, foo and bar. Within the main function, there is a reference to appropriate memory locations for appropriate functions. However, if you carefully take a look at the values in first column of bytecode (actual line numbers in source code file) they are still maintained at respective places.

Python files are executed from top to bottom. Everything in python is PyObject. PyFrameObject is used to manage the frames and scope with it’s own value stack. The code to the main interpreter can be found in directory “Python > ceval.c”. Below snippet shows the main infinite for loop which.

Function Bytecode

There is a difference between function object and code object. Function contains code and pointer to the environment. Code object doesn’t know how to look at globals and same code object can be used in different functions. For example, if the function body for 2 or more function is exactly the same - then this code is stored in the memory only once. All the functions would then point to this code.

As discussed earlier, everything in Python is PyObject. It is important to know that all the data types are derived from this PyObject. Every PyObject is associated with a set of functions/attributes some of which are common for all and others are specific to derived sub-types. However, PyObject itself is very thin - it only has type and refcount. These functions help key information related to the object handy for use by the interpreter. We have already taken a look at these functions in our first blog post by executing dir command. In order to understand this data model in more detail, you should check out - Python Data Model. Also you can find all the source code files for all the data types in “/Objects” directory.

Alright, that was a quick intro into Python internals. There are ton’s of resources available on the internet. Please make use of it.

Join Newsletter
Get the latest news right in your inbox!
Written by Sumeet Follow
Hi, I am Sumeet, and I believe the world belongs to the doers. Here, I publish my technical tinkering experiences. I hope you like it!