Some Intermediate Python Knowledge

I realized that even after coding in 6+ years in python, I am not fully aware of all of the python quirky knowledge. There are a lot of things that I know vaguely off of the top of my head that isn't really concrete in my mind. So this blog is tries to summarize all the more "niche" python knowledge into one doc for quick review.

Decorators (kinda metaprogramming?)

def my_decorator(func):
    def wrapper(*args, **kwargs):
        // do things before func call
        result = func(*args, **kwargs)
        // do things after
        return result
    return wrapper

Can be used to extend the behavior of functions of methods

Less common keywords

• yield: Used in generator functions to return values lazily.
• async / await: async defines a function that is asynchronous, await is used for calling an async function
• nonlocal: used inside of nested functions to modify variable from outer, non-global scope
• global: declares a variable as global inside a function
• del: delete anything from memory (more on this later)
• try/except/finally: finally just always occurs
• @: apply decorators
• exec: more of a function, but executes a string of Python code
• breakpoint: you can just add a breakpoint with a keyword woah

(i'm not sure why mdx colors some of the keywords and not others)

Dunder methods (ex. __thing__)

These are methods automatically invoked by python in certain operations

Object initialize and representation

• __init__ : constructor, initialize a new object
• __new__ : Controls object creation before __init__
• __del__ : Destructor, called when an object is deleted (using del keyword)
• __repr__ : Returns a string representation for an object (it's more supposed to be an unambiguous representation, called with repr(obj))
• __str__ : Returns a string representation for reading (defaultly used when printing)
• __bytes__ : Converts object to bytes

Comparison and stuff

• __eq__, __ne__, __lt__, __le__, etc: overloading ==, !=, <, <=, etc
• __bool__: truthiness of object
• __hash__: computes hash of object

Callable and context managers (also more on this)

• __call__ : make an object callable
• __enter__: used in with obj: (context manager)
• __exit__ : Helps cleanup in with statements

Attribute access and descriptors

• __getattr__
• __setattr__
• __delattr__
• __dir__
• __getattribute__
• __set__

Context managers

A construct in python that allows you to manage resources by setting it up and then cleaning up, using the with statement

How to write one

class MyContextManager:
    def __enter__(self):
        return self # value returned is assigned to 'as' variable
    
    def __exit__(self, exc_type, exc_value, traceback):
        return True

or more commonly

from contextlib import contextmanager

@contextmanager
def my_context():
    print("Entering context")
    try:
        yield "Hello"
    finally:
        print("Exiting context")

with my_context() as value:
    print("Inside context:", value)

Some examples

• with open(file) as f
• with lock (for threading)
• with torch.no_grad():

Positional and keyword arguments

*args and **kwargs are used in python function definitions to allow flexible arguments

• *args is known as positional arguments, it basically accepts any number of positional arguments as a tuple
• **kwargs is known as keyword arguments, it accepts any number of keyword arguments and is stored in a dictionary

If you want to use both regular positional arguments and keyword arguments along with *args and **kwargs, you must follow this structures

def function(pos1, pos2, *args, kw1, kw2, **kwargs): pass

Python data structures, and their inner implementations

• List: just a simple dynamic array
• Tuple: Immutable array
• Set: hash table, similar to dictionary but without values
• Dict: hash table

Hashing is implemented with open addressing (In which if a slot is not available, it would probe for the next slot. The other method is chaining where you use an array/linkedlist for an entry) and perturbation probing

• Str: Immutable array (when you concatenate, you create a new string)
  
  
• Deque: Double linked list
• Heap: Binary min-heap
• Counter: same as dictionary but for counting

(to be added topics)

Python packaging ecosystem

• pip, setup.py, pyproject.toml, wheels

Generator Functions

• yield

Garbage collection and memory management

Concurrency and parallelism

Advanced OOP

Registry design pattern