Monkey Patching is not too bad

Designed by JavierFeria / Freepik

History

A few months ago I’ve started to contribute with returns, an amazing Open Source Python library with a lot of containers to help us in many ways, I won’t cover those containers here but you can access the documentation page to know more about them.

Here I’m going to cover why we chose the monkey patching technique to implements a feature, Improve Failure Traceability, to the Result containers.

Just a brief explanation of what is a Result container, basically your code can take two ways:

  • Success, the function computation was successfully executed
  • Failure, the function computation broke due to business logic or an Exception was raised

That container abstracts those ways for us, see the example below:

Result container example

Using Result can be a great idea because you don’t have more to deal with raised Business Exceptions and put try...except everywhere in our code, we just have to return a Failure container.

Tracing Failure: feature explanation

Failure is great, but Exceptions give to us an important thing: where it was raised.

Inspired by the Tracing Failure feature from dry-rb, which is a collection of Ruby Libraries, we started the discussion to give this option to the returns users and one important thing was considered to implement it, users that will not use this feature can’t have their system/application performance affected!

The simple, easiest and unique (I guess) way to implement the feature is to get the call stack and make some manipulation with it. In Python is simple to get the call stack, but it’s a heavy operation that can affect the performance if it’s often called. Below you can see extracted metrics about memory consumption when creating Failure container getting the call stack and don’t getting it, respectively:

Memory consumption without trace implemented
Memory consumption with trace implemented

How can we implement the tracing feature?

We already know the tracing has to be optional because users that won’t use it can’t be affected, and as we can see when the trace is enabled we have a huge memory overhead!

To get the trace optional we had two options:

  • Use an environment variable
  • Use monkey patching

By the title of this article you know might we have chosen the second option.

Why monkey patching?

Monkey Patching is a more sophisticated and elegant approach than an environment variable, we can separate in the right way the tracing feature code from the class we want to be traced and we don’t depend on an external resource. Using an env variable will end up with something like this in our classes, we can decouple the if statement from the class but in somewhere of our code the if will be there:

Example class with environment variable

Monkey patching is a known friend of Python programmers, we use it a lot while writing tests to mock everything we want (e.g. API request, database interaction), but it’s not used too much in our “production” code because have some drawbacks like it’s not Thread Safe and can create many bugs since it can affect our entire code base at runtime. But we understood that tracing feature is for development purposes, we don’t care about thread safety problem and we know exactly we are monkey patching!

Can we turn the monkey patching technique thread safe in Python?

Yes, we can but it’s a subject for another article.

After some discussions, we finally delivered our Tracing Failures feature, and now users can active explicit in their code the tracing for the Result containers.

collect_traces example

The output should be something like:

/returns/returns/result.py:529 in `Failure` /example_folder/example.py:5 in `get_failure` /example_folder/example.py:1 in `<module>`

Extra

The main goal of the tracing feature is to give to the user the ability to find where the failure occurred, but if you don't want to analyze the call stack and know the scenario where the failure occurs use the returns pytest plugin to verify your hypothesis. We provide a fixture called returns with has_trace method, see the example below:

pytest plugin example

If test_if_failure_is_created_at_example_functions fails we know the failure is not created at example_function or in its internal calls.

Related links

Also published on:

--

--

--

Software Engineer at @globocom. Passionate for challenges, technology, good code and functional programming enthusiast. @dry_py core developer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

4 Exciting New CSS Features in 2022

How I used Scrum to Teach Myself Software Engineering

Functions in Python

Hexadecimal in programming

Terraform IAC: Google Cloud Build

Web Performance vs. User Engagement

How DBS dispelled the myths of Chaos Engineering

Self learning tips for hungry minds

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Pablo Aguilar

Pablo Aguilar

Software Engineer at @globocom. Passionate for challenges, technology, good code and functional programming enthusiast. @dry_py core developer

More from Medium

My TCS Elevate Wings 1 DCA Experience

Understanding List Comprehensions in Python

Debug Your Python Code like a Master

Build Array from Permutation — Day 103(Python)