Profiling Jupyter Notebook
Now let’s try to understand what are the steps needed to profile in Jupyter Notebook.
1. Installing the required libraries:
First we need to ensure that we have the necessary libraries installed in out system. As we need ‘line_profiler‘ for ‘%lprun’ and ‘cProfile‘ for ‘%prun‘. If it’s not installed already in our notebook, we can install them by using the following commad:
!pip install line_profiler
2. Load the Magic Commands
In our Jupyter Notebook, we then need to load the ‘%lprun’ and ‘%prun’ magic commands. We’ll now use the following commands to load them:
Python3
% load_ext line_profiler % load_ext cProfile |
3. Profiling the Code
Now we’ll use ‘%lprun’ to profile specific lines of code and ‘%prun’ to profile entire functions as we have dicussed about this in the introduction. Let’s understand this with a small example:
Python3
% load_ext line_profiler def fib(n): if n < 2 : return n else : return fib(n - 1 ) + fib(n - 2 ) % lprun - f fib fib( 20 ) |
Output:
Timer unit: 1e-09 s
Total time: 0.00622773 s
File: /tmp/ipykernel_601603/4220250964.py
Function: fib at line 3
Line # Hits Time Per Hit % Time Line Contents
==============================================================
3 def fib(n):
4 21891 3171786.0 144.9 50.9 if n < 2:
5 10946 1101316.0 100.6 17.7 return n
6 else:
7 10945 1954627.0 178.6 31.4 return fib(n-1) + fib(n-2)
When you run the above code, your output will be telling you about; Line, Hits, Time, Per Hit, % Time, and Line Contents. It’ll be somewhat looking like this:
Magic Commands for Profiling in Jupyter Notebook
Jupyter Notebook is a versatile tool or IDE widely used mostly by data scientists, researchers, and programmers for interactive computing and data analysis, dashboards, and visualizations. It offers a unique and rich set of features. Some topics are there it can perform such as: running the code in different languages, timing the code execution, debugging the code, and profiling. We’ll discuss the commands that work like magic for profiling in Jupyter Notebook. Magic Commands are the special commands in Jupyter Notebook that either start with the ‘% ‘ or ‘%% ‘ sign and perform different operations and tasks. Profiling helps in identifying bottlenecks, optimizing code, and improving overall performance. So, let’s explore the world of magic commands in Jupyter Notebook, specifically designed for profiling our Python code. Before we jump into it, we first need to understand some of the key concepts that we should know about profiling in Jupyter Notebook. So, what actually profiling is in Jupyter Notebook?