Colab, Jupyter or VS Code, and which one for which job

When I started Python I had never programmed, and the names Colab, Jupyter and VS Code kept coming up. I ended up using all of them, each for a different job. Four words come with them: notebook, cell, kernel and virtual environment. I understand them by things I already know as a chemical engineer.

Notebook, cell, kernel, virtual environment

A notebook is a document that keeps the code, the output of the code and your own notes on one page. On disk it is one file ending in .ipynb.11Jupyter Notebook documentation, “The Jupyter Notebook”. https://jupyter-notebook.readthedocs.io/en/stable/notebook.html The closest thing I know is a run sheet, where the instruction, the reading and the remark sit next to each other.

A cell is one box in that document. You write a few lines of code in it, you run it, and the result shows right under it.11Jupyter Notebook documentation, “The Jupyter Notebook”. https://jupyter-notebook.readthedocs.io/en/stable/notebook.html So a cell is like one step in a procedure. You do the step, you look at the reading, then you go to the next step.

The kernel is the thing that actually does the calculation. The page you see only sends the code of a cell to the kernel and shows what comes back.11Jupyter Notebook documentation, “The Jupyter Notebook”. https://jupyter-notebook.readthedocs.io/en/stable/notebook.html The Jupyter documentation describes kernels as separate processes that run on their own and talk to the notebook.22Jupyter documentation, “Kernels (programming languages)”. https://docs.jupyter.org/en/latest/projects/kernels.html I think of the page as the display and the kernel as the instrument behind it. The instrument keeps its state, so a data table you loaded in cell 3 is still in memory when you run cell 10. If you restart the kernel it is like switching the instrument off and on. Everything in memory is gone and you run the cells again, although the old outputs are still printed on the page.

A virtual environment, or venv, is a folder that holds one copy of Python and the libraries one project needs, kept apart from the other projects and from the Python of the computer itself.33Python 3 documentation, “venv, creation of virtual environments”. https://docs.python.org/3/library/venv.html It is like the toolbox you keep with one rig, so nothing gets borrowed for another job. The kernel of a notebook on your own computer runs from one of these folders, and that decides which libraries the notebook can see.

Colab for the first evening

For my first Python I asked for an online platform because I did not want to install anything, and I went with Google Colab. Colab is a notebook service hosted by Google that needs no setup, and the notebooks are stored in Google Drive.44Google, “Colaboratory frequently asked questions”. https://research.google.com/colaboratory/faq.html So the kernel runs on a machine at Google and you only need a browser. I ran the first cell and saw output, and the same evening I did a small tutorial problem on a first-order reactor in it. Later I did Python basics and NumPy in Colab too. Colab removes the install step entirely. The machine at Google is deleted after it sits idle for a while,44Google, “Colaboratory frequently asked questions”. https://research.google.com/colaboratory/faq.html so I would not leave a long run on it.

VS Code for the folder and the terminal

VS Code is a general editor. It shows a project folder and its files, and it has a terminal built in. I already had it for LaTeX. So when I wanted to do an analysis of public employment data on my own computer, step by step and by myself, VS Code was the place I already knew.

That evening was mostly a fight with the environment, a typo in a pip install and a Python that was not the one I meant, and when it was over the Jupyter server came up on localhost:8888, which is an address on your own computer and nothing on the internet.

VS Code can also open and run notebooks itself through its Jupyter extension.55Visual Studio Code documentation, “Jupyter Notebooks in VS Code”. https://code.visualstudio.com/docs/datascience/jupyter-notebooks I ran mine in a browser tab with VS Code beside it for the folder and the terminal.

Jupyter when I had to rerun the work myself

The next job was a modeling exercise. Claude was running it in the browser and it was taking too long, so I stopped it and moved the work to my own computer, one step at a time. I made a venv, installed the libraries, and started with a script, which is a single file that runs from top to bottom.

pip install pandas numpy scikit-learn xgboost shap matplotlib seaborn

Then I switched to Jupyter when I saw it would be easier cell by cell. With a script, if something fails near the end you run the whole file again. With a notebook the loaded data stays in the kernel, so you fix the one cell that failed and run only that. The next day I tried a script again in plain PowerShell and it failed because pandas and pip were not there outside the venv, so I went back to Jupyter in the venv. More than once at night I asked how to stop the notebook server and the venv.

What went wrong with long runs

One cell printed nothing for 25 minutes. I did not know if it was working, so I left it overnight, and I asked whether the PC going to sleep would stop it. When I checked the next evening it had finished, and no files had been written. All the results were only in the memory of the kernel, so I had them saved to a CSV right then.

Another long run died when I closed the browser by accident. The Jupyter documentation says the kernel stays active when the browser window is closed, and you can open the notebook again and reconnect.11Jupyter Notebook documentation, “The Jupyter Notebook”. https://jupyter-notebook.readthedocs.io/en/stable/notebook.html I do not know if my kernel was still running and I just did not reconnect.

Another time the PC went to sleep in the middle of a run. After the restart a cell gave a NameError on a variable that an earlier cell had defined, because the earlier cells had not been run again.

Footnotes

1. Jupyter Notebook documentation, “The Jupyter Notebook”. https://jupyter-notebook.readthedocs.io/en/stable/notebook.html ↩ ↩² ↩³ ↩⁴

2. Jupyter documentation, “Kernels (programming languages)”. https://docs.jupyter.org/en/latest/projects/kernels.html ↩

3. Python 3 documentation, “venv, creation of virtual environments”. https://docs.python.org/3/library/venv.html ↩

4. Google, “Colaboratory frequently asked questions”. https://research.google.com/colaboratory/faq.html ↩ ↩²

5. Visual Studio Code documentation, “Jupyter Notebooks in VS Code”. https://code.visualstudio.com/docs/datascience/jupyter-notebooks ↩