The very basics of what a Python package is

This short introduces the very basics of what a package looks like, how to create one, and how to install one. No Python knowledge needed.

Motivation for this micro-lesson

• Since our team will build software, including Python packages, to help streamline researchers’ work, this lesson aims to share some knowledge we’ve learned and help newcomers to on-board onto our work.

Assumed knowledge of reader

• No knowledge of Python is necessary.
• You know what a filesystem and file path are and should be able to answer questions like:
  • What is a parent or root directory?
  • What is a sub-folder?
  • What is a file path?
  • What is a file extension and what does it actually mean?
• You know how to open a Terminal (on Windows it would be through Git Bash) and how to type / paste into it.

Necessary software

• A Terminal (installed by default in Linux and MacOS, need to install Git Bash for Windows)
• Python 3

Learning goal

At the end, you will be able to identify if a folder and files inside are setup as a Python package. You will also be able to create a basic Python package yourself.

Take home messages

• A package is a set of files and folders that follow some simple rules

Lesson content

What is a Python package?

A package is a collection of Python files, structured in a fairly simple way, that can be installed on your computer and then used in other parts of your code in other projects. Unlike some other programming languages (like R), there is little standardization and “formal” structure to making files and folders a package. While this makes it fairly easy to just get started making a package, it also means there are several different ways and guides describing how to make them. So for a beginner, it can be a little confusing and challenging to understand if these different ways are describing the same thing (making a package) or not.

At the core of all of these different approaches, all packages must have files that look a bit like this:

packagename/
├── packagename/
│   ├── ___init___.py
│   └── module.py
└── pyproject.toml

The most important files to allow the Python code to be used as a package are the:

• __init__.py file in the sub-folder packagename/. This file is what tells Python that the folder and code within are meant to be used as a package. More technically, this is meant to tell Python the folder has “modules” to be included in the package. A module is a the term used to mean a Python file (like the module.py file in the file tree above) and will be covered in more detail another short. This file is used to tell Python what code found within the modules (files) in the folder to include within the package. If the file is empty, all code is included.
• pyproject.toml file in the parent folder. This file contains the necessary metadata for Python to properly install the Python package on your computer, metadata like version number, title, description, and dependencies.

Tip

The file pyproject.toml is the current way of declaring package metadata (see PEP 621). Before you needed to use the setuptools package and use a file called setup.py instead (or a similar name). So you may see this file used in other, older packages.

You might notice the duplicate packagename/ folder names being used, one for the parent folder and the other as a sub-folder. While technically only the folder that contains the __init__.py will be used as the name for the package (the packagename/ sub-folder here), it is common practice to name the parent folder the same name as the package name. This is why we have the packagename/packagename/ duplicate folder setup.

Since the __init__.py file tells Python the folder is a package, every sub-folder within the package must also have a __init__.py and is technically considered a “sub-package”.

Creating a basic package

Now that we know what a Python package actually is, let’s create one. While there are no standard ways or templates for making a Python package, a commonly used package to help with package creation is called cookiecutter.

We install cookiecutter by using pip3, which is the package installation manager for Python, and by opening the Terminal and pasting this code into it:

Terminal

pip3 install cookiecutter

Next we can create a starting Python package by pasting this code in the Terminal. The -o ~/Desktop tells cookiecutter to create the package in the Desktop/ folder, but you can change it to any other folder on your computer.

Terminal

cookiecutter https://github.com/kpj/cookiecutter-python -o ~/Desktop

There will be a series of questions cookiecutter will ask you, most of which you will need to answer, before it creates the package setup. After that, you now have a folder and file structure that looks almost the same as the one above (with minor differences)!

Installing a custom package

So now that we’ve created this package, how can we install it on own computer? We do that again by using pip3. Open the Terminal and run this command:

Terminal

pip3 install ~/Desktop/packagename

Replace the file path with the path you have the package in. Your package is now installed!

Tip

You can also install the package with . while inside the package folder. Check that you are in the folder by running:

Terminal

pwd

Which stands for print working directory. If it is not in the package folder, use this Terminal command to move into it:

Terminal

cd ~/Desktop/packagename

The cd command stands for change directory. You will need to replace packagename with the name you called your package. Once in the package directory, you can install the package by running:

Terminal

pip3 install .

If you want to keep developing the package while testing out how it works while installed, you can install it in “editable” or development mode using:

Terminal

pip3 install --editable ~/Desktop/packagename
# Or if in the folder itself
pip3 install --editable .

Summary

• A Python package is a folder with at least a pyproject.toml file, a folder with the name for the package, and an __init__.py file inside that folder.
• Create a template Python package using the cookiecutter package.
• Install your package using pip3 install . or pip3 install PATH.

Additional resources

• The Hitchhiker’s Guide to Python!, chapter on Structuring your project
• setuptools docs
• Official Python packaging docs
  • Packaging Python projects
  • Installing packages from a local path
  • Declaring project metadata