Code Room

Python Packages - Everything you need to know to get started

saurabh mehta's photo
saurabh mehta
·

7 min read

Python Packages - Everything you need to know to get started

Table of contents

1. Course Introduction

This course is for developers who have intermediate to expert-level skills in Python programming. In this course, we are going to learn :

  • what are Python packages

  • how to create a Python package

  • how to install a Python package locally on your machine

  • how to publish Python package on PyPI so others can install your package with this single command :

      pip install your_package

    I will show you how to build your own Python Package by building a package called pymeth which is a Python package for using algorithms, mathematical constants, etc. Similarly, you can start from scratch or modify my package to create your own package. You can access the source code for pymeth package from GitHub https://github.com/saurabhmehta1601/pymeth

    Also, this GitHub repository is set as a template repository which means you can use this template as a starting point for your package code.

2. Introduction to Python Packages

2.1 What are packages in Python?

In Python, a package is a way to organize related modules into a single directory hierarchy. Packages help you manage and structure your code, making it more organized and maintainable, especially for larger projects.

2.2 Some key points about packages :

  • Directory Structure: A package is essentially a directory that contains a special file called __init__.py. This file can be empty or contain an initialization code for the package. The presence of this file tells Python that the directory should be treated as a package.

  • Modules: Within a package (or subpackage), you can have regular Python modules. These are individual .py files that contain Python code. Modules within a package can import each other using relative import statements.

  • Importing Packages and Modules: You can import modules and packages using the import statement. For example:

      import package.module

    You can also use the from keyword to import specific objects from a module:

      from package.module import some_function

  • Namespace Separation: Packages help separate the namespaces of different modules. This means that if you have two modules with the same name in different packages, you can distinguish between them by specifying the package name.

  • Examples of Packages: Python's standard library is organized into packages. For instance, the os package contains modules related to interacting with the operating system, and the math package contains mathematical functions and constants.

3. Building our own Python package

We will be building our own Python package, I am creating my package called pymeth - A python package for using mathematical constants, algorithms, etc. You can also clone the source code from here on GitHub https://github.com/saurabhmehta1601/pymeth

3.1 Folder Structure for Our Package

The folder structure for our package looks like this:

.
├── LICENSE
├── README.md
├── pymeth
│   ├── __init__.py
│   ├── algorithms
│   │   ├── __init__.py
│   │   ├── binarysearch.py
│   |   └── quicksort.py
│   └── constants.py
└── setup.py

Since we already know all folders that contain __init__.py files are Python packages, that means we have these packages in our code - pymeth and pymeth.algorithms

3.2 Understanding the codebase

  • pymeth: This is our main package which includes module constants.py and other subpackage algorithms.

    • Module constants.py contains values of some mathematical constants.

    • Subpackage algorithms contain implementation for algorithms like binarysearch and quicksort.

    • Within module pymeth.algorithms.init.py we contain exports from other modules within the same algorithms folder which means when we import pymeth.algorithms it also loads all the algorithms from these modules.

  • README: Contains information about our project. We will also use it within the package we will deploy to PyPI.

  • setup.py: This file is a crucial component of Python packages. It plays a central role in the packaging, distribution, and installation of Python libraries and packages. This file looks like this:

      from setuptools import setup, find_packages

  with open("README.md", "r", encoding="utf-8") as f:
      long_description = f.read()

  setup(
      name="pymeth",
      version="1.0.1",
      author="Saurabh Mehta",
      author_email="saurabhmehta1601@gmail.com",
      maintainer="Saurabh Mehta",
      maintainer_email="saurabhmehta1601@gmail.com",
      url="https://github.com/saurabhmehta1601/pymeth.git",
      download_url="https://github.com/saurabhmehta1601/pymeth.git",
      project_urls={
          "Bug Tracker": "https://github.com/saurabhmehta1601/pymeth/issues"
      },
      python_requires=">=3.6",
      install_requires=[],
      packages=find_packages(),
      include_package_data=True,
      keywords=["Math", "Algorithm", "Binary Search"],
      license="MIT",
      description="A python package for using basic math algorithm",
      long_description=long_description,
      long_description_content_type="text/markdown",
  )

The important arguments in the setup function are:

  • name: The name of your package.

  • version: The version of your package. You cannot publish package with same version twice on PyPI.

  • packages: tells what packages and modules we want to include in our final library.

  • python_requires: The version of Python needed to run the package.

  • install_requires: List of dependencies required by our package to run. You can find all the needed dependencies using tools like either pip freeze or pipreqs.

    1. Pipreqs: It is the better option since it looks for the imports within the modules to see what exact external dependencies are imported in our package files. Also, it will save all the dependencies in the requirements.txt file automatically. To save all dependencies run the following commands from the root directory ( where setup.py file exists ).

       pip install pipreqs # install pipreq with pip if not already installed
 pipreqs . # search and saves all dependencies injected in the modules of current and nested directories

    2. pip freeze: This can be used to list all the dependencies installed in the project including those dependencies which are used in development mode only.

       pip freeze > requirements.txt # save dependencies in requirements.txt

4 Installing the Python package from GitHub

We will deploy our package to PyPI for distribution but for now, we will build and install the Python package from our source code on GitHub.

4.1 ( Method 1 )- Clone and install from the GitHub repository

So first of all, we will clone the source code for the Python package pymeth. The following command will clone the code into a folder by default called pymeth:

git clone https://github.com/saurabhmehta1601/pymeth.git

Open the folder pymeth in the terminal and run the following command from the root directory( where setup.py exists ):

python setup.py install

This will install the pymeth package on the system.

4.2 ( Method 2 ) - Install Directly from GitHub

We can install packages directly from the source code for our Python package from GitHub without cloning them locally. Also, you have the flexibility to install the Python package from the source code by specifying the branch name, tag, or commit hash like:

  • To install the package from the latest commit on the default branch use :
    pip install git+github.com<username>/<repo>.git

  • To install the package from a specific branch use :

    pip install git+github.com<username>/<repo>.git@your_branch_name

  • To install a package from a specific commit run command:

    pip install git+github.com<username>/<repo>.git@your_commit_hash

  • To install a package from a specific tag run command:

    pip install git+github.com<username>/<repo>.git@your_tag_name

  • To install using a GitHub personal access token ( useful in case of private repo ):
    pip install git+https://<token>@github.com/<username>/<repo>.git

Now, let's install the pymeth package from GitHub:

pip install git+https://github.com/saurabhmehta1601/pymeth.git

This will install the pymeth package on your system.

5. Running the pymeth Python package

You can run the following code on your machine to test the package installation :

from pymeth.algorithms import quick_sort

print( quick_sort([ 123, 24 , 90 , -599 , 150]) )

This will print the numbers in sorted order on the console.

6. Publishing Package to PyPI

6.1 What is PyPI?

PyPI stands for the Python Package Index. It is a repository of Python software packages and libraries that are freely available for developers to use in their Python projects. PyPI serves as a central hub for Python packages, making it easy for developers to discover, download, and install Python libraries and tools.

6.2 Prerequisites for Publishing Package

  • Firstly, check the package version and metadata in setup.py. Remember you cannot publish a package with the same version twice.

  • The name field in setup.py which is the package name should not be already taken. Go to https://pypi.org/ and check if the package name is available.

  • To upload the Python package you will need an account on PyPI with 2Factor Authentication enabled.

  • You will also need an access token from PyPI to authenticate while trying to upload the package.

6.3 Let's publish the package now

We will be using twine to upload the package.

So if you haven't installed twine already, run the following command to install it:

pip install twine==4.0.2

Before publishing the package we need to create a distribution folder which contains our files for the package to be uploaded to PyPI. Following command will create our distribution folder dist :

python setup.py bdist_wheel

Now finally replace YOUR_ACCESS_TOKEN placeholder with the access token you generated from PyPI and run this command to upload to PyPI :

twine upload --repository-url https://upload.pypi.org/legacy/  --skip-existing  -u __token__ -p YOUR_ACCESS_TOKEN dist/*
PythonPython Package Manager, pypi