Grow your own Python Package
I decided to write up this short post to help both me and some colleagues remember what the most basic way of constructing a python package is.
This is a bare-bones build - you should be doing more than this - for example putting in
README file and
LICENSE file and good docstrings.
The basic structure of a python package is quite straight forward:
basemypackage --> Base └── mypackage --> Actual Module ├── extras │ ├── multiply.py │ ├── divide.py ├── add.py ├── subtract.py
Here we have a dir called
mypackage which has two python files in it -
subtract.py but it also has a sub dir called
extras which contains two more python files. all of this will form up the basis of our imaginary python package. keep in mind that you will have different files and folders in your package and you can divide up the files/folders in as course or as fine a way as you like.
You will always find one or several
__init__.py files in python packages. this is because the
__init__.py files tell python to treat directories as modules (or sub-modules as we will see in a minute). this makes it an important part of our package and one we need to spend some time looking at.
__init__.py file contains two important things:
- a doc string for the available subpackages/submodules/methods in this dir (see the one in
numpyas an example:
- the names of all the methods in all the Python files that are in this immediate directory.
__init__.py looks like this:
""" some helpful info of what each method is/does... """ from file import method # 'method' is a function found in the python file called 'file.py'
You will need to put a
__init__.py in each sub dir of your package and so each sub-dir will become sub-modules of your main package.
here is the specific
__init__.py for our dummy example:
""" Provides 1. ability to add two numbers 2. ability to subtract two number. ... """ from add import add from subtract import subtract
as we have an
extras sub dir we need one in there too:
""" Provides 1. ability to add multiply numbers 2. ability to divide number. ... """ from multiply import multiply from divide import divide
Another attribute of a python package is the
setup.py which is a python file that contains information about your package, it’s version, its dependencies, and a whole whole lot more.
basemypackage dir (and in the same directory as our module
mypackage ) we will now add a
from setuptools import setup, find_packages VERSION = '0.0.1' DESCRIPTION = 'My absolutely gorgeous Python package' LONG_DESCRIPTION = 'My Python package, it is gorgeous, it makes you just want to import it.' # Setting up setup( # the name must match the folder name 'mypackage' name="mypackage", version=VERSION, author="Marty McFly", author_email="Marty.McFly@deloreantimemachines.com>", description=DESCRIPTION, long_description=LONG_DESCRIPTION, packages=find_packages(), install_requires=, # add any additional packages that # needs to be installed along with this package. Eg: 'numpy' keywords=['python', 'gorgeous'], url='https://gitlab.com/mygroup/gorgeous/basemypackage', license='MIT', classifiers= [ "Development Status :: 1 - Planning", "Intended Audience :: Religion", "Programming Language :: Python :: 3", "Operating System :: Unix", ] )
setuptools.setup() method accepts a variety of keyword arguments to specify additional metadata about your package. - info on this can be found here:
This includs a classifiers section - a full list of classifiers can be found here:
You can now build and install your package locally using the dir you have just set up. It is a better idea to store this package on-line somewhere where it can be version controlled and available whenever you need it. Many people build and distribute their package via PyPi - as it is the official Python repository where all Python packages are stored - but i find that a little scary and tend to just do my own private thing using gitlab so i’m not going to show you that. First we will cover just local build/install and then get to that other stuff later.
It’s common to locally install your project in “editable” or “developer” mode while you’re still developing/working on it. This allows your project to be both installed and editable so that updates are available when they get made.
basemypackage dir run:
python -m pip install -e .
Although somewhat cryptic, -e is short for –editable, and . refers to the current working directory, so it should install the current directory in editable mode. This will also install any dependencies declared in the
You should now be able to
import mypackage in your python environment.
It is more useful not to have a local copy of the package source-code but to use a version control system like gitlab to manage a lot of that for you. This is my preferred method of working and allows you to control who installs your package.
If you don’t already have one, create a new gitlab project and clone it to your local machine. Put all of the code that you want to include in your package including all the stuff shown above into this folder.
pip can talk git directly so it’s quite easy to build our python package from gitlab. Here is an example:
python3 -m pip install -e git+ssh://email@example.com/mygroup/gorgeous/basemypackage.git
Or if you want to specify a non default branch:
python3 -m pip install git+ssh://firstname.lastname@example.org/mygroup/gorgeous/basemypackage.git@dev
Or if you want to fetch a particular tag:
python3 -m pip install git+ssh://email@example.comfirstname.lastname@example.org
: at the end of gitlab.com must be changed to a
/ and you must have an ssh key registered with gitlab.com for the machine you’re working on.
There are a lot of other ways to install from version control that you can read about here:
What i haven’t covered here is testing and adding test running to your package. This will come in the future i expect.