How To Package Your Python Project

2017-01-18 21:15:15

I've recently started working on new open-source projects and decided to publish a post a basic tutorial on how to package your Python code.

This tutorial doesn't describe the only way of doing things, merely one specific approach that I use.

Package/Module Names

Python module/package names should generally follow the following constraints:

  • All lowercase
  • Unique on pypi, even if you don’t want to make your package publicly available (you might want to specify it privately as a dependency later)
  • Underscore-separated or no word separators at all (don’t use hyphens)

File Structure

The structure below

Example:

myproject/
    myproject/
        __init__.py
        a_module.py
    setup.py
    setup.cfg
    docs/
    tests/
    Authors
    README.rst
    LICENSE
    ChangeLog
    MANIFEST.in
    requirements.txt
  • myproject/myproject contains your package with modules.

requirements.txt

"Requirements files" are files containing a list of items to be installed using pip install like so:

System Message: WARNING/2 (<string>, line 40)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ pip install -r requirements.txt

In our case we use the requirements.txt file in the setup.py with setuptools to install dependencies.

MANIFEST.in

The myproject/MANIFEST.in includes additional files to the package.

include Authors
include README.rst
include LICENSE
include requirements.txt

setup.py

System Message: WARNING/2 (<string>, line 60)

Cannot analyze code. Pygments package not found.

.. code:: python

    import os

    try:
        from setuptools import setup
        from setuptools import find_packages
    except Exception as e:
        print("Requires 'setuptools'")
        print(" pip install setuptools")
        exit()

    config = {
        "name": "myproject",
        "version": "0.0.0",
        "author": "Christiaan F Rademan",
        "author_email": "chris@fwiw.co.za",
        "description": "MyProject Package Example",
        "license": "BSD 3-Clause",
        "keywords": "another example",
        "url": "http://www.fwiw.co.za",
        "packages": find_packages(),
        "include_package_data": True,
        "classifiers": [
            "Topic :: Software Development :: Libraries :: Application Frameworks",
            "Environment :: Other Environment",
            "Intended Audience :: Information Technology",
            "Intended Audience :: System Administrators",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: BSD License",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2.7"
            ]
        }

    # allow setup.py to be run from any path
    os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))

    if os.path.exists(os.path.join(os.path.dirname(__file__),
                                   'requirements.txt')):
        with open(os.path.join(os.path.dirname(__file__),
                               'requirements.txt')) as x:
            requirements = x.read().splitlines()
    else:
        requirements = []

    with open(os.path.join(os.path.dirname(__file__), 'README.rst')) as x:
        readme = x.read()

    print("%s %s\n" % (config['name']))

    setup(
        install_requires=requirements,
        long_description=readme,
        **config
    )

You can use setup.py to register your project on PyPI which is explained later or use it to remove, install your package.

System Message: WARNING/2 (<string>, line 120)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ python setup.py install

will install the package

System Message: WARNING/2 (<string>, line 126)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ python setup.py develop

The develop will not install the package but it will create a .egg-link in the deployment directory back to the project source code directory.

So it's like installing but instead of copying to the site-packages it adds a symbolic link (the .egg-link acts as a multiplatform symbolic link).

That way you can edit the source code and see the changes directly without having to reinstall every time that you make a little change. This is useful when you are the developer of that project hence the name develop.

setup.cfg

This tells PyPI where your README file is.

[metadata]
description-file = README.rst

LICENSE.txt

This file will contain whichver license you want your code to have. I tend to use the BSD 3-Clause license.

What is PyPI?

From the official website:

PyPI — the Python Package Index

The Python Package Index is a repository of software for the Python programming language.

Upload your code on PyPI. It's a big list of python packages that you absolutely must submit your package to for it to be easily one-line installable.

PyPI Account

On PyPI Live and also on PyPI Test, you must create an account in order to be able to upload your code. I recommend using the same email/password for both accounts, just to make your life easier when it comes time to push.

~.pypirc configuration file

[distutils]
index-servers =
  pypi
  pypitest

[pypi]
repository=https://pypi.python.org/pypi
username=your_username

[pypitest]
repository=https://testpypi.python.org/pypi
username=your_username

Upload your package to PyPI Test

This will attempt to register your package against PyPI's test server, just to make sure you've set up everything correctly.

System Message: WARNING/2 (<string>, line 186)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ python setup.py register -r pypitest

Now upload your package

System Message: WARNING/2 (<string>, line 192)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ python setup.py sdist upload -r pypitest

Upload your package to PyPI Test

Simply run above commands again and replace pypitest with pypi

Error while Uploading

If you receive an error TypeError: cannot concatenate 'str' and 'NoneType' objects this is a known bug. Its because the upload is not prompting for the password.

To work around this register and upload at the same time like so:

System Message: WARNING/2 (<string>, line 208)

Cannot analyze code. Pygments package not found.

.. code:: bash

    $ python setup.py sdist register -r pypi upload -r pypi

About

Service provider, telecoms, cloud, coding and automation.

Christiaan Rademan
JNCIE - #661