Python Module and Package Development Guide: From Basic Concepts to Installable Package Distribution

Fundamental Concepts of Python Modules and Packages

In Python development, modules and packages serve as the fundamental units for organizing code. A module is a file containing Python definitions and statements, where the filename corresponds to the module name with a .py extension. For instance, creating a file named hello.py with a simple function:

def helloworld():
    print "hello"

After creation, this module can be imported and used in the Python interpreter:

>>> import hello
>>> hello.helloworld()
'hello'

Package Creation and Organizational Structure

When organizing multiple related modules, packages provide enhanced namespace management. A package is essentially a directory containing an __init__.py file, which signals Python to treat the directory as a package. A typical package structure appears as follows:

|-HelloModule
  |_ __init__.py
  |_ hellomodule.py

In this structure, the __init__.py file can be empty or contain package initialization code. Users can import modules from the package in various ways:

>>> import mypackage
>>> from mypackage import mymodule
>>> mymodule.myfunc()
Hello!
>>> import mypackage.mymodule
>>> mypackage.mymodule.myfunc()
Hello!

Project Configuration and Packaging Preparation

To transform a Python package into a pip-installable format, project packaging information must be configured. Begin by establishing the basic project structure:

.
├── setup.py
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

Create a setup.py file in the project root directory to configure package metadata:

from setuptools import setup

setup(
    name='hellostackoverflow',
    version='0.0.1',
    description='a pip-installable package example',
    license='MIT',
    packages=['hellostackoverflow'],
    author='Your Name',
    author_email='your.email@example.com',
    keywords=['example'],
    url='https://github.com/yourusername/hellostackoverflow'
)

Additionally, include license and README files:

.
├── LICENCE.txt
├── README.rst
├── setup.py
└── hellostackoverflow/
    ├── __init__.py
    └── hellostackoverflow.py

Building and Local Testing

Use setuptools to build a source distribution package:

python setup.py sdist

This generates a dist directory containing the packaged tar.gz file. Test installation locally:

pip install ./dist/hellostackoverflow-0.0.1.tar.gz

After installation, verify package functionality in Python:

>>> from hellostackoverflow import hellostackoverflow
>>> hellostackoverflow.greeting()
'Hello Stack Overflow!'

Publishing to PyPI

First, register an account on TestPyPI and obtain an API token. Install the twine tool:

pip install twine

Upload the package to the test repository using twine:

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

Test installation from TestPyPI:

pip install --index-url https://test.pypi.org/simple/ hellostackoverflow

After successful testing, follow the same process to upload to the official PyPI repository.

Modern Packaging Configuration Methods

Beyond the traditional setup.py approach, modern Python packaging recommends using a pyproject.toml file. Basic configuration example:

[build-system]
requires = ["setuptools >= 77.0.3"]
build-backend = "setuptools.build_meta"

[project]
name = "example_package_your_username"
version = "0.0.1"
authors = [
    { name="Example Author", email="author@example.com" },
]
description = "A small example package"
readme = "README.md"
requires-python = ">=3.9"
classifiers = [
    "Programming Language :: Python :: 3",
    "Operating System :: OS Independent",
]
license = "MIT"
license-files = ["LICEN[CS]E*"]

[project.urls]
Homepage = "https://github.com/pypa/sampleproject"
Issues = "https://github.com/pypa/sampleproject/issues"

Use the build tool to create distribution packages:

python -m build

This generates both source distribution and built distribution files, preparing them for upload to PyPI.