Python Project Structure and Packaging

Typical project file structure

.
├── env
│   ├── bin
│   ...
│   └── pyvenv.cfg
├── .git
│   ├── branches
│   ├── config
|   ...
├── .gitignore
├── helloworld
│   ├── __init.py__
│   └── helloworld.py
├── Makefile
├── MANIFEST.in
├── README.md
├── requirements.txt
├── setup.py
└── tests
    ├── test_advanced.py
    └── test_basic.py

Brief summary

File / Folder	Content
env/	virtual development environment by `python -m venv env && source env/bin/activate`
helloworld/	contains project source code
Makefile	helps streamline development operations like tests, cleaning, etc.
MANIFEST.in	include non-python files into a package build
README.md	project introduction to end-users; I prefer Markdown for its simplicity
requirements.txt	development dependencies created by `pip freeze > requirements.txt`
setup.py	package and distribution management
test/	store test files; `pytest` runs for `test_.py` or `_test.py`

`Makefile`

requires make binary executable be available; $ make runs directives defined in Makefile
defines generic tasks as below and implement when appropriate with $make <directive>:
```
HOST_PORT=8080
init:
pip install -r requirements.txt
```

lint:
flake8 --exclude=.tox

run:
python manage.py runserver

test:
py.test tests

docker-run:
docker run
--detach=false
--name=my_project
--publish=$(HOST_PORT):8080
docker_image

.PHONY: init lint run test docker-run


### `setup.py`

- bare minimum:
```python
# setup.py
from setuptools import setup, find_packages
setup(
    name="HelloWorld",
    version="0.1",
    packages=find_packages(),
)

but need to include many more attributes

# from https://setuptools.readthedocs.io/en/latest/setuptools.html#developer-s-guide
from setuptools import setup, find_packages
setup(
    name="HelloWorld",
    version="0.1",
    packages=find_packages(),
    scripts=["say_hello.py"],

    # Project uses reStructuredText, so ensure that the docutils get
    # installed or upgraded on the target machine
    install_requires=["docutils>=0.3"],

    package_data={
        # If any package contains *.txt or *.rst files, include them:
        "": ["*.txt", "*.rst"],
        # And include any *.msg files found in the "hello" package, too:
        "hello": ["*.msg"],
    },

    # metadata to display on PyPI
    author="Me",
    author_email="me@example.com",
    description="This is an Example Package",
    keywords="hello world example examples",
    url="http://example.com/HelloWorld/",   # project home page, if any
    project_urls={
        "Bug Tracker": "https://bugs.example.com/HelloWorld/",
        "Documentation": "https://docs.example.com/HelloWorld/",
        "Source Code": "https://code.example.com/HelloWorld/",
    },
    classifiers=[
        "License :: OSI Approved :: Python Software Foundation License"
    ]

    # could also include long_description, download_url, etc.
)

One thing worth mentioning, quoted from Native namespace packages:

Python 3.3 added implicit namespace packages from PEP 420. All that is required to create a native namespace package is that you just omit init.py from the namespace package directory. Because mynamespace doesn’t contain an init.py, setuptools.find_packages() won’t find the sub-package. You must use setuptools.find_namespace_packages() instead or explicitly list all packages in your setup.py.
python setup.py install vs python setup.py develop
While the former, as equivalent to pip install ., installed compiled package into the python environment, the latter creates a symbolic link in the python environment which makes it possible modify the code while using it. In other words, there is no need to reinstall the package to make the modification go into effect. Although not 100% certain, it is regarded as equivalent to pip install -e ..

References

Structuring Your Project from The Hitchhiker's Guide to Python (link)
https://setuptools.readthedocs.io/en/latest/setuptools.html#developer-s-guide