Artificial Intelligence

Python Poetry - Simplifying Dependency Management and Packaging for Your Projects.

Jul 14, 2023
12 minute read

python-poetry

Python Poetry is a powerful tool and package manager for Python that simplifies the process of managing dependencies, packaging, and publishing Python projects. It allows developers to define project dependencies and configurations in a single, easy-to-read file. In this tutorial, we’ll walk through the basics of Python Poetry, including installation, project setup, dependency management, and packaging.

Table of Contents

  1. Installing Python Poetry
  2. Creating a New Project
  3. Adding Dependencies
  4. Installing Dependencies
  5. Packaging the Project
  6. Publishing to PyPI
  7. Removing Dependencies
  8. Managing Poetry Virtual Environments
  9. Containerize with Poetry

1. Installing Python Poetry

Before we get started, ensure you have Python installed on your system. Then, open your terminal or command prompt and run the following command to install Python Poetry:

curl -sSL https://install.python-poetry.org | python -

2. Creating a New Project

To create a new Python project using Poetry, navigate to the directory where you want to create the project and run the following command:

poetry new my_project

This will create a new directory called my_project with the basic structure of a Python project, including a pyproject.toml file, which is used to define the project’s metadata and dependencies.

3. Adding Dependencies

To add dependencies to your project, edit the pyproject.toml file and add them under the [tool.poetry.dependencies] section. For example:

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.31.0"

In this example, we added Python 3.9 as a required Python version and the requests library as a project dependency.

Another way to do this is using The poetry add command is used to add dependencies to your Poetry project. When you add a dependency using this command, it will automatically update your pyproject.toml file with the new dependency and also install it in your project’s virtual environment.

poetry add requests

4. Installing Dependencies

Once you’ve defined the dependencies in your pyproject.toml file, you can install them using Poetry by running:

poetry install

This command will create a virtual environment for your project and install all the required dependencies into it.

5. Packaging the Project

To package your Python project into a distributable format (e.g., a wheel or source distribution), you can use Poetry’s build command:

poetry build

This will create the distribution files in the dist directory within your project.

6. Publishing to PyPI

If you want to publish your Python package to the Python Package Index (PyPI), ensure you have registered an account on PyPI and have configured Poetry accordingly.

First, you need to create a pyproject.toml section for publishing, which includes your PyPI username and the name of the package. You can do this by running:

poetry config pypi-token.pypi <YOUR_PYPI_TOKEN>

Replace <YOUR_PYPI_TOKEN> with your PyPI token, which you can obtain from your PyPI account.

Next, update your pyproject.toml to include the necessary publishing information:

[tool.poetry.publishing]
name = "my_package_name"
version = "0.1.0"
description = "My Python package description."

Finally, to publish your package, use the following command:

poetry publish --build

This will create a distribution of your package and upload it to PyPI.

7. Removing Dependencies

The poetry remove command is used to remove dependencies from your Poetry project. When you remove a dependency using this command, it will update your pyproject.toml file by removing the specified package and uninstall it from your project’s virtual environment.

Syntax:

   poetry remove <package_name>

Example: To remove the requests package from your project, you can run:

   poetry remove requests

This will remove requests from the [tool.poetry.dependencies] section in your pyproject.toml file and uninstall it from your project’s virtual environment.

Remember to run poetry install after using poetry add or poetry remove to apply the changes and update your project’s virtual environment with the new dependencies or without the removed ones.

Another way to remove dependencies is remove it directly from the pyproject.toml file in Poetry, you can manually edit the file to remove the package you want to remove. Here’s a step-by-step guide:

  1. Open the pyproject.toml file in a text editor or an integrated development environment (IDE).

  2. Locate the [tool.poetry.dependencies] section in the file. This section contains a list of all the dependencies for your project.

  3. Find the package you want to remove from the list of dependencies.

  4. Remove the corresponding line that specifies the package you want to remove.

    For example, if you want to remove the requests package, the line will look like this:

    requests = "^2.26.0"

    Remove this line from the [tool.poetry.dependencies] section.

  5. Save the changes to the pyproject.toml file.

After removing the dependency from the pyproject.toml file, you need to update the project’s virtual environment to reflect the changes. To do this, run the following command in your project’s root directory:

poetry install

This will update your project’s virtual environment by removing the specified package from the dependencies.

Remember that directly editing the pyproject.toml file manually requires caution, as any mistakes could lead to syntax errors or other issues. Always make sure to keep a backup of the original file before making any changes. Additionally, using the poetry remove command is a safer and recommended approach for removing dependencies, as it handles the removal process automatically and ensures proper updates to the pyproject.toml file and virtual environment.

8.Managing Poetry environments

Managing environments with Poetry is straightforward, as it automatically creates and manages virtual environments for each of your projects. This helps ensure that your project’s dependencies are isolated from other projects and the system-wide Python installation. Here are some common tasks for managing Poetry environments:

1. Creating a new environment

When you create a new Poetry project, it automatically sets up a virtual environment for that project. If you want to create a new project in a specific Python version, you can use the following command:

poetry new my_project --python=<python_version>

Replace <python_version> with the desired Python version (e.g., 3.9) you want to use for the new project.

2. Activating the environment

To activate the virtual environment for your Poetry project, navigate to the project directory and run the following command:

poetry shell

This command will activate the virtual environment, and you’ll notice that your terminal prompt changes to indicate that you’re now working inside the virtual environment.

3. Deactivating the environment

To deactivate the virtual environment and return to the system-wide Python installation, simply type exit in the terminal where you activated the environment.

4. Running commands within the environment

If you need to run a command within the virtual environment without activating it, you can use the poetry run command. For example:

poetry run python my_script.py

This will execute my_script.py using the Python version and dependencies specified in your project’s pyproject.toml.

5. Installing dependencies

When you first create a new Poetry project or update the dependencies in your pyproject.toml, you need to install them in the virtual environment. Use the poetry install command to do this:

poetry install

This will ensure that all the required dependencies for your project are installed in the virtual environment.

6. Updating dependencies

To update the dependencies in your project’s virtual environment, use the poetry update command:

poetry update

This command will update all the packages listed in your pyproject.toml file to the latest compatible versions.

7. Viewing environment information

To view information about your project’s virtual environment, such as the Python version and installed packages, use the poetry env info command:

poetry env info

This command will display details about the active virtual environment, including the Python interpreter path and installed packages.

8. Managing multiple environments

If you have multiple projects with different dependencies, Poetry will automatically create separate virtual environments for each project, keeping them isolated from each other. This makes it easy to work on different projects without conflicts between dependencies.

Poetry simplifies the management of virtual environments by automatically creating and managing them for each project. This isolation ensures that each project has its own set of dependencies, making it easier to maintain and develop Python projects.

9. Making poetry to create the environment in the project directory

By default, Poetry creates virtual environments for each project in a separate directory within the Poetry home directory or your conda environment directory. However, if you want to create the virtual environment in the same directory as your project, you can configure Poetry to do so.

To configure Poetry to create the virtual environment in the same folder as your project, you need to set the virtualenvs.in-project configuration option to true. This can be done either for a specific project or globally for all projects.

Setting virtualenvs.in-project for a Specific Project

  1. Navigate to the root directory of your Poetry project.

  2. Open a terminal or command prompt in that directory.

  3. Run the following command to configure Poetry to create the virtual environment in the same folder as the project:

    poetry config virtualenvs.in-project true

After configuring Poetry to create the virtual environment in the same folder as your project, any time you run poetry install, poetry update, or any other commands that involve the virtual environment, it will be created inside your project’s root directory.

Keep in mind that creating the virtual environment in the same folder might clutter your project directory with additional files and folders. It’s generally recommended to keep virtual environments separate from your project files to maintain a cleaner project structure. However, this configuration option can be useful in certain scenarios where you prefer the virtual environment to be within the project directory.

10. Removing poetry environment

To remove a Poetry environment for a specific project, you can simply delete the virtual environment directory associated with that project. Poetry creates a virtual environment in the .venv directory within the project’s root directory.

Here are the steps to remove a Poetry environment:

  1. Deactivate the Environment: If you have the virtual environment activated (you’ll see the environment name in your terminal prompt), deactivate it first by typing exit in the terminal.

  2. Delete the Virtual Environment Directory: Navigate to the root directory of the project and delete the .venv directory. Be cautious not to delete any other important files or directories in the project.

    On Unix-based systems (Linux, macOS):

    rm -rf .venv

    On Windows (using Command Prompt):

    rmdir /s /q .venv

    On Windows (using PowerShell):

    Remove-Item -Recurse -Force .venv

  3. Remove the Lock File (Optional): Poetry also generates a poetry.lock file in the project’s root directory to lock the versions of your dependencies. If you no longer need this lock file, you can delete it as well.

  4. Update the pyproject.toml (Optional): If you plan to remove the project entirely or stop using Poetry for it, you might want to remove the [tool.poetry] section from the pyproject.toml file. This step is optional and can be skipped if you still intend to use Poetry for the project in the future.

After performing these steps, the Poetry environment for that specific project will be removed, and you’ll no longer have the virtual environment and its dependencies.

Remember that removing the virtual environment will not affect the installed dependencies on your system-wide Python installation, as the dependencies are isolated within the project’s environment. If you want to uninstall dependencies from your system-wide Python, you can use pip to uninstall them. For example:

pip uninstall package_name

Replace package_name with the name of the package you want to uninstall. Be cautious while using pip to avoid accidentally removing essential packages.

9. Containerize with Poetry - Dockerize a FastAPI app

Below is a sample Dockerfile for a Python project that uses Poetry for dependency management to run a FastAPI app. Ensure you have a FastAPI app in your project directory with the appropriate requirements for running the app.

# Use an official Python runtime as the base image
FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Install system dependencies
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        build-essential \
        curl \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install Poetry
RUN pip install poetry

# Set the working directory in the container
WORKDIR /app

# Copy the project files to the working directory
COPY pyproject.toml /app/

# Install project dependencies
RUN poetry config virtualenvs.create false \
    && poetry install --no-dev

# Copy the rest of the project files
COPY . /app/

# Expose the port on which FastAPI runs (replace 8000 with your app's port)
EXPOSE 8000

# Start the FastAPI application using uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Here’s what it does:

  1. It sets the working directory to /app.
  2. Copies the pyproject.toml and poetry.lock files into the container.
  3. Disables virtual environment creation in Poetry (since we’re using the system Python).
  4. Installs project dependencies without dev dependencies and root package.
  5. Copies the rest of the project files into the container (assuming your FastAPI app resides in the app directory).
  6. Exposes port 8000, which is the default port for FastAPI applications.
  7. Specifies the command to start the FastAPI application using uvicorn. It assumes your FastAPI app object is defined in main.py, and the App instance is named app.

To build and run the Docker image, navigate to the project directory containing the Dockerfile and the FastAPI app, and run the following commands:

docker build -t my_fastapi_app .
docker run -d -p 8000:8000 my_fastapi_app

Replace my_fastapi_app with the desired name for your Docker image. The docker run command will start the FastAPI application inside the container, and you can access it at http://localhost:8000. For production deployments, it’s recommended to have a poetry.lock file for consistency.

Conclusion

Python Poetry is a fantastic tool that simplifies the management of Python projects, dependencies, and packaging. With Poetry, you can easily create, maintain, and distribute Python packages with confidence. As you become more familiar with Poetry, you’ll discover many other features that can streamline your Python development workflow. Happy coding!