We’re pleased that you are interested in working on Warehouse.
Your first pull request¶
After you set up your development environment and ensure you can run the tests and build the documentation (using the instructions in this document), please look at our open issues that are labelled “good first issue”, find one you want to work on, comment on it to say you’re working on it, then submit a pull request. Use our Submitting patches documentation to help.
Setting up a development environment to work on Warehouse should be a
straightforward process. If you have any difficulty, please contact us
so we can improve the process. You can find us via a GitHub issue,
#pypa-dev on Freenode, or the pypa-dev mailing
list, to ask questions or get involved.
Quickstart for Developers with Docker experience¶
git clone email@example.com:pypa/warehouse.git cd warehouse make serve make initdb
View Warehouse in the browser at
Detailed Installation Instructions¶
Getting the Warehouse source code¶
Clone the Warehouse repository from GitHub:
git clone firstname.lastname@example.org:pypa/warehouse.git
Configure the development environment¶
Docker simplifies development environment set up.
Warehouse uses Docker and Docker Compose
to automate setting up a “batteries included” development environment.
The Dockerfile and
docker-compose.yml files include all the required steps
for installing and configuring all the required external services of the
- Install Docker Engine
The best experience for building Warehouse on Windows 10 is to use the Windows Subsystem for Linux (WSL) in combination with both Docker for Windows and Docker for Linux. Follow the instructions for both platforms, and see Docker and Windows Subsystem for Linux Quirks for extra configuration instructions.
Verifying Docker Installation¶
Check that Docker is installed:
Install Docker Compose¶
Install Docker Compose using the Docker-provided installation instructions.
Verifying Docker Compose Installation¶
Check that Docker Compose is installed:
Building the Warehouse Container¶
Once you have Docker and Docker Compose installed, run:
in the repository root directory.
This will pull down all of the required docker containers, build
Warehouse and run all of the needed services. The Warehouse repository will be
mounted inside of the Docker container at
Running the Warehouse Container and Services¶
You have to start the Docker services that make up the Warehouse application. These need ~4 GB of RAM dedicated to Docker to work. This is more than the default setting of the Docker Engine of 2 GB. Thus, you need to increase the memory allocated to Docker in Docker Preferences (on Mac) or Docker Settings (on Windows) by moving the slider to 4 GB in the GUI.
Then, in one terminal run the command:
Next, you will:
- create a new Postgres database,
- install example data to the Postgres database,
- run migrations, and
- load some example data from Test PyPI
In a second terminal, separate from the
make serve command above, run:
If you get an error about xz, you may need to install the
xz utility. This
is highly likely on Mac OS X and Windows.
Viewing Warehouse in a browser¶
Once the terminal running the
make serve command has logged that a
web service has started a reactor:
[twisted.application.runner._runner.Runner#info] Starting reactor...
the web container is listening on port 80. It’s accessible at
If you are using
docker-machine on an older version of Mac OS or
Windows, the warehouse application might be accessible at
https://<docker-ip>:80/ instead. You can get information about the
docker container with
Logging in to Warehouse¶
In the development environment, the password for every account has been set to
password. You can log in as any account at
To log in as an admin user, log in as
ewdurbin with the password
Stopping Warehouse and other services¶
In the terminal where
make serve is running, you can use
to gracefully stop all Docker containers, and thus the one running the
Or, from another terminal, use
make stop in the Warehouse
repository root; that’ll stop all the Docker processes with
warehouse in the name.
What did we just do and what is happening behind the scenes?¶
The repository is exposed inside of the web container at
/opt/warehouse/src/ and Warehouse will automatically reload
when it detects any changes made to the code.
The example data located in
dev/example.sql.xz is taken from
Test PyPI and has been sanitized to remove anything private.
Running your developer environment after initial setup¶
You won’t have to initialize the database after the first time you do
so, and you will rarely have to re-run
make build. Ordinarily, to
access your developer environment, you’ll:
View Warehouse in the browser at
Errors when executing
- If the
Dockerfileis edited or new dependencies are added (either by you or a prior pull request), a new container will need to built. A new container can be built by running
make build. This should be done before running
make servehangs after a new build, you should stop any running containers and repeat
- To run Warehouse behind a proxy set the appropriate proxy settings in the
sqlalchemy.exec.OperationalErroris displayed in
make servehas been executed, shut down the Docker containers. When the containers have shut down, run
make servein one terminal window while running
make initdbin a separate terminal window.
“no space left on device” when using
docker-compose may leave orphaned volumes during teardown. If you run into the message “no space left on device”, try running the following command (assuming Docker >= 1.9):
docker volume rm $(docker volume ls -qf dangling=true)
This will delete orphaned volumes as well as directories that are not
(Solution found and further details available at https://github.com/chadoe/docker-cleanup-volumes)
make initdb is slow or appears to make no progress¶
This typically occur when Docker is not allocated enough memory to perform the
migrations. Try modifying your Docker configuration to allow more RAM for each
container and run
make initdb again.
Docker and Windows Subsystem for Linux Quirks¶
Once you have installed Docker for Windows, the Windows Subsystem for Linux, and Docker and Docker Compose in WSL, there are some extra configuration steps to deal with current quirks in WSL. Nick Janetakis has a detailed blog post on these steps, including installation, but this is a summary of the required steps:
1. In WSL, run
sudo mkdir /c and
sudo mount --bind /mnt/c /c
to mount your root drive at
/c (or whichever drive you are
using). You should clone into this mount and run
docker-compose from within it, to ensure that when volumes
are linked into the container they can be found by Hyper-V.
2. In Windows, configure Docker to enable “Expose daemon on
tcp://localhost:2375 without TLS”. Note that this may expose your
machine to certain remote code execution attacks, so use with
export DOCKER_HOST=tcp://0.0.0.0:2375 to your
.bashrc file in WSL, and/or run it directly to enable for the
current session. Without this, the docker command in WSL
will not be able to find the daemon running in Windows.
Styles are written in the scss variant of Sass and compiled using
gulp. They will be automatically built when changed when
make serve is running.
Running the Interactive Shell¶
There is an interactive shell available in Warehouse which will automatically configure Warehouse and create a database session and make them available as variables in the interactive shell.
To run the interactive shell, simply run:
The interactive shell will have the following variables defined in it:
|db||The SQLAlchemy ORM
You can also run the IPython shell as the interactive shell. To do so export
the environment variable WAREHOUSE_IPYTHON_SHELL prior to running the
make build step:
Now you will be able to run the
make shell command to get the IPython
Running tests and linters¶
PostgreSQL 9.4 is required because of
The Warehouse tests are found in the
tests/ directory and are
designed to be run using make.
To run all tests, in the root of the repository:
This will run the tests with the supported interpreter as well as all of the additional testing that we require.
make tests from a clean checkout of
Warehouse (namely, before trying to compile any static assets) will
fail multiple tests because the tests depend on a file
/app/warehouse/static/dist/manifest.json) that gets
created during deployment. So until we fix bug 1536, you’ll need to
install Warehouse in a developer environment and run
before running tests; see Detailed Installation Instructions for instructions.
If you want to run a specific test, you can use the
T=tests/unit/i18n/test_filters.py make tests
You can run linters, programs that check the code, with:
Use make to build the documentation. For example:
The HTML documentation index can now be found at
Building the docs requires Python 3.6. If it is not installed, the make command will give the following error message:
make: python3.6: Command not found Makefile:53: recipe for target '.state/env/pyvenv.cfg' failed make: *** [.state/env/pyvenv.cfg] Error 127
Please look at our open issues that are labelled “good first issue”, find one you want to work on, comment on it to say you’re working on it, then submit a pull request. Use our Submitting patches documentation to help.