We’re pleased that you are interested in working on Warehouse.
Setting up a development environment to work on Warehouse should be a straightforward process. If you have any difficulty, please contact us so we can improve the process.
Quickstart for Developers with Docker experience¶
git clone firstname.lastname@example.org:pypa/warehouse.git cd warehouse make serve make initdb
View Warehouse in the browser at
Detailed Installation Instructions¶
Getting the warehouse source code¶
Clone the warehouse repository from GitHub:
git clone email@example.com:pypa/warehouse.git
Configure the development environment¶
Docker simplifies development environment set up.
Warehouse uses Docker and Docker Compose
to automate setting up a “batteries included” development environment.
The Dockerfile and
docker-compose.yml files include all the required steps
for installing and configuring all the required external services of the
Verifying Docker Installation¶
Check that Docker is installed:
Install Docker Compose¶
Install Docker Compose using the Docker-provided installation instructions.
Verifying Docker Compose Installation¶
Check that Docker Compose is installed:
Building the Warehouse Container¶
Once you have Docker and Docker Compose installed, run:
in the repository root directory.
This will pull down all of the required docker containers, build
Warehouse and run all of the needed services. The Warehouse repository will be
mounted inside of the docker container at
Running the Warehouse Container and Services¶
You have to start the Docker services that make up the Warehouse application. These need ~4 GB of RAM dedicated to Docker to work. This is more than the default setting of the Docker Engine of 2 GB. Thus, you need to increase the memory allocated to Docker in Docker Preferences (on Mac) or Docker Settings (on Windows) by moving the slider to 4 GB in the GUI.
Then, in one terminal run the command:
Next, you will:
- create a new Postgres database,
- install example data to the Postgres database,
- run migrations, and
- load some example data from Test PyPI
In a second terminal, separate from the
make serve command above, run:
If you get an error about xz, you may need to install the
xz utility. This
is highly likely on Mac OS X and Windows.
reCaptcha is featured in authentication and registration pages. To
enable it, pass
Viewing Warehouse in a browser¶
Once the terminal running the
make serve command has logged that a
web service has started a reactor:
[twisted.application.runner._runner.Runner#info] Starting reactor...
the web container is listening on port 80. It’s accessible at
If you are using
docker-machine on an older version of Mac OS or
Windows, the warehouse application might be accessible at
https://<docker-ip>:80/ instead. You can get information about the
docker container with
Stopping Warehouse and other services¶
In the terminal where
make serve is running, you can use
to gracefully stop all Docker containers, and thus the one running the
Or, from another terminal, use
make stop in the Warehouse
repository root; that’ll stop all the Docker processes with
warehouse in the name.
What did we just do and what is happening behind the scenes?¶
The repository is exposed inside of the web container at
/opt/warehouse/src/ and Warehouse will automatically reload when it detects
any changes made to the code.
The example data located in
dev/example.sql.xz is taken from
Test PyPI and has been sanitized to remove
anything private. The password for every account has been set to the string
Running your developer environment after initial setup¶
You won’t have to initialize the database after the first time you do
so, and you will rarely have to re-run
make build. Ordinarily, to
access your developer environment, you’ll:
View Warehouse in the browser at
Errors when executing
- If the
Dockerfileis edited or new dependencies are added (either by you or a prior pull request), a new container will need to built. A new container can be built by running
make build. This should be done before running
make servehangs after a new build, you should stop any running containers and repeat
- To run Warehouse behind a proxy set the appropriate proxy settings in the
“no space left on device” when using
docker-compose may leave orphaned volumes during teardown. If you run
into the message “no space left on device”, try running the following command
(assuming Docker >= 1.9):
docker volume rm $(docker volume ls -qf dangling=true)
This will delete orphaned volumes as well as directories that are not volumes in /var/lib/docker/volumes
(Solution found and further details available at https://github.com/chadoe/docker-cleanup-volumes)
Styles are written in the scss variant of Sass and compiled using Gulp. They
will be automatically built when changed when
make serve is running.
Running the Interactive Shell¶
There is an interactive shell available in Warehouse which will automatically configure Warehouse and create a database session and make them available as variables in the interactive shell.
To run the interactive shell, simply run:
The interactive shell will have the following variables defined in it:
|db||The SQLAlchemy ORM
You can also run the IPython shell as the interactive shell. To do so export
the environment variable WAREHOUSE_IPYTHON_SHELL prior to running the
make build step:
Now you will be able to run the
make shell command to get the IPython
Running tests and linters¶
PostgreSQL 9.4 is required because of pgcrypto extension
The Warehouse tests are found in the
tests/ directory and are designed to
be run using make.
To run all tests, all you have to do is:
This will run the tests with the supported interpreter as well as all of the additional testing that we require.
If you want to run a specific test, you can use the
T=tests/unit/i18n/test_filters.py make tests
You can run linters, programs that check the code, with:
make to build the documentation. For example:
The HTML documentation index can now be found at
Building the docs requires Python 3.6. If it is not installed, the
command will give the following error message:
make: python3.6: Command not found Makefile:53: recipe for target '.state/env/pyvenv.cfg' failed make: *** [.state/env/pyvenv.cfg] Error 127