2. Getting Started

Note

These instructions are for trying out DXR to see if you like it. If you plan to contribute code to DXR itself, please see Development instead.

The easiest way to get DXR working on your own machine is…

  1. Get the source code you want to index.
  2. If it a language analyzed at build time (like C++ or Rust), tell DXR how to build it.
  3. Run dxr index to index your code.
  4. Run dxr serve to present a web-based search interface.

But first, we have some installation to do.

2.1. Downloading DXR

Using git, clone the DXR repository:

git clone https://github.com/mozilla/dxr.git

2.2. Booting And Building

DXR runs only on Linux at the moment (and possibly other UNIX-like operating systems). The easiest way to get things set up is to use the included, preconfigured Docker setup. If you’re not running Linux on your host machine, you’ll need a virtualization provider. We recommend VirtualBox.

After you’ve installed VirtualBox (or ignored that bit because you’re on Linux), grab the three Docker tools you’ll need: docker, docker-compose, and, if you’re not on Linux, docker-machine. If you’re running the homebrew package manager on the Mac, this is as easy as…

brew install docker docker-compose docker-machine

Otherwise, visit the Docker Engine page for instructions.

Next, unless you’re already on Linux, you’ll need to spin up a Linux VM to host your Docker containers:

docker-machine create --driver virtualbox --virtualbox-disk-size 50000 --virtualbox-cpu-count 2 --virtualbox-memory 512 default
eval "$(docker-machine env default)"

Feel free to adjust the resource allocation numbers above as you see fit.

Note

Next time you reboot (or run make docker_stop), you’ll need to restart the VM:

docker-machine start default

And each time you use a new shell, you’ll need to set the environment variables that tell Docker how to find the VM:

eval "$(docker-machine env default)"

When you’re done with DXR and want to reclaim the RAM taken by the VM, run…

make docker_stop

Now you’re ready to fire up DXR’s Docker containers, one to run elasticsearch and the other to interact with you, index code, and serve web requests:

make shell

This drops you at a shell prompt in the interactive container. Now you can build DXR and run the tests to make sure it works. Type this at the prompt within the container:

# Within the docker container...
make test

2.3. Configuration

Before DXR can index your code, it needs to know where it is and, if you want to be able to do structural queries (like find-all-the-callers) for C, C++, or Rust, how to kick off a build. (Analysis of more dynamic languages like Python does not require a build step.) If you have a simple build process powered by make, a configuration like this might suffice. Place the following in a file called dxr.config. The location of the file doesn’t matter, but the usual place is adjacent to your source directory.

[DXR]
# Some global options here, if you like

[yourproject]
source_folder       = /code/my-checkout
build_command       = make clean; make -j {workers}

Note

Be sure to replace the placeholder paths in the above config. You’ll need to move your code to be indexed into the VM, either by downloading it from within the VM, or by moving it into your DXR repository folder, where it will be visible from within the VM in the shared ~/dxr folder. It’s possible to index your code from a folder within ~/dxr, but, if you are using a non-Linux host machine, moving it to /code will give you much faster IO by taking VirtualBox’s shared-folder machinery out of the mix.

By building your project with clang and under the control of dxr index, DXR gets a chance to interpose a custom compiler plugin that emits analysis data. It then processes that into an index.

If you have a non-C++ project and simply want to index it as text, the build_command can be set to blank:

build_command =

Though you shouldn’t need any of them yet, further config directives are described in Configuration.

2.4. Indexing

Now that you’ve told DXR about your codebase, it’s time to build an index:

dxr index --config dxr.config

Note

If you have a large codebase, the VM might run out of RAM. If that happens, wipe out the VM using docker-machine rm default, and then go back to the docker-machine create instruction and crank up the numbers. For example, this is plenty of space to build Firefox:

docker-machine create --driver virtualbox --virtualbox-disk-size 50000 --virtualbox-cpu-count 4 --virtualbox-memory 8000 default

# Reset your shell variables:
eval "$(docker-machine env default)"

# And drop back into the DXR container:
make shell

Note

If you have trouble getting your own code to index, step back and see if you can get one of the included test cases to work:

cd ~/dxr/tests/test_basic
dxr index

If that works, it’s just a matter of getting your configuration right. Pop into #static on irc.mozilla.org if you need a hand.

2.5. Serving Your Index

Congratulations; your index is built! Now, spin up DXR’s development server, and see what you’ve wrought:

dxr serve --all

If you’re using docker-machine, run docker-machine ip default to find the address of your VM. Then surf to http://that IP address:8000/ from the host machine, and poke around your fancy new searchable codebase.

If you’re not using docker-machine, your code should be accessible from http://localhost:8000/.