.. _getting_started: ****************************** Getting Started For Developers ****************************** .. warning:: the following guide is used only if you want to *develop* the `ctapipe` package, if you just want to write code that uses it externally, you can install `ctapipe` as a conda package with `conda install -c cta-observatory ctapipe`. This guide assumes you are using the *Anaconda* python distribution, installed locally (*miniconda* should also work). You can use *python 3.5* or above (we currently test on 3.5 and 3.6) ------------------------ Get the ctapipe software ------------------------ In order to checkout the software in such a way that you can read *and commit* changes, you need to `Fork and Clone `_ the main ctapipe repository (cta-observatory/ctapipe). First, it's useful to make a directory where you have can check out cta GIT repos (this is optinal - you can put it anywhere) .. code-block:: console $ mkdir ctasoft $ cd ctasoft ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Step 1: Fork the Master CTA-Observatory ctapipe repository ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Follow the instructions in the link above to make a *fork* of the ctapipe repo in your own GitHub userspace. That fork will be then called *yourusername*/ctapipe (it's as simple as clicking the fork button on `main ctapipe github page `_. You only need to make this fork once, when you first start developing, and you can use it from then on. +++++++++++++++++++++++++++++++++++++++++ Step 2: clone your forked version locally +++++++++++++++++++++++++++++++++++++++++ Next, you need to clone (copy to your local machine) the newly forked ctapipe repo (make sure you put in your own username there): .. code-block:: console $ git clone https://github.com/[YOUR-GITHUB-USERNAME]/ctapipe.git $ cd ctapipe You now need to tell Git that this repo where the master CTA version is: .. code-block:: console $ git remote add upstream https://github.com/cta-observatory/ctapipe.git If that worked, then you should see a *upstream* target in addition to *origin* when typing `git remote -v`. Later if you want to pull in any changes from the master repo, you just need to type `git pull upstream master`. +++++++++++++++++++++++++++++++++++++++ Step 4: Set up your package environment +++++++++++++++++++++++++++++++++++++++ Change to the directory where you cloned `ctapipe`, and type: .. code-block:: console $ conda env create -n cta-dev -f environment.yml This will create a conda virtual environment called `cta-dev` with all the ctapipe dependencies and a few useful packages for development and interaction. Next, switch to this new virtual environment: .. code-block:: console $ source activate cta-dev You will need to type that last command any time you open a new terminal to actiavate the virtual environment (you can of course install everything into the base Anaconda environment without creating a virtual environment, but then you may have trouble if you want to install other packages with different requirements on the dependencies) +++++++++++++++++++++++++++++++++++++ Step 5: Setup ctapipe for development +++++++++++++++++++++++++++++++++++++ Now setup this cloned version for development. The following command will make symlinks in your python library directory to your ctapipe installation (it creates a `.pth` file, there is no need to set PYTHONPATH, in fact it should be blank to avoid other problems). From then on, all the ctapipe binaries and the library itself will be usable from anywhere. .. code-block:: console $ pip install -e . .. note:: If the previous command fails with an error similar to "*no module named ctapipe._version_cache*", it is because the version tags are missing in your git repo (a problem with older versions of git before 1.9.0). To fix the problem, simply type `git fetch upstream --tags` and try `make develop` again) Run the tests to make sure everything is OK: .. code-block:: console $ pytest # if using an older version of python, type py.test instead Build the HTML docs locally and open them in your web browser: .. code-block:: console $ make doc Run the example Python scripts: .. code-block:: console $ cd examples $ python xxx_example.py try running some command line tools: .. code-block:: console $ ctapipe-info --all $ ctapipe-camdemo --camera=NectarCam # try --help for more info To update to the latest development version (merging in remote changes to your local working copy): .. code-block:: console $ git pull upstream master --------------------------------------- Developing a new feature or code change --------------------------------------- You should always create a branch when developing some new code (unless it is a very small change). Genearlly make a new branch for each new feature, so that you can make pull-requests for each one separately and not mix code from each. Remember that `git checkout ` switches between branches, `git checkout -b ` creates a new branch, and `git branch` on it's own will tell you which branches are available and which one you are currently on. First think of a name for your code change, here we'll use *implement_feature_1* as an example. +++++++++++++++++++++++++++ 1. Create a feature branch: +++++++++++++++++++++++++++ .. code-block:: sh $ git checkout -b implement_feature_1 ++++++++++++++++ 2. Edit the code ++++++++++++++++ and make as many commits as you want (more than one is generally better for large changes!). .. code-block:: sh $ git add some_changed_file.py another_file.py $ git commit [type descriptive message in window that pops up] and repeat. The commit message should follow the *GIT conventions*: the first line is a short description, followed by a blank line, followed by details if needed (in a bullet list if applicable). You may even refer to GitHub issue ids, and they will be automatically linked to the commit in the issue tracker. An example commit message:: fixed bug #245 - changed the order of if statements to avoid logical error - added unit test to check for regression Of course,make sure you frequently test via `make test` (or `py.test` in a sub-module), check the style, and make sure the docs render correctly (both code and top-level) using `make doc`. .. note:: A git commit should ideally contain one and only one feature change (e.g it should not mix changes that are logically different together). Therefore it's best to group related changes with `git add `. You may even commit only *parts* of a changed file using and `git add -p`. If you want to keep your git commit history clean, learn to use commands like `git commit --ammend` (append to previous commit without creating a new one, e.g. when you find a typo or something small) A clean history and a chain of well-written commit messages will make it easier on code reviews to see what you did. ++++++++++++++++++++++++++++++++++++++++++ 3. Push your branch to your fork on github ++++++++++++++++++++++++++++++++++++++++++ (sometimes refered to as "publishing" since it becomes public, but only in your fork) by running .. code-block:: sh git push You can do this at any time and more than once. It just moves the changes from your local branch on your development machine to your fork on github. ++++++++++++++++++++++++ 4. make a *Pull Request* ++++++++++++++++++++++++ When you're happy, you make PR on on your github fork page by clicking "pull request". You can also do this via *GitHub Desktop* if you have that installed, by pushing the pull-request button in the upper-right-hand corner. Make sure to describe all the changes and give examples and use cases! See the :ref:`pullrequests` section for more info. +++++++++++++++++++++++++ 5. Wait for a code review +++++++++++++++++++++++++ Keep in mind the following: * At least one reviewer must look at your code and accept your request. They may ask for changes before accepting. * All unit tests must pass. They are automatically run by Travis when you submit or update your pull request and you can monitor the results on the pull-request page. If there is a test that you added that should *not* pass because the feature is not yet implemented, you may `mark it as skipped temporarily `_ until the feature is complete. * All documentation must build without errors. Again, this is checked by Travis. It is your responsibility to run "make doc" and check that you don't have any syntax errors in your docstrings. * All code you have written should follow the style guide (e.g. no warnings when you run the `flake8` syntax checker) If the reviewer asks for changes, all you need to do is make them, `git commit` them and then run `git push` and the reviewer will see the changes. When the PR is accepted, the reviewer will merge your branch into the *master* repo on cta-observatory's account. +++++++++++++++++++++++++++++ 6. delete your feature branch +++++++++++++++++++++++++++++ since it is no longer needed (assuming it was accepted and merged in): .. code-block:: sh git checkout master # switch back to your master branch pull in the upstream changes, which should include your new features, and remove the branch from the local and remote (github). .. code-block:: sh git pull upstream master git branch --delete --remotes implement_feature_1 Note the last step can also be done on the GitHub website. --------------------- More Development help --------------------- For coding details, read the :ref:`guidelines` section of this documentation. To make git a bit easier (if you are on a Mac computer) you may want to use the `github-desktop GUI `_, which can do most of the fork/clone and remote git commands above automatically. It provides a graphical view of your fork and the upstream cta-observatory repository, so you can see easily what version you are working on. It will handle the forking, syncing, and even allow you to issue pull-requests.