Squashed 'third_party/ceres/' content from commit e51e9b4
Change-Id: I763587619d57e594d3fa158dc3a7fe0b89a1743b
git-subtree-dir: third_party/ceres
git-subtree-split: e51e9b46f6ca88ab8b2266d0e362771db6d98067
diff --git a/docs/source/derivatives.rst b/docs/source/derivatives.rst
new file mode 100644
index 0000000..bff6a29
--- /dev/null
+++ b/docs/source/derivatives.rst
@@ -0,0 +1,60 @@
+.. default-domain:: cpp
+
+.. cpp:namespace:: ceres
+
+.. _chapter-on_derivatives:
+
+==============
+On Derivatives
+==============
+
+Ceres Solver, like all gradient based optimization algorithms, depends
+on being able to evaluate the objective function and its derivatives
+at arbitrary points in its domain. Indeed, defining the objective
+function and its `Jacobian
+<https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant>`_ is
+the principal task that the user is required to perform when solving
+an optimization problem using Ceres Solver. The correct and efficient
+computation of the Jacobian is the key to good performance.
+
+Ceres Solver offers considerable flexibility in how the user can
+provide derivatives to the solver. She can use:
+
+#. :ref:`chapter-analytical_derivatives`: The user figures out the
+ derivatives herself, by hand or using a tool like `Maple
+ <https://www.maplesoft.com/products/maple/>`_ or `Mathematica
+ <https://www.wolfram.com/mathematica/>`_, and implements them in a
+ :class:`CostFunction`.
+#. :ref:`chapter-numerical_derivatives`: Ceres numerically computes
+ the derivative using finite differences.
+#. :ref:`chapter-automatic_derivatives`: Ceres automatically computes
+ the analytic derivative using C++ templates and operator
+ overloading.
+
+Which of these three approaches (alone or in combination) should be
+used depends on the situation and the tradeoffs the user is willing to
+make. Unfortunately, numerical optimization textbooks rarely discuss
+these issues in detail and the user is left to her own devices.
+
+The aim of this article is to fill this gap and describe each of these
+three approaches in the context of Ceres Solver with sufficient detail
+that the user can make an informed choice.
+
+For the impatient amongst you, here is some high level advice:
+
+#. Use :ref:`chapter-automatic_derivatives`.
+#. In some cases it maybe worth using
+ :ref:`chapter-analytical_derivatives`.
+#. Avoid :ref:`chapter-numerical_derivatives`. Use it as a measure of
+ last resort, mostly to interface with external libraries.
+
+For the rest, read on.
+
+.. toctree::
+ :maxdepth: 1
+
+ spivak_notation
+ analytical_derivatives
+ numerical_derivatives
+ automatic_derivatives
+ interfacing_with_autodiff