Equations

Equations are a vital part of exact science. You therefore need to practice with them, also/especially as a computer science student. 

Intended for: BSc, MSc, PhD

Equations can be very challenging for some computer science students, but you really need them for both explanation and conceptual understanding: 

• Explanation: You can never fully detail a computational procedure in words, it is simply impossible. You will need to use equations and algorithm boxes for it. The same applies the other way around: you need to be able to read equations to understand what someone else proposed/implemented. 

• Conceptual understanding: However, equation are much more than an explanation convention. They are a structured way of thinking about (computational) problems and solutions. Many students consider equations and mathematical thinking an unnecessary hassle, and getting them correct as only about begin 'neat'. However, this is fare from true. They are part of a systematic way of thinking about mathematical problems and their relations, and having a good grasp of sets, spaces, functions, probabilities, optimization, etc. will power your overall insight and conceptual understanding. 

1. Write equation yourself in Latex (never copy-paste them from screenshots)

• Learn Latex: First of all, you need to learn Latex. It is nearly impossible to write neat equations without it. 

• Never copy-paste: This naturally follows from the first point, but never screenshot equations from other sources. Your lay-out becomes a mess, your notation is almost by definition inconsistent when you screenshot from different sources, and you do not learn from it (and it's plagiarism). Always write your equations yourself. 

Example (below): Never screenshot an equation, and never put equations in figures (left). Never screenshot entire algorithms (Right), gives different font and notation. 

2. Use symbols (not text) 

Always write symbols in your equations, never use full words, they make them very hard to read. 

Example (below): define symbols like $\alpha$  for learning rate (an not 'lr') and $\theta$ for the set of parameters. 

3. Introduce every symbol

The third common mistake is that people often use symbols without explaining what they mean. You need to tell the reader what every certain symbol represents. 

4. Write variables in Italic, operators in Roman

The most important convention of formulas is that variables are written in Italic, and other operators/labels are written in Roman (non Italic). 

• Variables in italic: Put all symbols you write in Latex math mode, which automatically renders them in italic.  Choose between: 

  • Inline math mode: Use $ .. $ for every symbol equation within a sentence. For example, never write "we run the algorithm for n iterations", but write "we run the algorithm for $n$ iterations". This will correctly display 'n' as a symbol (in italic). 

  • Display math mode: Put bigger equation in display mode with  $$ ... $$ or \begin{equation} ... \end{equation}. 

• Operators\labels in roman: Operators and labels are text (not variables), and should not appear in Roman (non Italic):  

  • Operators: Operators, such as 'max', 'min', 'argmax', 'argmin', 'log',  'exp', 'sin', 'cos', etc., specify operations on variables. Therefore,  do not just write 'max' in your math mode equation, since Latex will interpret this as three variables ('m', 'a', and 'x'). Instead, use the '\max' , '\min', '\log', etc. operators. For 'argmax' and 'argmin', define the following operators yourself in you document preamble by adding: \usepackage{amsmath},  \DeclareMathOperator*{\argmin}{arg\,min} and \DeclareMathOperator*{\argmax}{arg\,max}. 

  • Labels: When you want to label a variable, do not write $x_{train}$, since this will make Latex write 'train' as if they are variables 't', 'r', 'a', 'i', 'n' again. Instead, write $x_\text{train}$ to indicate 'train' is a text label (not a variable). 

5. Stick to notational conventions

Make sure you stick to notational conventions, which makes it much easier to read your equations: 

• Sets: use italic uppercase letters. 

  • Example: "$\mathcal{S}$ denotes the state space."

  • For sets of the basic number system, use blackboard bold ($\mathbb{..}$) uppercase letters. Example: $\mathbb{R}$ (real line), $\mathbb{N}$ (natural numbers), etc. 

• Functions: use lower (possibly upper) case letters with brackets. 

  • When a function depends on parameters, you may use a subscript. Example: "$f_\theta(x)$ denotes a neural network parameterized by $\theta$". 

• Vectors and matrices: Technically, you should use bold lower case letters for vectors, and bold upper case letters for matrices. 

  • Example: $\boldsymbol{x} denotes the vector of inputs$. However, in when the vector representation is not necessary for your story, bold notation is often omitted. Index into a vector/matrix through subscripts, e.g. $\boldsymbol{A}_{i,j}$ denotes the element in the i-th row and j-th column of matrix A. 

• Probability distributions: 

  • When you work with probability distribution, the convention is to use upper case letters for the random variables, and lower case letters for a realized particular value. Example: The random variable $X$ takes value $x=2$ with probability $P(X=2)=0.3$. In practice, probability distributions are often written with either lower or upper case letter, i.e., $P(X=x)$ is typically abbreviated to either $p(X)$ or p(x), or even P(X) or P(x), depending on your preference.   

  • To indicate sampling from a distribution, use a tilde (\sim). Example: $s' \sim \pi(s'|s,a)$. 

  • For expectation, use an uppercase E, preferably in blackboard bold: $\mathbb{E}]$. 

• For operators with an argument, indicate the argument:

  • Max/min: write $\max_{a \in \mathcal{A}}$ instead of only $\max$. Can be abbreviated to only $\max_a$. 

  • Expectations: write $\mathbb{E}_{x \sim p(x)} [...]}$ instead of only $\mathbb{E} [...]$. Can be abbreviated to only $\mathbb{E}_x$ or $\mathbb{E}_{p(x)}$. 

• Multiplication & division: use \cdot (and never * or x) for multiplication, and \frac{}{} (and preferably not /) for division.  

6. Keep notation consistent

• Stick to the same notation throughout your whole document. Check for this by explicitly rereading your document for notation!

• Example from a single report: 

7. Practice!

Students that struggle with equations, often have the tendency to not practice with them, and ignore them as much as possible. However, in the long run this is a very bad tactic: it will make the problem worse and worse. Understanding and writing equations is really about practice, and you should start practicing both as soon as possible.