OWL ONLINE TUTORIALFROM THE BOOK OCAML SCIENTIFIC COMPUTING

APPENDIX A

Conventions of Owl

All software systems have their own rules and conventions with which developers must comply, and Owl is no exception. In this appendix we cover function naming and various other conventions in the Owl library.

A.1 Pure vs Impure

The Ndarray module contains functions to manipulate and perform mathematical operations over multi-dimensional arrays. Here, pure refers to operations such as addition that leave their input arrays unchanged and return a new array. In contrast, their impure counterparts write the result into an existing array. The arrays themselves remain mutable; this naming convention does not make every function without an underscore pure.

Functional programming in general promotes the use of immutable data structures as impure functions make it more difficult to reason about the correctness of code. On the other hand, generating a fresh 1000 × 1000 matrix every time you modify a single element does not seem very practical.

The introduction of impure functions into Owl is thus only done under careful, practical consideration. In-place modification can reduce expensive memory allocation and deallocation, which can significantly improve the runtime performance of a numerical application, especially when large ndarrays and matrices are involved.

Thus, many pure functions in the Ndarray module have corresponding impure versions indicated by an underscore “_” added at the end of the function name. For example, the following pure functions in the Arr module, Arr.sin and Arr.add, have corresponding impure functions, Arr.sin_, Arr.add_.

For unary operators such as Arr.sin_ x, x is modified in place by default. However, binary operators are more complex. For example, in Arr.add_scalar_ x a, x will be modified in place to store the final result – relatively straightforward as a is a scalar. In Arr.add_ x y, the question is where to store the final result when both inputs are ndarrays. Let’s look at the type of the Arr.add_ function:

OCAML SESSION
val add_ : ?out:Owl.Arr.arr -> Owl.Arr.arr -> Owl.Arr.arr -> unit

As we can see from the type signature, the output is specified in the optional out parameter. If out is not provided, the result is stored in the first operand (x). The destination must have the shape of the broadcast result. With the default destination x, this means that, after aligning dimensions from the right, y must have the same size or size one in each dimension of x. Merely having smaller dimensions is not sufficient. With a separate out, both inputs may be broadcast to the result shape. Unary and scalar operations such as sin_ and add_scalar_ also accept an optional out.

Most binary math functions have an associated shorthand operator, such as +, -, *, and /. The impure versions also have associated operators, e.g., rather than Arr.(x + y) which returns the result in a new ndarray, you can write Arr.(x += y) which adds x and y and saves the result into x. These operators are listed in Tbl. A.1.

Table A.1: Aliases of pure and impure binary math functions
Function Name Pure Impure
add + +=
sub - -=
mul * *=
div / /=
add_scalar +$ +$=
sub_scalar -$ -$=
mul_scalar *$ *$=
div_scalar /$ /$=

A.2 Ndarray vs Scalar

Three common classes of ndarray operations are map, scan, and reduce. Many Ndarray functions can be categorised as reduce operations, such as Arr.sum, Arr.prod, Arr.min, Arr.mean, and Arr.std. For example, we can sum all the elements in an ndarray using the sum function.

OCAML
let x = Arr.sequential [|3;3;3|];;
let a = Arr.sum x;;

The result is a one-element ndarray so to treat it as a scalar, you must retrieve the single value by calling the get function.

OCAML
let b = Arr.get a [|0|] in
b +. 10.;;

This becomes inconvenient in OCaml if we must always extract the scalar value from the return of reduce operations. In languages like Python and Julia, a function can return an array or a scalar depending on its arguments. OCaml’s static typing requires a single result type for each such function; an array-or-scalar result would need a common representation. Owl instead provides two versions of these reduction functions:

  • one allowing you to reduce along the specified axis or to reduce all the elements, but always returning an ndarray;
  • one that reduces all the elements, always returning a scalar value.

The difference between the two is indicated by naming those returning a scalar with an extra “’” character in their names. For example, those functions returning an ndarray are named Arr.sum, Arr.prod, Arr.mean, etc, while those returning a scalar are named Arr.sum’, Arr.prod’, Arr.mean’, etc.

A.3 Infix Operators

In each Ndarray and Matrix module, some frequently used functions have the corresponding infix operators as convenient aliases. Tbl. A.2 summarises the operators in the current dense modules, together with the prefix functions min2 and max2. In the table, both x and y represent either a matrix or an ndarray while a represents a scalar value.

Table A.2: Operators and helpers in dense ndarray and matrix modules
Operator Example Operation Ndarray/Matrix
+    x + y    element-wise add both
-    x - y    element-wise sub both
*    x * y    element-wise mul both
/    x / y    element-wise div both
+$   x +$ a   add scalar both
-$   x -$ a   sub scalar both
*$   x *$ a   mul scalar both
/$   x /$ a   div scalar both
$+   a $+ x   scalar add both
$-   a $- x   scalar sub both
$*   a $* x   scalar mul both
$/   a $/ x   scalar div both
=    x = y    comparison both
!=   x != y   comparison both
<>   x <> y   same as != both
>    x > y    comparison both
<    x < y    comparison both
>=   x >= y   comparison both
<=   x <= y   comparison both
=.   x =. y   element-wise cmp both
!=.  (!=.) x y  element-wise cmp both
<>.  x <>. y  same as !=. both
>.   x >. y   element-wise cmp both
<.   x <. y   element-wise cmp both
>=.  x >=. y  element-wise cmp both
<=.  x <=. y  element-wise cmp both
=$   x =$ a   comp to scalar both
!=$  (!=$) x a  comp to scalar both
<>$  x <>$ a  same as !=$ both
>$   x >$ a   compare to scalar both
<$   x <$ a   compare to scalar both
>=$  x >=$ a  compare to scalar both
<=$  x <=$ a  compare to scalar both
=.$  x =.$ a  element-wise cmp both
!=.$ (!=.$) x a element-wise cmp both
<>.$ x <>.$ a same as !=.$ both
>.$  x >.$ a  element-wise cmp both
<.$  x <.$ a  element-wise cmp both
>=.$ x >=.$ a element-wise cmp both
<=.$ x <=.$ a element-wise cmp both
=~   x =~ y   approx = both
=~$  x =~$ a  approx =$ both
=~.  x =~. y  approx =. both
=~.$ x =~.$ a approx =.$ both
%    x  % y   element-wise remainder both
%$   x %$ a   remainder with scalar both
**   x ** y   element-wise power both
*@   x *@ y   matrix multiply Matrix
/@   x /@ y   solve linear system Matrix
**@  x **@ a  matrix power Matrix
min2 min2 x y element-wise min both
max2 max2 x y element-wise max both
@=   x @= y   concatenate on first axis both
@|| x @|| y concatenate on last axis both

For matrices, concatenation on the first and last axes means vertical and horizontal concatenation, respectively. The matrix-power operator **@ takes an integer-valued float exponent, such as 2.; fractional powers are not implemented in the current version.

The names !=., !=$, and !=.$ are parsed as prefix operators in OCaml, so the table applies them as parenthesised functions. Use <>., <>$, and <>.$ for infix notation.

We draw your attention to several things in this table:

  • +$ has its corresponding operator $+ if we flip the order of parameters. However, it is important to be very careful about operator precedence in these cases: OCaml generally determines precedence from the first character of an infix operator, with a separate, higher-precedence class for operators beginning with **. +$ has the same precedence as +, whereas $+ has lower precedence. We thus recommend using $+ with great care, and always using parentheses to explicitly specify precedence. The same applies to $-, $*, and $/.
  • For comparison operators, e.g. both = and =. compare elements of arrays x and y of the same shape. The difference is that = returns a boolean value whereas =. returns a matrix or ndarray of the same shape and type as x and y. In the returned result, the value in a given position is 1 if the values of the corresponding position in x and y satisfy the predicate, otherwise it is 0.
  • The comparison operators ending with $ are used to compare a matrix/ndarray to a scalar value.

A.4 Module Structures

Owl’s current Dense module contains Dense.Ndarray and Dense.Matrix. The matrix modules are a special case built upon the corresponding ndarray modules. Earlier releases also exposed sparse modules, but they are not exported by the current Owl version used in the book container.

With Dense.Ndarray, we can create a dense n-dimensional array of no more than 16 dimensions. This constraint originates from the underlying Bigarray.Genarray module in OCaml. Storage grows with the product of the axis sizes, so adding nontrivial axes can quickly become expensive; the number of dimensions alone does not determine the storage requirement. The current Bigarray limit is checked in the companion examples; the former Sparse.Ndarray API is not available as an alternative in this environment.

The dense modules contain five submodules to handle different types of numbers:

  • S module supports single precision floating-point numbers float32;
  • D module supports double precision floating-point numbers float64;
  • C module supports single precision complex numbers complex32 (32 bits per component);
  • Z module supports double precision complex numbers complex64 (64 bits per component);
  • Generic module supports all aforementioned number types via a GADT.

Technically, S, D, C, and Z wrap shared generic operations with explicit type information and some type-specific functions. Using these submodules directly lets you omit the element-kind argument needed by functions such as Generic.zeros. The precision describes array storage: for example, reading an element of an S array still returns an OCaml float.

In practice, we often work with double precision numbers so Owl provides shortcuts to the data structures of double precision floating point numbers. Arr is equivalent to double precision real Dense.Ndarray.D, and similarly, Mat is equivalent to double precision real Dense.Matrix.D. These two modules are frequently used in this book.

You can convert an array or matrix from one element kind to another by using the cast_* functions in the Generic module. For example, Generic.cast_s2d casts from float32 to float64, and Generic.cast_c2z casts from complex32 to complex64.

Many functions in the Generic module can handle the four number types mentioned above. This polymorphism is achieved by pattern matching on generalised algebraic data types (GADTs) in OCaml. In the code below, we use the sum function in the Dense.Matrix.Generic module as an example.

OCAML
open Owl;;

let x = Dense.Matrix.S.eye 5 in
Dense.Matrix.Generic.sum x;;

let x = Dense.Matrix.D.eye 5 in
Dense.Matrix.Generic.sum x;;

let x = Dense.Matrix.C.eye 5 in
Dense.Matrix.Generic.sum x;;

let x = Dense.Matrix.Z.eye 5 in
Dense.Matrix.Generic.sum x;;

As we can see, no matter what kind of numbers are held in an identity matrix, we can always pass it to the Dense.Matrix.Generic.sum function. Similarly, we can do the same thing for other modules (such as Dense.Ndarray) and other functions (add, mul, neg, etc.).

When combining matrices with different precisions, convert them explicitly to the same precision first. For example, we can convert a single-precision matrix to double precision, then use the usual matrix operators:

OCAML
let x32 = Dense.Matrix.S.eye 3;;
let x64 = Dense.Matrix.Generic.cast_s2d x32;;
let z = Mat.(x64 + ones 3 3);;

Here the conversion produces a separate double-precision matrix; it does not change the original single-precision matrix.

Text from OCaml Scientific Computing by Liang Wang, Jianxin Zhao and Richard Mortier. Code: springer-book-code.