APPENDIX A
Conventions of Owl
All software systems have their own rules and conventions with which developers must comply, and Owl is no exception. In this appendix we cover function naming and various other conventions in the Owl library.
A.1 Pure vs Impure
The Ndarray module contains functions to manipulate and perform mathematical
operations over multi-dimensional arrays. Here, pure refers to operations such as
addition that leave their input arrays unchanged and return a new array. In contrast,
their impure counterparts write the result into an existing array. The arrays
themselves remain mutable; this naming convention does not make every function
without an underscore pure.
Functional programming in general promotes the use of immutable data structures as impure functions make it more difficult to reason about the correctness of code. On the other hand, generating a fresh matrix every time you modify a single element does not seem very practical.
The introduction of impure functions into Owl is thus only done under careful, practical consideration. In-place modification can reduce expensive memory allocation and deallocation, which can significantly improve the runtime performance of a numerical application, especially when large ndarrays and matrices are involved.
Thus, many pure functions in the Ndarray module have corresponding
impure versions indicated by an underscore “_” added at the end of the
function name. For example, the following pure functions in the Arr module,
Arr.sin and Arr.add, have corresponding impure functions, Arr.sin_,
Arr.add_.
For unary operators such as Arr.sin_ x, x is modified in place by default. However,
binary operators are more complex. For example, in Arr.add_scalar_ x a, x will
be modified in place to store the final result – relatively straightforward
as a is a scalar. In Arr.add_ x y, the question is where to store the final
result when both inputs are ndarrays. Let’s look at the type of the Arr.add_
function:
val add_ : ?out:Owl.Arr.arr -> Owl.Arr.arr -> Owl.Arr.arr -> unit
As we can see from the type signature, the output is specified in the optional out
parameter. If out is not provided, the result is stored in the first operand (x). The
destination must have the shape of the broadcast result. With the default destination
x, this means that, after aligning dimensions from the right, y must have the same
size or size one in each dimension of x. Merely having smaller dimensions is not
sufficient. With a separate out, both inputs may be broadcast to the result shape.
Unary and scalar operations such as sin_ and add_scalar_ also accept an optional
out.
Most binary math functions have an associated shorthand operator, such as +, -,
*, and /. The impure versions also have associated operators, e.g., rather than
Arr.(x + y) which returns the result in a new ndarray, you can write Arr.(x += y)
which adds x and y and saves the result into x. These operators are listed in
Tbl. A.1.
A.2 Ndarray vs Scalar
Three common classes of ndarray operations are map, scan, and reduce. Many
Ndarray functions can be categorised as reduce operations, such as Arr.sum,
Arr.prod, Arr.min, Arr.mean, and Arr.std. For example, we can sum all the
elements in an ndarray using the sum function.
let x = Arr.sequential [|3;3;3|];;
let a = Arr.sum x;;
The result is a one-element ndarray so to treat it as a scalar, you must retrieve
the single value by calling the get function.
let b = Arr.get a [|0|] in
b +. 10.;;
This becomes inconvenient in OCaml if we must always extract the scalar value from the return of reduce operations. In languages like Python and Julia, a function can return an array or a scalar depending on its arguments. OCaml’s static typing requires a single result type for each such function; an array-or-scalar result would need a common representation. Owl instead provides two versions of these reduction functions:
- one allowing you to reduce along the specified axis or to reduce all the elements, but always returning an ndarray;
- one that reduces all the elements, always returning a scalar value.
The difference between the two is indicated by naming those returning
a scalar with an extra “’” character in their names. For example, those
functions returning an ndarray are named Arr.sum, Arr.prod, Arr.mean, etc,
while those returning a scalar are named Arr.sum’, Arr.prod’, Arr.mean’,
etc.
A.3 Infix Operators
In each Ndarray and Matrix module, some frequently used functions have the
corresponding infix operators as convenient aliases. Tbl. A.2 summarises the
operators in the current dense modules, together with the prefix functions min2 and
max2. In the table, both x and y represent either a matrix or an ndarray while a
represents a scalar value.
| Operator | Example | Operation | Ndarray/Matrix |
+ | x + y | element-wise add | both |
- | x - y | element-wise sub | both |
* | x * y | element-wise mul | both |
/ | x / y | element-wise div | both |
+$ | x +$ a | add scalar | both |
-$ | x -$ a | sub scalar | both |
*$ | x *$ a | mul scalar | both |
/$ | x /$ a | div scalar | both |
$+ | a $+ x | scalar add | both |
$- | a $- x | scalar sub | both |
$* | a $* x | scalar mul | both |
$/ | a $/ x | scalar div | both |
= | x = y | comparison | both |
!= | x != y | comparison | both |
<> | x <> y | same as != | both |
> | x > y | comparison | both |
< | x < y | comparison | both |
>= | x >= y | comparison | both |
<= | x <= y | comparison | both |
=. | x =. y | element-wise cmp | both |
!=. | (!=.) x y | element-wise cmp | both |
<>. | x <>. y | same as !=. | both |
>. | x >. y | element-wise cmp | both |
<. | x <. y | element-wise cmp | both |
>=. | x >=. y | element-wise cmp | both |
<=. | x <=. y | element-wise cmp | both |
=$ | x =$ a | comp to scalar | both |
!=$ | (!=$) x a | comp to scalar | both |
<>$ | x <>$ a | same as !=$ | both |
>$ | x >$ a | compare to scalar | both |
<$ | x <$ a | compare to scalar | both |
>=$ | x >=$ a | compare to scalar | both |
<=$ | x <=$ a | compare to scalar | both |
=.$ | x =.$ a | element-wise cmp | both |
!=.$ | (!=.$) x a | element-wise cmp | both |
<>.$ | x <>.$ a | same as !=.$ | both |
>.$ | x >.$ a | element-wise cmp | both |
<.$ | x <.$ a | element-wise cmp | both |
>=.$ | x >=.$ a | element-wise cmp | both |
<=.$ | x <=.$ a | element-wise cmp | both |
=~ | x =~ y | approx = | both |
=~$ | x =~$ a | approx =$ | both |
=~. | x =~. y | approx =. | both |
=~.$ | x =~.$ a | approx =.$ | both |
% | x % y | element-wise remainder | both |
%$ | x %$ a | remainder with scalar | both |
** | x ** y | element-wise power | both |
*@ | x *@ y | matrix multiply | Matrix |
/@ | x /@ y | solve linear system | Matrix |
**@ | x **@ a | matrix power | Matrix |
min2 | min2 x y | element-wise min | both |
max2 | max2 x y | element-wise max | both |
@= | x @= y | concatenate on first axis | both |
@|| | x @|| y | concatenate on last axis | both |
For matrices, concatenation on the first and last axes means vertical and
horizontal concatenation, respectively. The matrix-power operator **@ takes an
integer-valued float exponent, such as 2.; fractional powers are not implemented in
the current version.
The names !=., !=$, and !=.$ are parsed as prefix operators in OCaml, so the
table applies them as parenthesised functions. Use <>., <>$, and <>.$ for infix
notation.
We draw your attention to several things in this table:
+$ has its corresponding operator $+if we flip the order of parameters. However, it is important to be very careful about operator precedence in these cases: OCaml generally determines precedence from the first character of an infix operator, with a separate, higher-precedence class for operators beginning with**.+$ has the same precedence as+, whereas $+has lower precedence. We thus recommend using $+with great care, and always using parentheses to explicitly specify precedence. The same applies to $-, $*, and $/.- For comparison operators, e.g. both
=and=.compare elements of arraysxandyof the same shape. The difference is that=returns a boolean value whereas=.returns a matrix or ndarray of the same shape and type asxandy. In the returned result, the value in a given position is1if the values of the corresponding position inxandysatisfy the predicate, otherwise it is0. - The comparison operators ending with $ are used to compare a matrix/ndarray to a scalar value.
A.4 Module Structures
Owl’s current Dense module contains Dense.Ndarray and Dense.Matrix. The
matrix modules are a special case built upon the corresponding ndarray modules.
Earlier releases also exposed sparse modules, but they are not exported by the
current Owl version used in the book container.
With Dense.Ndarray, we can create a dense
-dimensional
array of no more than 16 dimensions. This constraint originates from the underlying
Bigarray.Genarray module in OCaml. Storage grows with the product of the axis
sizes, so adding nontrivial axes can quickly become expensive; the number of
dimensions alone does not determine the storage requirement. The current Bigarray
limit is checked in the companion examples; the former Sparse.Ndarray API is not
available as an alternative in this environment.
The dense modules contain five submodules to handle different types of numbers:
Smodule supports single precision floating-point numbersfloat32;Dmodule supports double precision floating-point numbersfloat64;Cmodule supports single precision complex numberscomplex32(32 bits per component);Zmodule supports double precision complex numberscomplex64(64 bits per component);Genericmodule supports all aforementioned number types via a GADT.
Technically, S, D, C, and Z wrap shared generic operations with explicit type
information and some type-specific functions. Using these submodules directly lets
you omit the element-kind argument needed by functions such as Generic.zeros.
The precision describes array storage: for example, reading an element of an S array
still returns an OCaml float.
In practice, we often work with double precision numbers so Owl provides
shortcuts to the data structures of double precision floating point numbers. Arr is
equivalent to double precision real Dense.Ndarray.D, and similarly, Mat is equivalent
to double precision real Dense.Matrix.D. These two modules are frequently used in
this book.
You can convert an array or matrix from one element kind to another by using
the cast_* functions in the Generic module. For example, Generic.cast_s2d casts
from float32 to float64, and Generic.cast_c2z casts from complex32 to
complex64.
Many functions in the Generic module can handle the four number types
mentioned above. This polymorphism is achieved by pattern matching on generalised
algebraic data types (GADTs) in OCaml. In the code below, we use the sum function
in the Dense.Matrix.Generic module as an example.
open Owl;;
let x = Dense.Matrix.S.eye 5 in
Dense.Matrix.Generic.sum x;;
let x = Dense.Matrix.D.eye 5 in
Dense.Matrix.Generic.sum x;;
let x = Dense.Matrix.C.eye 5 in
Dense.Matrix.Generic.sum x;;
let x = Dense.Matrix.Z.eye 5 in
Dense.Matrix.Generic.sum x;;
As we can see, no matter what kind of numbers are held in an identity matrix, we
can always pass it to the Dense.Matrix.Generic.sum function. Similarly, we can do
the same thing for other modules (such as Dense.Ndarray) and other functions (add,
mul, neg, etc.).
When combining matrices with different precisions, convert them explicitly to the same precision first. For example, we can convert a single-precision matrix to double precision, then use the usual matrix operators:
let x32 = Dense.Matrix.S.eye 3;;
let x64 = Dense.Matrix.Generic.cast_s2d x32;;
let z = Mat.(x64 + ones 3 3);;
Here the conversion produces a separate double-precision matrix; it does not change the original single-precision matrix.