1 Introduction

1.1 What is ALGLIB

ALGLIB is a cross-platform numerical analysis and data mining library. It supports several programming languages (C++, C#, Delphi, VB.NET, Python) and several operating systems (Windows, *nix family).

ALGLIB features include:

Data analysis (classification/regression, including neural networks)
Optimization and nonlinear solvers
Interpolation and linear/nonlinear least-squares fitting
Linear algebra (direct algorithms, EVD/SVD), direct and iterative linear solvers, Fast Fourier Transform and many other algorithms (numerical integration, ODEs, statistics, special functions)

ALGLIB Project (the company behind ALGLIB) delivers to you several editions of ALGLIB:

ALGLIB Free Edition - full functionality but limited performance and license
ALGLIB Commercial Edition - high-performance version of ALGLIB with business-friendly license

Free Edition is a serial version without multithreading support or extensive low-level optimizations (generic C or C# code). Commercial Edition is a heavily optimized version of ALGLIB. It supports multithreading, it was extensively optimized, and (on Intel platforms) - our commercial users may enjoy precompiled version of ALGLIB which internally calls Intel MKL to accelerate low-level tasks. We obtained license from Intel corp., which allows us to integrate Intel MKL into ALGLIB, so you don't have to buy separate license from Intel.

1.2 ALGLIB license

ALGLIB Free Edition is distributed under license which favors non-commmercial usage, but is not well suited for commercial applications:

ALGLIB for C++ and ALGLIB for C# are distributed under GPL 2+, GPL license version 2 or at your option any later version. A copy of the GNU General Public License is available at http://www.fsf.org/licensing/licenses
ALGLIB for Delphi and ALGLIB for CPython are distributed under ALGLIB Personal and Academic Use License Agreement.

ALGLIB Commercial Edition is distributed under license which is friendly to commericial users. A copy of the commercial license can be found at http://www.alglib.net/commercial.php.

1.3 Documentation license

This reference manual is licensed under BSD-like documentation license:

Redistribution and use of this document (ALGLIB Reference Manual) with or without modification, are permitted provided that such redistributions will retain the above copyright notice, this condition and the following disclaimer as the first (or last) lines of this file.

THIS DOCUMENTATION IS PROVIDED BY THE ALGLIB PROJECT "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE ALGLIB PROJECT BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

1.4 Reference Manual and User Guide

ALGLIB Project provides two sources of information: ALGLIB Reference Manual (this document) and ALGLIB User Guide.

ALGLIB Reference Manual contains full description of all publicly accessible ALGLIB units accompanied with examples. Reference Manual is focused on the source code: it documents units, functions, structures and so on. If you want to know what unit YYY can do or what subroutines unit ZZZ contains Reference Manual is a place to go. Free software needs free documentation - that's why ALGLIB Reference Manual is licensed under BSD-like documentation license.

Additionally to the Reference Manual we provide you User Guide. User Guide is focused on more general questions: how fast ALGLIB is? how reliable it is? what are the strong and weak sides of the algorithms used? We aim to make ALGLIB User Guide an important source of information both about ALGLIB and numerical analysis algorithms in general. We want it to be a book about algorithms, not just software documentation. And we want it to be unique - that's why ALGLIB User Guide is distributed under less-permissive personal-use-only license.

1.5 Acknowledgements

ALGLIB was not possible without contribution of the next open source projects:

We also want to thank developers of the Intel's local development center (Nizhny Novgorod branch) for their help during MKL integration.

2 ALGLIB structure

2.1 Packages

ALGLIB is a C++ interface to the computational core written in C. Both C library and C++ wrapper are automatically generated by code generation tools developed within ALGLIB project. Pre-3.0 versions of ALGLIB included more than 100 units, but it was difficult to work with such large number of files. Since ALGLIB 3.0 all units are merged into 11 packages and two support units:

alglibmisc.cpp - contains different algorithms which are hard to classify
dataanalysis.cpp - contains data mining algorithms
diffequations.cpp - contains differential equation solvers
fasttransforms.cpp - contains FFT and other related algorithms
integration.cpp - contains numerical integration algorithms
interpolation.cpp - contains interpolation algorithms
linalg.cpp - contains linear algebra algorithms
optimization.cpp - contains optimization algorithms
solvers.cpp - contains linear and nonlinear solvers
specialfunctions.cpp - contains special functions
statistics.cpp - statistics
alglibinternal.cpp - contains internal functions which are used by other packages, but not exposed to the external world
ap.cpp - contains publicly accessible vector/matrix classes, most important and general functions and other "basic" functionality

One package may rely on other ones, but we have tried to reduce number of dependencies. Every package relies on ap.cpp and many packages rely on alglibinternal.cpp. But many packages require only these two to work, and many other packages need significantly less than 13 packages. For example, statistics.cpp requires two packages mentioned above and only one additional package - specialfunctions.cpp.

2.2 Subpackages

There is one more concept to learn - subpackages. Every package was created from several source files. For example (as of ALGLIB 3.0.0), linalg.cpp was created by merging together 14 .cpp files (C++ interface) and 14 .c files (computational core). These files provide different functionality: one of them calculates triangular factorizations, another generates random matrices, and so on. We've merged source code, but what to do with their documentation?

Of course, we can merge their documentation (as we've merged units) in one big list of functions and data structures, but such list will be hard to read. Instead, we have decided to merge source code, but leave documentation separate.

If you look at the list of ALGLIB packages, you will see that each package includes several subpackages. For example, linalg.cpp includes trfac, svd, evd and other subpackages. These subpackages do no exist as separate files, namespaces or other entities. They are just subsets of one large unit which provide significantly different functionality. They have separate documentation sections, but if you want to use svd subpackage, you have to include linalg.h, not svd.h.

2.3 Open Source and Commercial versions

ALGLIB comes in two versions - open source (GPL-licensed) and commercial (closed source) one. Both versions have same functionality, i.e. may solve same set of problems. However, commercial version differs from open source one in following aspects:

License. Commercial ALGLIB is licensed under non-copyleft license which is friendly to commercial users.
Performance. Many algorithms in commercial ALGLIB are multi-threaded and SIMD-optimized (when used on Intel systems). Open source ALGLIB is single-threaded and can not efficiently use modern multicore CPU's.
You have to study comments on specific functions if you want to know whether they have multithreaded versions or not.

This documentation applies to both versions of ALGLIB. Detailed description of commercial version can be found below.

3 Compatibility

3.1 CPU

ALGLIB is compatible with any CPU which:

supports double precision arithmetics
complies with IEEE 754 floating point standard (especially in its handling of IEEE special values)
either big-endian or little-endian (but not mixed-endian)

Most mainstream CPU's (in particular, x86, x86_64, ARM and SPARC) satisfy these requirements.

As for Intel architecture, ALGLIB works with both FPU-based and SIMD-based implementations of floating point math. 80-bit internal representation used by Intel FPU is not a problem for ALGLIB.

3.2 OS

ALGLIB for C++ (both open source and commercial versions) can be compiled in OS-agnostic mode (no OS-specific preprocessor definitions), when it is compatible with any OS which supports C++98 standard library. In particular, it will work under any POSIX-compatible OS and under Windows.

If you want to use multithreaded capabilities of commercial version of ALGLIB, you should compile it in OS-specific mode by #defining either AE_OS=AE_WINDOWS or AE_OS=AE_POSIX at compile time, depending on OS being used. Former corresponds to any modern OS (32/64-bit Windows XP and later) from Windows family, while latter means almost any POSIX-compatible OS. It applies only to commercial version of ALGLIB. Open source version is always OS-agnostic, even in the presence of OS-specific definitions.

3.3 Compiler

ALGLIB is compatible with any C++ compiler which:

supports 32-bit and 64-bit signed integer datatypes
emits code which handles comparisons with IEEE special values without raising exception. We don't require that x/0 will return INF. But at least we must be able to compare double precision value with infinity or NAN without raising exception.

All modern compilers satisfy these requirements.

However, some very old compilers (ten years old version of Borland C++ Builder, for example) may emit code which does not correctly work with IEEE special values. If you use one of these old compilers, we recommend you to run ALGLIB test suite to ensure that library works.

3.4 Optimization settings

ALGLIB is compatible with any kind of optimizing compiler as long as:

volatile modifier is correctly handled (i.e. compiler does not optimize volatile reads/writes)
optimized code correctly handles IEEE special values

Generally, all kinds of optimization that were marked by compiler vendor as "safe" are possible. For example, ALGLIB can be compiled:

under MSVC: with /O1, /O2, /Og, /Os, /Ox, /Ot, /Oy, /fp:precise, /fp:except, /fp:strict
under GCC: with -O1, -O2, -O3, -Os

From the other side, following "unsafe" optimizations will break ALGLIB:

under MSVC: /fp:fast
under GCC: -Ofast, -ffast-math

4 Compiling ALGLIB

4.1 Adding to your project

Adding ALGLIB to your project is easy - just pick packages you need and... add them to your project! Under most used compilers (GCC, MSVC) it will work without any additional settings. In other cases you will need to define several preprocessor definitions (this topic will be detailed below), but everything will still be simple.

By "adding to your project" we mean that you should a) compile .cpp files with the rest of your project, and b) include .h files you need. Do not include .cpp files - these files must be compiled separately, not as part of some larger source file. The only files you should include are .h files, stored in the /src folder of the ALGLIB distribution.

As you see, ALGLIB has no project files or makefiles. Why? There are several reasons:

first, because many ALGLIB users don't need separate static library (which will be created by invoking makefile) - they prefer to integrate source code in their projects. We have provided script-based build system before, but majority of our users prefer to build ALGLIB themselves.
second, because we want ALGLIB to be usable in any programming environment, whether it is Visual Studio, GNU build system or something else. The best solution is to write package which doesn't depend on any particular programming environment.

In any case, compiling ALGLIB is so simple that even without project file you can do it in several minutes.

4.2 Configuring for your compiler

If you use modern versions of MSVC or GCC, you don't need to configure ALGLIB at all. But if you use outdated versions of these compilers (or something else), then you may need to tune definitions of several data types:

alglib_impl::ae_int32_t - signed integer which is 32 bits wide
alglib_impl::ae_int64_t - signed integer which is 64 bits wide
alglib_impl::ae_uint64_t - unsigned integer which is 64 bits wide
alglib_impl::ae_int_t - signed integer which has same width as pointer

ALGLIB tries to autodetect your compiler and to define these types in compiler-specific manner:

ae_int32_t is defined as int, because this type is 32 bits wide in all modern compilers.
ae_int64_t is defined as _int64 (MSVC) or as signed long long (GCC, Sun Studio).
ae_uint64_t is defined as unsigned _int64 (MSVC) or as unsigned long long (GCC, Sun Studio).
ae_int_t is defined as ptrdiff_t.

In most cases, it is enough. But if anything goes wrong, you have several options:

if your compiler provides stdint.h, you can define AE_HAVE_STDINT conditional symbol
alternatively, you can manually define AE_INT32_T and/or AE_INT64_T and/or AE_UINT64_T and/or AE_INT_T symbols. Just assign datatype name to them, and ALGLIB will automatically use your definition. You may define all or just one/two types (those which are not detected automatically).

4.3 Improving performance (CPU-specific and OS-specific optimizations)

You can improve performance of ALGLIB in a several ways:

by compiling with advanced optimization turned on
by telling ALGLIB about CPU it will run on

ALGLIB has two-layered structure: some set of basic performance-critical primitives is implemented using optimized code, and the rest of the library is built on top of these primitives. By default, ALGLIB uses generic C code to implement these primitives (matrix multiplication, decompositions, etc.). This code works everywhere from Intel to SPARC. However, you can tell ALGLIB that it will work under particular architecture by defining appropriate macro at the global level:

defining AE_CPU=AE_INTEL - to tell ALGLIB that it will work under Intel

When AE_CPU macro is defined and equals to the AE_INTEL, it enables SIMD support. ALGLIB will use cpuid instruction to determine SIMD presence at run-time and use SIMD-capable code. ALGLIB uses SIMD intrinsics which are portable across different compilers and efficient enough for most practical purposes.

If you want to use multithreaded capabilities of commercial version of ALGLIB, you should compile it in OS-specific mode by #defining either AE_OS=AE_WINDOWS, AE_OS=AE_POSIX or AE_OS=AE_LINUX (POSIX with Linux-specific extensions) at compile time, depending on OS being used. Former corresponds to any modern OS (32/64-bit Windows XP and later) from Windows family, while latter means almost any POSIX-compatible OS (or any OS from the Linux family). It applies only to commercial version of ALGLIB. Open source version is always OS-agnostic, even in the presence of OS-specific definitions.

4.4 Examples (free and commercial editions)

4.4.1 Introduction

In this section we'll consider different compilation scenarios for free and commercial versions of ALGLIB - from simple platform-agnostic compilation to compiling/linking with MKL extensions.

We assume that you unpacked ALGLIB distribution in the current directory and saved here demo.cpp file, whose code is given below. Thus, in the current directory you should have exactly one file (demo.cpp) and exactly one subdirectory (cpp folder with ALGLIB distribution).

4.4.2 Compiling under Windows

File listing below contains the very basic program which uses ALGLIB to perform matrix-matrix multiplication. After that program evaluates performance of GEMM (function being called) and prints result to console. We'll show how performance of this program continually increases as we add more and more sophisticated compiler options.

demo.cpp (WINDOWS EXAMPLE)

#include <stdio.h>
#include <windows.h>
#include "LinAlg.h"

double counter()
{
    return 0.001*GetTickCount();
}

int main()
{
    alglib::real_2d_array a, b, c;
    int n = 2000;
    int i, j;
    double timeneeded, flops;
    
    // Initialize arrays
    a.setlength(n, n);
    b.setlength(n, n);
    c.setlength(n, n);
    for(i=0; i<n; i++)
        for(j=0; j<n; j++)
        {
            a[i][j] = alglib::randomreal()-0.5;
            b[i][j] = alglib::randomreal()-0.5;
            c[i][j] = 0.0;
        }
    
    // Set global threading settings (applied to all ALGLIB functions);
    // default is to perform serial computations, unless parallel execution
    // is activated. Parallel execution tries to utilize all cores; this
    // behavior can be changed with alglib::setnworkers() call.
    alglib::setglobalthreading(alglib::parallel);
    
    // Perform matrix-matrix product.
    flops = 2*pow((double)n, (double)3);
    timeneeded = counter();
    alglib::rmatrixgemm(
        n, n, n,
        1.0,
        a, 0, 0, 0,
        b, 0, 0, 1,
        0.0,
        c, 0, 0);
    timeneeded = counter()-timeneeded;
    
    // Evaluate performance
    printf("Performance is %.1f GFLOPS\n", (double)(1.0E-9*flops/timeneeded));
    
    return 0;
}

Examples below cover Windows compilation from command line with MSVC. It is very straightforward to adapt them to compilation from MSVC IDE - or to another compilers. We assume that you already called %VCINSTALLDIR%\bin\amd64\vcvars64.bat batch file which loads 64-bit build environment (or its 32-bit counterpart). We also assume that current directory is clean before example is executed (i.e. it has ONLY demo.cpp file and cpp folder). We used 3.2 GHz 4-core CPU for this test.

First example covers platform-agnostic compilation without optimization settings - the most simple way to compile ALGLIB. This step is same in both open source and commercial editions. However, in platform-agnostic mode ALGLIB is unable to use all performance related features present in commercial edition.

We starts from copying all cpp and h files to current directory, then we will compile them along with demo.cpp. In this and following examples we will omit compiler output for the sake of simplicity.

OS-agnostic mode, no compiler optimizations

> copy cpp\src\*.* .
> cl /I. /EHsc /Fedemo.exe *.cpp
> demo.exe
Performance is 0.7 GFLOPS

Well, 0.7 GFLOPS is not very impressing for a 3.2GHz CPU... Let's add /Ox to compiler parameters.

OS-agnostic mode, /Ox optimization

> cl /I. /EHsc /Fedemo.exe /Ox *.cpp
> demo.exe
Performance is 0.9 GFLOPS

Still not impressed. Let's turn on optimizations for x86 architecture: define AE_CPU=AE_INTEL. This option provides some speed-up in both free and commercial editions of ALGLIB.

OS-agnostic mode, ALGLIB knows it is x86/x64

> cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL *.cpp
> demo.exe
Performance is 4.5 GFLOPS

It is good, but we have 4 cores - and only one of them was used. Defining AE_OS=AE_WINDOWS allows ALGLIB to use Windows threads to parallelize execution of some functions. Starting from this moment, our example applies only to Commercial Edition.

ALGLIB knows it is Windows on x86/x64 CPU (COMMERCIAL EDITION)

> cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL /DAE_OS=AE_WINDOWS *.cpp
> demo.exe
Performance is 16.0 GFLOPS

Not bad. And now we are ready to the final test - linking with MKL extensions.

Linking with MKL extensions differs a bit from standard way of linking with ALGLIB. ALGLIB itself is compiled with one more preprocessor definition: we define AE_MKL symbol. We also link ALGLIB with appropriate (32-bit or 64-bit) alglib???_??mkl.lib static library, which is an import library for special lightweight MKL distribution, shipped with ALGLIB. We also should copy to current directory appropriate alglib???_??mkl.dll binary file which contains Intel MKL.

Linking with MKL extensions (COMMERCIAL EDITION)

> copy cpp\addons-mkl\alglib*64mkl.lib .
> copy cpp\addons-mkl\alglib*64mkl.dll .
> cl /I. /EHsc /Fedemo.exe /Ox /DAE_CPU=AE_INTEL /DAE_OS=AE_WINDOWS /DAE_MKL *.cpp alglib*64mkl.lib
> demo.exe
Performance is 33.1 GFLOPS

From 0.7 GFLOPS to 33.1 GFLOPS - you may see that commercial version of ALGLIB is really worth it!

4.4.3 Compiling under Linux

demo.cpp (LINUX EXAMPLE)

#include <stdio.h>
#include <sys/time.h>
#include "LinAlg.h"

double counter()
{
    struct timeval now;
    alglib_impl::ae_int64_t r, v;
    gettimeofday(&now, NULL);
    v = now.tv_sec;
    r = v*1000;
    v = now.tv_usec/1000;
    r = r+v;
    return 0.001*r;
}

int main()
{
    alglib::real_2d_array a, b, c;
    int n = 2000;
    int i, j;
    double timeneeded, flops;
    
    // Initialize arrays
    a.setlength(n, n);
    b.setlength(n, n);
    c.setlength(n, n);
    for(i=0; i<n; i++)
        for(j=0; j<n; j++)
        {
            a[i][j] = alglib::randomreal()-0.5;
            b[i][j] = alglib::randomreal()-0.5;
            c[i][j] = 0.0;
        }
    
    // Set global threading settings (applied to all ALGLIB functions);
    // default is to perform serial computations, unless parallel execution
    // is activated. Parallel execution tries to utilize all cores; this
    // behavior can be changed with alglib::setnworkers() call.
    alglib::setglobalthreading(alglib::parallel);
    
    // Perform matrix-matrix product.
    flops = 2*pow((double)n, (double)3);
    timeneeded = counter();
    alglib::rmatrixgemm(
        n, n, n,
        1.0,
        a, 0, 0, 0,
        b, 0, 0, 1,
        0.0,
        c, 0, 0);
    timeneeded = counter()-timeneeded;
    
    // Evaluate performance
    printf("Performance is %.1f GFLOPS\n", (double)(1.0E-9*flops/timeneeded));
    
    return 0;
}

Examples below cover x64 Linux compilation from command line with GCC. We assume that current directory is clean before example is executed (i.e. it has ONLY demo.cpp file and cpp folder). We used 2.3 GHz 2-core Skylake CPU with 2x Hyperthreading enabled for this test.

OS-agnostic mode, no compiler optimizations

> cp cpp/src/* .
> g++ -I. -o demo.out *.cpp
> ./demo.out
Performance is 0.9 GFLOPS

Let's add -O3 to compiler parameters.

OS-agnostic mode, -O3 optimization

> g++ -I. -o demo.out -O3 *.cpp
> ./demo.out
Performance is 2.8 GFLOPS

Better, but not impressed. Let's turn on optimizations for x86 architecture: define AE_CPU=AE_INTEL. This option provides some speed-up in both free and commercial editions of ALGLIB.

OS-agnostic mode, ALGLIB knows it is x86/x64

> g++ -I. -o demo.out -O3 -DAE_CPU=AE_INTEL *.cpp
> ./demo.out
Performance is 5.0 GFLOPS

It is good, but we have 4 cores (in fact, 2 cores - it is 2-way hyperthreaded system) and only one of them was used. Defining AE_OS=AE_POSIX allows ALGLIB to use POSIX threads to parallelize execution of some functions. You should also specify -pthread flag to link with pthreads standard library. Starting from this moment, our example applies only to Commercial Edition.

ALGLIB knows it is POSIX OS on x86/x64 CPU (COMMERCIAL EDITION)

> g++ -I. -o demo.out -O3 -DAE_CPU=AE_INTEL -DAE_OS=AE_POSIX -pthread *.cpp
> ./demo.out
Performance is 9.0 GFLOPS

Not bad. You may notice that performance growth was ~2x, not 4x. The reason is that we tested ALGLIB on hyperthreaded system: although we have 4 logical cores, they share computational resources of just 2 physical cores. And now we are ready to the final test - linking with MKL extensions.

We should note that on typical Linux system shared libraries are not loaded from current directory by default. Either you install them into one of the system directories, or use some way to tell linker/loader that you want to load shared library from some specific directory. For our example we choose to update LD_LIBRARY_PATH environment variable.

Linking with MKL extensions (COMMERCIAL EDITION, relevant for ALGLIB 3.13)

> cp cpp/addons-mkl/libalglib*64mkl.so .
> ls *.so
libalglib313_64mkl.so
> g++ -I. -o demo.out -O3 -DAE_CPU=AE_INTEL -DAE_OS=AE_POSIX -pthread -DAE_MKL -L. *.cpp -lalglib313_64mkl
> LD_LIBRARY_PATH=.
> export LD_LIBRARY_PATH
> ./demo.out
Performance is 33.8 GFLOPS

Final result: from 0.9 GFLOPS to 33.8 GFLOPS!

5 Using ALGLIB

5.1 Thread-safety

Both open source and commercial versions of ALGLIB are 100% thread-safe as long as different user threads work with different instances of objects/arrays. Thread-safety is guaranteed by having no global shared variables.

However, any kind of sharing ALGLIB objects/arrays between different threads is potentially hazardous. Even when this object is seemingly used in read-only mode!

Say, you use ALGLIB neural network NET to process two input vectors X0 and X1, and get two output vectors Y0 and Y1. You may decide that neural network is used in read-only mode which does not change state of NET, because output is written to distinct arrays Y. Thus, you may want to process these vectors from parallel threads.

But it is not read-only operation, even if it looks like that! Neural network object NET allocates internal temporary buffers, which are modified by neural processing functions. Thus, sharing one instance of neural network between two threads is thread-unsafe!

5.2 Global definitions

ALGLIB defines several conditional symbols (all start with "AE_" which means "ALGLIB environment") and two namespaces: alglib_impl (contains computational core) and alglib (contains C++ interface).

Although this manual mentions both alglib_impl and alglib namespaces, only alglib namespace should be used by you. It contains user-friendly C++ interface with automatic memory management, exception handling and all other nice features. alglib_impl is less user-friendly, is less documented, and it is too easy to crash your system or cause memory leak if you use it directly.

5.3 Datatypes

ALGLIB (ap.h header) defines several "basic" datatypes (types which are used by all packages) and many package-specific datatypes. "Basic" datatypes are:

alglib::ae_int_t - signed integer type used by library
alglib::complex - double precision complex datatype, safer replacement for std::complex
alglib::ap_error - exception which is thrown by library
boolean_1d_array - 1-dimensional boolean array
integer_1d_array - 1-dimensional integer array
real_1d_array - 1-dimensional real (double precision) array
complex_1d_array - 1-dimensional complex array
boolean_2d_array - 2-dimensional boolean array
integer_2d_array - 2-dimensional integer array
real_2d_array - 2-dimensional real (double precision) array
complex_2d_array - 2-dimensional complex array

Package-specific datatypes are classes which can be divided into two distinct groups:

"struct-like" classes with public fields which you can access directly.
"object-like" classes which have no public fields. You should use ALGLIB functions to work with them.

5.4 Constants

The most important constants (defined in the ap.h header) from ALGLIB namespace are:

alglib::machineepsilon - small number which is close to the double precision &eps;, but is slightly larger
alglib::maxrealnumber - very large number which is close to the maximum real number, but is slightly smaller
alglib::minrealnumber - very small number which is close to the minimum nonzero real number, but is slightly larger
alglib::fp_nan - NAN (non-signalling under most platforms except for PA-RISC, where it is signalling; but when PA-RISC CPU is in its default state, it is silently converted to the quiet NAN)
alglib::fp_posinf - positive infinity
alglib::fp_neginf - negative infinity

5.5 Functions

The most important "basic" functions from ALGLIB namespace (ap.h header) are:

alglib::randomreal() - returns random real number from [0,1)
alglib::randominteger(mx) - returns random integer number from [0,nx); mx must be less than RAND_MAX
alglib::fp_eq(v1,v2) - makes IEEE-compliant comparison of two double precision numbers. If numbers are represented with greater precision than specified by IEEE 754 (as with Intel 80-bit FPU), this functions converts them to 64 bits before comparing.
alglib::fp_neq(v1,v2) - makes IEEE-compliant comparison of two double precision numbers.
alglib::fp_less(v1,v2) - makes IEEE-compliant comparison of two double precision numbers.
alglib::fp_less_eq(v1,v2) - makes IEEE-compliant comparison of two double precision numbers.
alglib::fp_greater(v1,v2) - makes IEEE-compliant comparison of two double precision numbers.
alglib::fp_greater_eq(v1,v2) - makes IEEE-compliant comparison of two double precision numbers.
alglib::fp_isnan - checks whether number is NAN
alglib::fp_isposinf - checks whether number is +INF
alglib::fp_isneginf - checks whether number is -INF
alglib::fp_isinf - checks whether number is +INF or -INF
alglib::fp_isfinite - checks whether number is finite value (possibly subnormalized)

5.6 Working with vectors and matrices

ALGLIB (ap.h header) supports matrixes and vectors (one-dimensional and two-dimensional arrays) of variable size, with numeration starting from zero.

Everything starts from array creation. You should distinguish the creation of array class instance and the memory allocation for the array. When creating the class instance, you can use constructor without any parameters, that creates an empty array without any elements. An attempt to address them may cause the program failure.

You can use copy and assignment constructors that copy one array into another. If, during the copy operation, the source array has no memory allocated for the array elements, destination array will contain no elements either. If the source array has memory allocated for its elements, destination array will allocate the same amount of memory and copy the elements there. That is, the copy operation yields into two independent arrays with indentical contents.

You can also create array from formatted string like "[]", "[true,FALSE,tRUe]", "[[]]]" or "[[1,2],[3.2,4],[5.2]]" (note: '.' is used as decimal point independently from locale settings).

alglib::boolean_1d_array b1;
b1 = "[true]";

alglib::real_2d_array r2("[[2,3],[3,4]]");
alglib::real_2d_array r2_1("[[]]");
alglib::real_2d_array r2_2(r2);
r2_1 = r2;

alglib::complex_1d_array c2;
c2 = "[]";
c2 = "[0]";
c2 = "[1,2i]";
c2 = "[+1-2i,-1+5i]";
c2 = "[ 4i-2,  8i+2]";
c2 = "[+4i-2, +8i+2]";
c2 = "[-4i-2, -8i+2]";

After an empty array has been created, you can allocate memory for its elements, using the setlength() method. The content of the created array elements is not defined. If the setlength method is called for the array with already allocated memory, then, after changing its parameters, the newly allocated elements also become undefined and the old content is destroyed.

alglib::boolean_1d_array b1;
b1.setlength(2);

alglib::integer_2d_array r2;
r2.setlength(4,3);

Another way to initialize array is to call setcontent() method. This method accepts pointer to data which are copied into newly allocated array. Vectors are stored in contiguous order, matrices are stored row by row.

alglib::real_1d_array r1;
double _r1[] = {2, 3};
r1.setcontent(2,_r1);

alglib::real_2d_array r2;
double _r2[] = {11, 12, 13, 21, 22, 23};
r2.setcontent(2,3,_r2);

You can also attach real vector/matrix object to already allocated double precision array (attaching to boolean/integer/complex arrays is not supported). In this case, no actual data is copied, and attached vector/matrix object becomes a read/write proxy for external array.

alglib::real_1d_array r1;
double a1[] = {2, 3};
r1.attach_to_ptr(2,a1);

alglib::real_2d_array r2;
double a2[] = {11, 12, 13, 21, 22, 23};
r2.attach_to_ptr(2,3,_r2);

To access the array elements, an overloaded operator() or operator[] can used. That is, the code addressing the element of array a with indexes [i,j] can look like a(i,j) or a[i][j].

alglib::integer_1d_array a("[1,2,3]");
alglib::integer_1d_array b("[3,9,27]");
a[0] = b(0);

alglib::i