Performance¶
Computing preprocessing data and models, both linear and nonlinear ones, requires a lot of processor time. Programs from the ShapeKit Suite try hard to reduce the time to process data, while still retaining accuracy.
Algorithms¶
This is the most important point when looking for performance. ShapeKit Suite uses a comprehensive set of algorithms, with automatic selection to adapt to context (such as memory available, training set sizes, miscellaneous degenerated cases).
We designed and tuned our optimization algorithms with respect to latest results on the topic, and we keep on updating them. We believe all of our algorithms are as fast as it is possible with current state of the art. Our tests show variations of several order of magnitude between different algorithms.
Multiple cores/processors¶
ShapeKit Suite detects and use if the computer has several cores and/or processors. This is especially useful for the supervisor. Most kind of processing do not scale well with multicores, but computing several model in parallel does, and does very well. This is because this kind of computing is completely CPU-bound, and does not require data exchange between cores. You can actually expect a n-times improvement in speed if you multiply the number of cores/processors by n.
We also have a version performing parallel computing accross several computers. It requires some setup and is not included in the common retail product for simplicity. Very few customers actually need this. If you do, please contact us. All details are on our website: https://www.sylphide-consulting.com.
Vectorization¶
Most current processors have some kind of vectorization instructions (such as MMX, SSE, Altivec, Neon, ...). ShapeKit Suite uses those instructions sets on the following architecture:
- x86/amd64: All versions of MS-Windows, Mac-OS and Linux.
- powerpc : Linux on power PC platform
- ARM: Linux on ARM platform
Accuracy¶
Accuracy of the implementation is of paramount importance, because optimization algorithms are very sensitive to this regard. Keeping good accuracy may allow to find a better extremism point, hence a better model. It also often allow to find results more faster.
Our core libraries are tested for accuracy using both our own test cases and well-known standard test-suites.
Reliability and testing¶
All of our software is thoroughly tested using an exhaustive suite of unit tests, ensuring correctness, and providing protection against regressions.
All algorithms are tested for reproducibility, speed, and accuracy. All code generators and spreadsheet exports are tested for reproducibility.
