Applied Cytometry - Pioneering Software Simplicity

VenturiCore Introduction

Almost all computer software operates on a single processor and therefore takes the same time, or sometimes even longer, to do the same task on a modern multi-core computer. VenturiCore simplifies access to the processing power of modern x86 processors through in-memory data processing. Using VenturiCore an organisation can reduce the total cost of ownership required for high performance computing and “Big Data” applications, for example:

  • Use your existing x86 code base – no project to rewrite GPGPU code.
  • Deploy your work non-geographically ( you don't need specialist hardware) it's all x86.
  • Is being used by commercial and academic customers worldwide;
  • Delivers "Grid-in-a-Box" performance

A. Big Data Processing

When data is processed by computer information is read from the main system memory, which is relatively slow in throughput terms, held on cache memory, which is relatively fast, ready to be processed by the CPU. Where large amounts of data have to be processed, this mechanism needs to be streamlined to increase throughput. When the CPU needs to read data from system memory, the CPU will stop and wait for the data to be delivered so that it can continue – during this time the CPU does nothing.

VenturiCore delivers the orderly processing of data which is necessary to increase throughput in many Big Data applications. When it comes to processing “Big Data” the VenturiCore system optimises processor memory access times to increase throughput because the processor need not wait for information to be loaded from memory.

B. How does it work?

VenturiCore works on x86 based computers; where a computer comprises multiple processor sockets it strives to optimise the performance of all processors to achieve maximum throughput. VenturiCore is an in-memory solution and can use huge amounts of memory to minimise processing time.

The VenturiCore process management system aggregates tasks to create a series of micro-cascades, where data is read from system memory into processor cache memory, sequentially processed by a number of tasks and then when all of the tasks have been completed the results are written back out to system memory. By dividing the work into different generations VenturiCore ensures that the maximum number of tasks can be executed in parallel making optimum use of all the cores in a given processor.

C. Technical Description

The VenturiCore software components represent a variant of a Kahn Processing Network, "KPN", because the processing of data needs to be as stateless as possible, within the constraints of buffer size, to ensure optimum parallelization. This primer discusses the relationship between VenturiCore and KPN plus a brief introduction showing to how to use VenturiCore in an application.

D. Applications of VenturiCore

VenturiCore can be used in applications which require high throughput data processing, for example, in the estimation of the "Value at Risk" and the visualization of large data sets. Typically VenturiCore applications can be used to collapse grid processing systems to a handful of nodes.

 

Click here to download the VenturiCore presentation

 

Back To Top