Friday, July 20, 2007

Some fascinating videos I found on the Internet about Lost Civilizations, Archeology, strange artifacts...

Might be improbable, but not impossible. Enjoy!

Wednesday, July 18, 2007

StreamCruncher 2.1 Release Candidate

The follow up to the 2.0 Beta release is now ready. This release has some minor bug-fixes and feature enhancements:

1) Pre-Filter for Input Event Streams support <, >, !=, =, *, /, +, -, in (..), not in (..), and, or. The in clause can refer to an SQL Sub-Query. Such Sub-Queries are cached by the Kernel to improve performance

2) An additional property cacherefresh.threads.num can be configured to specify the number of Sub-Query Cache processing Threads to use

3) 2 new Test cases have been added to test the new features - H2StartupShutdown3Test and ThreeEventOrderTest

Saturday, July 07, 2007

There's a Website called LibraryThing where you can create your own Reading list and look at what others are reading, search for books based on what you've read and a lot of other nice things. You can even share your list for others to see.

Wednesday, July 04, 2007

StreamCruncher 2.0 Beta is ready!

This 2.0 version is the result of a major refactoring job.

1) The API has been greatly simplified. The internal architecture has changed considerably, resulting in a vast improvement in performance. The TimeWindowFPerfTest (single Query on a Stream) can do 25,500 Events per second on a 1.6 GHz Centrino! More details and log files in my next Blog

2) Plain Windows constructs that were available in previous version has been removed entirely. Partitions are the only construct now. The syntax for Simple (anonymous) Window Partitions has changed slightly, in that there is no by keyword between partition and store

3) An additional property db.schema can be specified, as the Database Schema in which the Kernel creates its internal artifacts

4) Chained Partitions from now on, must always have a Pre-Filter clause starting with $row_status is new/dead

5) The Kernel can now accommodate Events that arrive out-of-order. OutOfOrderEventTest demonstrates this new ability

6) Pre-Filter temporarily does not support the complete SQL grammar like in, exists

Phew! It took me more than a month for make these changes. The internal Event stores for Input Event Streams and Partitions have changed considerably. The idea was to keep the Events inside the Kernel's process for as long as possible and delay inserting the Events into the Database to the last stages. As a result, Events get passed around as references most of the time. The latency per Event has dropped dramatically.

There are a few things that need to be completed/cleaned up - like the Pre-Filter clause syntax, Kernel restarts do not work correctly. This version comes with the latest version of the H2 Database, which now supports Table-level concurrency - a much needed feature for StreamCruncher.