Last modified 2 years ago
QuerySplitter
The QuerySplitter is one of the real break-throughs of MetaModel. With QuerySplitter it is possible to split a query into several "partial queries", ie. queries that in total yield the same result(set).
How does it work?
The way that QuerySplitter works is that it utilizes the Schema API and Query API of MetaModel to determine which columns could be used to split a query up by introducing new WHERE-clauses. A simple example for this could be splitting this query:
- SELECT name, email FROM persons
Into these partial queries:
- SELECT name, email FROM persons WHERE age < 20 OR age IS NULL
- SELECT name, email FROM persons WHERE age < 40 AND age >= 20
- SELECT name, email FROM persons WHERE age >= 40
Why should I want to split up queries?
There are basicly three reasons why it can be a good idea to split up queries, but let this be said first: It is only beneficial for queries that yield large resultsets! Here are the reasons:
- Splitting up queries mean that you can distribute query execution onto different processes which can enable load-balancing and parallelization.
- Some JDBC drivers don't implement the setFetchSize(int) method correctly which means that a query's resulting ResultSet will be loaded into memory instead of being streamed (and thus keeping memory down). This means you can very easily run out of memory.
- A lot of databases only dedicate a single thread to the execution of a query which means that you can actually tune you database by executing several small queries instead of one big, because this will yield a thread for every single query.
