Feed on
Posts
Comments

Archive for the ‘SQLite’ Category

Spending time on Excel-SQLite, C, VBA Callbacks & Twitter

Posted in BI, ETL, Palo, SQLite, VBA, Web2.0, excel, xLite, tagged , on November 20, 2008 | No Comments »

Haven’t posted here in a while as my spare time has been soaked up programing, well actually refactoring would be more exact.  My xLite “SQLite empowered Excel” codebase has grown over the years and required a serious makeover to get rid of stuff I no longer use and to generally make it more robust.  I [...]

Read Full Post »

Why Larry hates the cloud, and my data trinity.

Posted in AmazonAWS, ETL, Palo, SQLite, cloud, excel, olap, tagged , on October 4, 2008 | No Comments »

Last week Oracle certified Amazon EC2 as a supported platform, that same week Larry Elison attacked the concept of cloud computing as pure hype. Obviously, Larry is not happy with this whole cloud thing, and I think it’s not just the threat it poses to the software industry’s traditional licensing model that worries him, rather, as Robert X. Cringely [...]

Read Full Post »

Talend + SQLite + Groovy the new Oracle …

Posted in BI, EC2, ETL, Groovy, Palo, SQLite, Talend, data, excel, olap, tagged , on August 2, 2008 | 5 Comments »

… well, at least for me.  Let me explain.
For most of my datasmithing career, I’ve had access to corporate Oracle databases and now with the availability of  Oracle10g  Express I can even run my own Oracle instances at home or on EC2.  The combination of a powerful SQL engine, expressive scripting language (PL/SQL) ,OS independence, [...]

Read Full Post »

New universal SQLite JDBC library.

Posted in ETL, Java, SQLite, Talend, kettle, news, tagged , , on July 21, 2008 | No Comments »

Both Talend (Java) and Kettle distribute the Zentus.com pure-Java SQLite JDBC driver and for most purposes this run-anywhere version is fine. But, if you really need to take advantage of SQLite’s speed then connecting using the native JNI version is a must.  Doing this was easy enough, just change over to using a generic JDBC [...]

Read Full Post »

Groovy as Talend’s scripting language

Posted in ETL, Groovy, Java, Palo, SQLite, Talend, data, tagged , on July 20, 2008 | 5 Comments »

Although I had decided to use Talend (Java version) as my primary datasmithing tool I still had one major problem with it, its lack of a scripting tool.  Kettle (Pentaho PDI) has Javascript, Excel has VBA, Picalo has (well OK, is) Python and Talend in its Perl version has Perl.  I could have gone (and [...]

Read Full Post »

Palo ETL Server - Not for me …

Posted in BI, ETL, Palo, SQLite, excel, tagged , on May 1, 2008 | 2 Comments »

Jedox have just released V1.0 of their Palo-centric ETL Server. I had been looking forward to this, not so much for its ETL ability (which is somewhat limited when compared to the likes of Pentaho PDI or Talend) but for the drill-through capability it would add to Palo. Alas, there’s a catch, you [...]

Read Full Post »

SQLite - the ultimate data-smithing tool!

Posted in AmazonAWS, ETL, SQLite, Talend, data, excel, kettle, tagged , on April 26, 2008 | 1 Comment »

Image via Wikipedia
Although my data-smithing tool box is full to the brim with powerful tools such as Talend, Kettle PDI, Picalo and Excel, all backed by the cloud infrastructure of Amazon’s S3, SImpleDB and EC2, there’s one simple yet powerful tool that I always seem to gravitate back to, that tool is SQLite.
Now obviously being [...]

Read Full Post »

Python the new VBA ?

Posted in BI, ETL, Palo, Ruby, SQLite, Web2.0, data, excel, news, tagged , , , , on April 11, 2008 | 6 Comments »

These last two weeks, Python has been on my mind. First off, last week I decided to make time to fully investigate Picalo, an open-source Python-based data analysis tool, and then, this week, Google announced their long awaited cloud-computing offering, Google Apps Engine, with the language at its core.
Python was the first of [...]

Read Full Post »

xlAWS - 100,000 downloads?

Posted in AmazonAWS, Proto, S3, SQLite, SimpleDB, VBA, programming, xLite, xlAWS, tagged , on April 2, 2008 | 2 Comments »

Not sure, but this morning I received my monthly AWS bill, and it was double its usual amount! When I investigated the extra cost it was due to 133GBs of downloads from my www2.gobansaor.com bucket. This is the S3 bucket in which I store the xlAWS zip file, xlAWS being a “library-of-sorts” of [...]

Read Full Post »

Postgres Plus Cloud Edition is boring …

Posted in AmazonAWS, BI, EC2, ETL, S3, SQLite, SimpleDB, olap, tagged , , , on March 27, 2008 | 2 Comments »

… and that’s good. That’s how I like my databases, boring, reliable, consistent, easy to use.
SimpleDB on the other hand is not boring, it’s an exciting new shiny thing that opens up a myriad of new possibilities; but first, I and the rest of the developer community, need to tool up and cast aside [...]

Read Full Post »

Dublin Bus and PALO ETL - the connection!

Posted in AmazonAWS, BI, ETL, Palo, S3, SQLite, SimpleDB, Talend, VBA, excel, kettle, olap, tagged , , , , on January 26, 2008 | 5 Comments »

Dublin buses, as is the norm with most road-based public transport systems in our increasingly car-choked cities, tend to operate on the basis of “no sign of a bus for ages, then two or three arrive at the same time”. Palo MOLAP ETL options appear to be following the same pattern; we’ve been waiting for [...]

Read Full Post »

SimpleDB + S3 = distributed document-centric database

Posted in AmazonAWS, EC2, S3, SQLite, SimpleDB, Web2.0, data, excel, news, tagged , on December 14, 2007 | 3 Comments »

I’m a database man. I’ve worked on or about most variations on the theme, from roll-your-own flat files, to hierarchical, to CODASYL network databases, to the current crop of relational and MOLAP platforms. Of late, I’ve being investigating what I think will be the future of database technology, the distributed document-centric database. [...]

Read Full Post »

Firefox tune up time again …..

Posted in EC2, Firefox, S3, SQLite, Web2.0, tagged , , , , , on December 5, 2007 | No Comments »

This morning Firefox just got slower and slower; clicking on a link or a text box took ages to respond; using online WYSIWYG editors became next to impossible; I was also getting an error when attempting to connect to Google Sync.
I checked the usual suspects; internet connection OK; did a quick HijackThis scan and analysis [...]

Read Full Post »

Take Mind Mapping offline with Google Gears

Posted in GoogleApps, SQLite, Web2.0, education, tagged , on November 5, 2007 | 2 Comments »

I’ve been a long time fan of mind maps (the pencil and paper type) and have also occasionally used the excellent and free computer based FreeMind to good effect. Over the last year or so a number of online mind mapping tools have appeared and I see that one of the better ones, www.mindmeister.com, [...]

Read Full Post »

Using the latest Pure Java SQLite JDBC driver in Kettle

Posted in ETL, Java, SQLite, kettle, tagged , on October 5, 2007 | 3 Comments »

The bug in the pure Java SQLiteJDBC driver that caused an “out of memory” error when trying to connect to a SQLite database using standard windows drive letters (e.g. c:\kettle\mydata.db) is now fixed. The current version (V037) has also been updated to SQLite version 3.4.2. To use the latest driver within Kettle, download [...]

Read Full Post »

CouchDB - document centric ODS

Posted in BI, ETL, S3, SQLite, data, tagged , , on September 14, 2007 | 3 Comments »

While the potential of column-oriented DBMSs within BI projects is obvious given the popularity of MOLAP ( a form of column-oriented data store) the potential for the other new kid on the block, the document-oriented database, is less so. One such DBMS,CouchDb, is the latest wunderkid to bubble to the surface, helped by the [...]

Read Full Post »

SQLite as a Column Oriented Database

Posted in BI, ETL, Palo, SQLite, data on September 7, 2007 | 1 Comment »

According to Michael Stonebraker , one of the pioneers of relational database technology, the future of  DBMSs lies with column-oriented databases such as C-Store or Google’s BigTable. In the BI sphere, MOLAP column-oriented data-stores are increasingly the norm. But the fact table implementations of most ROLAP star-schemas tend to favour a row-oriented “wide [...]

Read Full Post »

Google Spreadsheets - ETL tool

Posted in BI, ETL, GoogleApps, RSSBus, Ruby, SQLite, Talend, VBA, Web2.0, excel, kettle, tagged on September 6, 2007 | 1 Comment »

Although I’m a total Excel fanboy, I most admit I rarely use it any longer for personal stuff such as home budgets, tax calculations, what-ifs, to-do lists etc.; I now tend to use Google Spreadsheets. Likewise, personal notes, drafts and useful bits of code are stored using Google Docs rather than MS Word. [...]

Read Full Post »

In Memory OLAP

Posted in BI, ETL, Palo, SQLite, VBA, data, excel, olap, xLite on September 5, 2007 | 4 Comments »

The consolidation within the BI market continues, this time with the purchase of Applix by Cognos. As Timo Elliott points out, the interesting bit is the Applix TM1 memory-centric OLAP product. For the vast majority of OLAP users (i.e. the millions of Excel Pivot table jockeys) in-memory OLAP is nothing new, but traditionally [...]

Read Full Post »

SQLite Star Query Part II

Posted in BI, ETL, Palo, SQLite, VBA, excel, olap, xLite on August 31, 2007 | No Comments »

In my previous post I looked at simulating a bitmap-join in SQLite using a sub-query and the INTERSECT command. The problem is of course, this is a simulation, SQLite lacks bitmap indices and although the sub-query will read only the fact table’s index B-trees (avoiding accessing the fact table proper) and should be [...]

Read Full Post »

Older Posts »