The wording used in statements like below suggest a bias to me as well. This and the lack of experimental detail leaves me waiting for a study that validates the results here before I’d accept any of it as fact. There are too many missing variables.
“Our main reason for this assessment of Pentaho is
mostly linked to the many parameters that need to be learnt. However, we think that if you *invest lots of time in it*, it could become an powerful tool.”

Some examples: they didn’t use v3.0 (milestone versions of v3.0 where available since July
and we made some huge enhancements on a lot of components on tFileInput, tAggregate, t…)), in numerous jobs, the basic approach is used, it would be much better (efficiency speaking) to leverage dedicated components (tJoin, tFilterColumns…) but I think theses observations are also good for others tools).
No really, Talend experts are not working like that.

Fabrice | talend

I don’t salute their initiative at all. It’s misinformation at best, slander at worst.

We all know that:
- a benchmark is a benchmark and remains subjective (you always know a tool more than another one…).
- benchmarks are never pleasing enough for everyone.

This being said, even if certain results need to be looked at closer (especially Informatica’s results in my opinion), this is a really good job from the Manapps team. It takes a lot of time and skill to produce this kind of paper and I salute their initiative.

Great job!

I have not used Kettle or Data Stage at all. But I am very familiar with Informatica. I could see that they developer seemed a bit unsure with INFA as a number of places, he/she could have used a ROUTER rather than a FILTER and saved some time. Also it is a good practice to user a SORTER before an AGGREGATOR as the sorter transform is considerably faster than the aggregator in INFA. Another thing in Informatica’s defense is that this is being done on an XP laptop/desktop. INFA probably is significantly faster when deployed on an AIX type machine with large memory. And I also agree that the best way to aggregate is in the DB if possible. Even with INFA, they have taken the output to an aggregator (again without a sorter) rather that doing it in the Source Qualifier via SQL query.

As far as bias is concerned, they did show Datastage PX to be as fast or faster than Talend. So this might be inexperience with the other tools rather than deliberate bias.

I feel that one is going to use the tool that one is most comfortable in (for me Informatica and Talend at this point) unless you just cannot achieve something at all with that tool.

Your right as regards SQL, can’t understand this aversion to using it. In fact, even when I’m mainly dealing with text or Excel data I load the data into a SQLite database (in memory preferably) and let it do the heavy lifting; quicker and less error prone than using tool based transformations.

Anyway this speed thing is a red herring, people will use the tool (and programming language) that they “want” to use (even to the point of paying big bucks when there’s an OSS solution that could equally do the job!).

If there are speed problems these days it’s often easier to throw hardware (or another EC2 instance) at the problem.

Tom