I’ve written before about Scratch, a teaching platform developed by MIT to introduce kids to the art of programming. My son has been playing around with Scratch for over a year and although he still enjoys it, he’s showing signs of needing to move to the next level, a ‘real’ programming language. I decided that Python, being one of my own favourite languages, would be an ideal next step, particularly when I discovered PyGame, a Python library based on SDL.
Using Pygame with its similar problem domain to that of Scratch would, I figured, make the transition to a grown-up platform easier, and so it has; concepts such as sprite, coordinates, animation etc. are common to both. I took him through the “Pummel the Chimp” tutorial, expecting his young eyes to glaze over within 10 minutes, but no, a hour later he was still engaged and learning. Why? He already has a deep understanding of programming, particularly object oriented programming, all thanks to Scratch.
Most of this knowledge he acquired without any help for me, I simply introduced him to Scratch and explained one or two concepts (variables and messages/method calls) which he initially had trouble with, the rest he picked up from looking at other Scratch projects and from writing his own.
So if your kids (or even you) have an itch to learn the essence of programming in a fun and effective way, then Scratch it.
Not sure, but this morning I received my monthly AWS bill, and it was double its usual amount! When I investigated the extra cost it was due to 133GBs of downloads from my www2.gobansaor.com bucket. This is the S3 bucket in which I store the xlAWS zip file, xlAWS being a “library-of-sorts” of VBA/VB6 helper code for accessing Amazon S3 and SimpleDB.
It’s linked to from this page on my blog (which has had 200 or so hits this month) and from this AWS Community Code page. The excessive hits on the bucket started on the 28th of Feb , the day the xlAWS code was published on Amazon and continued through most of March. Talking the size of the zip file, 133GB represents approximately 100,000 downloads. I don’t have server logging enabled on the bucket, so I can’t be sure how much is due to the other public files in the bucket (all belonging to the VBA/Proto SQLite xLite project), but as that project has been available for months and is accessible only through my website (who’s stats show a consistent 5-10 downloads per week) I’m guessing the downloads are for xlAWS.
Who would have though that there would be such interest in VBA/VB6 code for accessing AWS services! I wonder was it the Excel VBA side of the house or the dispossessed (and p*ssed off) VB6 developer hoards who downloaded it the most? Leave a comment if you downloaded and used the library, I’d love to know.
I’ve been using Amazon’s S3 service from within Excel for sometime now and as there are no libraries or examples for calling AWS services from VBA (or VB6) I had to roll my own. As with most things Excel, getting the job done always triumphs over elegance and industrial strength implementations, in other words it was all a bit of a “dog’s dinner”. To remedy this and to share my experience of using S3 from within a VBA/VB6 environment, I decided to re-factor the code and to assemble it into a more re-usable form; the end result is xlAWS.
It was going to be called xlS3, but while doing the exercise SimpleDB appeared on the scene, so I decided to try accessing it from Excel, particularly as both products have a lot in common; both “simple”, both “schema-less” data stores. Like the S3Helper code, the simpleDBHelper module is less of comprehensive library, more a collection of useful functions which (hopefully) make working with AWS a bit easier.
To use this code library, you’ll need to have a good grasp of the S3 and SimpleDB APIs and be reasonably proficient with VBA. This is not an end-user tool, it’s for VBA (or VB6) developers. There’s a README and some basic examples within the Excel VBA project to help you get started. Code is released “in the spirit” of LGPL, you can use it how you wish, but if you add something new to the “library” (or find/fix a bug) do let the rest of us know.
As I’ve not been able to find a pure VBA implementation of the HMAC-SHA1 hash algorithm (and I couldn’t see an implementation within the standard “Microsoft Enhanced Cryptographic Provider” ) I’ve wrapped the open source XySSL SHA1 HMAC C code in a VBA friendly DLL. This DLL (and the source, under LGPL) is included in the zip file as AWS authentication requires SHA1 HMAC signatures.
You’ll also obviously require an AWS account. Credentials are stored within the workbook’s custom properties and can be encrypted via a “key file” if required. If you intend to use this code within VB6 (or Proto) you’ll need to provide your own implementation of the AWSKeyData class in order to use a non-Excel persistence store.
Although Zimki is to shut down on Christmas Eve, the ideas behind the service live on. Two new offerings, Horuku and AppJet, offer variations on the idea of hosted application development/deployment.
AppJet, funded by Paul Graham‘s Y-Combinator, is very similar to Zimki, being a server-side JavaScript platform. No details yet as to what sort of paid options will be offered (all accounts are free at the moment). Unlike Zimki there’s no plans to create an open-source version. I like the easy “build a Facebook app” feature; and I guess this is the sort of light-weight applications that they hope to attract.
Although Heroku uses Ruby-on-Rails technology, rather than JavaScript, it is closer to the original Zimki idea; but rather than take the hard (and ultimately unsuccessful in Zimki’s case) road of building an open-source platform from scratch, Heroku takes an already popular open-source project and offers it wrapped in a full on-line development and deployment environment. Again, being in beta, there’s no indication as to what pricing model it will operate under, but I would think that it will attract more “serious” projects than AppJet since anything developed under Heroku is pure Rails which means it can be migrated to any other Rails hosting environment; so no lock-in. The online editor is excellent and whatever about its merits as a hosting service it’s by far the easiest way to learn and explore Ruby and Rails, even easier than this…
If Facebook apps are your goal but you wish to use Ruby rather than AppJet’s JavaScript then not to panic, as being Ruby some bright young spark (no, not me I’m afraid) will already have done a lot of the hard graft for you…
Having mastered JavaScript (OK master is too strong a word – having become comfortable with both its syntax and usage patterns) my next port of call is JavaFX the recently announced Flash/Silverlight competitor. What led me to JavaFX Script was not its role in this Flash/AJAX alternative platform (which unless Sun improves the JRE download experience is dead in the water they may have a chance with the promised consumer JRE ) but its status as the 2nd scripting language to be supported by the Java 6 ScriptEngineManager – the first being JavaScript. Although the JavaFX platform is still in alpha, the important elements of the scripting language appear to be usable.
Why do I need another scripting language? Well JavaFX Script is a DSL (domain-specific language) and the domain in question is the GUI and as a GUI DSL, JavaFX is impressive.
In my quest for a micro ETL environment the lack of a fast and powerful GUI tool has been a problem. Admittedly, in my Excel/Sqlite xLite suite GUI generation is not a problem as VBA forms are both fast to develop in and feature rich, but the other two options (Java and Ruby) both lack a cost-effective GUI tool (I know there’s Swing and Tk but for somebody who’s used to the speed of development of tools such as VB forms or Oracle’s Application Express, neither appealed). JavaFX changes all that, it’s elegant, powerful, fast to develop in and portable. Further pushes me down the road of using Talend generated Java as my micro-ETL environment of choice.
It’s easy to dismiss JavaFX as hype but the technology behind the hype looks sound. The language borrows powerful features from other languages, object literals from JavaScript, list-comprehensions from Python/Erland/Haskell and first-class functions from JavaScript/Lisp all combined with the full power and glory of Java. The end result:
..is a declarative and statically typed programming language. It has first-class functions, declarative syntax, list-comprehensions, and incremental dependency-based evaluation. The JavaFX Script language makes intensive use of the Java2D swing GUI components and allows for easy creation of GUIs.
More Douglas Crawford JavaScript/DOM related presentations can be found here.
The feature that impressed me most about JavaScript is its functional programming and closure support, a good introduction to the functional programming capabilities of JavaScript can be found here.
What have Javascript and VBA in common? Not much on the surface and their respective user bases rarely if ever overlap. What they do share are their roles as the imperative (the-if-then-else-loop-etc) programming languages of the “I’m not a programmer” programmers, the great unwashed, the “normal” people out there who are just trying to get something done. VBA adds that “something special” to millions of Excel spreadsheets and MS Access databases. JavaScript does likewise for millions of web pages. Most of the code is copied from existing examples or generated by macro recorders. Often the code is not pretty, not well structured, not adequately tested but it is usually “good enough”.
Both languages can also transcend their “ugly duckling” status. If you want to take VBA development to a new level have a look at Professional Excel Development by Bullen et al. GMail, Google Docs & Spreadsheets and the other AJAX powered applications of Web2.0 show the power of JavaScript. Both languages are now starting to move beyond their original home bases, particularly in the case of JavaScript.
VBA has been used in the past as the scripting language for corporate applications such as Siebel; now in an inspired move, Proto opted to use VBA as the scripting environment in their corporate focused mashup tool.
JavaScript is however showing the real expansionist streak. Along side its traditional client-side role in web page DOM manipulation, Mozilla Firefox extensions, widgets and within Flash applications under its ActionScript implementation, it can now be found providing the imperative flourish to many client and server side declarative tools (e.g. Kettle ETL, OpenLaszlo and Grazr). You can even use JavaScript to provide the complete server-side logic for a web application/service by using Zimki.(deadpool)
My decision to spend time to improve my JavaScript skills was as a result of my experience with Zimki. I initially created a web service to sign requests for uploading files to my Amazon S3 account; allowing me to give clients a small executable (developed in VBA’s big brother VB6) with which to securely upload data to S3 without the need to expose my Amazon account credentials. This turned out to be so simple and elegant with Zimki’s Rhino based JavaScript environment that I decided this language deserved some quality time. I had I guess, like many others, confused the mess that is the browser-wars scared DOM with JavaScript (or more correctly, EMACScript) the language. Having looked further, I’ve discovered an elegant,simple yet powerful language that I’m tempted to think may become the universal glue-language.
UPDATE:
As Tom from Fotango (the Canon Europe company behind Zimki also in deadpool) points out, Zimki uses Mozilla’s C implementation of JavaScript, SpiderMonkey rather than Rhino.
UPDATE: Sep 2007
Cannon Europe are to close the Zimki service on Dec 24th, 2007
Okay I’m exaggerating, there’s no debate, those using tables (mainly business-orientated techies/power-users) are blissfully unaware of the charms of XML; while those whose only answer to every data exchange problem is XML (or its Javascript cousin JSON) think tables are something people used to design HTML web sites before they discovered the delights of CSS – or worse, the things that DBAs (those killjoys of the development world) are always going on about.
Tables, aka datasets, aka datagrids, aka CSV or TSV files and the peoples favourite, the Excel Range, are alive and well and continue to drive most IT based business processes.
Look at this recent comparison of mashup tools over on TechCrunch, if you look at the comparison table only one product uses tables (datagrids) as its “pipe transport protocol” (Input/Output feeds) the rest are all XML variations. The product in question is Proto which is significant as it’s the only mashup tool I know (other than RSSBus which wasn’t on the Techcrunch list) that deals with the current reality of Excel/”behind-the-firewall data sources” while also looking to the near-future where more and more (but not all) data migrates to the cloud.
I not saying that XML is always the wrong choice; RSS, OPML, MicroFormats and the unloved but powerful HTML <table> tag are here to stay and more and more tools offer out-of-the-box parsing for these standards (did you know that Excel can parse a HTML table?). Likewise for formal robust interfaces between systems a bespoke XML protocol can be useful for professional developers. But if a mashup product ignores tables as an input/output format, that product is ignoring the most common business focused data format there is (and the skills of those who day-in day-out manipulate such datasets).
One other line on the TechCrunch table (see how useful tables are) that intrigued me was the assigned level of Advanced to the skill level required to use Proto. There are at least 2 million VBA users out there without any formal programming training who construct corporate internal “mashups” everyday, for that constituency Proto would be by far the easiest tool to use.
The excellent Alistair Cockburn discuses why software development is remarkably like manufacturing and how we can learn from modern manufacturing theory to better structure our software delivery processes (or indeed any business process where the internal inventory item is the ‘yet-to-be-validated decision’).
Having looked in the past for a suitable introduction to programming for my 10 year old son I had come to the conclusion that the existing options (such as KPL) where too ‘wordy’ and not able to compete with the point and click powered online/gaming worlds that youngsters now inhabit. That was until I discovered Scratch, an education oriented programming language developed by MIT.
As an introduction to programing it is superb and it’s also fun. I could not drag my son away from it and not only that, within two days he had come in contact with (and understood) most of the important structures of any programming language, including loops, if-then-else statements, variables, encapsulation, events, messages (the tool is built in Smalltalk and it shows).
When he outgrows Scratch, he would then be capable of moving on to other more powerful education oriented languages such as Phrogram (the next version of KPL) or Squekland (built using another Smalltalk variation) or indeed a ‘real’ language such as Ruby.
I could have done with this 15 years or so ago when I was trying to get my head around OOP concepts (when I started coding, procedural languages such as Cobol, PL1, Fortran, MUMPS and various types of BASIC where the weapons of choice for commercial development).
I also think end-user focused language designers could also learn a lot from the design of the scratch IDE; perhaps if Microsoft were ever to retire the venerable VBA, they should replace it with a Scratch-like macro language?
Check out the video and definitely download it if you have kids in the house…
… you guessed it, pipes. The classic idea of the unix pipe has been given a new life in these three easy-to-use products, but this time the target audience is not the sysadmins and professional programmers of the past, this is programming for the masses.
Oh, in case you really were looking for Danny Boy.
I see Peter Cooper has sold the excellent Code Snippets site, congratulations Peter. This site and Google CodeSearch are a must-have in the toolbox of all lazy developers; a category in which I proudly place myself. If fact, I agree with Jonathan ‘Wolf’ Rentzsch, who believes the best programmers don’t really want to program at all, they just want to solve problems.
Convert lists (database tables, spreadsheets, CSV files, XML files into another format (pivot tables, PowerPivot cubes, PALO cubes, one-to-many, many-to-one, XML to CSV, XML to SQL, etc.). Delivered either as a ...
SERVICE: Send the data to me, I'll return the finished dataset or... DIY: I'll send you the "code" - usually an Excel Add-in - which you can then apply locally to transform your data.
Excel Based BI
Excel-based Business Intelligence
(PowerPivot, PivotTables & Palo OLAP).
Star-Schema Design, the underlying data design for optimal PowerPivot, Palo OLAP or PivotTable cubes.
Amazing, near riot on O'Connell St. over Tony Blair's war record; ignoring his huge contribution to peace in NI; but why no riots over Anglo 2 hours ago
@mcgconor I'd brave Mulvey's back bar but never the Club House, no wonder I'd missed it; must give it a try. 2 hours ago
